AlphaGo wasn’t just a victory for artificial intelligence. When millions of people across the world tuned in to watch DeepMind’s machine beat the human Go world champion Lee Sedol, they also witnessed a historic victory for open-source.
DeepMind used a scientific computing framework called Torch extensively in the development and execution of AlphaGo’s neural networks. Torch was first released back in 2002 under a BSD open-source license with algorithms that are still commonly used by data scientists such as multi-layer perceptrons, support vector machines and K-nearest neighbours. Torch also supported ensembles – a popular technique that combines the output of multiple algorithms, usually with a weighted average.
It’s not just open-source software that contributed to the growth of machine learning. Long before startups and the enterprise became obsessed with artificial intelligence the academic world was openly researching and sharing and building upon their learnings. Christopher Bishop published a research paper in 1995 called Neural Networks for Pattern Recognition that presented the corpus of techniques that took machine learning from a statistical science to one inspired by the biological networks in our brains. Geoffrey Hinton noted in the foreward that Bishop “has wisely avoided the temptation to try to cover everything and has omitted interesting topics such as reinforcement learning, Hopfield Networks and Boltzmann machines in order to focus on the types of neural networks that are most widely used in practical applications”. DeepMind famously employed these techniques almost 20 years later to create a generalised AI that can learn to play Atari games at a superhuman level. When Bishop published his 2006 paper he was at Microsoft research.
So, why the history lesson? To make the point that if you look close artificial intelligence has always been open-source, and open R&D is a core reason why AI is where it is today.
I have been building technology start-ups since 2003. Throughout the years I observed a trend towards the commoditization of machine learning algorithms and the data wrangling tools to deploy these techniques in the real world. The team at Seldon had been hand-crafting recommendation algorithms for many years. We adopted Hadoop back in 2011 to scale our data processing capabilities beyond programmatic and relational databases. Hadoop had a sister called project Apache Mahout that bundled a variety of machine learning algorithms. A few years later Apache Spark revolutionised the computation of streaming data and came bundled with a machine learning library called MLlib. By mid–2014, PredictionIO had released an open-source machine learning server – the first to provide a full stack solution with tools to build, deploy and optimise machine learning models. The talented team at PredictionIO are now part of Salesforce, where they most likely played a significant role in the development of Einstein — the new AI platform baked into the Salesforce cloud ecosystem.
Following Seldon’s first release in February 2015, our open-source project quickly built a community of thousands of data scientist and developers from around the world. Through the lens of the business world, open-sourcing something as advanced as a machine learning platform was new and exciting. It contradicted the start-up playbook of locking down your code and providing solutions from a black box. I had observed signals of a trend towards commoditization, but 2015 became the year that open-source machine learning hit the prime time.
Towards the end of 2015, Google, Microsoft and IBM all threw down their cards. Google received the most press and developer attention with TensorFlow, which comes as no surprise as machine learning has been at the core of the search and advertising products that enabled Google to build one of the most innovative and successful technology companies in history. At an earnings call in October 2015, Google CEO Sundar Pichai announced that they were “rethinking everything in machine learning”. By the time the year drew to a close, an A-list of Silicon Valley entrepreneurs, investors and world class researchers had formed OpenAI, a nonprofit with an ultimate aim of solving AI in a way that Elon Musk’s well cited fear of strong AI under the control of bad actors.
From my perspective, there are three reasons for tech giants release open-source machine learning projects:
- To recruit engineers who have already started to engage and build empathy via the open-source project.
- To control a popular machine learning platform that works best and plays into their broader SDK or cloud platform strategy.
- To grow the entire market because their market share has reached a saturation point and the best way to increase their market cap is to expand the market in which it operates.
When a startup releases an open-source deep tech project, it generates awareness, some of which converts into paid customers and recruitment. Start-ups by their very definition are trying to get a foothold in a market instead of growing an existing market. But is there anything wrong with an early stage startup seeking the expand the market with open-source? It may sound delusional, but I believe open-source start-ups and projects are better able to adopt the same mindset that anything that expands the market is also a good thing. Open-source is frictionless; it costs nothing to serve another organic user and enable organisations to solve real problems, which allows the code to make a much bigger impact. Quite often when we meet companies for the first time they already have already started testing the platform and have an internal technical evangelist.
Instead of disrupting the start-ups building proprietary technologies, open-source has given the world a taller pair of shoulders to stand on. One of the knock-on effects, I hope, will be a shift in focus on where the value lies. With the commoditization of the full AI technology stack, the focus shifts from core machine learning technologies to building the best models – and this requires a vast amount of data and domain experts to create to train the models. Large incumbent businesses with an existing network effect have a natural advantage.
Seldon has been focusing this year on solving problems in the financial sector because banks are data-rich and have a multitude of high-value use cases. Being part of Barclays Accelerator enabled us to sandbox ideas very quickly across Barclays. I was able to announce at our Demo Day in April that Seldon is working on a project to help Barclaycard predict which of its customers are likely to go into arrears so that the bank can take measures to support the customer before they get into financial difficulty. Machine learning models are much more effective at spotting patterns of behaviour earlier in the transaction history of a customer. While banks are now moving toward using open-source technologies, it will still take a while for the culture the change to the extent that they also contribute directly to projects in the same way that employees of technology companies now regularly do.
So, with all of these new open-source machine learning tools in the wild, how is Seldon unique? The short answer is that Seldon is independent and platform-agnostic. One of our core principles is to give developers and data scientists the best tools for the job, regardless of whether the component technologies are built by Seldon engineers or come part of another project. For example, our most recent release supports TensorFlow neural network models, and we created a demo and tutorial to show you how to create a digit classifier. And unlike most machine learning APIs, Seldon is not tied to any cloud platform, and you are free to deploy on premise.
There have never been so many freely available tools for data scientists and developers to accelerate progress. So where will open-source take us? When Sam Altman asked Elon Musk in a recent interview, he casually prophesied a good outcome where we could become “AI-human symbiotes” by improving the neural link between our context and your digital extension. I can safely say superintelligent androids aren’t yet in Seldon’s roadmap, but we sure are living in an exciting time. How are you using open-source technologies in your project or business? And what do you think the future will hold? I’d love to hear from you – please leave your comments below.
To get started with Seldon, check out our technical documentation and open-source project.