OpenAI Finds Machine Learning Efficiency Is Outpacing Moore’s Law
OpenAI Finds Machine Learning Efficiency Is Outpacing Moore’s Law
OpenAI Finds Machine Learning Efficiency Is Outpacing Moore’s Law
By Jason Dorrier
May 17, 2020
Summary
Eight years ago a machine learning algorithm learned to identify a cat-and it stunned the world.
Now, machine learning has begun to excel at complex multiplayer video games like Starcraft and Dota 2 and subtle games like poker.
How fast is fast, and what's driving the pace? While better computer chips are key, AI research organization OpenAI thinks we should measure the pace of improvement of the actual machine learning algorithms too.
In a blog post and paper-authored by OpenAI's Danny Hernandez and Tom Brown and published on the arXiv, an open repository for pre-print studies-the researchers say they've begun tracking a new measure for machine learning efficiency.
Why track algorithmic efficiency? The authors say that three inputs drive progress in machine learning: available computing power, data, and algorithmic innovation.
Is there a kind of algorithmic Moore's Law in machine learning? Maybe.
The Future of AI. It's worth noting the study focuses on deep learning algorithms, the dominant AI approach at the moment.
Whether deep learning continues to make such dramatic progress is a source of debate in the AI community.
Whereas growth in the amount of computing power used by AI programs prior to 2012 largely tracked Moore's Law, the computing power used by machine learning algorithms since 2012 has been growing seven times faster than Moore's Law.
If machine learning algorithms are getting more expensive to train, for example, it's important to increase funding to academic researchers so they can keep up with private efforts.
Reference
Dorrier, J. (2020, May 17). OpenAI Finds Machine Learning Efficiency Is Outpacing Moore's Law. Retrieved May 20, 2020, from https://singularityhub.com/2020/05/17/openai-finds-machine-learning-efficiency-is-outpacing-moores-law/