Breaking News
Government shutdown triggers travel nightmare as controller shortages force ground stops...
How a natural PEPTIDE helped me REGENERATE injured tissue
No Vacation For You!
Asteroid Threat Detection and Planetary Defense Can Be Complete and Ready by 2035
Top Tech News
Graphene Dream Becomes a Reality as Miracle Material Enters Production for Better Chips, Batteries
Virtual Fencing May Allow Thousands More Cattle to Be Ranched on Land Rather Than in Barns
Prominent Personalities Sign Letter Seeking Ban On 'Development Of Superintelligence'
Why 'Mirror Life' Is Causing Some Genetic Scientists To Freak Out
Retina e-paper promises screens 'visually indistinguishable from reality'
Scientists baffled as interstellar visitor appears to reverse thrust before vanishing behind the sun
Future of Satellite of Direct to Cellphone
Amazon goes nuclear with new modular reactor plant
China Is Making 800-Mile EV Batteries. Here's Why America Can't Have Them
China Innovates: Transforming Sand into Paper
Three Eras of Machine Learning and Predicting the Future of AI
They show :
before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months.
Deep Learning started in the early 2010s and the scaling of training compute has accelerated, doubling approximately every 6 months.
In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute.
Based on these observations they split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era . Overall, the work highlights the fast-growing compute requirements for training advanced ML systems.
They have detailed investigation into the compute demand of milestone ML models over time. They make the following contributions:
1. They curate a dataset of 123 milestone Machine Learning systems, annotated with the compute it took to train them.