Breaking News
Australia's central bank has equity wiped out by billions in bond losses
Epic Discussion Feat. Dr. Judy Mikovits, Patrick Byrne And Mikki Willis
Is it Time to Dissolve the British Commonwealth?
Poland Begins Distribution of Iodine Pills for "Nuclear" Emergencies;
Top Tech News
SpaceX's Starlink internet reaches Antarctica, touching all 7 continents
It's Insane. My Solar Generates up to 180kWh in a Day + Passive Solar Greenhouse Tour
'World's Most Advanced' Humanoid Robot Promises Not To 'Take Over The World'
Do Authoritarians 'Care' About You?
DARPA Vertical Takeoff and Landing X-Plane
Biden is blocking DeSantis from letting Floridians access cheaper medication
How salt and sand could replace lithium batteries
Quantum Heat Pump Made From Particles of Light
Three Eras of Machine Learning and Predicting the Future of AI
Outrageous armored submarine claims 50-km/h underwater top speed
Three Eras of Machine Learning and Predicting the Future of AI
They show :
before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months.
Deep Learning started in the early 2010s and the scaling of training compute has accelerated, doubling approximately every 6 months.
In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute.
Based on these observations they split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era . Overall, the work highlights the fast-growing compute requirements for training advanced ML systems.
They have detailed investigation into the compute demand of milestone ML models over time. They make the following contributions:
1. They curate a dataset of 123 milestone Machine Learning systems, annotated with the compute it took to train them.