Breaking News
U.S. Strengths and Weaknesses in Production
Washington Lost the Equivalent of Two Apples. Did Anyone Notice?
The Light at The End of the Tunnel - It's Not All Doom & Gloom
Iran has published a naive letter to the American People
Top Tech News
A laser just photographed objects through six feet of concrete
Elon Musk's Next-Gen Motor Destroy Entire EV Industry
China's disputed satellite refueling heralds new space war era
BYD Will Put Solid-State Batteries In An EV Next Year: Executive
FDA-cleared exoskeleton puts spinal-cord patients back on their feet
Your Robotic Vacuum Is Watching You -- Could It Someday Testify Against You in Court?
Why Unigrid's Sodium-Ion Batteries Are the Game-Changer for Off-Grid Energy Storage
This Battery On Wheels Makes Any Diesel Truck Electric In 5 Minutes
Mass-Production of AI Infrastructure – Speed Up by 10X
3D-printed Martian mud homes look critical for life beyond Earth
Three Eras of Machine Learning and Predicting the Future of AI
They show :
before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months.
Deep Learning started in the early 2010s and the scaling of training compute has accelerated, doubling approximately every 6 months.
In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute.
Based on these observations they split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era . Overall, the work highlights the fast-growing compute requirements for training advanced ML systems.
They have detailed investigation into the compute demand of milestone ML models over time. They make the following contributions:
1. They curate a dataset of 123 milestone Machine Learning systems, annotated with the compute it took to train them.