>
Down with Big Brother: Warrantless Surveillance Makes a Mockery of the Constitution
Skynet 2024: The Infrastructure is Complete!
Derivatives are a $200 trillion bubble right now...
Patrick Mahomes reveals why he has NOT called for tighter gun laws after shooting at Kansas City...
Blazing bits transmitted 4.5 million times faster than broadband
Scientists Close To Controlling All Genetic Material On Earth
Doodle to reality: World's 1st nuclear fusion-powered electric propulsion drive
Phase-change concrete melts snow and ice without salt or shovels
You Won't Want To Miss THIS During The Total Solar Eclipse (3D Eclipse Timeline And Viewing Tips
China Room Temperature Superconductor Researcher Had Experiments to Refute Critics
5 video games we wanna smell, now that it's kinda possible with GameScent
Unpowered cargo gliders on tow ropes promise 65% cheaper air freight
Wyoming A Finalist For Factory To Build Portable Micro-Nuclear Plants
They show :
before 2010 training compute grew in line with Moore's law, doubling roughly every 20 months.
Deep Learning started in the early 2010s and the scaling of training compute has accelerated, doubling approximately every 6 months.
In late 2015, a new trend emerged as firms developed large-scale ML models with 10 to 100-fold larger requirements in training compute.
Based on these observations they split the history of compute in ML into three eras: the Pre Deep Learning Era, the Deep Learning Era and the Large-Scale Era . Overall, the work highlights the fast-growing compute requirements for training advanced ML systems.
They have detailed investigation into the compute demand of milestone ML models over time. They make the following contributions:
1. They curate a dataset of 123 milestone Machine Learning systems, annotated with the compute it took to train them.