[[{“value”:”What has surprised you the most this year? The development of decentralized training for AI models, from DiLoCo and DiPaCo from Google DeepMind to Distro coming up from Nous Research. It completely changes the game in terms of how we ought to think about large models and what we can do to train them. This means pure compute thresholds are not
The post What surprised you the most this year? appeared first on Marginal REVOLUTION.”}]]
What has surprised you the most this year?
The development of decentralized training for AI models, from DiLoCo and DiPaCo from Google DeepMind to Distro coming up from Nous Research. It completely changes the game in terms of how we ought to think about large models and what we can do to train them. This means pure compute thresholds are not going to be very useful, and that we will have even less of a way to centrally control the means of knowledge production.
That is from Rohit Krishnan, who answers other questions as well. Interviewed by Derek Robertson, via Mike Doherty.
The post What surprised you the most this year? appeared first on Marginal REVOLUTION.
Uncategorized, Web/Tech
Leave a Reply