Research Scientist - Pre-training

من الإعلان الأصلي

San Francisco Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report https://arxiv.org/abs/). While many of the core research problems have been solved, Protocol Learning unlocks a series of new…