PULSELoCo applies the PULSE compute-visibility rule to DiLoCo-style pseudo-gradients: it matches DiLoCo learning behavior while sending over 17x less data than DiLoCo and about 138x less than DDP on Qwen2.5-7B.
A 72B-parameter language model trained with open, permissionless participation over commodity internet links, using Gauntlet incentives and SparseLoCo compression to keep unreliable peers useful.
RL fine-tuning changes only about 1% of model weights per step. PULSE exploits that sparsity to synchronize policy weights losslessly with over 100x less bandwidth.
A framework for letting resource-limited participants form a SparseLoCo replica through compressed pipeline parallelism while better-connected replicas stay full precision.
The architecture, verification pipeline, and incentives behind one of the first open decentralized RL networks with real-time transparency — miners generate rollouts, validators verify them with the grail proof, and a trainer learns from a live network.
SparseLoCo replaces dense pseudo-gradient exchange with chunked TOP-k sparsification, local error feedback, and 2-bit values while matching or improving DiLoCo-style pre-training.
Gauntlet scores pseudo-gradient contributions from untrusted peers with fast reliability checks, loss-before/after validation, and persistent OpenSkill ratings.
Every paper above started as an open problem. We hire researchers and engineers to work on the next one: optimizers, incentives, and systems for training frontier models over the open internet. We also collaborate with external researchers and labs, and offer research compute grants for promising work on decentralized intelligence.