NousResearch/DisTrO
Distributed Training Over-The-Internet
trainingdistributedoptimizerllm
주요 지표
스타 성장
스타
1.1k
포크
60
주간 성장
+1
이슈
1
5001k
2025년 9월2025년 11월2026년 1월2026년 3월2026년 6월2026년 8월2026년 10월
README
DisTrO
This is the repository for DisTrO (Distributed Training Over-The-Internet), a family of low latency distributed optimizers that reduce inter-GPU communication requirements by three to four orders of magnitude.
- Aug. 26th, 2024: DisTrO (Preliminary Report)
- Dec. 2nd, 2024: DeMo Optimization (Paper) (Code), original seed research/idea for DisTrO
- Dec. 2nd, 2024: Nous trains a 15b model using DisTrO
- May 14th, 2025: Psyche Network
- May 14th, 2025: Nous Consilience 40b LLM, Huggingface
- Oct. 14th, 2025: DeMo Optimization (Paper v2), (Production Code)
Join us on Discord if you're interested in helping research and build the future of distributed training.