LIGHTRON
// A light implementation of megatron for research and study
Lightron
A light implementation of megatron for research and study
13EmergingUnknown
What it does
Lightron is a lightweight, educational, yet modern distributed training framework for LLMs. Lightron aims to bridge the gap between minimal implementations and modern production features such as 4-D Parallelism, including Tensor Parallelism, Pipeline Parallelism, Data Parallelism, and Context Parallelism. - Distributed Ready: Support 4-D Parallelism(TP, PP, DP, CP), EP and FSDP V2. - Modern
Getting Started
git
git clone https://github.com/lwj2015/lightron