Framework
Megatron-LM
Megatron-LM is an open-source AI framework from NVIDIA, launched in 2019, made for developers and businesses.
A library for training transformer models at scale across thousands of GPUs.
Main jobs: Model training. It is open source and free to use. You can use it on your own server and in the terminal.
Markdown version of https://godooo.ai/en/framework/megatron-lm. Every page on this site has one: add .md to its address.
At a glance
- Type
- Framework
- Made by
- NVIDIA
- Launched
- 2019
- Galaxy
- Models & APIs
- Open source
- Yes
- Source code
- github.com/NVIDIA/Megatron-LM
- GitHub stars
- 17,996
- How you run it
- Self-host · Terminal
- Made for
- Developers · Businesses
- Website
- docs.nvidia.com
- Also known as
- Megatron Core
What it can do
Model trainingmain jobunconfirmed
“GPU-optimized library for training transformer models at scale”— github.com, 2026-09-24
unconfirmedproposed by a machine, awaiting calibration
Alternatives
- PyTorchOpen source, freeAlso does: Model training; runs offline
- TensorFlowOpen source, freeAlso does: Model training; runs offline
- TransformersOpen source, freeAlso does: Model training; runs offline
- JAXOpen source, freeAlso does: Model training; runs offline
- DeepSpeedOpen source, freeAlso does: Model training; runs offline
- MLXOpen source, freeAlso does: Model training; runs offline
- nanoGPTOpen source, freeAlso does: Model training; runs offline
- Hugging FaceFree plan + paid upgradesAlso does: Model training; runs offline
- ModelScopeOpen source, freeAlso does: Model training; runs offline
- AudioCraftOpen source, freeAlso does: Model training; runs offline
- DiffusersOpen source, freeAlso does: Model training; runs offline
- Kohya's GUIOpen source, freeAlso does: Model training; runs offline
Connections
- Applies
- Mixture of Experts (MoE)
- Built with
- PyTorch
Questions
What is Megatron-LM?
Megatron-LM is an open-source AI framework from NVIDIA, launched in 2019, made for developers and businesses. A library for training transformer models at scale across thousands of GPUs.
Is Megatron-LM free?
Yes. Megatron-LM is open source and free to use.
Is Megatron-LM open source?
Yes. The source code is at https://github.com/NVIDIA/Megatron-LM.
Can Megatron-LM run locally?
Megatron-LM can be self-hosted on your own server.
What can Megatron-LM do?
Main jobs: Model training.
Who makes Megatron-LM?
Megatron-LM is made by NVIDIA.
What are open-source alternatives to Megatron-LM?
Open-source ones: PyTorch, TensorFlow, Transformers, JAX, DeepSpeed, MLX, nanoGPT and ModelScope. Others: Hugging Face.