# Megatron-LM

Framework · https://godooo.ai/en/framework/megatron-lm

> Megatron-LM is an open-source AI framework from NVIDIA, launched in 2019, made for developers and businesses.

A library for training transformer models at scale across thousands of GPUs.

Main jobs: Model training. It is open source and free to use. You can use it on your own server and in the terminal.

## At a glance

- **Type:** [Framework](https://godooo.ai/en/browse#framework)
- **Made by:** [NVIDIA](https://godooo.ai/en/org/nvidia)
- **Launched:** 2019
- **Galaxy:** [Models & APIs](https://godooo.ai/en/galaxy/builders)
- **Open source:** Yes
- **Source code:** [github.com/NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
- **GitHub stars:** 17,996
- **How you run it:** Self-host · Terminal
- **Made for:** Developers · Businesses
- **Website:** [docs.nvidia.com](https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/quickstart.html)
- **Also known as:** Megatron Core

## What it can do

- **[Model training](https://godooo.ai/en/capability/model-training)** — main job, _proposed by a machine, awaiting calibration_. “GPU-optimized library for training transformer models at scale” (read on [github.com](https://github.com/NVIDIA/Megatron-LM), 2026-09-24)

## Alternatives

- **[PyTorch](https://godooo.ai/en/framework/pytorch)** (Open source, free): Also does: Model training; runs offline
- **[TensorFlow](https://godooo.ai/en/framework/tensorflow)** (Open source, free): Also does: Model training; runs offline
- **[Transformers](https://godooo.ai/en/framework/transformers)** (Open source, free): Also does: Model training; runs offline
- **[JAX](https://godooo.ai/en/framework/jax)** (Open source, free): Also does: Model training; runs offline
- **[DeepSpeed](https://godooo.ai/en/framework/deepspeed)** (Open source, free): Also does: Model training; runs offline
- **[MLX](https://godooo.ai/en/framework/mlx)** (Open source, free): Also does: Model training; runs offline
- **[nanoGPT](https://godooo.ai/en/framework/nanogpt)** (Open source, free): Also does: Model training; runs offline
- **[Hugging Face](https://godooo.ai/en/product/huggingface)** (Free plan + paid upgrades): Also does: Model training; runs offline
- **[ModelScope](https://godooo.ai/en/product/modelscope)** (Open source, free): Also does: Model training; runs offline
- **[AudioCraft](https://godooo.ai/en/framework/audiocraft)** (Open source, free): Also does: Model training; runs offline
- **[Diffusers](https://godooo.ai/en/framework/diffusers)** (Open source, free): Also does: Model training; runs offline
- **[Kohya's GUI](https://godooo.ai/en/product/kohya-ss)** (Open source, free): Also does: Model training; runs offline

## Connections

- **Applies:** [Mixture of Experts (MoE)](https://godooo.ai/en/concept/mixture-of-experts)
- **Built with:** [PyTorch](https://godooo.ai/en/framework/pytorch)

## Questions

### What is Megatron-LM?

Megatron-LM is an open-source AI framework from NVIDIA, launched in 2019, made for developers and businesses. A library for training transformer models at scale across thousands of GPUs.

### Is Megatron-LM free?

Yes. Megatron-LM is open source and free to use.

### Is Megatron-LM open source?

Yes. The source code is at https://github.com/NVIDIA/Megatron-LM.

### Can Megatron-LM run locally?

Megatron-LM can be self-hosted on your own server.

### What can Megatron-LM do?

Main jobs: Model training.

### Who makes Megatron-LM?

Megatron-LM is made by NVIDIA.

### What are open-source alternatives to Megatron-LM?

Open-source ones: PyTorch, TensorFlow, Transformers, JAX, DeepSpeed, MLX, nanoGPT and ModelScope. Others: Hugging Face.

## Sources

- [github.com](https://github.com/NVIDIA/Megatron-LM) — checked 2026-09-24
- [docs.nvidia.com](https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/quickstart.html)

---

Last updated 2026-09-24 · sources checked 2026-09-24 · Markdown version of https://godooo.ai/en/framework/megatron-lm. Every page on this site has one: add .md to its address.