# TRL

Framework · https://godooo.ai/en/framework/trl

> TRL is an open-source AI framework from Hugging Face, launched in 2020, made for developers.

A library for post-training language models with SFT, DPO, GRPO and other methods.

Main jobs: Fine-tuning. It is open source and free to use. You can use it on your own server, offline on your own machine and in the terminal.

## At a glance

- **Type:** [Framework](https://godooo.ai/en/browse#framework)
- **Made by:** [Hugging Face](https://godooo.ai/en/org/hugging-face)
- **Launched:** 2020
- **Galaxy:** [Models & APIs](https://godooo.ai/en/galaxy/builders)
- **Open source:** Yes · Apache-2.0
- **Source code:** [github.com/huggingface/trl](https://github.com/huggingface/trl)
- **GitHub stars:** 19,377
- **How you run it:** Self-host · Runs offline · Terminal
- **Made for:** Developers
- **Website:** [hf.co](https://hf.co/docs/trl)
- **Also known as:** Transformers Reinforcement Learning

## What it can do

- **[Fine-tuning](https://godooo.ai/en/capability/fine-tuning)** — main job, _proposed by a machine, awaiting calibration_. “TRL is a cutting-edge library designed for post-training foundation models using advanced techniques like Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), and Direct Preference Optimization (DPO).” (read on [github.com](https://github.com/huggingface/trl), 2026-09-24)
- **[Model training](https://godooo.ai/en/capability/model-training)** — supported, _proposed by a machine, awaiting calibration_. “TRL is a full stack library where we provide a set of tools to train transformer language models with methods like Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), Direct Preference Optimization (DPO), Reward Modeling, and more.” (read on [huggingface.co](https://huggingface.co/docs/trl/index), 2026-09-24)

## Alternatives

- **[nanoGPT](https://godooo.ai/en/framework/nanogpt)** (Open source, free): Also does: Fine-tuning, Model training
- **[Kohya's GUI](https://godooo.ai/en/product/kohya-ss)** (Open source, free): Also does: Fine-tuning, Model training
- **[LlamaFactory](https://godooo.ai/en/framework/llamafactory)** (Open source, free): Also does: Fine-tuning, Model training
- **[Unsloth](https://godooo.ai/en/framework/unsloth)** (Open source, free): Also does: Fine-tuning, Model training
- **[Axolotl](https://godooo.ai/en/framework/axolotl)** (Open source, free): Also does: Fine-tuning
- **[MLX](https://godooo.ai/en/framework/mlx)** (Open source, free): Also does: Model training, Fine-tuning
- **[PEFT](https://godooo.ai/en/framework/peft)** (Open source, free): Also does: Fine-tuning
- **[Hugging Face](https://godooo.ai/en/product/huggingface)** (Free plan + paid upgrades): Also does: Fine-tuning, Model training
- **[ModelScope](https://godooo.ai/en/product/modelscope)** (Open source, free): Also does: Fine-tuning, Model training
- **[Sentence Transformers](https://godooo.ai/en/framework/sentence-transformers)** (Open source, free): Also does: Fine-tuning, Model training
- **[Together AI](https://godooo.ai/en/product/together)** (Paid): Also does: Fine-tuning, Model training
- **[Modal](https://godooo.ai/en/product/modal)** (Paid): Also does: Fine-tuning, Model training

## Connections

- **Applies:** [RLHF](https://godooo.ai/en/concept/rlhf)
- **Built with:** [Transformers](https://godooo.ai/en/framework/transformers)

## Questions

### What is TRL?

TRL is an open-source AI framework from Hugging Face, launched in 2020, made for developers. A library for post-training language models with SFT, DPO, GRPO and other methods.

### Is TRL free?

Yes. TRL is open source (Apache-2.0) and free to use.

### Is TRL open source?

Yes, under the Apache-2.0 license. The source code is at https://github.com/huggingface/trl.

### Can TRL run locally?

Yes. TRL can run offline on your own machine. It can also be self-hosted on your own server.

### What can TRL do?

Main jobs: Fine-tuning. It can also do: Model training.

### Who makes TRL?

TRL is made by Hugging Face.

### What are open-source alternatives to TRL?

Open-source ones: nanoGPT, Kohya's GUI, LlamaFactory, Unsloth, Axolotl, MLX, PEFT and ModelScope. Others: Hugging Face, Together AI and Modal.

## Sources

- [github.com](https://github.com/huggingface/trl) — checked 2026-09-24
- [huggingface.co](https://huggingface.co/docs/trl/index) — checked 2026-09-24
- [hf.co](https://hf.co/docs/trl)

---

Last updated 2026-09-24 · sources checked 2026-09-24 · Markdown version of https://godooo.ai/en/framework/trl. Every page on this site has one: add .md to its address.