# LMDeploy

Framework · https://godooo.ai/en/framework/lmdeploy

> LMDeploy is an open-source AI framework, launched in 2023, made for developers.

A toolkit for compressing, deploying and serving LLMs with fast inference engines.

Main jobs: LLM API, Model serving. It is open source and free to use. You can use it on your own server, in the terminal and through an API.

## At a glance

- **Type:** [Framework](https://godooo.ai/en/browse#framework)
- **Launched:** 2023
- **Galaxy:** [Models & APIs](https://godooo.ai/en/galaxy/builders)
- **Open source:** Yes · Apache-2.0
- **Source code:** [github.com/InternLM/lmdeploy](https://github.com/InternLM/lmdeploy)
- **GitHub stars:** 8,097
- **How you run it:** Self-host · Terminal · API
- **Made for:** Developers
- **Website:** [lmdeploy.readthedocs.io](https://lmdeploy.readthedocs.io/en/latest)

## What it can do

- **[LLM API](https://godooo.ai/en/capability/llm-api)** — main job, _proposed by a machine, awaiting calibration_. “OpenAI Compatible Server” (read on [lmdeploy.readthedocs.io](https://lmdeploy.readthedocs.io/en/latest/), 2026-09-24)
- **[Model serving](https://godooo.ai/en/capability/model-serving)** — main job, _proposed by a machine, awaiting calibration_. “LMDeploy is a toolkit for compressing, deploying, and serving LLM” (read on [lmdeploy.readthedocs.io](https://lmdeploy.readthedocs.io/en/latest/), 2026-09-24)

## Alternatives

- **[vLLM](https://godooo.ai/en/framework/vllm)** (Open source, free): Also does: Model serving, LLM API; runs offline
- **[LocalAI](https://godooo.ai/en/framework/localai)** (Open source, free): Also does: LLM API, Model serving; runs offline
- **[exo](https://godooo.ai/en/framework/exo)** (Open source, free): Also does: Model serving, LLM API; runs offline
- **[MLC LLM](https://godooo.ai/en/framework/mlc-llm)** (Open source, free): Also does: Model serving, LLM API; runs offline
- **[Replicate](https://godooo.ai/en/product/replicate)** (Paid): Also does: LLM API, Model serving
- **[SGLang](https://godooo.ai/en/framework/sglang)** (Open source, free): Also does: Model serving, LLM API; runs offline
- **[TensorRT LLM](https://godooo.ai/en/framework/tensorrt-llm)** (Open source, free): Also does: Model serving, LLM API
- **[Together AI](https://godooo.ai/en/product/together)** (Paid): Also does: LLM API, Model serving
- **[Claude](https://godooo.ai/en/model/claude)**: Also does: LLM API; no code needed
- **[Gemini](https://godooo.ai/en/model/gemini)**: Also does: LLM API; no code needed
- **[GPT](https://godooo.ai/en/model/gpt)**: Also does: LLM API; no code needed
- **[GPT-3](https://godooo.ai/en/model/gpt-3)**: Also does: LLM API

## Connections

- **Applies:** [Quantization](https://godooo.ai/en/concept/quantization)

## Questions

### What is LMDeploy?

LMDeploy is an open-source AI framework, launched in 2023, made for developers. A toolkit for compressing, deploying and serving LLMs with fast inference engines.

### Is LMDeploy free?

Yes. LMDeploy is open source (Apache-2.0) and free to use.

### Is LMDeploy open source?

Yes, under the Apache-2.0 license. The source code is at https://github.com/InternLM/lmdeploy.

### Can LMDeploy run locally?

LMDeploy can be self-hosted on your own server.

### What can LMDeploy do?

Main jobs: LLM API, Model serving.

### What are open-source alternatives to LMDeploy?

Open-source ones: vLLM, LocalAI, exo, MLC LLM, SGLang and TensorRT LLM. Others: Replicate, Together AI, Claude, Gemini, GPT and GPT-3.

## Sources

- [lmdeploy.readthedocs.io](https://lmdeploy.readthedocs.io/en/latest/) — checked 2026-09-24
- [github.com](https://github.com/InternLM/lmdeploy)

---

Last updated 2026-09-24 · sources checked 2026-09-24 · Markdown version of https://godooo.ai/en/framework/lmdeploy. Every page on this site has one: add .md to its address.