Framework
LMDeploy
LMDeploy is an open-source AI framework, launched in 2023, made for developers.
A toolkit for compressing, deploying and serving LLMs with fast inference engines.
Main jobs: LLM API, Model serving. It is open source and free to use. You can use it on your own server, in the terminal and through an API.
Markdown version of https://godooo.ai/en/framework/lmdeploy. Every page on this site has one: add .md to its address.
At a glance
- Type
- Framework
- Launched
- 2023
- Galaxy
- Models & APIs
- Open source
- Yes · Apache-2.0
- Source code
- github.com/InternLM/lmdeploy
- GitHub stars
- 8,097
- How you run it
- Self-host · Terminal · API
- Made for
- Developers
- Website
- lmdeploy.readthedocs.io
What it can do
LLM APImain jobunconfirmed
“OpenAI Compatible Server”— lmdeploy.readthedocs.io, 2026-09-24
Model servingmain jobunconfirmed
“LMDeploy is a toolkit for compressing, deploying, and serving LLM”— lmdeploy.readthedocs.io, 2026-09-24
unconfirmedproposed by a machine, awaiting calibration
Alternatives
- vLLMOpen source, freeAlso does: Model serving, LLM API; runs offline
- LocalAIOpen source, freeAlso does: LLM API, Model serving; runs offline
- exoOpen source, freeAlso does: Model serving, LLM API; runs offline
- MLC LLMOpen source, freeAlso does: Model serving, LLM API; runs offline
- ReplicatePaidAlso does: LLM API, Model serving
- SGLangOpen source, freeAlso does: Model serving, LLM API; runs offline
- TensorRT LLMOpen source, freeAlso does: Model serving, LLM API
- Together AIPaidAlso does: LLM API, Model serving
- ClaudeAlso does: LLM API; no code needed
- GeminiAlso does: LLM API; no code needed
- GPTAlso does: LLM API; no code needed
- GPT-3Also does: LLM API
Connections
- Applies
- Quantization
Questions
What is LMDeploy?
LMDeploy is an open-source AI framework, launched in 2023, made for developers. A toolkit for compressing, deploying and serving LLMs with fast inference engines.
Is LMDeploy free?
Yes. LMDeploy is open source (Apache-2.0) and free to use.
Is LMDeploy open source?
Yes, under the Apache-2.0 license. The source code is at https://github.com/InternLM/lmdeploy.
Can LMDeploy run locally?
LMDeploy can be self-hosted on your own server.
What can LMDeploy do?
Main jobs: LLM API, Model serving.
What are open-source alternatives to LMDeploy?
Open-source ones: vLLM, LocalAI, exo, MLC LLM, SGLang and TensorRT LLM. Others: Replicate, Together AI, Claude, Gemini, GPT and GPT-3.
Sources
- lmdeploy.readthedocs.io/en/latest/checked 2026-09-24
- github.com/InternLM/lmdeploy