Framework
MLC LLM
MLC LLM is an open-source AI framework, launched in 2023, made for developers.
A compiler and engine that deploys LLMs natively on GPUs, phones and in the browser.
Main jobs: Model serving, Run models locally. It is open source and free to use. You can use it in the browser, as a desktop app, on phones, in the terminal, through an API, on your own server and offline on your own machine.
Markdown version of https://godooo.ai/en/framework/mlc-llm. Every page on this site has one: add .md to its address.
At a glance
- Type
- Framework
- Launched
- 2023
- Galaxy
- Models & APIs
- Open source
- Yes · Apache-2.0
- Source code
- github.com/mlc-ai/mlc-llm
- GitHub stars
- 23,185
- How you run it
- Web · Desktop · Mobile · Terminal · API · Self-host · Runs offline
- Made for
- Developers
- Protocols
- OpenAI-compatible API
- Website
- llm.mlc.ai
- Also known as
- MLCEngine
What it can do
Model servingmain jobunconfirmed
“MLCEngine -- a unified high-performance LLM inference engine across the above platforms”— github.com, 2026-09-24
Run models locallymain jobunconfirmed
“enable everyone to develop, optimize, and deploy AI models natively on everyone's platforms”— github.com, 2026-09-24
LLM APIsupportedunconfirmed
“MLCEngine provides OpenAI-compatible API available through REST server, python, javascript, iOS, Android”— github.com, 2026-09-24
unconfirmedproposed by a machine, awaiting calibration
Alternatives
- llama.cppOpen source, freeAlso does: Run models locally, Model serving
- LocalAIOpen source, freeAlso does: LLM API, Model serving, Run models locally
- exoOpen source, freeAlso does: Model serving, Run models locally, LLM API
- OllamaOpen source, freeAlso does: Run models locally, LLM API, Model serving
- WebLLMOpen source, freeAlso does: LLM API, Run models locally
- LMDeployOpen source, freeAlso does: LLM API, Model serving
- GemmaOpen source, freeAlso does: Run models locally, LLM API
- vLLMOpen source, freeAlso does: Model serving, LLM API
- gpt-ossOpen source, freeAlso does: Run models locally, LLM API
- JanOpen source, freeAlso does: Run models locally, LLM API; no code needed
- LM StudioFreeAlso does: Run models locally, LLM API; no code needed
- PhiOpen source, freeAlso does: Run models locally, LLM API
Connections
- Used to build
- WebLLM
Questions
What is MLC LLM?
MLC LLM is an open-source AI framework, launched in 2023, made for developers. A compiler and engine that deploys LLMs natively on GPUs, phones and in the browser.
Is MLC LLM free?
Yes. MLC LLM is open source (Apache-2.0) and free to use.
Is MLC LLM open source?
Yes, under the Apache-2.0 license. The source code is at https://github.com/mlc-ai/mlc-llm.
Can MLC LLM run locally?
Yes. MLC LLM can run offline on your own machine. It can also be self-hosted on your own server.
What can MLC LLM do?
Main jobs: Model serving, Run models locally. It can also do: LLM API.
What are open-source alternatives to MLC LLM?
Open-source ones: llama.cpp, LocalAI, exo, Ollama, WebLLM, LMDeploy, Gemma and vLLM. Others: LM Studio.
Sources
- github.com/mlc-ai/mlc-llmchecked 2026-09-24
- llm.mlc.ai