# MLC LLM

Framework · https://godooo.ai/en/framework/mlc-llm

> MLC LLM is an open-source AI framework, launched in 2023, made for developers.

A compiler and engine that deploys LLMs natively on GPUs, phones and in the browser.

Main jobs: Model serving, Run models locally. It is open source and free to use. You can use it in the browser, as a desktop app, on phones, in the terminal, through an API, on your own server and offline on your own machine.

## At a glance

- **Type:** [Framework](https://godooo.ai/en/browse#framework)
- **Launched:** 2023
- **Galaxy:** [Models & APIs](https://godooo.ai/en/galaxy/builders)
- **Open source:** Yes · Apache-2.0
- **Source code:** [github.com/mlc-ai/mlc-llm](https://github.com/mlc-ai/mlc-llm)
- **GitHub stars:** 23,185
- **How you run it:** Web · Desktop · Mobile · Terminal · API · Self-host · Runs offline
- **Made for:** Developers
- **Protocols:** [OpenAI-compatible API](https://godooo.ai/en/protocol/openai-compatible-api)
- **Website:** [llm.mlc.ai](https://llm.mlc.ai)
- **Also known as:** MLCEngine

## What it can do

- **[Model serving](https://godooo.ai/en/capability/model-serving)** — main job, _proposed by a machine, awaiting calibration_. “MLCEngine -- a unified high-performance LLM inference engine across the above platforms” (read on [github.com](https://github.com/mlc-ai/mlc-llm), 2026-09-24)
- **[Run models locally](https://godooo.ai/en/capability/local-inference)** — main job, _proposed by a machine, awaiting calibration_. “enable everyone to develop, optimize, and deploy AI models natively on everyone's platforms” (read on [github.com](https://github.com/mlc-ai/mlc-llm), 2026-09-24)
- **[LLM API](https://godooo.ai/en/capability/llm-api)** — supported, _proposed by a machine, awaiting calibration_. “MLCEngine provides OpenAI-compatible API available through REST server, python, javascript, iOS, Android” (read on [github.com](https://github.com/mlc-ai/mlc-llm), 2026-09-24)

## Alternatives

- **[llama.cpp](https://godooo.ai/en/framework/llama-cpp)** (Open source, free): Also does: Run models locally, Model serving
- **[LocalAI](https://godooo.ai/en/framework/localai)** (Open source, free): Also does: LLM API, Model serving, Run models locally
- **[exo](https://godooo.ai/en/framework/exo)** (Open source, free): Also does: Model serving, Run models locally, LLM API
- **[Ollama](https://godooo.ai/en/product/ollama)** (Open source, free): Also does: Run models locally, LLM API, Model serving
- **[WebLLM](https://godooo.ai/en/framework/webllm)** (Open source, free): Also does: LLM API, Run models locally
- **[LMDeploy](https://godooo.ai/en/framework/lmdeploy)** (Open source, free): Also does: LLM API, Model serving
- **[Gemma](https://godooo.ai/en/model/gemma)** (Open source, free): Also does: Run models locally, LLM API
- **[vLLM](https://godooo.ai/en/framework/vllm)** (Open source, free): Also does: Model serving, LLM API
- **[gpt-oss](https://godooo.ai/en/model/gpt-oss)** (Open source, free): Also does: Run models locally, LLM API
- **[Jan](https://godooo.ai/en/product/jan)** (Open source, free): Also does: Run models locally, LLM API; no code needed
- **[LM Studio](https://godooo.ai/en/product/lm-studio)** (Free): Also does: Run models locally, LLM API; no code needed
- **[Phi](https://godooo.ai/en/model/phi)** (Open source, free): Also does: Run models locally, LLM API

## Connections

- **Used to build:** [WebLLM](https://godooo.ai/en/framework/webllm)

## Questions

### What is MLC LLM?

MLC LLM is an open-source AI framework, launched in 2023, made for developers. A compiler and engine that deploys LLMs natively on GPUs, phones and in the browser.

### Is MLC LLM free?

Yes. MLC LLM is open source (Apache-2.0) and free to use.

### Is MLC LLM open source?

Yes, under the Apache-2.0 license. The source code is at https://github.com/mlc-ai/mlc-llm.

### Can MLC LLM run locally?

Yes. MLC LLM can run offline on your own machine. It can also be self-hosted on your own server.

### What can MLC LLM do?

Main jobs: Model serving, Run models locally. It can also do: LLM API.

### What are open-source alternatives to MLC LLM?

Open-source ones: llama.cpp, LocalAI, exo, Ollama, WebLLM, LMDeploy, Gemma and vLLM. Others: LM Studio.

## Sources

- [github.com](https://github.com/mlc-ai/mlc-llm) — checked 2026-09-24
- [llm.mlc.ai](https://llm.mlc.ai)

---

Last updated 2026-09-24 · sources checked 2026-09-24 · Markdown version of https://godooo.ai/en/framework/mlc-llm. Every page on this site has one: add .md to its address.