# LiteRT-LM

Google's open-source inference framework for deploying large language models on edge devices.

## Agent Decision Summary
- Risk level: low
- Source confidence: high
- Recommended workflows: Local or private AI stack
- Permission surface: external services
- Agent JSON: https://www.openagent.bot/models/litert-lm.agent.json

## Summary
LiteRT-LM is Google's open-source, production-oriented inference framework for running LLMs on edge devices. It is relevant for teams evaluating local, mobile, and on-device agent stacks where latency, privacy, and hardware constraints matter.


## Guide
LiteRT-LM is Google's open-source inference framework for deploying large language models on edge devices.

### What it is
It is a model runtime layer for local and on-device LLM applications rather than a complete agent framework.

### Why it matters
As agents move closer to devices, local inference becomes a core deployment choice.

### How it works
Start from the official repository and documentation, then evaluate latency, supported models, device targets, and integration fit.


### FAQ
- Is LiteRT-LM open source?
  - Yes. The GitHub repository is listed under the Apache-2.0 license.
- Who should evaluate LiteRT-LM?
  - Teams building edge, mobile, desktop, or privacy-sensitive AI applications should evaluate it.
## What It Does
It is a model runtime layer for local and on-device LLM applications rather than a complete agent framework.

## How To Evaluate
Start from the official repository and documentation, then evaluate latency, supported models, device targets, and integration fit.

## Why It Matters
On-device agents need a model layer that can run close to the user. LiteRT-LM matters because it brings Google's edge AI work into an open-source framework for local LLM inference.


## Best For
- Developers building on-device or edge AI applications
- Teams evaluating local LLM deployment for privacy-sensitive agents
- Researchers comparing mobile and embedded inference frameworks

## Not For
- Teams that only need hosted API access to frontier models
- Users looking for a complete agent framework rather than an inference layer

## What It Actually Does
- Edge-first inference: LiteRT-LM focuses on deploying LLMs on edge and on-device environments.
  - Why it matters: Local inference can reduce latency, preserve privacy, and keep agents useful when cloud access is constrained.
- Google AI Edge ecosystem: The project sits under Google's AI Edge GitHub organization.
  - Why it matters: Teams already watching Google's mobile and edge AI stack get a relevant open-source inference option to evaluate.
- Production-oriented model serving: The repository describes LiteRT-LM as a production-ready inference framework.
  - Why it matters: Agent builders need model layers that can move beyond notebooks and into real devices.

## Typical Use Cases
- On-device assistants: Evaluate LiteRT-LM when an assistant needs local responses on mobile, desktop, or embedded hardware.
- Private local inference: Use edge deployment to reduce dependence on cloud APIs for sensitive workflows.
- Model runtime comparison: Compare LiteRT-LM with other local inference projects before choosing an agent model layer.

## How It Compares
- When to choose LiteRT-LM: Compare it with nearby models by looking at hosting model, integration surface, license, and whether the official docs show the workflow you need.

## Fit Matrix
- Local or private AI stack: strong. LiteRT-LM has multiple signals for local or private ai stack, including matching tags, capabilities, category, or positioning. Required check: Verify hardware requirements, data path, storage, and whether all calls stay in your environment.
- Coding agent workflow: partial. LiteRT-LM has at least one signal for coding agent workflow, but should be checked against a real task before adoption. Required check: Run a small repository change and inspect the diff, tests, and rollback path.
- Connector or protocol layer: partial. LiteRT-LM has at least one signal for connector or protocol layer, but should be checked against a real task before adoption. Required check: Connect one low-risk service, then inspect schemas, auth scope, errors, and logs.
- Evaluation and observability: partial. LiteRT-LM has at least one signal for evaluation and observability, but should be checked against a real task before adoption. Required check: Add one repeatable test case and confirm results can run again in review or CI.
- Browser automation: weak. LiteRT-LM is not primarily positioned for browser automation in the current metadata. Required check: Run one non-sensitive website task and inspect clicks, waits, retries, and changed URLs.
- Memory or RAG workflow: weak. LiteRT-LM is not primarily positioned for memory or rag workflow in the current metadata. Required check: Create, update, retrieve, correct, and delete memory or retrieval objects with real data.

## Evidence
- verified: LiteRT-LM is listed as open source. Source: License metadata: Apache-2.0
- verified: LiteRT-LM has a recorded GitHub repository: google-ai-edge/LiteRT-LM. Source: Resource facts and GitHub source link.
- inferred: LiteRT-LM supports these recorded deployment modes: local, cloud. Source: OpenAgent decision signal metadata.
- inferred: LiteRT-LM is tagged with local inference, inference capabilities. Source: OpenAgent capability taxonomy.

## Missing Checks
- Dedicated docs link is missing.
- Repository freshness has not been recorded.

## Next Actions
- Inspect repository: https://github.com/google-ai-edge/LiteRT-LM
- Open Homepage: https://ai.google.dev/edge/litert-lm

## Facts
- Category: models
- Resource type: model
- Open source: yes
- License: Apache-2.0
- Last verified: 2026-06-10
- GitHub repo: google-ai-edge/LiteRT-LM
- GitHub stars: 5524

## Capabilities
- local-inference
- inference

## Structured Use Case Tags
- local-ai

## Getting Started
- Open the GitHub repository: https://github.com/google-ai-edge/LiteRT-LM
- Read the project docs: https://ai.google.dev/edge/litert-lm

## Links
- GitHub: https://github.com/google-ai-edge/LiteRT-LM
- Homepage: https://ai.google.dev/edge/litert-lm

## Structured Outputs
- JSON: https://www.openagent.bot/models/litert-lm.json
- Markdown: https://www.openagent.bot/models/litert-lm.md
- Agent JSON: https://www.openagent.bot/models/litert-lm.agent.json
- Canonical: https://www.openagent.bot/models/litert-lm