Instructions to use Ajay0007/receipt-extractor-lfm2.5-350m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Ajay0007/receipt-extractor-lfm2.5-350m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Ajay0007/receipt-extractor-lfm2.5-350m") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Ajay0007/receipt-extractor-lfm2.5-350m") model = AutoModelForCausalLM.from_pretrained("Ajay0007/receipt-extractor-lfm2.5-350m", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - LiteRT
How to use Ajay0007/receipt-extractor-lfm2.5-350m with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- LiteRT-LM
How to use Ajay0007/receipt-extractor-lfm2.5-350m with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=Ajay0007/receipt-extractor-lfm2.5-350m \ --prompt="Write me a poem"
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Ajay0007/receipt-extractor-lfm2.5-350m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Ajay0007/receipt-extractor-lfm2.5-350m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ajay0007/receipt-extractor-lfm2.5-350m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Ajay0007/receipt-extractor-lfm2.5-350m
- SGLang
How to use Ajay0007/receipt-extractor-lfm2.5-350m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Ajay0007/receipt-extractor-lfm2.5-350m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ajay0007/receipt-extractor-lfm2.5-350m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Ajay0007/receipt-extractor-lfm2.5-350m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ajay0007/receipt-extractor-lfm2.5-350m", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Ajay0007/receipt-extractor-lfm2.5-350m with Docker Model Runner:
docker model run hf.co/Ajay0007/receipt-extractor-lfm2.5-350m
receipt-extractor-lfm2.5-350m
Turn a receipt into structured JSON on the phone. This small model reads the OCR text of a receipt photo and returns line items, subtotal, tax, service charge/tip, discount and total as JSON. Built for an on-device, Splitwise-style "snap the bill and split it by item" app.
- Base model:
LiquidAI/LFM2.5-350M(354M parameters), fully fine-tuned - Input: OCR text of a receipt (one visual line per row), from Google ML Kit (Android), Apple Vision (iOS), PaddleOCR, etc.
- Output: compact JSON:
{"items":[{"name","qty","price"}], "subtotal", "tax", "service", "discount", "total"} - On-device: includes
model.litertlm(int8, 364 MB) for Google's LiteRT-LM runtime (Android, iOS, desktop) - Speed: median 0.8 s, p90 1.9 s per receipt in the official LiteRT-LM runtime (int8, 4 CPU threads on a server CPU, excluding OCR). Expect similar or better on a recent phone; please benchmark on your target devices.
How it fits in an app
photo ──► on-device OCR (ML Kit / Apple Vision, ~0.3 s) ──► this model (~1 s) ──► JSON ──► sum check ──► split UI
└─ fails → ask the user to review
The sum check (do item prices + tax + service − discount equal the total?) is a strong confidence signal: in our
tests, receipts that pass it were fully correct ~89% of the time, versus ~25% for receipts that fail it.
receipt_utils.sum_check() in this repo implements it.
Quick start (Python / transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import hf_hub_download
import importlib.util, sys
repo = "Ajay0007/receipt-extractor-lfm2.5-350m"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
# helper functions shipped with the model (prompt, OCR line grouping, JSON parsing, sum check)
spec = importlib.util.spec_from_file_location("receipt_utils", hf_hub_download(repo, "receipt_utils.py"))
ru = importlib.util.module_from_spec(spec); spec.loader.exec_module(ru)
ocr_text = """1 EGG TART 13,000
2 CHOCO CUSTARD PASTRY 24,000
REDBEAN BREAD 9,000
TOTAL 46,000
CASH 50,000
CHANGE 4,000"""
messages = [{"role": "user", "content": ru.build_prompt(ocr_text)}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_dict=True,
return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
receipt = ru.parse_output(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
print(receipt)
print("sum check passed:", ru.sum_check(receipt))
Output for the example above:
{"items": [{"name": "EGG TART", "qty": "1", "price": "13,000"}, {"name": "CHOCO CUSTARD PASTRY", "qty": "2", "price": "24,000"}, {"name": "REDBEAN BREAD", "qty": "1", "price": "9,000"}], "total": "46,000"}
Use the prompt exactly as build_prompt writes it. The model was trained on that wording. Use greedy decoding
(do_sample=False).
Getting OCR text in the right format
The model expects one visual receipt line per row, with the tokens on a line ordered left to right (so an item
name and its price sit on the same row). Raw OCR engines return boxes, not lines. Use
receipt_utils.ocr_boxes_to_text([(x_min, y_min, y_max, text), ...]), which is the same grouping used to build the
training data. Port this ~20-line function to Kotlin/Swift for your app.
On-device: LiteRT-LM (Android / iOS)
model.litertlm is an int8 build (weights int8, activations fp32) made with
litert-torch (dynamic_wi8_afp32, prefill 256, KV cache 1024 tokens).
Measured on 100 receipts per set in the LiteRT-LM runtime (4 CPU threads): CORD split-ready 70% / total 94%, WildReceipt split-ready 31% / total 88%. That is within ~1 point of the bf16 weights on the same receipts, so int8 conversion costs essentially no accuracy. Latency: median 0.82 s, p90 1.90 s.
import litert_lm # pip install litert-lm-api
from huggingface_hub import hf_hub_download
path = hf_hub_download("Ajay0007/receipt-extractor-lfm2.5-350m", "model.litertlm")
engine = litert_lm.Engine(path, backend=litert_lm.Backend.CPU(thread_count=4), max_num_tokens=2048)
conv = engine.create_conversation(sampler_config=litert_lm.SamplerConfig(top_k=1, temperature=0.0),
max_output_tokens=768)
reply = conv.send_message(ru.build_prompt(ocr_text))
print(reply["content"] if isinstance(reply, dict) else reply)
For apps, load the same file with the LiteRT-LM Kotlin API (Android, stable) or Swift API (iOS), or use the Google AI Edge Gallery app to try it on a phone and see its speed.
Training data
| Dataset | Receipts used | Region / language | Labels used |
|---|---|---|---|
| CORD-v2 (CC-BY-4.0), train split | 800 | Indonesia (Indonesian/English, IDR) | items, priced add-ons, subtotal, tax, service, discount, total |
| WildReceipt, train split | 1,013 | mostly US/EU photos (English, some German/French) | items, subtotal, tax, tip→service, total |
| Total | 1,813 |
Inputs were produced by running RapidOCR (PP-OCR) on the images, so the model learned from realistic, noisy OCR, not clean ground-truth text. CORD labels were corrected: priced add-ons stored as sub-items in CORD (e.g. "Extra Jelly 5,000") were restored, and a few annotation errors fixed. Store names, addresses, dates and payment lines (cash, change, card) are deliberately not extracted.
Training procedure
Full fine-tune (all weights) with plain supervised next-token loss on the assistant JSON only (prompt tokens masked).
| Setting | Value |
|---|---|
| Epochs | 5 |
| Learning rate / schedule | 5e-5, cosine, 5% warmup |
| Batch size | 8 |
| Max sequence length | 2,048 tokens |
| Precision | bf16, gradient checkpointing |
| Hardware / time | 1× NVIDIA A40, ~6 min |
| Target format | compact JSON, null fields dropped (shorter output = faster on device) |
We also tested LoRA (r=16, all linear layers, lr 2e-4): it matched the full fine-tune on CORD (70% vs 72% split-ready) and is a good option if you want small swappable adapters.
Evaluation
All numbers use OCR from RapidOCR, greedy decoding and the bf16 weights unless noted. Split-ready = every line
item (name + price) and the total are exactly right, so the bill can be split with no edits. Item F1 = item
matched on name (fuzzy ≥ 80) and exact price. Amounts are compared by value (60.000 = 60,000, 525 = 525.00).
This model
| Test set | Item F1 | Total correct | Split-ready |
|---|---|---|---|
| CORD-v2 test+val, corrected labels (200, Indonesia) | 86.3% | 95.9% | 77.5% |
| WildReceipt test (472, US/EU photos) | 57.3% | 88.4% | 30.9% |
| SROIE test (347, Malaysia, total only) — unseen dataset | – | 85.0% | – |
| Real-world photos, Latin script (30, Wikimedia Commons, hand-verified) — unseen | 29.5% | 50.0% | 13.3% |
| Real-world photos, CJK/Thai/Cyrillic (33, VLM-labeled) — unseen | 17.1% | 54.5% | 12.1% |
Compared with other small models (same pipeline, same test sets)
| Model | Trained on | CORD split / total | WildReceipt split / total | SROIE total | Photos split / total | Size |
|---|---|---|---|---|---|---|
| Donut-base CORD-v2 (public, image→JSON) | CORD | 82 / 98 | 1 / 54 | 31 | 0 / 13 | ~200M params |
| Gemma 3 270M | CORD | 64 / 93 | 4 / 48 | 33 | 0 / 20 | ~290 MB |
| LFM2.5-350M | CORD | 73 / 94 | 2 / 40 | 19 | 0 / 13 | 364 MB |
| Qwen3-0.6B | CORD | 76 / 93 | 10 / 64 | 49 | 3 / 47 | 614 MB |
| LFM2-VL-450M (image→JSON) | CORD | 74 / 92 | 3 / 52 | 43 | 7 / 47 | ~450M params |
| LFM2.5-350M — ours | CORD-fixed + WildReceipt | 78 / 96 | 31 / 88 | 85 | 13 / 50 | 364 MB |
| Qwen3-0.6B — ours | CORD-fixed + WildReceipt | 76 / 94 | 37 / 91 | 86 | 37 / 57 | 614 MB |
| LFM2-350M-Extract, zero-shot | — | 2 / 34 | 2 / 16 | 26 | – / – | 364 MB |
Key takeaways:
- Models trained only on CORD don't generalize. Every CORD-only model, including the public Donut-CORD, gets ≤10% split-ready on US/EU receipts. Adding WildReceipt raised WildReceipt totals from ~40–65% to ~90%, and SROIE (never seen in training) from ~20–50% to ~85%.
- The int8 LiteRT build was evaluated separately; see the on-device section.
- Seed-to-seed variation is about ±1–3 points; the 30-photo set has about ±9 points of noise.
Limitations
- Real-world photos are still hard. On real phone photos, about 1 in 3 come out fully split-ready and about 1 in 2
have the right total. Typical failures:
- totals labeled with keywords unseen in training ("Celkem", "SUMA PLN", "NET A PAYER") get omitted;
- the OCR misses or garbles the total (OCR quality caps accuracy, and the model copies OCR errors);
- item names and prices on separate lines of tall receipts get mis-paired.
- Non-Latin scripts (Japanese, Chinese, Thai, Cyrillic) were not in the training data: totals are often right, but items rarely are.
- Amounts are copied as printed (strings with the receipt's own separators and currency marks). Normalize them in
the app (
receipt_utils.to_number). - Handwritten bills, invoices and multi-page documents are out of scope.
- Do not use the output for accounting or tax purposes without review.
Improve it for your users: fine-tune further on a few hundred labeled receipts from your target countries. In our experiments, adding diverse data mattered far more than model size.
License
LFM Open License v1.0: free for commercial use by organizations with < USD 10M annual revenue; larger organizations need a separate license from Liquid AI. Fine-tuning data: CORD-v2 (CC-BY-4.0) and WildReceipt. This repository contains modified weights
(fine-tuned) of LiquidAI/LFM2.5-350M.
- Downloads last month
- 319