Measured August 2026 on Amazon ESCI

Frontier accuracy.A fraction of the cost.

We forge frontier-model capability into small private models: open weights fine-tuned on your highest-volume task, proven head-to-head against the API you pay for today, served on infrastructure you control.

Start a Pilot Run the Numbers

A fine-tuned 4B model beat GPT-4o-mini on 5,000 held-out examples at 14.3× lower cost. Every knob was tuned on a separate dev split. The full method is below.

Proof before promises.

Frontier-level is a claim we prove, not assert. Every engagement begins with a benchmark you can interrogate. This is ours: a Qwen3-4B fine-tuned on Amazon’s ESCI relevance task, against GPT-4o-mini, identical prompts, one held-out test set.

  • Head-to-head, on your data. Your fine-tuned model against the API you use today, on a test set your team defines.
  • Honest methodology. Tuning decisions are made on a separate development set, never on the data we report. Baselines run the way a skeptic would run them.
  • The report is yours either way. If your current setup wins, the benchmark says so and you keep the full analysis.

0.6424micro-F1

Higher accuracy

GPT-4o-mini: 0.6116

A 4-billion-parameter model, fine-tuned on the task, outscoring the frontier API on the held-out test set.

$1.98per 1M calls

Self-hosted cost

GPT-4o-mini: $28.26

One rented RTX 4090, model resident in memory, continuous batching. Real GPU rate, measured throughput.

14.3×lower cost

At higher accuracy

With the confidence router: 0.6536 micro-F1

The model answers with a calibrated confidence score. The unsure slice escalates to your frontier API automatically, so the system as a whole scores higher than either model alone. In production the router is one if statement on first-token logprobs.

Built for repetitive work. Honest about the rest.

Our specialty is high-volume, structured work where you already know the output shape: one fixed prompt, short outputs, millions of calls.

What we take on
  • Document and product classification
  • Entity and attribute extraction
  • Metadata tagging and moderation
  • Request routing and dedupe
What stays with frontier models
Open-ended conversation, coding, research, agentic reasoning: anything where every request is different. They win there, and we will tell you to keep them. Use frontier models for exploration. Use ForgeModel for production execution.
What happens to the unusual cases
Every answer comes with a confidence score. Below your threshold, the request escalates to your frontier API automatically, so weird inputs never get a confident wrong answer from a small model.
Start a Pilot
Example requestESCI relevance

Query wireless earbuds noise cancelling
Product Sony WF-1000XM5 earbuds

Exact match, confidence 0.94Answered locally
Example requestESCI relevance

Query gift for a ten year old
Product LEGO Technic crane, 1,200 pieces

Substitute, confidence 0.41Escalated to API
Measured, 5,000-example test setAugust 2026
Setupmicro-F1$ per 1M
GPT-4o-mini only0.6116$28.26
ForgeModel only0.6424$1.98
ForgeModel with router0.6536varies by threshold

Run your numbers.

Pick the API model you use today. Its published token prices, a matched open-weights replacement, and that model’s GPU prefill automatically. Every field stays editable, and the math updates live.

Projected impact

$—

Estimated monthly savings

$—

Annualized savings

Current API cost per month—
ForgeModel cost per month—
GPUs needed at target utilization—
Actual utilization—

Directional, not a quote. The token prices and GPU rates above are real and current; the throughput and utilization figures are honest estimates. Every workload is different, which is exactly what the fixed-price pilot is for: it replaces every estimate on this page with numbers measured on your data, before you commit to anything.

Forge, prove, ship.

A disciplined path to production, in the order it actually happens.

1

Fine-tune

We tune a small open model on your task using labeled examples you choose to share: your historical labels, or answers generated by the API you already pay for. Hours of GPU time, not weeks.

2

Prove

Head-to-head against your current model on a held-out test set you define, tuning knobs only on a separate dev split. Per-class results and confusion matrices before anything ships.

3

Route and serve

The model answers with a calibrated confidence score; the unsure slice escalates to your frontier API automatically. Served behind an OpenAI-compatible endpoint, so switching is a one-line change.

Start your pilot.

Tell us about one high-volume pipeline and we’ll scope a fixed-price pilot that proves the savings on your actual data. The report is yours either way.

or email abhishekg@forgemodel.org directly