# ForgeModel

> ForgeModel forges private fine-tuned open-weights models for a company's highest-volume AI task, proves them head-to-head against the API the company already pays for on a held-out test set, and serves them on infrastructure the customer controls. Measured result: a fine-tuned Qwen3-4B beat GPT-4o-mini on Amazon's ESCI relevance benchmark at 14.3x lower cost.

ForgeModel is a small consultancy. The site is a single page at https://forgemodel.org/ with a benchmark report, a scope statement, an interactive cost calculator, a three-step process, and a pilot-request form. Contact: abhishekg@forgemodel.org.

## Key facts

- Benchmark (August 2026, Amazon ESCI query-to-product relevance, 5,000 held-out examples, knobs tuned only on a separate dev split): ForgeModel fine-tuned Qwen3-4B micro-F1 0.6424 vs GPT-4o-mini 0.6116. Cost per 1M calls: $1.98 self-hosted on one rented RTX 4090 vs $28.26 for GPT-4o-mini. With the confidence router, system micro-F1 0.6536.
- What ForgeModel takes on: high-volume, structured work with a known output shape. Document and product classification, entity and attribute extraction, metadata tagging and moderation, request routing and dedupe.
- What stays with frontier models: open-ended conversation, coding, research, agentic reasoning. ForgeModel says so and recommends keeping them for that.
- Confidence router: the fine-tuned model answers with a calibrated confidence score; requests below a threshold escalate to the customer's frontier API automatically. In production it is one comparison on first-token logprobs, served behind an OpenAI-compatible endpoint.
- Process: (1) fine-tune a small open model on labeled examples the customer chooses to share, (2) prove it head-to-head on a held-out test set the customer defines, (3) route and serve. The benchmark report is the customer's either way, including when the incumbent wins.
- Engagement model: a fixed-price pilot that replaces every estimate with numbers measured on the customer's own data before any commitment.

## Pages

- [ForgeModel home](https://forgemodel.org/): the full site, one page.
- [Full text for language models](https://forgemodel.org/llms-full.txt): every section of the site in plain Markdown, including the calculator's assumptions.
- [Request a pilot](https://forgemodel.org/#contact): the form. Fields are name, work email, company, and the highest-volume task.

## Optional

- [Cost calculator](https://forgemodel.org/#savings): interactive. Prefills use providers' published token prices and market GPU rates; all figures are directional until measured on the customer's workload.