Decision Models · demos
A small model trained for one job beats a generalist on that job.
Jev 1.14.0 is TypeSafe's general model. Here it meets our use-case models, each on its own task, on held-out examples (for the two offline models we also show their public dev set, marked as such). You can try the first one live; the other two were measured offline.
A. LLM request router · B. Demand forecasting and risk assessment
A. s1-llm-auto-router
A router classifier for LLM gateways. For one user request it answers seven typed questions in a single call: what kind of request it is, how hard, how high the stakes, whether it needs tools, vision or a long context, and whether it is a follow-up. It runs on CPU, about 26 ms inside the model.
Which of 9 kinds of request. Higher is better.
Every one of the seven answers correct for the same request.
List prices of the two APIs.
Where the difference comes from
Difficulty and stakes: exact level matched (difficulty 5 levels, stakes 4 levels). Flags: yes/no accuracy at p ≥ 0.5. Category calibration error (ECE): how far stated confidence is from actual accuracy, lower is better.