Learning on small approved data

Big models learn from everything. A model of your own learns from very little: the work your people approve, one example at a time. That is the point. It is also the hardest part of what we do.

Our rule is strict. A correction is not training data until someone approves it, and a training run starts only with at least 50 approved examples.1 Fifty is enough to start. How far a small expert can go from there is open, and it is the first of four problems we are working on.

1. Learning from little data

What we know so far comes from one model. In Benchmark 001, 200 steps of small adapters on an open base of 24.7 million parameters cut forecast error 17.8% on days the model never saw, in all four runs. Domain training moves a small base. It did not move it past three simple baselines.

The methods are open. LoRA trains small adapters on a frozen base, with up to 10,000 times fewer trainable parameters.2 QLoRA fine-tunes a 65B model on one 48 GB GPU.3 GRPO lets a model learn from its own best attempts, scored by a grader, without a separate critic model.4 Distillation from a larger teacher, only where its licence allows, is planned. What is open is how far they go on 50, 500 or 5,000 approved examples of one company’s work.

Not every gain needs training. In published work, a search over the harness around a model, its tools, memory and context, added 7.7 points with four times fewer context tokens.5 We plan to learn at both levels: skills kept only when they beat the live version, and adapters retrained on approved work.

2. A router that knows its limits

Most requests should never need the big model. A small decision model reads each request, answers when it is confident and passes it on when it is not. Our targets: under 10 ms per decision, and 80 to 90% of requests without the main expert. Today the open 0.42B model we start from, Laya, measures 33 to 40 ms. The hard part is calibration per client, so the fast path passes on exactly what it should.

Decision model figures are targets, to be measured in pilots.

3. Evaluation that stays cheap

Every update meets the gate: sealed tests, three trials per item and a shadow run next to the live version. If each test costs more than the training it judges, the loop stops paying for itself. We want evaluation that stays cheaper than training, without testing less.

4. Models that fit a box

An Eigna Box has desk-sized memory: 36 to 256 GB in today’s tiers. A strong expert has to fit inside without losing quality. Compression is the fourth problem.

Two engineering measurements for scale. On a Mac Studio M5 Max, LoRA training on an 8B language model runs at about 1,200 tokens a second. One end-to-end pipeline run of our first model on one rented H200 GPU cost $5.22.6

First on the list

Before any of this becomes a product claim, one thing comes first: beat the simple baselines. Benchmark 001 does not yet. If you work on learning from little data, routing or evaluation, we would like to hear from you.