How AI models are made, fine-tuned and used

No maths and no jargon you will not meet again below. By the end you will know what a model really is, how it learns, what "fine-tuning" means, why a small model can learn from a big one, and how we did it for real.

• 1. The big picture
• 2. Words become numbers
• 3. What a model is
• 4. How it learns
• 5. Pretraining
• 6. Fine-tuning
• 7. LoRA and QLoRA
• 8. Good training data
• 9. Knobs and overfitting
• 10. Testing honestly
• 11. Using a model
• 12. Big teaches small
• 13. Putting it together
• 14. Real numbers
• Glossary

1The big picture

Every AI chatbot goes through the same three stages. Think of a person: they grow up and read everything, then they train for a job, then they go to work.

2Computers only understand numbers

A model cannot read letters. First, the text is chopped into small pieces called tokens (often a word, or part of a word). Each token gets an ID number from a fixed list, and the model works on those numbers.

Then each ID is turned into a longer list of numbers (called an embedding) that captures its meaning: the lists for "cat" and "kitten" end up close to each other, and "cat" and "tax" end up far apart. This is how a model can feel that words are related.

3What a model is, and what weights are

A model is one very large calculation. The input numbers go in, pass through many layers, and the output is a list of probabilities for the next token: "after these words, 'customers' is 62% likely, 'users' 11%, ...". To write a sentence, the model picks one token, adds it, and asks again, over and over.

Inside each layer are adjustable numbers called weights. Think of them as dials. Together they decide how the model turns input into output.

The saved file with all dial settings is "the model weights". "Open weights" means anyone can download that file. The weights are the model: the code that runs them is generic.

4How a model learns: guess, measure the mistake, nudge the dials

At the start the dials are random, so the model outputs nonsense. Training repeats one small loop millions of times:

How does it know which way to turn each dial? A bit of calculus (called gradient descent) tells every dial: "if you go up a little, the mistake grows; if you go down, it shrinks." Every dial moves a tiny step in the helpful direction. The size of that step is the learning rate.

5Pretraining: where a model gets its general skills

Pretraining is the same loop, but the examples are enormous amounts of text: "given these words, predict the next one". That is the whole task. To do it well across books, code and websites, the model has to pick up grammar, facts and reasoning patterns along the way.

It takes thousands of GPUs and months, costs millions, and only a few companies do it. The result is a base model. We did not do this. We started from NVIDIA's Nemotron 3 Nano 4B.

A base model is a bit like a very well-read person who has never held a job: knowledgeable, but not yet good at the exact thing you need, and sometimes it rambles or answers in the wrong style.

6Fine-tuning: practice one job

Fine-tuning is the same learning loop again, but with a small, focused set of examples: each one is a prompt (what the user asks) and the right answer you want. A few thousand good examples can change how the model behaves on that task.

During fine-tuning the loss is measured only on the answer part, because that is what we want the model to learn to produce.

7LoRA and QLoRA: fine-tuning on a small, cheap GPU

Changing all 4 billion dials needs a lot of memory, because for every dial the computer also keeps notes on how it is changing. Two tricks make it cheap:

LoRA keeps the big model frozen and trains a small add-on next to its layers (two thin grids of numbers per layer, instead of changing one giant grid). The add-on is a file of only about 40 MB, called an adapter. Take it off, and you have the original model back. The same base model can carry many adapters for different jobs.

That is exactly what we did: QLoRA on a 4-billion-dial model, on one small rented GPU.

8Good training data is most of the work

A model copies what it is shown. If the examples contain wrong answers, it learns wrong answers. So the quality of the examples matters more than their number.

Ours: a big model (Nemotron 3 Ultra) invented small databases and wrote questions with answers. We ran every answer and kept only the queries that worked. 821 examples in round 1, 4,322 in round 2. We also removed any practice example that matched an exam question.

9The knobs, and the trap called overfitting

Epoch

One full pass over all the examples. We did 2 passes.

Batch size

How many examples the model looks at before it nudges the dials.

Learning rate

How big each nudge is. Too big and it overshoots, too small and it barely learns.

Rank

How big the LoRA add-on is. Ours was 16: small but enough.

Overfitting is memorising the practice examples instead of learning the skill, like a student who memorises last year's answers. Practice scores keep improving, but new questions get worse.

10Testing honestly

The only fair test is on questions the model has never seen. We use an exam of 172 database questions, 40% from industries the model never practised on. A query is marked right only if running it returns exactly the same rows as the stored reference query.

We also check for leaks: no exam question, answer or database may appear in the training examples. If it did, the score would be fake.

11Using a model, and how it can know when it is unsure

Using a trained model is called inference. For each next word the model gives probabilities, so we can also read how sure it is: a 95% word choice is very different from a 30% one. We can also ask the same question several times and see whether the answers agree. Together this gives a confidence score.

12Big teaches small (distillation)

Running a 550-billion-dial model for every question is slow and costly. A 4-billion one is cheap but less skilled. Distillation uses the big model as a teacher: its good answers become the small model's lesson book. The student cannot match the teacher everywhere, but on the questions it has practised it can be close, at a fraction of the price.

13Putting it together: Ratchet

Ratchet is a gateway between an app and the models. The app talks to it exactly like it talks to OpenAI. Behind it, the small model answers first, and the big one steps in only when the small one is unsure. Then the system learns from what the big model answered.

The lessons saved in step 5 have personal data removed first. The audits matter: they measure how often the small model is truly right, so the system never just trusts it.

Every night: the system improves itself

Simulation The nightly release and automatic rollback have been tested with mock models only; they have not yet run live. Real The data, the training and the exam below did happen.

14The real numbers

Exam of 172 database questions the model never trained on, scored strictly (the same rows as the reference query).

Glossary