A Different Question From Which One to Use
We have already written about which approach to choose, in RAG versus fine-tuning. That article answers an architecture question: should knowledge live in the model's weights or in an index outside it? It is a good question and the answer is fairly stable. This article answers the question that follows it and that gets asked far less often, usually by whoever signs the budget: what does fine-tuning actually cost, and at what point does it pay for itself? Those are different questions with different answers, and a project can pass the first test and fail the second badly.
The most common way teams get this wrong is to evaluate fine-tuning as a capability decision. Someone runs an experiment, the fine-tuned model performs better on a sample, and the decision is treated as made. What that experiment did not measure is the annotation effort behind a production-grade dataset, the evaluation harness needed to detect regressions, the retraining that follows every base-model change, and the serving arrangement for a model that is now yours to operate. The experiment measured whether it works. The budget question is whether it keeps working at an acceptable cost for two years.
The useful reframe is this: fine-tuning is not a one-time purchase, it is a commitment to a recurring maintenance obligation in exchange for a lower unit cost and tighter behavioural control. That trade is excellent under some conditions and terrible under others, and the conditions are almost entirely about request volume and task narrowness. The rest of this article works through the cost stack, the techniques, the break-even arithmetic and the decision rule we apply. None of it requires a machine learning background to follow, which is deliberate, because the person who needs the answer is usually the person holding the budget.
The Four Things Fine-Tuning Actually Buys
The first is format and style adherence. Prompting will get a model to produce a given output shape most of the time, and in our experience the residual failure rate on a complex structured format sits somewhere in the low single digits even after careful prompt work. Fine-tuning pushes that further down and, just as importantly, removes the several hundred tokens of formatting instruction you were re-sending on every call. If the output feeds a downstream system that breaks on malformed input, the difference between a 3 percent malformation rate and a 0.3 percent one is the difference between a manual review queue and no queue.
The second is domain vocabulary and register. Jargon, entity naming conventions, abbreviations, and the specific voice of a professional field are things a general model handles unevenly. This is a particularly strong case in Turkish: legal, insurance, medical and heavy-industry registers each have conventions that general-purpose models approximate rather than reproduce, and the gap is wider in Turkish than in English simply because there is less of the register in general training data. Fine-tuning on a few thousand in-domain examples closes that gap in a way no amount of prompt instruction reliably does.
The third is the commercially important one: moving work to a smaller model. A small model fine-tuned on a narrow task can match a much larger general model's quality on that task, at a fraction of the per-request cost and with substantially lower latency. This is where fine-tuning stops being a quality technique and becomes a cost technique, and it is the natural end point of a model routing programme: once you have identified a route that is called constantly and does one narrow thing, that route is the candidate. It is also the argument for small language models generally.
The fourth is behaviour that prompting cannot reach. Consistent refusal boundaries, a specific reasoning shape, an output convention too intricate to describe in words, or a house style that reviewers recognise but cannot specify. There is also a quiet fourth-and-a-half: replacing a long system prompt with weights. If you are carrying a three-thousand-token system prompt on every call, encoding that behaviour into the model removes those input tokens permanently, which on a high-volume workload is a saving in its own right. That saving is available on the first day of deployment and does not depend on the fine-tune improving quality at all.
The One Thing It Does Not Buy: Fresh Facts
Training bakes in a snapshot. A model fine-tuned on this quarter's pricing, policies, product catalogue or regulations will confidently state this quarter's version of them for as long as it is deployed, including after they change. There is no mechanism inside the weights for expiry. This is the single most common misconception we encounter in scoping conversations, and it is usually expressed as a hope that fine-tuning will teach the model the company's knowledge. It will teach the model the company's language. It will not keep the model current. The distinction is worth settling in the first ten minutes of a scoping conversation, because everything downstream depends on it.
Updating facts by retraining is the most expensive database update ever devised. Every change requires a new dataset revision, a training run, an evaluation pass and a deployment, which is days to weeks of turnaround for information that a retrieval system updates in seconds. So the division is clean and it is worth stating as a rule: facts belong in retrieval, behaviour belongs in weights. Read that way, retrieval and fine-tuning are not competing options at all. They are complements that solve different halves of the same problem, and most mature systems use both.
There is a second thing fine-tuning does not buy, and it disappoints people more quietly. It does not create capability that the base model lacks. Fine-tuning shapes and reliably elicits behaviour the model can already produce; it does not install new reasoning ability. The diagnostic is straightforward: if the base model fails the task even when handed perfect context and a carefully written prompt, and fails it in a way that looks like not understanding rather than not complying, then a dataset of five thousand examples will not rescue it. Choose a stronger base model instead, or reconsider whether the task is well-posed.
The Cost Stack: Dataset Construction Dominates
The naive cost model is that you rent some GPUs, run a job, and collect a model. GPU time is real, but in the projects we run it is frequently the smallest line in the budget, and on adapter-based training it can be close to trivial. The dominant cost is dataset construction, which in our experience accounts for somewhere between half and two-thirds of the total effort on a first fine-tuning project. That is why a budget conversation opening with GPU pricing has started in the wrong place; the right opening question is annotation capacity.
Break that cost down and it becomes concrete. Sourcing and extracting the raw material. Cleaning, de-duplication and removal of near-identical examples that will teach the model nothing. Formatting into the training schema. Splitting into training, validation and test sets that do not leak into each other. And above all, annotation: a domain expert producing genuinely high-quality examples manages perhaps 20 to 60 per day depending on task complexity, which means a modest 3,000-example dataset is 50 to 150 expert-days of work. That is the real bill, and it is paid in the time of the most expensive people in the organisation.
Synthetic generation reduces this but does not remove it. Using a strong model to draft examples that experts then correct can raise throughput considerably, and it is the standard approach now. But you still need human-verified evaluation data, which cannot be synthetic without becoming circular, and synthetic training data faithfully reproduces whatever biases and blind spots the generating model has. The evaluation set in particular is a separate, non-optional cost: without it you have no instrument capable of telling you whether the fine-tune helped, which means you cannot justify the spend even after making it. This is the same data readiness work that determines whether any AI project succeeds.
Then there is the treadmill. Base models improve, get deprecated, or change licence terms, and each of those events puts some fraction of your work back on the table. Budget for two to four retraining cycles per year as a standing cost, not an exception. And finally, serving: an API model is somebody else's operational problem, while a custom model is yours. Self-hosting means provisioning GPU capacity for peak rather than average demand, and every hour of idle capacity is pure waste that does not appear in any comparison built from per-request numbers.
Full Fine-Tuning, LoRA and Distillation Have Different Economics
Full fine-tuning updates every weight in the model. It has the highest compute and memory requirements, it produces a complete model artifact for each version you train, and it multiplies your storage and serving footprint by the number of variants you keep. It can reach behaviours that lighter methods struggle with, and for genuinely large-scale or research work it remains the right tool. For the overwhelming majority of commercial projects it is not, and choosing it by default is an expensive mistake. Before selecting it, show with measurement that the lighter methods genuinely fall short, which in most commercial cases they do not.
Parameter-efficient methods are the practical default. LoRA and its quantised variant QLoRA train a small set of adapter parameters while the base weights stay frozen, which cuts compute requirements dramatically and produces adapters measured in megabytes rather than gigabytes. The serving consequence is more important than the training one: you can host many adapters against a single base model and switch between them per request, which turns a fleet of custom models into one model plus a set of small files. The tooling is mature, with Hugging Face PEFT, Axolotl and Unsloth all in common use.
Distillation inverts the cost profile. You use a large capable model to generate a substantial volume of training data, then train a small model to reproduce its behaviour on that narrow task. Data generation cost is high, since you pay the expensive model to produce thousands of examples, but serving cost is the lowest of the three by a wide margin. It fits one situation very well: a large model doing a narrow, high-volume task where you want the quality and not the bill. Preference-based methods such as DPO layer on top of supervised fine-tuning where the desired behaviour is easier to rank than to write out.
Break-Even Is a Function of Request Volume
The arithmetic has three inputs. The one-time cost, which is dataset construction plus training plus building the evaluation harness. The recurring cost, which is retraining, serving, monitoring and the ownership overhead. And the per-request saving, which is the difference between what the step costs now and what it will cost on the fine-tuned model. Break-even in months is the one-time cost divided by the quantity monthly volume times per-request saving minus monthly recurring cost. It is not complicated arithmetic, and the striking thing is how rarely anyone does it before starting.
The most important property of that formula is that the denominator can be negative. If the monthly recurring cost exceeds the monthly saving, the project never pays back, no matter how long you run it. That is not an edge case; it is the normal outcome for low-volume applications, because the recurring costs of owning a model are largely fixed while the savings scale with volume. Teams should be actively testing for this condition before committing, and in our experience most do not compute it at all until somebody asks why the model still has not paid for itself.
Two variables determine which side of the line you land on. Volume is the obvious one: high volume on a single task is where fine-tuning pays, and low volume on a broad task is where it never does. Narrowness matters just as much and is more often overlooked. A narrow task needs a smaller dataset, keeps a stable definition, and retrains cheaply. A broad task needs a large dataset, keeps shifting definition as stakeholders discover new cases, and turns retraining into a rolling project. As a working threshold from our scoping, below roughly 100,000 requests a month on a single well-defined task, the cost case rarely closes on its own.
A Worked Amortisation Example
The following is an illustrative model with hypothetical figures, not a quotation and not a measurement. Its purpose is to show the shape of the calculation so you can run it with your own numbers. Take a document classification and field extraction task inside a mid-sized Turkish company, currently running on a large general-purpose API model, handling 500,000 requests a month. The task is narrow, the categories are stable, and there is an existing manual process producing labelled examples as a by-product, which is the ideal starting position and also a rarer one than people expect.
One-time cost. A 3,000-example dataset at roughly 40 usable annotations per expert-day is about 75 expert-days, reduced to perhaps 45 if synthetic drafting with expert correction works well on this task. Add 10 to 15 engineering days for the data pipeline, training runs and deployment, and 8 to 12 days building the evaluation and regression suite. Call the total 70 to 100 person-days. The GPU bill for LoRA training on a task this size is small enough that it does not change the total materially, which is the point of the whole section above.
Recurring and saving. Assume two retraining cycles a year at 5 to 10 person-days each, plus serving and monitoring overhead. Assume the fine-tuned small model performs the step at somewhere between a fifth and a fifteenth of the current per-request cost, which is a realistic band when moving from a large general model to a small task-specific one. If the resulting monthly saving is worth the equivalent of 12 to 30 person-days, the 70 to 100 person-day investment is recovered in roughly three to eight months, and everything after that is margin.
Now change one input. Hold everything else identical and drop the volume to 20,000 requests a month. The one-time cost is unchanged, because the dataset does not get smaller when the traffic does, but the monthly saving falls by a factor of twenty-five. Break-even moves out past any horizon a business will accept, and the recurring cost may well exceed the saving outright, in which case the project loses money forever. This is why volume is the first question we ask, before any discussion of technique, base model or dataset design.
The Hidden Costs Nobody Puts in the Proposal
The evaluation regression suite is the first, and it is permanent. Once you own a fine-tuned model you own a quality baseline that must be re-verified on every retrain, every base-model change and every dataset revision. Without it you cannot distinguish a better model from a differently broken one, and quality drift becomes invisible until users complain. This is the same instrumentation described in AI observability and eval, and if you are not willing to build and maintain it, that is a legitimate reason to decide against fine-tuning entirely. Fine-tuning without that suite is an investment you will never be able to measure.
The second is model lifecycle management. You need a registry recording which adapter was trained on which dataset revision against which base model, and where each version is deployed. You need the ability to roll back to the previous adapter or to the base model within minutes, which in turn requires that model selection is centralised in a gateway rather than hard-coded across your application. Teams that skip this discover the gap during an incident, which is the most expensive possible moment to discover it. This is platform infrastructure that should exist before the first fine-tune, and its cost belongs in the platform budget rather than the model budget.
The third is dependency on a base model you do not control. Your adapter is worthless without its base. Licences change, hosted models get deprecated on the provider's schedule rather than yours, and terms can restrict commercial use or the training of derivative models. Read the licence before you build on it, and weigh the tradeoffs discussed in open versus closed models, since an open-weight base you can keep a copy of is a materially different risk profile from a hosted one that can disappear. Keeping your own copy of an open-weight base largely removes this risk, and that choice belongs on day one.
The fourth is data governance, and in Turkey it has statutory teeth. Your training set now contains data that came from somewhere, and if any of it is personal data then Law No. 6698 applies: Articles 5 and 6 on lawful basis and special-category data, Article 10 on the duty to inform, and Article 12 on data security. The practical difficulty is that deleting a record from a database is trivial while removing its influence from trained weights is not, so the lawful basis and retention position has to be settled before training rather than after.
The Decision Rule
Start with prompting plus retrieval. Every time, without exception, regardless of how confident anyone is that fine-tuning is required. This is not conservatism for its own sake; it is that the prompting-plus-retrieval baseline is the only thing that can tell you what gap actually remains, and without knowing the gap you cannot size the value of closing it. It is also fast: a baseline that takes days to build routinely resolves the requirement entirely, and the same logic applies when weighing context against retrieval in long context or RAG. It also produces the baseline you need to prove the value later.
Build the evaluation set before deciding anything. It is the instrument, and buying the treatment before owning the thermometer is how projects end up unable to prove their own value. Then fine-tune only when four conditions hold together. One: the evaluation set shows a specific, reproducible gap that prompting and retrieval do not close. Two: the task is narrow and its definition is stable. Three: volume is high enough that the break-even lands inside a horizon the business will accept. Four: a named person owns the retraining cycle. If even one condition is missing, waiting costs less than starting wrong.
Two anti-patterns are worth naming because they recur constantly. The first is fine-tuning to fix hallucination. Hallucination is a grounding problem, and training on more examples of correct answers does not install a mechanism for checking claims against sources; retrieval, citation validation and evaluation do that. The second is fine-tuning before the task definition has stabilised. If stakeholders are still discovering new categories and edge cases, you will retrain three times in two months and each retrain will cost most of what the first one did. Wait for the definition to stop moving, then build once.
How We Approach Fine-Tuning
Fine-tuning is one of the services we sell, and the first thing we do on a fine-tuning enquiry is try to talk the client out of it. That is not modesty, it is sequencing: an enquiry that survives a serious attempt to solve the problem with prompting, retrieval and routing is an enquiry where fine-tuning will actually pay, and one that does not survive would have become an expensive disappointment. A meaningful share of the fine-tuning conversations we scope resolve into a prompt restructure, a retrieval fix or a routing change instead, and those clients are better off for it.
Where it does go ahead, the shape is consistent. We build the evaluation set first and establish a prompting-plus-retrieval baseline against it, so the gap is measured rather than assumed. Training is LoRA on an open-weight base in almost every case, because the serving economics of one base with many adapters are so much better than a fleet of full models, and because keeping a copy removes the dependency risk described above. The regression suite goes into CI at the same time as the first training run, not afterwards, and retraining is reviewed on a quarterly cycle rather than triggered by whoever noticed something.
The clearest genuine wins we see are in Turkish domain language. Legal drafting conventions, insurance policy language, industrial and technical documentation, and regulatory correspondence all have registers that general models approximate rather than reproduce, and the gap is measurably wider in Turkish than in English. That is the case where the evaluation set shows a real difference rather than a marginal one. In caseon.ai and DiligenceAI the same division applies that we recommend to clients: the language and the output conventions live in the model, and the facts live in retrieval, because the facts change and the language does not.
If you are weighing this decision, the most useful thing you can do before talking to anyone is write down your monthly request volume for the specific task, and be honest about whether its definition is finished. Those two numbers decide most of it. Our LLM training and fine-tuning work starts from exactly that measurement, and it sits alongside the broader budgeting picture in AI project cost, because a fine-tuning decision made without the volume number in front of you is a decision made on hope. Conversations that start without those two numbers usually get lost in technical detail and end without a decision.