Model Fine-Tuning: A Short Guide

Fine-tuning sounds like a way to "make the model your own"

: take an open LLM, train it on your data, and get an expert in your domain.

Fine-tuning solves a narrow but real class of tasks: when the model must run inside a closed perimeter on accessible hardware, without the cloud and without corpus search on every request.

Beyond that, cloud RAG or llm-wiki discipline is cheaper and more reliable.

This text is about how to tell one from the other, without extra theory and without made-up economics.

This is a breakdown of one way to give an LLM your knowledge from the broader cluster

A map of all approaches and how they combine is in the overview ["How to Give LLM Your Knowledge"

](/blog/kak-dat-llm-vashi-znaniya).

Key point

Fine-tuning changes the model's weights. Knowledge and behavior are "built into" the model rather than provided as input - this is what distinguishes it from RAG (runtime retrieval) and llm-wiki (compiling knowledge into a readable corpus).
The main gain is not "intelligence," but where it runs: a small fine-tuned model works locally, offline, inside your perimeter, on available hardware, with low latency and no large prompt on each call.
The main price is frozen knowledge: what is built in becomes outdated; you cannot update a single fact by "reindexing" - you have to retrain.
Fine-tuning is justified for privacy/air-gap setups, edge/offline use, high call volume, and a stable narrow domain with specific behavior. In other cases, use cloud + RAG/llm-wiki.
The common right answer is both: fine-tune for domain and behavior, use RAG for fresh facts.

What fine-tuning is

In a base LLM, knowledge and response style are encoded in the weights - billions of numbers tuned on a vast general corpus.

Fine-tuning continues that adjustment on your data: the model studies examples from your domain and shifts its weights so it answers closer to them.

The key difference from the other two approaches is where the knowledge lives.

RAG leaves the weights untouched and, for each query, adds only the relevant retrieved fragments to the context - that is its token economy; the knowledge stays outside, in storage. llm-wiki also does not touch the weights: the knowledge is precompiled into a coherent set of short markdown files (the README points to the right file, one fact in one place, git versions), and the agent reads the project map instead of searching random chunks; the knowledge stays outside, in a structured corpus.

Fine-tuning embeds knowledge and behavior into the weights - at runtime there is neither search nor a large prompt, and the model responds immediately.

There is also a third external layer - a knowledge graph / GraphRAG: for global and multi-step questions where standard RAG is weak, an entity-and-relationship graph is built and answers are produced by traversing the graph. And for a small, stable body of knowledge for one task, it is sometimes enough to simply put it in the prompt (in-context), without any infrastructure - though then it burns tokens on every call.

So there are not two external layers, but several; fine-tuning stands apart because it is the only approach that changes the model itself.

Types of fine-tuning by depth of intervention

01
SFT / instruction-tuning
Behavior adaptation: we teach the model the domain's format, style, and reasoning pattern using input-to-desired-answer pairs. This is about how it should answer.
02
LoRA / adapters
Light fine-tuning: we leave the main weights mostly untouched and train a small adapter on top. It is cheaper, faster, and easier to maintain multiple versions for different tasks on one base model.
03
Continued pretraining
Domain adaptation: we spend longer fine-tuning the model on a large volume of domain texts so it absorbs the vocabulary and substance of the industry. This is the hardest path and requires the largest, cleanest corpus.

What fine-tuning gives

For most applied tasks, this means the first two types - behavior and format, sometimes with a lightweight adapter.

Continued pretraining is a separate, expensive topic. The strengths of fine-tuning are not that the "model becomes smarter".

These are the concrete consequences of knowledge being inside the weights.

Three consequences, not three adjectives

On-prem execution

A small fine-tuned open-source model runs locally and offline - on available hardware, including edge devices, without going to the cloud. Data does not leave the perimeter; you own the system and the weights, without being tied to a cloud vendor or its availability. This matters where privacy is not a preference but a requirement.

Low latency and cheap calls

There is no search through storage and no large prompt for every request - the knowledge is already in the weights. Fewer tokens per call, lower latency. At high call volume, the economics are determined by hardware utilization and the absence of repeated cloud charges for each call - a lower IT inference budget under heavy traffic (the mechanism is in the economics section).

Behavior, not just facts

The model adopts the domain's reasoning style - how to structure an answer, what to look at first, and what language to use. RAG and prompting also influence style through examples, but they pay for it with context on every call; fine-tuning fixes the style in the weights - consistently and without token cost.

Example: AI lawyer

A model fine-tuned on a company's corpus of similar contracts reasons like that company's specialist lawyer: it knows their contract structure, standard wording, and what to check first.
It runs within your perimeter - contracts never leave the cloud - and does not require RAG at runtime.
Two conditions without which the example does not work. First: the patterns and style must be stable.
If contract templates and interpretations change every month, the advantage fades, and the question shifts to RAG/llm-wiki over a live corpus. Second: fine-tuning fixes the style and structure of reasoning, but does not guarantee factual or legal correctness - a human still has to check that.

Fine-tuning does not buy intelligence, but execution location and style - the model responds locally and without prompting.

What you pay for it

Built-in knowledge has a mirror cost. It is worth understanding before the project starts, not after.

What you pay for fine-tuning

Training is expensive and time-consuming. It requires compute, engineering time, and training iterations - this is not a "upload files and done" process.
Knowledge is frozen at the time of training. A fine-tuned model knows the world as it was in the dataset; if reality changes, the model will not learn about it until it is retrained.
You need a high-quality labeled dataset. Model behavior is a consequence of the data; most companies simply do not have a clean, consistent, representative dataset, and collecting one is a separate project.
It is hard to update a single fact. In RAG, that means reindexing the affected documents, usually quickly; in a fine-tuned model, it means a new training cycle.
Risk of catastrophic forgetting and overfitting. When fine-tuning for a narrow task, you can wash out part of the model's general capabilities or teach it to reproduce the dataset instead of generalizing.
Auditing where an answer came from is worse. RAG shows the source of the chunk; a fine-tuned model answers "from the weights" - it is harder to trace why it answered that way.

When to use fine-tuning, and when not to

It is worth saying this directly: fine-tuning is needed less often than it seems. For most tasks, cloud + RAG or llm-wiki discipline is enough - and that is not a compromise, but the right choice on total cost of ownership. The most underestimated downside is freezing: fine-tuning punishes frequent changes; the more dynamic the knowledge, the more often you will need retraining and the faster the training investment loses value.

Fine-tuning: to use it or not

Choose fine-tuning

Privacy / air-gap: data must not physically leave the perimeter.
Edge / offline / weak connectivity: the model must work without the cloud and without a stable connection.
High call volume where context cost dominates: removing a large prompt/RAG from runtime is the main savings.
Stable narrow domain + specific behavior or format: knowledge changes rarely, and response style matters.
Latency is critical: an extra retrieval step is unacceptable.

Cloud + RAG/llm-wiki is enough

Knowledge changes often (prices, catalog, documentation, procedures): reindexing is cheaper than retraining.
Source links and answer auditing are needed.
The corpus is large, and each query needs only a slice.
No clean labeled dataset (most organizations do not have one).
A quick update to one fact is needed without a training cycle.

Assess where AI can deliver impact in your process

clients@kt.team Telegram @kt_team_it

Most often the answer is both layers

The most common practical answer is both layers.
Fine-tune the model for domain behavior and format, and use RAG for fresh facts.
Then what is stable is built in (style, structure, vocabulary), while what changes lives outside (numbers, documents, rules).
The tree below helps determine which way a specific task leans.

How to choose: decision tree

What matters most in the task?

Indicator

Privacy / air-gap, edge/offline, high call volume, or latency is criticalleads to fine-tuning

Indicator

Stable narrow domain + specific behavior or formatleads to fine-tuning

Indicator

Knowledge changes often, citations and auditability are needed, and there is no datasetleads to cloud + RAG

Solution

Fine-tuningoften + RAG for fresh facts

Cloud + RAG / llm-wikireindexing instead of retraining

Signals such as privacy, edge/offline, high call volume, or latency push toward fine-tuning; a stable narrow domain with specific behavior does too. Changing knowledge, a requirement for source links and auditing, or the absence of a clean dataset push toward cloud with RAG/llm-wiki. In practice, these branches are often combined: fine-tuned behavior plus RAG for fresh facts.

Three industries, three forks

The same question - "fine-tune or not" - is answered differently in real environments, even within one company. The separator is almost always the same: how often the knowledge changes and whether there is connectivity.

Factories, construction, retail

Factories

Assistant at a production line: a workshop without stable network access, an air-gap requirement, and limited hardware. Stable safety and equipment procedures -> fine-tune a small local model; it works offline inside the perimeter. But technical regulations and manuals that are updated regularly -> move into RAG/llm-wiki so the model does not need retraining for every change.

Construction

A tablet on a job site where connectivity may be unavailable: regulations, safety rules, and standard procedures are stable, so a fine-tuned local model makes sense. But drawings and specifications change almost daily - embedding them into the weights is pointless; that is a job for RAG over a live document corpus.

Retail

The catalog and prices change constantly - for this moving part, fine-tuning is not rational: reindexing in RAG is cheaper than any training cycle. But a stable return policy, brand voice, and offline POS at the checkout on the network edge mean fine-tuning is justified: behavior is fixed, and it works without the cloud.

The boundary is based on the rate of change, not the industry

The conclusion is not "fine-tuning for manufacturing, RAG for retail." The conclusion is that in every industry, stable behavior belongs in the weights, while changing facts belong in the external layer. The boundary is not by industry, but by how quickly knowledge changes.

Fine-tuning, RAG, and llm-wiki side by side

Criterion	Fine-tuning	RAG	llm-wiki
In brief	knowledge is embedded in the weights	runtime search over storage	knowledge is compiled into a readable corpus
Freshness of knowledge	frozen at training time	instant (reindexing)	by commit in the repository
Privacy / locality	strong point: offline inside the perimeter on affordable hardware	depends on the storage infrastructure	depends on where the corpus is stored
Single-fact updateability	hard (a new training cycle)	easy (reindex the affected documents)	easy (edit one file)
A dataset is needed	yes, high-quality labeled	no, you need a corpus for the index	no, you need structural discipline
Audit / source links	weakly (“from the weights”)	strong (chunk link)	strong (file link)
Tokens per request	minimum (knowledge in the weights)	moderate (we pull in only what is relevant)	from reading the README map to the needed summary - the structure minimizes volume

Read the table by the "Freshness" and "Updateability" rows: if the knowledge changes often, external layers win; if it is stable and must live inside the perimeter without the cloud, fine-tuning wins. A detailed RAG breakdown is in the article about the RAG tool; the llm-wiki "No RAG" approach as corporate memory - in Sloy.

Economics: measure utilization, not token price

It is tempting to compare "fine-tuning vs cloud"

That is the wrong axis

For a fine-tuned model in your perimeter, there is no cloud per-token fee - the cost of each call is hidden in hardware utilization and operations.

On-prem economics are determined by GPU utilization, not token price: an idle accelerator is expensive with any “cheap” model, while a well-loaded one pays for itself under high request volume.

That is why we deliberately do not print ruble figures for hardware or inference here - they depend on your configuration, workload profile, and ownership horizon.

The setup cost is calculated for the task.

Calculator: cost and payback of the setup

All figures are order-of-magnitude estimates, not a commercial offer.

Inference is modeled at the Opus 4.8 price as a frontier-model benchmark, using the same token volume for every deployment contour. Prices come from the 2026 LLM comparison as of 2026-06-27; calculator amounts are displayed in USD at a fixed rate of 1 USD = 80 RUB.
Implementation cost is modeled only per process: 300,000 RUB per process at a 60% automatable share. If the share is lower or higher, process cost scales proportionally. The calculator has no one-off launch fee.
Processes with high variability or strict legal responsibility are automated partially. That is normal.

Two pricing models

Two commercial models: fixed price for a working process, or an outstaff contract for a dedicated ai-native team. The inference contour (cloud, on-premise, or your perimeter) is selected separately. More detail: pricing approach.

Check with your data →

On-prem wins on utilization, not on token price

What we do

Yes, KT.Team fine-tunes models, including so the model can run locally, on available hardware, and inside your perimeter.
This is appropriate when the task falls into the "right" branch of the tree: privacy, edge/offline, high call volume, a stable domain with specific behavior.
Because these are open models in your own environment, you own the system and the weights - without dependence on a cloud vendor, its pricing, or its availability. And just as honestly: if your knowledge changes often, you have no dataset, and you need links and auditability, you do not need this.
Then we will build a cloud setup with RAG or an llm-wiki discipline, and it will be cheaper to own and easier to operate.
Often the result is a combination: fine-tuned behavior plus RAG for fresh facts.
If data must be kept under Federal Law 152 and anonymized before entering the model, this is the layer LLM Gateway, it works with both a fine-tuned and a cloud model.
We work in short iterations: first a lightweight adapter and an effect measurement, then expansion - so you pay for the result, not the promise.

Fine-tuning for domain and perimeter

A model that reasons like your specialist - and runs inside your perimeter

Local · offline · on-prem

We will choose what to embed in the weights and what to keep in the external layer, assemble the dataset and the deployment setup - and tell you honestly if standard cloud RAG is enough.

Design a fine-tuning setup

Next steps

How we provide model fine-tuning as a service - dataset, LoRA, before/after metrics, and deployment loop - [

Model fine-tuning (ML)](/solutions/ai-for-business/ml-fine-tuning)

A complete map of ways to give an LLM knowledge and how they combine - ["How to Give LLMs Your Knowledge"

](/blog/kak-dat-llm-vashi-znaniya).

Understanding versus execution, weights versus context - "Karpathy's Lens". RAG as a runtime retrieval mechanism - RAG tool.

Corporate memory based on the "No RAG" principle - Sloy.

Models for Federal Law 152 and anonymization at input - LLM Gateway.

AI implementation in short iterations - for business.

Model cost figures and a deployment calculator - LLM 2026 breakdown.

Sources

Reviewed: 30.06.2026

Model Fine-Tuning: A Short Guide

This is a breakdown of one way to give an LLM your knowledge from the broader cluster

What fine-tuning is

Types of fine-tuning by depth of intervention

SFT / instruction-tuning

LoRA / adapters

Continued pretraining

What fine-tuning gives

Three consequences, not three adjectives

On-prem execution

Low latency and cheap calls

Behavior, not just facts

Example: AI lawyer

What you pay for it

What you pay for fine-tuning

When to use fine-tuning, and when not to

Fine-tuning: to use it or not

Choose fine-tuning

Cloud + RAG/llm-wiki is enough

Assess where AI can deliver impact in your process

Most often the answer is both layers

How to choose: decision tree

Three industries, three forks

Factories, construction, retail

Factories

Construction

Retail

The boundary is based on the rate of change, not the industry

Fine-tuning, RAG, and llm-wiki side by side

Economics: measure utilization, not token price

That is the wrong axis

Calculator: cost and payback of the setup

Two pricing models

On-prem wins on utilization, not on token price

What we do

A model that reasons like your specialist - and runs inside your perimeter

Next steps

Model fine-tuning (ML)](/solutions/ai-for-business/ml-fine-tuning)

Sources

Discuss the article: Fine-Tuning Models: A Short Guide

Continue on the topic

Related Cases

Subscription grocery delivery service for Danone

Brandquad PIM for Campari Rus

Akeneo PIM for the Norgau B2B store

Related solutions

Articles on the topic

Related videos

News on the topic