What Are the Different Types of Fine-Tuning? A Practical Selection Guide for SFT, RFT, Distillation, and LoRA

Fine-tuning is not one technique. It is a family of ways to adapt an existing model by training it further for a target task. The most useful way to choose among those methods is not to start with GPU count, model size, or a training framework. Start with a harder and more fundamental question:

Can you write the ideal output for each input, or can you only tell whether a generated output is good after you see it?

If you can consistently write target outputs, supervised fine-tuning (SFT) is usually the sensible starting point. If producing a gold answer is difficult but verifying candidate answers is reliable, reinforcement fine-tuning (RFT), including group-based approaches such as GRPO, can be appropriate. If a stronger model can create outputs you trust more than your current model, distillation is an option. LoRA and QLoRA are different: they are primarily ways to make training cheaper, rather than alternatives to SFT or RFT.

This guide is deliberately practical. Fine-tuning is a poor repair for changing factual knowledge, missing product data, weak retrieval, vague requirements, or an evaluation suite that does not measure the user outcome. Treat it as a behavior-learning tool.

Problem Breakdown: Separate the Learning Objective From the Training Mechanics

Teams often compare “LoRA versus SFT” or “QLoRA versus RFT.” That comparison mixes two separate decisions.

1. Learning objective: What signal tells the model how to improve?

   • SFT learns from input/output demonstrations.
   • RFT learns from a reward or verifier.
   • Distillation learns from outputs generated by a teacher model.

2. Parameter-update strategy: Which parameters are allowed to change, and how much hardware is needed?

   • Full fine-tuning updates the model’s weights.
   • LoRA freezes the base model and trains small adapter weights.
   • QLoRA adds quantization of the frozen base model to reduce memory pressure further.

This means a real training plan can be described in two layers:

• “SFT using QLoRA”: learn from authored demonstrations while training adapters on a quantized frozen model.
• “RFT using LoRA”: optimize toward a reward while updating adapter weights rather than all base weights.
• “Distill into a LoRA adapter”: have a teacher produce training examples, then train an adapter on the selected examples.

Do not choose a parameter-efficient technique before deciding whether your supervision is actually usable.

The four questions that determine the method

Before training anything, answer these questions in writing.

The phrase “correct answer” needs care. For structured extraction, code with executable tests, or classification, correctness can be relatively concrete. For legal advice, emotional support, product strategy, and open-ended writing, a single ideal answer may not exist. SFT can still work in those areas, but only if annotation policy makes quality criteria consistent.

Supervised Fine-Tuning (SFT): Teach by Worked Example

SFT trains a model on prompt-and-answer pairs. The model sees an input and a desired output, then learns to reproduce that pattern. It is the default starting point because it is conceptually direct, relatively easy to inspect, and compatible with a wide range of dataset and deployment setups.

When SFT is a good fit

Use SFT when you can build examples that reliably answer all of these:

• What should the model say or produce?
• What formatting and level of detail are required?
• What must it refuse, omit, or escalate?
• How should it behave when required information is absent?

Typical tasks include structured extraction, consistent support-response style, controlled JSON generation, classification with rationale policy, format conversion, narrow domain workflows, and tool-call selection where correct calls can be authored.

A narrow task may need thousands of useful examples, but treat any count as a planning hypothesis rather than a guarantee. Dataset diversity, label consistency, base-model capability, output complexity, and the strictness of the evaluation all matter more than a generic example-count target.

Inputs and outputs for an SFT dataset

A training item should include the same instruction structure the model will receive in production. If your application uses a system instruction, retrieved context, user request, and strict schema, represent those conditions in the examples.

Input

System: You extract invoice fields. Return JSON only. Use null when a field is absent.

User: Extract the invoice number, invoice date, currency, and total from:
Invoice 001-728
Issued: 14 March 2026
Total due: EUR 1,249.50

Target output

{
  "invoice_number": "001-728",
  "invoice_date": "2026-03-14",
  "currency": "EUR",
  "total": 1249.50
}

The target must be something your production system genuinely wants. Do not train a model to emit prose around JSON if downstream code requires JSON only. Do not include an invented value merely because empty fields are inconvenient. Inconsistent missing-data behavior is a common way to create a fine-tuned model that seems unreliable.

Verified prompt template: SFT example-authoring review

Use this template with a subject-matter expert or a capable reviewing model to turn task rules into candidate training examples. It is verification-oriented: it requires the reviewer to identify ambiguity before producing a final target.

You are reviewing one proposed supervised fine-tuning example.

Task policy:
[PASTE THE CURRENT, VERSIONED TASK POLICY]

Production input format:
[PASTE THE EXACT INPUT SHAPE THE MODEL WILL RECEIVE]

Proposed user input:
[INPUT]

Proposed target output:
[TARGET OUTPUT]

Perform these checks:
1. Is every output field supported by the input or explicitly required by policy?
2. Does the output exactly follow the production schema and formatting rules?
3. Is the target the single preferred answer under the policy?
4. Is any instruction ambiguous, contradictory, or missing?
5. Would a competent annotator likely create the same target?

Return exactly this format:
DECISION: ACCEPT | REVISE | REJECT
ISSUES:
- [specific issue or NONE]
REVISED_TARGET:
[correct target, or N/A if rejection requires policy clarification]
POLICY_GAP:
[gap or NONE]

SFT failure modes

• Training examples are cleaner than production traffic. The model does well in validation and fails on malformed, partial, adversarial, multilingual, or multi-intent inputs.
• Annotators follow unstated personal preferences. This creates conflicting targets for similar inputs.
• The task is really a knowledge-freshness problem. Fine-tuning does not automatically maintain current prices, policies, documentation, or account data.
• The dataset copies bad legacy behavior. A model can learn terse answers, unsupported claims, or incorrect refusals with high fidelity.
• The team measures only loss. A falling training loss does not prove production behavior is acceptable.

Reinforcement Fine-Tuning (RFT) With GRPO: Optimize What You Can Judge

RFT is useful when writing a complete gold response is expensive or subjective, but scoring candidate responses is manageable. Instead of supplying one target completion per prompt, you define a reward signal. The training procedure produces candidate outputs and uses the reward to push behavior toward higher-scoring answers.

GRPO, or Group Relative Policy Optimization, is a group-based reinforcement approach. At a high level, multiple responses for the same prompt are assessed relative to one another, allowing learning from their comparative quality. The key operational point is not the acronym: your reward must distinguish genuinely useful behavior from shortcuts that merely score well.

When RFT is a better fit than SFT

Consider RFT when any of the following are true:

• There are many legitimate answer paths and no single reference response is ideal.
• Correctness can be checked by a test runner, compiler, policy engine, structured validator, simulator, or reliable human rubric.
• The goal is to improve a measurable outcome, such as satisfying constraints, passing tests, following a sequence, or avoiding policy violations.
• Experts can rank several candidates much faster or more consistently than they can write a polished answer from scratch.

For example, a coding task may have many correct implementations. A deterministic test suite can assess functional behavior. An SFT dataset of authored solutions can still help, but a reward based on test outcomes may better reflect the actual objective.

Inputs and outputs for an RFT workflow

Input

Task: Write a function named normalize_phone that accepts a string and returns
only its digits. Raise ValueError when the input contains no digits.

Constraints:
- Use Python.
- Do not use external packages.
- Return code only.

Generated outputs

Candidate A: [model-generated solution]
Candidate B: [model-generated solution]
Candidate C: [model-generated solution]

Reward output

{
  "candidate_a": {"score": 1, "reasons": ["all tests passed"]},
  "candidate_b": {"score": 0, "reasons": ["returns empty string instead of raising ValueError"]},
  "candidate_c": {"score": 0, "reasons": ["uses an external package"]}
}

A reward can be binary, graded, or a mixture. Prefer deterministic checks where possible. If a model grades another model, treat that judgment as a noisy component, audit it against human review, and isolate it from the only reward signal whenever feasible.

Verified prompt template: reward-rubric design

You are designing a reward rubric for reinforcement fine-tuning.

Task:
[TASK]

Production constraints:
[SAFETY, FORMAT, LATENCY, TOOL, AND POLICY CONSTRAINTS]

Available verifiers:
[TESTS, SCHEMAS, HUMAN REVIEW, POLICY CHECKS, OR NONE]

Define a reward specification that:
1. Scores the user outcome rather than superficial wording.
2. Separates hard failures from quality preferences.
3. Identifies likely reward-hacking behavior.
4. States which checks are deterministic and which require judgment.
5. Rejects outputs that violate mandatory constraints even if they are otherwise useful.

Return:
- HARD_FAILURES: numbered list
- DETERMINISTIC_CHECKS: numbered list
- HUMAN_RUBRIC: 0 to 4 scale with anchored definitions
- REWARD_HACKS_TO_TEST: numbered list
- UNRESOLVED_AMBIGUITIES: numbered list

RFT’s real cost: reward engineering

RFT shifts work from writing answers to designing and maintaining the reward. That is not automatically cheaper. A flawed reward can train behavior that appears excellent in a dashboard yet harms the actual user experience. For example, if brevity is rewarded without sufficiency checks, the model may omit necessary caveats. If tool-call success is rewarded without outcome validation, it may call tools unnecessarily or exploit a weak success condition.

Build a held-out evaluation set that the reward cannot see during iteration. Include deliberate edge cases and examples designed to expose reward hacking. Review sampled outputs manually, especially after any reward change.

Distillation: Use a Stronger Teacher to Create Training Signal

Distillation transfers behavior from a stronger teacher model to a smaller, cheaper, more controllable, or self-hosted student model. In practice, the teacher generates candidate outputs; those outputs are filtered, reviewed, or scored; the selected set becomes training data for the student, often through SFT.

Distillation is attractive when high-quality human demonstrations are scarce but a teacher is capable of solving much of the task. It can also turn expensive inference from a large model into cheaper inference from a smaller model after the student reaches acceptable quality.

Inputs and outputs for a distillation pipeline

Input corpus

User request: Summarize the incident report for an on-call engineer.
Incident report: [redacted operational report]

Teacher output

Impact: [specific operational impact]
Timeline: [ordered events]
Root cause: [evidence-backed cause or unknown]
Mitigation: [completed actions]
Follow-up: [owned next actions]

Selected student-training example

{
  "input": "Summarize the incident report for an on-call engineer...",
  "output": "Impact: ...\nTimeline: ...\nRoot cause: ..."
}

Do not blindly train on all teacher generations. The student will inherit teacher mistakes, unsupported assertions, stylistic quirks, and any systematic bias in the prompt. Filter outputs with task validators, sample them for expert review, and retain provenance: which teacher, prompt version, policy version, and filtering rules produced each record.

Verified prompt template: teacher-output acceptance gate

You are a strict acceptance gate for a candidate distillation example.

Original task input:
[INPUT]

Required output contract:
[FORMAT, FACTUALITY, SAFETY, AND STYLE RULES]

Candidate teacher output:
[OUTPUT]

Assess only what is supported by the original task input and the output contract.

Return valid JSON with:
{
  "accept": true | false,
  "failure_categories": ["unsupported_claim", "format_error", "missing_requirement", "policy_violation", "other"],
  "evidence": ["brief, input-grounded explanation"],
  "repairable": true | false
}

Where distillation goes wrong

The most dangerous assumption is that the teacher is a label oracle. It is not. If the teacher is uncertain, has incomplete context, or interprets the task incorrectly, its output is not ground truth. The benefit of distillation depends heavily on acceptance gates and held-out evaluation. It is often most useful as a data-creation accelerator, not as a replacement for quality ownership.

LoRA and QLoRA: Reduce the Cost of SFT or RFT

Full fine-tuning updates every model weight. This can be slow and memory-intensive. LoRA freezes the base model and trains a relatively small set of adapter weights. QLoRA further reduces memory requirements by quantizing the frozen base model while training adapters.

The key distinction is simple:

LoRA and QLoRA answer “how do we train efficiently?” SFT, RFT, and distillation answer “what supervision teaches the behavior?”

Choose LoRA when

• You need to train task-specific variants without storing a complete new base model for each task.
• Compute and memory are constrained.
• You want to ship, swap, or version smaller task adapters.
• You have a stable base model and want specialization rather than a wholesale model rewrite.

Choose QLoRA when

• The same adapter-based approach is appropriate, but base-model memory usage is the limiting constraint.
• Your training stack supports the selected model, quantization configuration, and target hardware reliably.
• You are prepared to validate quality and operational behavior against a non-quantized reference where that comparison matters.

QLoRA makes more training workloads feasible on constrained hardware, but it does not cure poor data, an unclear target behavior, or a broken reward. It may also introduce setup complexity around compatible runtimes, precision choices, quantization support, and reproducibility. Treat the environment as part of the experiment specification: record base-model revision, tokenizer revision, prompt format, adapter configuration, dataset version, and evaluation version.

A Practical Decision Workflow

Use this sequence before committing to a large training run.

Step 1: Define a non-training baseline

Write the best production prompt you can, including a clear output contract. Add retrieval or tools if the task depends on changing facts. Evaluate this baseline on representative, held-out cases. If the baseline has not been measured, a fine-tuning result has no meaningful comparator.

Step 2: Classify the supervision you possess

• Gold outputs available: Begin with SFT.
• Reliable tests, validators, or rankings available but gold outputs are hard: Investigate RFT/GRPO.
• A capable teacher is available but labeled data is limited: Distill carefully, then usually use SFT on accepted examples.
• Hardware limits dominate: Apply LoRA or QLoRA to the chosen learning objective.

Step 3: Build an evaluation set before training

Split examples by real-world scenario, not only at random. Prevent near-duplicates, templates, or documents from leaking between training and test sets. Include failures that matter operationally: missing information, conflicting instructions, unusual formats, prohibited requests, multilingual inputs where relevant, and schema-breaking outputs.

Step 4: Define release gates

A model should not ship because it “looks better.” Decide thresholds in advance, such as schema validity, task success rate, mandatory safety behavior, human-quality rating, or error rate for high-severity cases. Compare against the baseline and inspect regressions.

Verified prompt template: method-selection worksheet

Act as a fine-tuning planning reviewer. Do not recommend training unless the evidence supports it.

Target task:
[TASK]

Current baseline behavior and measured failures:
[BASELINE RESULTS]

Can experts author ideal outputs? Explain:
[YES/NO/PARTIALLY + DETAILS]

Can outputs be scored reliably? Explain available tests, validators, or rubric:
[DETAILS]

Is a stronger teacher model available? Explain expected use and review process:
[DETAILS]

Operational constraints:
[MODEL, HARDWARE, DATA SENSITIVITY, LATENCY, DEPLOYMENT]

Return:
1. PRIMARY_METHOD: SFT | RFT_WITH_GROUP_BASED_OPTIMIZATION | DISTILLATION_THEN_SFT | DO_NOT_FINE_TUNE_YET
2. TRAINING_MECHANISM: FULL_FINE_TUNING | LORA | QLORA | UNDECIDED
3. RATIONALE: concise, evidence-based explanation
4. MINIMUM_DATA_ARTIFACTS: required dataset, rubric, validators, and provenance fields
5. EVALUATION_GATES: measurable pre-release gates
6. TOP_THREE_RISKS: specific failure modes and mitigations
7. WHAT_WOULD_CHANGE_THIS_DECISION: missing evidence or conditions

Tips That Prevent Expensive Fine-Tuning Mistakes

1. Train the production prompt shape. Differences in role messages, delimiters, schemas, tool descriptions, or context formatting can undermine a dataset that otherwise looks good.
2. Version every artifact. Record model and tokenizer identity, prompt template, data snapshot, annotation guideline, training settings, adapter settings, reward definition, and evaluation set version.
3. Keep an untouched test set. Do not repeatedly improve the training pipeline based on the same test items until it becomes another training signal.
4. Use failure buckets. Track errors by type: unsupported answer, wrong schema, missed escalation, weak extraction, refusal error, tool misuse, or verbosity mismatch. Aggregate scores alone hide useful diagnosis.
5. Preserve negative and boundary cases. A model must learn not only ideal answers but also how to handle absent evidence, impossible requests, conflicting requirements, and unsafe requests.
6. Make annotation disagreement visible. If reviewers disagree often, improve the policy before adding more examples. More contradictory data does not create clarity.
7. Do not use fine-tuning as a database. Current and customer-specific information generally belongs in a controlled retrieval or tool layer, with explicit grounding and validation.
8. Run a rollback-capable deployment. A fine-tuned model can improve average performance while creating a severe regression in a critical edge case.

FAQ

Is SFT always the first fine-tuning method to try?

Usually, when you can author consistent ideal outputs. SFT is easier to reason about because each example visibly demonstrates desired behavior. Do not force SFT onto a task where correct outputs are highly variable and reviewers cannot agree on a target. In that case, a robust verifier or ranking process may provide better supervision for RFT.

What is the practical difference between RFT and GRPO?

RFT is the broader idea of improving a model with reward signals instead of only fixed target outputs. GRPO is a group-based optimization approach within that family, using relative quality among multiple generated responses for a prompt. The practical prerequisite remains the same: a reward, test, or ranking signal that reflects genuine task success and resists shortcuts.

Does LoRA replace SFT or RFT?

No. LoRA is a parameter-efficient way to update a model. You can use LoRA with SFT, RFT, or a distillation-derived training dataset. Choose the learning objective based on your supervision; choose LoRA or QLoRA based on resource, deployment, and adapter-management constraints.

When should I use QLoRA instead of LoRA?

Use QLoRA when the memory required to keep the frozen base model available is a serious constraint and your chosen stack supports the configuration reliably. Validate the resulting model on the same held-out evaluation set used for any alternative approach. The memory benefit does not remove the need for careful setup and quality checks.

Can distillation eliminate human review?

No. Teacher-generated outputs are candidate labels, not guaranteed truth. Automated filters can catch structural failures, but human review and task-specific evaluation remain necessary for high-impact behavior, subtle policy requirements, or factual claims that require judgment.

How do I know that fine-tuning is the wrong solution?

Fine-tuning is likely the wrong first move when the main problem is stale knowledge, inaccessible private data, missing tools, unclear business policy, an unmeasured baseline, or insufficiently defined evaluation criteria. Fix the information and workflow architecture first; then fine-tune only if a persistent behavior gap remains.

Related Reading

• Hybrid AI Support Prompts: RAG and Fine-Tuning Workflows Developers Can Use Today
• Kimi K3 on a 16x GB10 Cluster: Ready-to-Copy Prompts for Benchmarking, vLLM Packaging, and Performance Tuning
• API Gateway Prompt Collection: Turn Real LLM Usage Data Into Better Onboarding, Quotas, and Model Selection