Fine-Tuning Data Preparation in Practice

The previous article, “Hands-On Introduction to LoRA/QLoRA Fine-Tuning,” covered the training workflow. This article focuses on the step that was mentioned only briefly but determines success or failure: data preparation.

There is a saying in fine-tuning circles: garbage in, garbage out. No matter how good the model architecture is, how precisely the learning rate is tuned, or how carefully the LoRA rank is chosen, garbage input still produces garbage output. Conversely, sufficiently good data can produce remarkable results even with a basic training workflow. This article explains how to prepare fine-tuning data: formatting, collection, cleaning, quality control, synthetic data, augmentation, and scale. Every step includes practical guidance.

1. Quality Is Everything: Lessons from LIMA and “Less but Better”

First, establish a potentially counterintuitive principle: for fine-tuning data, quality matters far more than quantity.

The strongest evidence comes from the LIMA (Less Is More for Alignment) paper. Its authors used only 1,000 carefully written, high-quality human-authored examples to fine-tune a base model to conversational performance approaching GPT-4. One thousand examples, fewer than you might collect in a single scraping run. But every example was polished word by word by domain experts, covered diverse scenarios, and provided a model answer worthy of being called a “gold standard.”

By comparison, many teams train on hundreds of thousands of uneven-quality instruction-response pairs scraped from the web, yet achieve worse results than LIMA. Why?

• Fine-tuning fundamentally adjusts the model's output distribution, moving it toward the behavior you want. Noisy examples pull that distribution in the wrong direction.
• The model learns every pattern in the data, including bad ones: verbosity, digressions, irrelevant answers, and inconsistent formatting. Once learned, these patterns are difficult to correct.
• Redundant examples waste training signal. Ten thousand distinct, high-quality examples contain far more information than 100,000 repetitive, mediocre ones.

The first rule of data preparation is therefore: maximize quality before discussing quantity. Two thousand excellent examples are better than 50,000 weeds. Every subsequent step, including cleaning, filtering, and deduplication, is fundamentally about protecting this baseline.

2. Instruction-Pair Formats: What the Data Looks Like

The core structure of instruction-tuning data is a triplet: instruction + input (optional) + output (expected output). The classic Alpaca format looks like this:

{
  "instruction": "Translate the following sentence into formal Chinese",
  "input": "hello world",
  "output": "Hello, world"
}

When there is no input, simply leave it empty, as in “Write a poem about autumn.” This is a single-turn instruction format, and early versions of Alpaca and Vicuna both used variants of it.

Real-world scenarios more often involve multi-turn conversations, which require a conversational sequence format. ChatML is the current de facto standard, widely adopted by Qwen, Llama, InternLM, and others. It uses special tokens to identify roles:

<|im_start|>user
Summarize the following text for me:<Article content><|im_end|>
<|im_start|>assistant
This passage is about...<|im_end|>
<|im_start|>user
Now say it again in English<|im_end|>
<|im_start|>assistant
The passage is about...<|im_end|>

For storage, a more structured JSON representation is usually used, with one conversation list per example:

{
  "messages": [
    {"role": "user", "content": "Summarize the following text for me:<Article>"},
    {"role": "assistant", "content": "This passage is about..."},
    {"role": "user", "content": "Now say it again in English"},
    {"role": "assistant", "content": "The passage is about..."}
  ]
}

Three practical reminders:

1. The format must match the training framework. LLaMA-Factory, Unsloth, and Axolotl each have their own dataset schema. Check the required format before choosing a framework rather than discovering after generating the data that it cannot be ingested.
2. Compute loss only on the assistant portion. During multi-turn conversation training, user content is context and should not contribute to loss; otherwise, the model learns to “generate the user's words.” Mainstream frameworks do this by default, but always confirm it in custom training scripts.
3. Do not omit the EOS token. Every output must end with a termination token. Otherwise, the model will not know when to stop during inference and may continue rambling until it reaches max_tokens.

3. Where Data Comes From: Three Approaches

Where do instruction-response pairs come from? There are three mainstream approaches, each with its own tradeoffs.

1. Human annotation: expensive but accurate. Ask domain experts to write the instruction and output. LIMA and Dolly both followed this approach. It offers the highest quality ceiling because people can write excellent answers that models would not produce on their own, while naturally providing diversity. The drawbacks are that it is expensive, slow, and difficult to scale. A high-quality human-authored example may cost anywhere from a few yuan to dozens of yuan, so producing several thousand can easily cost tens of thousands. This approach is suitable for core seed data and high-value tasks.

2. Real user logs: the most realistic, but they must be de-identified. If you already have a running product, such as a customer-service bot or coding assistant, real user questions paired with ideal answers are a gold mine. Their distribution naturally matches production scenarios; nothing is more realistic. But there are two pitfalls:

• De-identification is non-negotiable. User logs contain PII such as names, phone numbers, ID numbers, and internal code. This information must be cleaned and replaced before entering the training set. Otherwise, it creates compliance risks and may cause the model to leak data.
• Raw answers may not be good enough. The bot's answers in user conversations may be poor, especially if they came from the old model you want to replace. Either use only the user side and have humans rewrite the answers, or select only highly rated or accepted conversations.

3. Synthetic data: inexpensive but biased. Use a strong model such as GPT-4 or Claude to generate training pairs. This approach is cheap, fast, and highly scalable, making it the prevailing way to expand datasets today. It also carries the greatest risks, which the next section covers in detail.

A common practical recipe is: human-authored seeds (hundreds of examples) to establish quality + synthetic data (thousands to tens of thousands) to expand scale + user logs (continuously) to align with the real-world distribution. Mix all three in suitable proportions rather than relying on a single source.

4. Cleaning: Deduplication, Denoising, Filtering, and Consistency

Regardless of where the data comes from, cleaning is mandatory. It is the critical process that turns “garbage into usable data” and usually has four steps.

1. Deduplication. There are two types:

• Exact deduplication: the entire example string is identical. One line of Python handles it: df.drop_duplicates(subset=['instruction','output']).
• Near-duplicate removal: the content is highly similar, differing only by a few words or a rephrased question. This accounts for most duplication because scraped data contains many superficial rewrites of the same template. Use MinHash + LSH from the datasketch library or SimHash to calculate similarity. Treat examples above a threshold such as 0.8 as duplicates and keep only one.

Near-duplicate removal is often overlooked, but it can dramatically improve quality. A dataset containing 100,000 “seemingly diverse” examples may retain only 30,000 genuinely distinct ones after near-duplicate removal. The other 70,000 redundant examples are not merely useless; they can make the model overfit to boilerplate phrasing.

2. Denoising. Remove examples with obvious problems:

• Garbled text: encoding errors, invisible characters, or residual HTML tags.
• Truncation: an output ends halfway through a sentence because it was truncated during scraping.
• Empty or extremely short content: an output contains only a few characters and almost no information.
• Mixed languages: a sample that should be Chinese contains large amounts of English.

A practical cleaning-script skeleton:

import re, pandas as pd
from datasketch import MinHash, MinHashLSH

def is_clean(text, min_len=10, max_len=4000):
    if not text or len(text) < min_len or len(text) > max_len:
        return False
    if re.search(r'<[^>]+>', text):          # HTML tags
        return False
    if len(re.findall(r'[\x00-\x1f]', text)) > 5:  # Control characters
        return False
    return True

df = df[df['output'].apply(is_clean)]

# Near-duplicate removal
def make_minhash(text, num_perm=64):
    m = MinHash(num_perm=num_perm)
    for w in set(text.split()):
        m.update(w.encode('utf-8'))
    return m

lsh = MinHashLSH(threshold=0.8, num_perm=64)
keep = []
for i, row in df.iterrows():
    mh = make_minhash(row['instruction'])
    if not lsh.query(mh):           # No similar item has been seen; keep it
        lsh.insert(str(i), mh)
        keep.append(i)
df = df.loc[keep]

3. Filtering. Apply business rules to remove examples that should not be used for training:

• Length filtering: content that is too short contains little information, while excessively long content may be an accidentally included document.
• Toxicity and safety: use a classifier such as Perspective API or OpenAI moderation to remove harmful content. Do not teach the model behaviors you do not want.
• Language: use fastlid or langdetect to retain the target language.
• Task relevance: use embedding similarity to remove examples unrelated to the target domain.

4. Consistency checks. Many people skip this step, but it is critical: do the instruction and output actually match? Synthetic data often “asks A but answers B”; for example, the instruction requests a translation, but the output is a summary. Use a strong model to evaluate consistency, as described in the next section, or apply simple rules for coarse filtering. For example, if the instruction contains “translate,” the output language should match the request. Inconsistent examples are strong noise and directly mislead the model.

5. Quality Control: Sampling, Model Judges, and Bad Cases

Cleaning is not the end. You must also perform active quality assurance. Use these three methods together.

1. Manual sample review. This is the simplest and most effective method. Randomly sample 100-200 examples from the cleaned data and read each one: Is the instruction clear? Is the output correct and good? Are there typos, factual errors, or formatting problems?

This step can reveal systemic problems you never anticipated. For example, every synthetic example may begin with “First... Second... Finally...,” or 80% of user logs may consist of greetings. The purpose of sample review is not to correct every example; it is to identify dataset-wide bias. When you find bias, go back and clean the dataset again.

2. Use a strong model as a judge (LLM-as-a-judge). Manual review cannot cover tens of thousands of examples, so use a strong model for batch scoring. Give GPT-4 or Claude a scoring prompt:

You are a strict data-quality reviewer. Score the following instruction-tuning example from 1 to 5:

Instruction: {instruction}
Output: {output}

Scoring criteria:
1. Whether the Instruction is clear and independently understandable
2. Whether the Output correctly answers the Instruction
3. Output quality (accuracy, completeness, and lack of redundancy)
4. Whether the formatting is standardized

First provide a brief rationale, then output a final line: "SCORE: X".

Run the dataset through it, then manually review or directly remove examples with SCORE ≤ 2. The judge model does not need to be perfect; it only needs to filter out obviously poor examples. Note that the judge model should ideally be one tier more capable than the base model you are fine-tuning. Asking a model of the same tier to evaluate its peers can lead to overly lenient judgments.

3. Remove bad cases. Prepare a “negative checklist” of behaviors you do not want the model to exhibit. For example:

• Do not respond with evasive phrases such as “I am an AI, so I cannot...”
• Do not restate the user's question at the beginning of every answer.
• Do not end with filler such as “Hope this helps!”

Scan the data with rules or a small classifier, then remove or rewrite examples containing bad cases. If the model never learns bad patterns, it will naturally be less likely to produce them. This is far cheaper than correcting those behaviors with RLHF after training.

6. Synthetic Data: Inexpensive, but Not Free of Cost

Synthetic data is now the primary way to expand datasets, but it is a double-edged sword. Used correctly, it is powerful; used incorrectly, it plants hidden problems.

The right approach:

1. Provide good seeds. Do not ask a strong model to generate everything from scratch, because it will collapse toward a few common question patterns. First prepare a diverse set of seed instructions manually, then have the model expand from them. Seed quality determines the ceiling.
2. Explicitly require diversity. In the prompt, require that “topic, difficulty, length, style, and language style must all differ.” Apply the self-instruct approach: ask the model to generate many candidate instructions, then cluster and sample them to ensure topic coverage.
3. Use a strong model to generate the output, then filter it with a judge model using the workflow from the previous section. Generation plus filtering provides two layers of protection.
4. The Evol-Instruct approach: “evolve” existing instructions by adding constraints, increasing depth, or changing complexity. This produces harder variants and improves the dataset's difficulty distribution.

You must understand the risks:

• Mode collapse. Examples generated by a strong model tend to be highly homogeneous in style; they all sound like ChatGPT. After fine-tuning, your model may also speak only in a “ChatGPT-like” voice and lose its individuality.
• Bias amplification. A strong model's built-in biases, including its positions, tone, and avoidance tendencies, transfer into your model through synthetic data and may even be amplified. This is why some fine-tuned models become increasingly “politically correct” and repetitive.
• Model collapse. If model A generates data to train model B, then B generates data to train C, and so on, the tail of the data distribution disappears after several generations and the models become increasingly mediocre. Synthetic data is best used only to “increase volume”; real data must still anchor the base dataset's diversity.
• Knowledge blind spots. Knowledge that the strong model lacks will also be absent from synthetic data. Domain-expert knowledge still requires human annotation or real documents.

My practical recommendation is: do not let synthetic data exceed 60-70% of the dataset. Core seeds and critical domain examples must be human-authored. A model trained entirely on synthetic data may look impressive at first, but long-tail production cases will expose its weaknesses.

7. Data Augmentation: Increase Scale Without Sacrificing Quality

If you lack data but do not want to rely too heavily on synthetic generation, use data augmentation. It expands existing high-quality examples in different ways without reducing information density.

Three common techniques are:

1. Paraphrasing. Use a model to express the same example differently: rephrase the instruction and rewrite the output without changing the meaning. One original example becomes 3-5 examples, improving robustness to different phrasings. Only paraphrase high-quality examples. Rewriting garbage five times still produces garbage.

2. Back-translation. Translate Chinese → English → Chinese to obtain natural and diverse Chinese expressions. This longstanding NLP technique remains effective for instruction data and can break the syntactic habits of the original dataset. In multilingual scenarios, it can also produce multilingual training pairs along the way.

3. Format transformation. Present the same task using different output formats:

• “List three reasons” → use a Markdown list, numbered list, table, or paragraphs.
• Summary → a one-sentence version, three-sentence version, or bullet-point version.

This teaches the model that “content remains stable while formatting is controllable,” which is one of the primary goals of fine-tuning.

Augmentation discipline: Every augmented example should still pass quality filtering, including deduplication and judge-model scoring. Augmented data does not deserve an exemption. Augmentation is an amplifier: it amplifies quality, but it also amplifies defects.

8. Scale Guidelines: How Many Examples Do You Actually Need?

One final frequently asked question: how much data does fine-tuning require? There is no standard answer, but these practical ranges provide a reference:

Several dimensions to consider:

• Task complexity: open-ended generation, such as article writing, requires more examples to cover diverse outputs. Closed tasks, such as classification and extraction, need fewer.
• Base-model capability: the stronger the base model, especially if it is already a capable instruct model, the smaller the “correction” required from fine-tuning and the less data you need. A base model requires more.
• Rate of change: if the data changes frequently, such as customer-service scripts or product rules, train on less data and iterate often. For stable targets such as writing style, you can invest more.
• Overfitting threshold: instruction tuning generally uses 2-3 epochs. Be especially careful about overfitting when the dataset contains only a few hundred examples. The model may memorize the examples and lose its ability to generalize. Below 500 examples, consider using augmentation and synthetic data to reach 1k+ before training.

A simple rule of thumb is: first train one version using the highest-quality 1k-2k examples you can obtain, evaluate the results, and then decide whether to add more. Do not begin by piling up tens of thousands of uncleaned examples. You will be unable to tell whether any change came from having more data or from the side effects of added noise.

Summary

Preparing fine-tuning data fundamentally means one thing: use the fewest, most accurate, and cleanest examples possible to show the model how it should behave.

• Quality > quantity: LIMA used 1,000 examples to outperform datasets containing hundreds of thousands. Maximize quality before discussing scale.
• Match the framework's format: use Alpaca triplets or ChatML multi-turn conversations as required by the training tool. Only the assistant portion should contribute to loss, and do not forget EOS.
• Three data sources: human annotation (accurate but expensive), user logs (realistic but requiring de-identification), and synthetic data (inexpensive but biased). Use a mixture.
• Four cleaning steps: deduplication, especially near-duplicate removal; denoising; filtering; and consistency checks. Skip this stage, and everything that follows is wasted effort.
• Three quality-control tools: manual sample review, strong-model judge scoring, and bad-case removal.
• Synthetic data has limits: seeds must be diverse, generated examples must be filtered, and the proportion should not exceed 60-70%. Watch for mode collapse and bias amplification.
• Scale depends on the task: begin with 1k-10k examples for instruction following and potentially 10k+ for domain adaptation, but always start small with high-quality data and iterate.

Remember this maxim: fine-tuning is 70% data and only 30% training. Adjusting the learning rate, changing the LoRA rank, and adding epochs are all merely fine-tuning, pun intended, once the data is good enough. Build the data pipeline properly and everything afterward becomes much easier. Cut corners on the data, and even the flashiest training techniques cannot rescue the result. That is why people who truly understand fine-tuning spend most of their time on data.