— Small open models
Fine-tuning is worthwhile when tests show a recurring quality gap and you have enough curated data. I work with small open models, not with a promise to retrain a very large public model.
The path starts with your data: audit, curation, and a clean split between training and evaluation. Only then come the fine-tune, serving, and a rollback path. If retrieval and clear prompts are enough, I say so before any training run.
In the lab, stacking Cisco-router first-day configuration and live diagnosis into one 1.5-billion-parameter model adapter failed. The next experiment is a human-switched sidecar: you name the job, paste the site, the matching model adapter types on the serial console. Live wiring is parked.
— From dataset to operation
I review provenance, rights, duplicates, gaps, sensitive content, and whether the examples actually match the workflow you want. The result is a clear statement on whether fine-tuning is justified.
We clean, align, and label instruction pairs so a human can judge quality. Weak examples are removed instead of mixed into training.
Training examples and test examples stay separate. Evaluation measures the same workflow you expect in production — not only an internal loss curve.
I adapt a small open model with a traceable method in the QLoRA class. Checkpoints, hyperparameters, and comparison runs against the base model are recorded.
The adapted model is served so your team can call it, measure it, and revert to the previous version if needed. Documentation and acceptance tests are part of handover.
— When you should not fine-tune
Many projects first need better prompts and document retrieval. Fine-tuning comes only after those steps fail to close the quality gap.
If answers live mainly in files, policies, or knowledge bases, retrieval with citations is usually faster, cheaper, and easier to review.
If the workflow can be steered with instructions, templates, and approval gates, start with prompt design and measurable test cases — without changing model weights.
Without enough curated instruction pairs and a real evaluation set, you only buy an expensive overfit. The next step is then dataset work or a different approach.
— What you receive
You know whether fine-tuning, retrieval, or prompts is the right next step — from tests, not from a demonstration.
Curated instruction pairs, separate training and evaluation sets, and a short record of what was deliberately excluded.
Comparison against the base model on the same test cases, including known weaknesses.
Serving, measurement, and rollback to the previous version so your team can repeat the step.
— Frequently asked questions
Describe the workflow, the data you have, and what you already tried with prompts or retrieval. I will reply with an initial assessment.