Running a LoRA fine-tune on a consumer GPU at home turns out to be more forgiving than the AI Twitter discourse would suggest — and more informative about what actually matters for production than a cloud run ever is.
The setup: a single-GPU box with 24GB VRAM, a 7B-parameter open-source base model, and a target corpus of about 30K domain-specific instruction pairs. Total training time for a meaningful LoRA adapter: roughly 4 hours. Total cost: electricity.
Three things surprised me, in order:
1. Data quality dominates everything else. I spent two weeks preparing training pairs and 4 hours running the fine-tune. If I had it to do over, I would spend three weeks on data and 4 hours on compute. The returns on cleaner, more specific, more consistent training data are enormous. The returns on another hyperparameter sweep are marginal.
2. Evaluation is harder than training. The fine-tune finished in an afternoon. Evaluating whether it was actually better than the base model on realistic tasks took a week. Most public benchmarks do not test what I actually wanted the model to do, so I ended up hand-building a task-specific eval. That work was more valuable than the training itself.
3. Local fine-tuning changes the economics of experimentation. When each run costs nothing but time and a few kilowatt-hours, you iterate more. I ran 14 experiments in a month. That would have cost thousands in cloud compute. The fact that I could try genuinely bad ideas quickly is why I eventually found the good ones.
Next: move to a larger base model and a more rigorous eval harness. The research practice is catching up to the infrastructure.
Leave a Reply