LoRA vs QLoRA vs full fine-tuning: cost, quality and when each makes sense

Pranjul Rathour
Pranjul Rathour

Originally published at https://pranjulrathour.scult.in/blog/lora-vs-qlora-vs-full-fine-tuning. That copy is the canonical version and gets updates first.

Every fine-tuning tutorial assumes hardware most students do not have. The choice between LoRA, QLoRA and full fine-tuning is mostly a hardware and data question, so let me lay out what each one actually does before you pick.

Full fine-tuning: every weight moves

You update all parameters of the model. It gives the most capacity to change behaviour and is what labs do for major model versions. It also needs memory for weights, gradients and optimiser states — several times the model size — which puts even a 7B model out of reach for a consumer GPU. For a student project it is almost never the right tool.

Presenting KrishGyan, farming advice in your voice and language
Presenting KrishGyan, farming advice in your voice and language

LoRA: train small adapters instead

LoRA freezes the base weights and trains low-rank matrices injected into attention (and often MLP) layers. The trainable parameters drop to a fraction of a percent, memory drops with them, and the result is a small adapter file you can swap in and out. Quality on narrow tasks is usually close to full fine-tuning.

QLoRA: LoRA on a quantised base

QLoRA loads the frozen base model in 4-bit precision and trains LoRA adapters on top. That is how FineTune Studio trains Qwen3-1.7B at around 3.2 GB of VRAM — hardware a student can borrow or rent for almost nothing. The adapters are trained in higher precision, so quality holds up well; the cost is slower steps because of dequantisation.

At an Integral Startup Foundation hackathon
At an Integral Startup Foundation hackathon

A decision guide

  • Under 8 GB VRAM, or a free cloud GPU → QLoRA. Nothing else fits.
  • 16–24 GB VRAM and a model up to 7–8B → LoRA in 16-bit for faster steps, QLoRA if you want a bigger base.
  • A narrow behaviour (format, tone, classification) → adapters are enough; full fine-tuning is waste.
  • Fundamentally new capabilities or a new language → full fine-tuning on serious hardware, and probably not your project this semester.

The part that matters more than the method

Whichever you choose, evaluate base against tuned on held-out examples before you believe anything — the workflow in how to evaluate a fine-tuned model honestly. Most disappointing fine-tunes were not the wrong method; they were the wrong dataset.

From my carousels
1,211 Prompts, Every One Stamped
1,211 Prompts, Every One Stamped, slide 11,211 Prompts, Every One Stamped, slide 2
1,211 Prompts, Every One Stamped, slide 31,211 Prompts, Every One Stamped, slide 4
Full carousel on Instagram and LinkedIn.
Pranjul Rathour
Pranjul Rathour
GenAI engineer, Kanpur · 3x first-prize hackathon winner · campus mentor
I ship production RAG pipelines, fine-tune LLMs and build agentic AI products end to end. I lead engineering at SCULT INDIA for a 14-member team and have mentored 200+ students through TechVerse Enclave.
Open to: GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges.
On stage, at hackathons and on campus
Pranjul Rathour
Pranjul Rathour
Presenting to a room
Presenting to a room
Trophy and certificate of merit at Vividhotsava 2025
Trophy and certificate of merit at Vividhotsava 2025

Comments

Post a Comment

Popular posts from this blog

Forming a hackathon team: roles, skills and the mistake most teams make

Image
At an Integral Startup Foundation hackathon Originally published at https://pranjulrathour.scult.in/blog/hackathon-team-roles-and-formation . That copy is the canonical version and gets updates first. Team TechVerse — Raman Shukla, Saksham Gupta and me — won three first prizes in 2025 not because we were the strongest coders in the room, but because we had stopped overlapping. When I mentor teams before a hackathon, formation is the first thing I fix. In a packed college auditorium The four roles Builder — owns the core feature and the demo path. Usually backend or the AI piece. Interface — owns what the judge sees: the front end, the flow, the seeded data. Storyteller — owns the problem statement, the pitch, the slides and the Q&A plan; also does user research in hour one. Integrator — keeps the pieces working together, manages the repo, runs the demo rehearsals. In a team of three, the builder or interface person takes this. The mistake Five friends who all write R...
Read more

Hello from Kanpur: what I build, and what I'll write about here

Image
On the mic Originally published at https://dev.to/pranjulrathour/hello-from-kanpur-what-i-build-and-what-ill-write-about-here-f2j . That copy is the canonical version and gets updates first. I'm Pranjul Rathour, a GenAI engineer from Kanpur, India. This is the first post on this blog, so here is who is writing, what I have actually shipped, and what you can expect if you follow along. What I build Between February and July 2026, at Next Upgrad Web Solutions, I designed, built and shipped five production AI applications end to end, alone, and deployed all of them with Docker, CI and observability: RAG.NextUpgrad : a retrieval-augmented generation service with hybrid dense plus BM25 retrieval, cross-encoder reranking, streaming cited answers, and a confidence gate that refuses to call the model when the evidence is weak. It runs in about 220 MB of memory, inside a free 512 MB instance. FineTune Studio : fine-tune open models with QLoRA in a browser. Upload a dataset, validate ...
Read more

I built 15 free tools, 1,211 prompts and a 50,000-skill library — here's what's inside tools.scult.in

Image
Carousel · 12 slides I built every part of tools.scult.in myself — 15 browser tools, 1,211 version-stamped prompts, a 50,000+ agent-skills library, a public MCP server, and 40 retro photo prompts where I'm the face in every example. All of it free. No signup, no ads, no upload step. Swipe through for what's inside and why I built it this way. Link in the last slide. Slide 1 of 12 Slide 2 of 12 Slide 3 of 12 Slide 4 of 12 Slide 5 of 12 Slide 6 of 12 Slide 7 of 12 Slide 8 of 12 Slide 9 of 12 Slide 10 of 12 Slide 11 of 12 Slide 12 of 12 Pranjul Rathour GenAI engineer, Kanpur · 3x first-prize hackathon winner · campus mentor I ship production RAG pipelines, fine-tune LLMs and build agentic AI products end to end. I lead engineering at SCULT INDIA for a 14-member team and have mentored 200+ students through TechVerse Enclave. Open to: GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email: pranjulrathour41@gmail.com Portfolio & ...
Read more