Running ML models in the browser with ONNX Runtime Web: a practical guide

Presenting to a room
Presenting to a room

Originally published at https://pranjulrathour.scult.in/blog/onnx-runtime-web-browser-ml-guide. That copy is the canonical version and gets updates first.

FaceVision does face detection, recognition and liveness checks without sending a single frame to a server, because every model runs in the browser through ONNX Runtime Web. That architecture is the reason its privacy story is simple. It is also the part students find hardest to get working, so here is the path that worked.

Export, then verify

Export the model to ONNX from its training framework and immediately run the same input through both the original and the exported graph. Compare outputs numerically. Every model in FaceVision was verified against its actual ONNX graph and, where possible, the reference implementation — not assumed from documentation. Half the "the model is broken in the browser" reports I have seen were pre-processing mismatches that a five-minute comparison would have caught.

Requirements gathering and user flows, on stage
Requirements gathering and user flows, on stage

Choose the execution provider deliberately

  • WebGPU — fastest where available; check support and fall back gracefully.
  • WASM with SIMD and threads — the reliable default; needs the right headers for multi-threading.
  • WebGL — legacy; avoid for new work.

Pre- and post-processing is where bugs live

Models expect a specific channel order, normalisation and input size. Browser image data arrives as RGBA bytes in a different layout. Write the conversion once, test it against a known image, and keep the output tensors' shapes in a comment next to the code. Post-processing — decoding anchors for a detector, normalising an embedding before cosine similarity — deserves the same care.

Performance that feels live

  1. Warm up the session with one dummy inference on page load; the first run is always slow.
  2. Run inference in a Web Worker so the camera preview never stutters.
  3. Downscale frames before detection; run recognition only on detected crops.
  4. Skip frames when the queue is full rather than letting latency grow.

What the backend does when the model does not

In FaceVision the FastAPI and PostgreSQL backend stores only vector embeddings and matches them at enrolment and verification. Because no images are ever persisted, the privacy claim is easy to explain and audit. I wrote about the design reasoning in face recognition that never uploads a face.

Presenting KrishGyan, farming advice in your voice and language
Presenting KrishGyan, farming advice in your voice and language

Browser-side inference is not a gimmick. For anything involving faces, documents or health data, it is often the architecture that makes the product acceptable to the people using it.

From my carousels
5 Production AI Apps, All Open Source
5 Production AI Apps, All Open Source, slide 15 Production AI Apps, All Open Source, slide 2
5 Production AI Apps, All Open Source, slide 35 Production AI Apps, All Open Source, slide 4
Full carousel on Instagram and LinkedIn.
Pranjul Rathour
Pranjul Rathour
GenAI engineer, Kanpur · 3x first-prize hackathon winner · campus mentor
I ship production RAG pipelines, fine-tune LLMs and build agentic AI products end to end. I lead engineering at SCULT INDIA for a 14-member team and have mentored 200+ students through TechVerse Enclave.
Open to: GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges.
On stage, at hackathons and on campus
Taking questions during a session
Taking questions during a session
Pranjul Rathour
Pranjul Rathour
Presenting Annapurna on stage
Presenting Annapurna on stage

Comments

Post a Comment

Popular posts from this blog

Forming a hackathon team: roles, skills and the mistake most teams make

Image
At an Integral Startup Foundation hackathon Originally published at https://pranjulrathour.scult.in/blog/hackathon-team-roles-and-formation . That copy is the canonical version and gets updates first. Team TechVerse — Raman Shukla, Saksham Gupta and me — won three first prizes in 2025 not because we were the strongest coders in the room, but because we had stopped overlapping. When I mentor teams before a hackathon, formation is the first thing I fix. In a packed college auditorium The four roles Builder — owns the core feature and the demo path. Usually backend or the AI piece. Interface — owns what the judge sees: the front end, the flow, the seeded data. Storyteller — owns the problem statement, the pitch, the slides and the Q&A plan; also does user research in hour one. Integrator — keeps the pieces working together, manages the repo, runs the demo rehearsals. In a team of three, the builder or interface person takes this. The mistake Five friends who all write R...
Read more

Hello from Kanpur: what I build, and what I'll write about here

Image
On the mic Originally published at https://dev.to/pranjulrathour/hello-from-kanpur-what-i-build-and-what-ill-write-about-here-f2j . That copy is the canonical version and gets updates first. I'm Pranjul Rathour, a GenAI engineer from Kanpur, India. This is the first post on this blog, so here is who is writing, what I have actually shipped, and what you can expect if you follow along. What I build Between February and July 2026, at Next Upgrad Web Solutions, I designed, built and shipped five production AI applications end to end, alone, and deployed all of them with Docker, CI and observability: RAG.NextUpgrad : a retrieval-augmented generation service with hybrid dense plus BM25 retrieval, cross-encoder reranking, streaming cited answers, and a confidence gate that refuses to call the model when the evidence is weak. It runs in about 220 MB of memory, inside a free 512 MB instance. FineTune Studio : fine-tune open models with QLoRA in a browser. Upload a dataset, validate ...
Read more

I built 15 free tools, 1,211 prompts and a 50,000-skill library — here's what's inside tools.scult.in

Image
Carousel · 12 slides I built every part of tools.scult.in myself — 15 browser tools, 1,211 version-stamped prompts, a 50,000+ agent-skills library, a public MCP server, and 40 retro photo prompts where I'm the face in every example. All of it free. No signup, no ads, no upload step. Swipe through for what's inside and why I built it this way. Link in the last slide. Slide 1 of 12 Slide 2 of 12 Slide 3 of 12 Slide 4 of 12 Slide 5 of 12 Slide 6 of 12 Slide 7 of 12 Slide 8 of 12 Slide 9 of 12 Slide 10 of 12 Slide 11 of 12 Slide 12 of 12 Pranjul Rathour GenAI engineer, Kanpur · 3x first-prize hackathon winner · campus mentor I ship production RAG pipelines, fine-tune LLMs and build agentic AI products end to end. I lead engineering at SCULT INDIA for a 14-member team and have mentored 200+ students through TechVerse Enclave. Open to: GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email: pranjulrathour41@gmail.com Portfolio & ...
Read more