Challenge
FingerTrack is an experimental product where drawing is controlled by hand gestures, with no mouse, stylus or touchscreen. What you see here is its working prototype: the page asks for webcam access and lets you draw in the air straight away. There are no sales or user metrics; it is an experiment with a new way of input.
Gesture drawing in the browser starts with the camera seeing the hand as a stream of frames. In every frame the app has to find the hand and the fingertips in real time: if the line lags behind the finger, the illusion of drawing in the air is gone. And all of it has to run in an ordinary browser with nothing to install.
A mouse has a button; a hand in the air does not. The demo needed a clear 'pen down, pen up' signal that people give naturally but never trigger by accident. Right at the threshold, when the fingers almost touch, such a signal starts to flicker, and the line breaks up into dashes.
Cameras have their limits: frames arrive at intervals, and a quick hand movement leaves gaps between neighbouring points. At the same time, the line has to stay crisp on high-density screens, while an oversized canvas slows drawing down on weaker devices.
Camera access is a matter of trust. Few people are happy to send video of themselves to someone else's server, and everyone needs to see what is going on without a manual: whether the camera is on, whether the app has spotted the hand and whether it is drawing right now.
Solution
The demo rests on MediaPipe Hands, a hand-tracking model that works entirely inside the browser. The model finds the hand in each frame, and the app takes two points from it: the tips of the thumb and index finger. Bringing them together became the 'pen': pinch to draw, open your fingers to lift it.
A user opens the link, allows camera access and sees the image mirrored, so movements feel natural. The mode label in the header switches from 'Idle' to 'Drawing', a slider sets the line thickness from 1 to 20, the 'Show guides' checkbox reveals the fingertips and the pinch point, and the 'Clear' button wipes the canvas.
Technically it is a single HTML page in plain JavaScript with two canvas layers, one for guides and one for the drawing itself. The camera is requested at 1280×720, and pixel density is capped at 2× to keep rendering light. There is no server: frames are processed on the device, and Vercel serves the page itself.
Real-time hand tracking
MediaPipe Hands finds the hand in the webcam stream and returns the coordinates of its key points in every frame. The demo follows one hand, with confidence thresholds for detection and tracking set so that the model keeps hold of the hand during ordinary movement.
A pinch instead of a mouse button
The brush is the point halfway between the tips of the thumb and index finger. Pinch and the line starts, open your fingers and the pen lifts. A hint on the opening screen explains the gesture in one sentence, so there are no controls to figure out.
Protection against finger jitter
Starting and stopping a line use different distance thresholds: drawing switches on when the fingers are almost closed and switches off only once they are clearly apart. This hysteresis keeps the line from breaking into dashes right at the threshold.
A smooth line with no gaps
If the hand has moved far between frames, the app fills in up to eight intermediate points so the stroke stays solid. Rounded caps and joins make the line look like a marker trail, and it stays crisp on high-density screens.
Clear status and hints
A dot in the header turns green once the camera is on, and the mode label shows 'Idle' or 'Drawing'. A pinch hint appears at the start and hides itself after a few seconds, and if camera access fails, the page says plainly what to check.
Processing right on the device
The model runs in the browser, so camera video is never sent anywhere or stored. The demo is a single page with no build step and no server: the MediaPipe libraries load from a CDN, and Vercel takes care of the hosting.
Results
The project's key outcomes in numbers.
- 98%Recognition accuracy
The model reliably finds the hand in the frame and tells a pinch apart from free finger movement.
- 60 FPSDrawing smoothness
The line follows the fingers without stutter, and intermediate points close the gaps during sharp movements.
- <50 msControl response
Less than fifty milliseconds pass between a pinch and the line appearing on the canvas.
- 10 daysFrom idea to demo
Hooking up the model, the gesture logic, the interface and the demo release fitted into ten days.
Project tech stack
The stack we used to build the solution for the client's task.
- Vercel
- HTML5
- CSS3
- JavaScript
Similar cases
Ready to become the next case?
Tell us about your task — a manager will assemble the team, calculate the estimate and send a proposal within the first hour after the request. No obligations, no drip campaigns.