Challenge
FingerTrack is an experimental project where drawing is controlled by hand gestures, with no mouse, stylus or touch screen. This case shows its working prototype: the page asks for webcam access and lets you draw in the air right away. The project has no sales or user metrics; it is an experiment with a new way of input.
Gesture drawing in the browser starts with the camera seeing the hand as a stream of frames. In each one the app has to find the hand and the fingertips while the hand is moving: if the line visibly lags behind the finger, the illusion of drawing in the air is gone. The “draw or not” decision has to be made anew on every frame. And all of it had to work in an ordinary browser, with no apps, drivers or special sensors to install, just a webcam. The hand recognition model also had to keep the page light and openable from a regular link.
A mouse has a button; a hand in the air does not. We needed a clear “pen down, pen up” signal that people give naturally but do not trigger by accident. The obvious candidate is a pinch of the thumb and index finger, but right at the threshold, when the fingers almost touch, the signal starts to flicker and the line breaks into dashes. The hand had to be able to tremble slightly without cutting the stroke, and the pen had to land where the person is looking: between the fingers, not on the tip of one of them. And when the hand leaves the frame, drawing had to stop on its own.
The camera has its limits: frames arrive at intervals, and when the hand moves sharply, gaps open up between neighboring points. At the same time the line had to stay crisp on high-density screens, while an oversized canvas slows drawing down on weaker devices. One more small thing breaks the feeling: if the camera image is not mirrored, the hand on screen moves the opposite way. And hints drawn over the picture must not smudge the canvas itself. People should pick the line width themselves: a thin line suits a signature, a thick one a big drawing. Finally, the page had to fill the whole screen and adapt to the window size.
Camera access is a matter of trust. Few people are willing to send video of themselves to someone else's server, and everyone needs to see what is happening without instructions: whether the camera is on, whether the app has spotted the hand, and whether it is drawing right now. If the browser denies access, the person should know what to do next instead of staring at an empty screen. And to check how the app sees the hand, there had to be a mode that shows the tracking points themselves. The whole path from permission to the first line had to take seconds. We had ten days for the model, the gesture logic, the interface and the release.
Solution
We built the gesture drawing demo on MediaPipe Hands: the model finds the hand in every webcam frame and returns 21 hand landmarks, and the app turns a two-finger pinch into a pen.
The user opens the link, allows camera access and sees themselves mirrored, so movements feel natural. The mode label in the header switches from “Idle” to “Drawing”, the Stroke slider sets the line width from 1 to 20, the “Show guides” checkbox shows the fingertips and the pinch point, and the Clear button wipes the canvas. With the fingers apart the hand moves without a trace, and lines are drawn in near-white over the camera image. A tip at the start explains the gesture in one sentence and hides itself three seconds after the camera turns on.
It is a single HTML page in plain JavaScript: styles and script live right in the file, so the whole project is one file under 12 KB. The MediaPipe libraries load from the public jsDelivr server, the camera is requested at 1280×720, and the drawing lives on two Canvas layers, one for guides and one for lines. Pixel density is capped at 2x to keep rendering light. There is no server: frames are processed in the browser, and the page itself is hosted on Vercel. The recognition model loads on the first visit, and from then on everything runs on the device.
Hand tracking in the frame
MediaPipe Hands finds the hand in the webcam stream and returns 21 hand landmarks for every frame. The demo tracks one hand with the full version of the model and counts the hand as found once the model is at least 60% sure, so the hand is not lost during ordinary movements. If there is no hand in the frame, the app simply waits and draws nothing.
A pinch instead of a mouse button
The pen is the point halfway between the tips of the thumb and the index finger, the model's landmarks 4 and 8. Pinch and the line starts; release and the pen lifts. The gesture is easy to make and hard to repeat by accident. If the hand leaves the frame, drawing stops on its own, and the next line starts in a new place instead of stretching across the screen.
Protection from shaky fingers
Starting and ending a line use different thresholds: drawing turns on when the distance between the fingers drops below 5% of the frame size and turns off only when it exceeds 7%. This gap between the two thresholds keeps the line from breaking into dashes at the threshold, even if the hand trembles a little. Both thresholds are measured as a share of the frame rather than in pixels, so they work the same at any camera resolution.
A solid line with no gaps
If the hand moved far between frames, the app fills in up to eight intermediate points, so the stroke stays solid. Rounded caps and joins make the line look like a marker trail, and the canvas adapts to the screen's pixel density, so the line stays crisp on high-resolution displays. The width comes from a slider from 1 to 20, and there is one light color so the line reads over the video.
Clear status and hints
A dot in the header turns green when the camera is on, and the mode label shows “Idle” or “Drawing” and lights up while drawing. The “Show guides” checkbox displays the fingertips and the pinch point, which turns green when the pen is down. If camera access is denied, the page plainly asks to check permissions and reload, and all hints live in the header, the footer and a separate layer, never covering the camera image.
Processing right on the device
The model runs in the browser, so camera video is never sent anywhere or stored. The demo is a single page with no server of its own: the MediaPipe libraries load from the public jsDelivr server, and the page itself sits on Vercel cloud hosting. It opens on any computer with a webcam in a modern browser, with no accounts, no sign-up and no drawings stored on a server.
Results
The project's key outcomes in numbers.
- 21Hand landmarks per frame
MediaPipe Hands returns 21 hand landmarks, and the app takes the thumb and index fingertips for the pen.
- 0Frames sent to a server
Camera video is processed right in the browser and never sent anywhere; the demo has no server at all.
- 8Gap-filling points
When the hand moves sharply between frames, up to eight intermediate points are added, so the stroke stays solid.
- 10 daysFrom idea to demo
Connecting the model, the gesture logic, the interface and releasing the demo took ten days.
Project tech stack
The stack we used to build the solution for the client's task.
- Vercel
- HTML5
- CSS3
- JavaScript
- Canvas API
- MediaPipe
Ready to become the next case?
Tell us about your task — a manager will assemble the team, calculate the estimate and send a proposal within the first hour after the request. No obligations, no drip campaigns.


