How Camera-Based Punch Tracking Works on Your iPhone

It sounds like it shouldn't work. You prop your iPhone on a table, start throwing punches, and the app counts them — jab, cross, hook — without a single sensor strapped to your body. No gloves, no wearables, no Bluetooth pairing dance.

Here's the plain-English version of how camera-based punch tracking actually works.

Step 1: The camera finds your body

Modern iPhones run pose estimation — the phone's camera and on-device machine learning locate key points on your body: wrists, elbows, shoulders, hips. Many times per second, the app gets a live "skeleton" of where your joints are in space.

Crucially, this runs on the device. Your video never has to leave your phone for the tracking to work.

Step 2: Motion becomes a punch

A skeleton alone isn't a punch. The app watches how those joints move over time:

• A wrist extending fast and straight from guard, then snapping back → a jab or cross.
• A wrist arcing horizontally around → a hook.
• A wrist driving upward → an uppercut.

By reading the path, speed, and which hand is moving, the app classifies each punch — not just "motion detected," but which punch you threw.

Step 3: Turning it into a score

Once punches are classified, the rest is game design: count output, measure speed, reward accuracy and clean combinations, and stack it all into rounds. That's how shadowboxing in your living room becomes a boxing game with a scoreboard.

Why this beats wearables

• Nothing to buy or charge — the camera you already own is the sensor.
• Nothing to strap on — set the phone down and box.
• Sees your whole body — a wrist sensor can't tell if your guard is up; a camera can.

The catch (and how to get clean tracking)

Camera tracking needs to see you, so a few basics matter:

• Good, even lighting — avoid strong backlight.
• The phone placed so your upper body and hands are in frame.
• A bit of space to extend your punches fully.

Get those right and the tracking feels almost invisible — you just box, and the numbers show up.

This is the engine under StrikeLab: on-device pose tracking that reads real punches and turns them into scored rounds, using nothing but the iPhone in front of you.