Guides  /  App development

How AI form scoring works in a boxing app

Boxerium, our boxing app, scores your form through the phone's camera. It uses Google ML Kit's body tracking to find points on your body, then measures the angles and distances between them to judge whether your base is solid, your torso is over your stance, or your chin is exposed. It also detects which punches you throw and checks them against the workout. All of that feeds a scoring engine for each round. Here is how each layer works.

Quick answer
  • The phone's camera finds 33 points on the body using Google's ML Kit pose detection
  • The app measures angles and distances between those points to judge base, posture and guard
  • It also detects which punches are thrown, checks them against the workout, and scores each round

What does the camera actually see?

Not a boxer. It sees a set of dots.

Google's ML Kit includes a ready-made pose detection model. Point a camera at a person and, for each frame, it returns the positions of landmarks on the body. Google describes it as producing "a full-body 33 point skeletal match that includes facial landmarks (ears, eyes, mouth, and nose) and points on the hands and feet."

33landmarks

Points on the body that ML Kit returns for each camera frame. Google ML Kit

3form checks

Base, posture and guard, each built from those points. Boxerium

We did not train this model. That is the opposite of our basketball app, where we trained our own. A human body is something a general model already recognises well, so the work in Boxerium went into what to do with the dots.

How do dots become a judgement about form?

With geometry. Once you know where the shoulders, elbows, wrists, hips, knees and ankles are, you can measure things between them:

  • Angles. The angle at the elbow, between the shoulder, elbow and wrist. The angle of the torso against vertical.
  • Distances. How far apart the ankles are. How far a wrist is from the chin.
  • Ratios. A raw distance in pixels means little, because it changes with your height and how far you stand from the phone. Comparing one distance with another on the same body, such as stance width against shoulder width, removes that problem.
Diagram of a boxer in a guard stance drawn as body landmarks joined by lines, with the base, posture and guard checks markedGUARDAre the hands up,or is the chin exposed?POSTUREIs the torso overthe stance?BASEAre the feet setin a solid stance?
A boxer as the app sees them: landmarks joined into a skeleton. Each form check is a measurement between a few of those points.

Google's own documentation describes recognising poses with angle heuristics, calculated from the joint angles between landmarks. We use the same idea to judge quality, not just to name a pose.

What does Boxerium check for?

Three things a coach in your corner would watch, each expressed as a measurement:

CheckThe coaching pointWhat is measured
BaseIs your stance solid?The position and spread of the feet, against your own proportions
PostureIs your torso over your stance?Where the shoulders and hips sit, relative to the feet
GuardIs your chin exposed?Where the hands are, relative to the head

Each check runs many times a second for the whole round. A single bad frame means nothing: people blink, cameras blur. What counts is a pattern, such as a rear hand that drops every time you throw the jab.

Boxerium Form Coach round in progress: round 3 of 3, 17 punches counted and 1:39 left on the timer
During the round: the punch count runs while the camera watches.
Boxerium round results scoring guard, stance, mechanics and output, with a list of flagged moments
After the round: scores for guard, stance, mechanics and output, with flagged moments.

How does it know which punch you threw?

By watching how the landmarks move over time, not where they are in one frame.

A jab and a cross both end with an arm extended. What separates them is which hand moved, from where, and what the shoulders and hips did on the way. A hook travels around; an uppercut travels up. The app follows the wrists, elbows and shoulders across a run of frames and matches the movement to a punch.

Then it compares what you threw with what you were asked to throw. Boxerium calls combos out loud during a round. If the call was jab, cross, hook and you threw jab, cross, the app knows. That is what lets it score a workout and not just count arm movements.

How does the scoring engine work?

It combines the layers into a result for each round:

01

Landmarks

ML Kit finds the body in each frame.

02

Measurements

Angles and distances are calculated from the points.

03

Form checks

Base, posture and guard are judged across the round.

04

Punches

Each punch is detected and checked against the workout.

05

Score

Everything feeds one score for the round.

The scoring had to be something a boxer would agree with. Throwing a lot of punches with your hands down should not beat throwing fewer with a tight guard, so output and technique are scored separately and both count.

The same engine makes online fights possible. Two players get the same random workout, both are scored, and the higher score wins. Nobody reports their own result. We explain why that matters in why competition keeps people using apps.

What are the limits of camera form scoring?

It is one camera looking from one side, and it is honest to say what that means.

  • Framing. Your whole body has to be in shot. Feet cut off means no base check.
  • Light. Dim rooms and strong backlight make the landmarks jump around.
  • Angle. Something hidden behind your own body cannot be measured.
  • Clothing and background. Very loose clothing and cluttered backgrounds reduce accuracy.
  • Depth. A single camera judges distance towards the lens poorly.
  • It is not a coach. It checks what can be measured. It cannot see intent, timing against an opponent, or pain.

Good design works with those limits. Boxerium asks you to prop the phone where it can see all of you, and it scores patterns over a round, not single moments.

Where else does this approach work?

Anywhere a movement has a correct shape: squats, golf swings, yoga, physiotherapy exercises, dance. The pattern is the same each time. Use a general pose model for the body, write the measurements that matter for your movement, and decide how to turn them into feedback a person can act on.

The hard part is rarely the AI. It is knowing the activity well enough to say what good looks like in numbers. Ethan boxes, which is why Boxerium checks the things it does.

Boxerium is built in Flutter and runs on iPhone and Android from one codebase. See Flutter vs native apps for when that is the right choice, or read more on the Boxerium page.

Want an app that watches how someone moves?

Form checking works for far more than boxing. Email us the movement you want to measure and we will tell you what a phone camera can and cannot judge.