Guides / App development
How AI form scoring works in a boxing app
Boxerium, our boxing app, scores your form through the phone's camera. It uses Google ML Kit's body tracking to find points on your body, then measures the angles and distances between them to judge whether your base is solid, your torso is over your stance, or your chin is exposed. It also detects which punches you throw and checks them against the workout. All of that feeds a scoring engine for each round. Here is how each layer works.
- The phone's camera finds 33 points on the body using Google's ML Kit pose detection
- The app measures angles and distances between those points to judge base, posture and guard
- It also detects which punches are thrown, checks them against the workout, and scores each round
What does the camera actually see?
Not a boxer. It sees a set of dots.
Google's ML Kit includes a ready-made pose detection model. Point a camera at a person and, for each frame, it returns the positions of landmarks on the body. Google describes it as producing "a full-body 33 point skeletal match that includes facial landmarks (ears, eyes, mouth, and nose) and points on the hands and feet."
We did not train this model. That is the opposite of our basketball app, where we trained our own. A human body is something a general model already recognises well, so the work in Boxerium went into what to do with the dots.
How do dots become a judgement about form?
With geometry. Once you know where the shoulders, elbows, wrists, hips, knees and ankles are, you can measure things between them:
- Angles. The angle at the elbow, between the shoulder, elbow and wrist. The angle of the torso against vertical.
- Distances. How far apart the ankles are. How far a wrist is from the chin.
- Ratios. A raw distance in pixels means little, because it changes with your height and how far you stand from the phone. Comparing one distance with another on the same body, such as stance width against shoulder width, removes that problem.
Google's own documentation describes recognising poses with angle heuristics, calculated from the joint angles between landmarks. We use the same idea to judge quality, not just to name a pose.
What does Boxerium check for?
Three things a coach in your corner would watch, each expressed as a measurement:
| Check | The coaching point | What is measured |
|---|---|---|
| Base | Is your stance solid? | The position and spread of the feet, against your own proportions |
| Posture | Is your torso over your stance? | Where the shoulders and hips sit, relative to the feet |
| Guard | Is your chin exposed? | Where the hands are, relative to the head |
Each check runs many times a second for the whole round. A single bad frame means nothing: people blink, cameras blur. What counts is a pattern, such as a rear hand that drops every time you throw the jab.


How does it know which punch you threw?
By watching how the landmarks move over time, not where they are in one frame.
A jab and a cross both end with an arm extended. What separates them is which hand moved, from where, and what the shoulders and hips did on the way. A hook travels around; an uppercut travels up. The app follows the wrists, elbows and shoulders across a run of frames and matches the movement to a punch.
Then it compares what you threw with what you were asked to throw. Boxerium calls combos out loud during a round. If the call was jab, cross, hook and you threw jab, cross, the app knows. That is what lets it score a workout and not just count arm movements.
How does the scoring engine work?
It combines the layers into a result for each round:
Landmarks
ML Kit finds the body in each frame.
Measurements
Angles and distances are calculated from the points.
Form checks
Base, posture and guard are judged across the round.
Punches
Each punch is detected and checked against the workout.
Score
Everything feeds one score for the round.
The scoring had to be something a boxer would agree with. Throwing a lot of punches with your hands down should not beat throwing fewer with a tight guard, so output and technique are scored separately and both count.
The same engine makes online fights possible. Two players get the same random workout, both are scored, and the higher score wins. Nobody reports their own result. We explain why that matters in why competition keeps people using apps.
What are the limits of camera form scoring?
It is one camera looking from one side, and it is honest to say what that means.
- Framing. Your whole body has to be in shot. Feet cut off means no base check.
- Light. Dim rooms and strong backlight make the landmarks jump around.
- Angle. Something hidden behind your own body cannot be measured.
- Clothing and background. Very loose clothing and cluttered backgrounds reduce accuracy.
- Depth. A single camera judges distance towards the lens poorly.
- It is not a coach. It checks what can be measured. It cannot see intent, timing against an opponent, or pain.
Good design works with those limits. Boxerium asks you to prop the phone where it can see all of you, and it scores patterns over a round, not single moments.
Where else does this approach work?
Anywhere a movement has a correct shape: squats, golf swings, yoga, physiotherapy exercises, dance. The pattern is the same each time. Use a general pose model for the body, write the measurements that matter for your movement, and decide how to turn them into feedback a person can act on.
The hard part is rarely the AI. It is knowing the activity well enough to say what good looks like in numbers. Ethan boxes, which is why Boxerium checks the things it does.
Boxerium is built in Flutter and runs on iPhone and Android from one codebase. See Flutter vs native apps for when that is the right choice, or read more on the Boxerium page.