Solving glove data onto a hand skeleton

The Smartgloves stream per-sensor orientations, not joint angles or a finished pose. To drive a rigged hand you have to solve those orientations onto a skeleton every frame. This guide recaps the steps that solve involves, independent of any particular language or engine, so you can implement it on your own stack. The RGMP Viewer sample is a complete, working reference for everything described here.

The SDK is deliberately low-level: it hands you the raw sensor data and stays out of your way, so the solver is yours to build and tune for your own rig, accuracy, and performance needs. Our sample is here to show one approach and get you started; treat it as a learning reference rather than a drop-in, production-grade implementation, and don’t be shy about doing things differently.

If you would rather not implement any of it, the SDK also ships a ready-made solver you can run as a separate process — see Hand solver below.

Hand solver

rkk-hand-solver is a sidecar binary, included in the SDK package, that performs the solve described on this page for you. It consumes the driver’s RGMP stream, solves every connected glove into a 26-joint hand, and re-emits the result as RGMP on a port of its own:

flowchart LR
    Driver["SDK Driver<br/>rokoko-sdk"]
    Solver["rkk-hand-solver<br/>(solve)"]
    App["Your application"]

    Driver -->|"TCP :12276 (RGMP)"| Solver
    Solver -->|"TCP :12277 (RGMP)"| App

Re-emitting RGMP is the whole point: a solved hand is just another RGMP device, so you read it with the client you already have. Point it at the solver’s port instead of the driver’s and nothing else changes:

python3 rgmp_stream.py --host 127.0.0.1 --port 12277

Running it

By default the solver starts the driver itself, so this is the whole setup:

rkk-hand-solver

If something else already manages the driver, tell the solver to use the running one instead:

rkk-hand-solver --no-driver

Flag	Default	Purpose
--listen	127.0.0.1:12277	Address to serve solved hands on. Use :0 to let the OS pick a free port; the startup banner reports what was bound.
--rgmp-addr	127.0.0.1:12276	The driver’s RGMP endpoint to consume.
--emit	local	What each hand carries: local, world, or both. See below.
--no-driver	—	Assume the driver is already running.
--driver-path	rokoko-sdk	Driver executable to launch.
--driver-arg	—	Extra argument passed to the driver, after the defaults. Repeatable.
--driver-quiet	—	Discard the driver’s output instead of sharing the solver’s.
--driver-timeout	15	Seconds to wait for the driver to open its RGMP port.

What the solver emits

One synthetic device per solved hand, with device_type of solved_hand. Its device_id is the source glove’s, and timestamp_epoch and every timestamp_us are passed through unchanged, so solved hands stay correlated with the raw gloves they came from and remain monotonic.

The skeleton is 26 joints: forearm → hand → five fingers, with the thumb carrying metacarpal, proximal, and distal joints and the other fingers adding a medial one. Because a solve only ever changes joint rotations, the rest pose — bone lengths and rest orientations, fixed for the session — is sent once in static_data as one TRANSFORM per joint, and the per-frame groups carry only what actually moves.

--emit chooses that per-frame shape:

--emit	Group	Per frame, per hand	Pick it when
local (default)	joints_local	444 bytes	You are retargeting onto your own skeleton. Local rotations against a known rest pose are what that takes.
world	joints_world	744 bytes	You want to draw the hand or attach props directly, with no forward kinematics on your side.
both	both of the above	1188 bytes	You need each of the above for different consumers.

The output is solved hands only

Any non-Smartgloves device on the input stream is not republished — the solver’s port carries solved hands and nothing else. To read other devices at the same time, connect to the driver’s own :12276 in parallel; it accepts multiple clients.

A frame that cannot be decoded, or whose solve produces a non-finite joint, is dropped rather than emitted, and each cause is logged once per device rather than once per frame. The solve is stateless, so the next good frame recovers. If the upstream stream drops, every solved device gets a Disconnect frame and the solver reconnects with backoff; a consumer that connects mid-session receives every current definition before any data.

What it does not do for you

Values arrive in the glove’s own right-handed sensor frame (+X right, +Y toward the wrist, +Z down). There is no conversion to Y-up, no handedness flip, and no yaw offset — so both frame cautions below still apply to the solved output, and rebasing onto your engine’s frame remains a single rotation you apply to the whole skeleton. This is deliberate: yaw is referenced to magnetic north with an arbitrary per-session zero, so no fixed rotation could cancel it and choosing one here would be guessing on your behalf.

Early release

The sidecar is at version 0.1.0. The solve itself is verified against the reference implementation, but a few behaviors are worth knowing before you build on it:

• Calibration is not applied. The solve runs with the model’s default finger spread and no per-sensor calibration beyond the hub’s fixed mounting offset, so resting spread and knuckle alignment may not match a Studio-calibrated pipeline.
• Sensor validity is not acted on. The data-valid flag is read but not used; a blocked or invalid sensor still contributes its last orientation to the pose.
• Coil Pro world anchoring is the least exercised path. World-space re-anchoring is implemented but has had the least validation against real hardware, so verify it against your own setup before relying on it.

If you need a solve you can tune for these yourself, implement the steps below against the raw stream instead.

What you receive

Each glove exposes 8 sensors over the RGMP stream:

• one per finger (5),
• a hub on the back of the hand,
• a forearm sensor, and
• a world-anchored forearm pose, available only when a Coil Pro is tracking.

Every sensor streams an orientation quaternion; some also carry a position and status flags (data-valid, Coil-present). The orientations live in a right-handed, gravity-referenced frame: one axis points along gravity, and the hub references its heading to magnetic north. The finger and forearm sensors are expressed relative to the hub, so you compose them with the hub orientation to get a hand-space pose. See the Smartgloves reference for the exact frames and sensor list.

What you need first

• A rigged hand skeleton with a known bone hierarchy and a neutral (rest) pose.
• A sensor-to-bone mapping: which bone each sensor drives, and that sensor’s mounting offset: where it physically sits relative to the bone.
• Quaternion math (compose, rotate a vector, build a rotation that aims one axis at a target) and forward kinematics to turn local rotations into world transforms.

The solve, step by step

Run this each frame, from the latest sample:

1. Reset to the rest pose. Start from a known neutral pose (fingers slightly fanned, joints at identity) so the result is deterministic and doesn’t drift.
2. Anchor the hand. Orient the hand from the hub sensor and offset the hand root by the hub’s mounting position on the back of the hand.
3. Apply calibration. Compose each sensor’s orientation with its per-sensor calibration offset (identity if you aren’t calibrating).
4. Aim each finger. For each finger, aim the driven bone’s parent at a target derived from the sensor orientation and that sensor’s mounting offset, then set the driven bone from the sensor orientation itself.
5. Constrain to anatomy. Fingers (except the thumb) bend on a hinge: clamp backward hyper-extension to a small limit, and distribute the remaining curl onto the fingertip joint so the chain reads naturally.
6. Orient the forearm from the forearm sensor (composed through the hub).
7. World-anchor (optional). When a Coil Pro is tracking, re-anchor the whole arm in world space from the world-anchored forearm pose: a single rotation + translation alignment.
8. Propagate transforms. Run forward kinematics so every local rotation becomes a world transform your renderer can consume.

Two things that trip people up

Left vs. right is a mirror, not a rotation

A left hand is the mirror image of a right hand. Mirror the mounting offsets and rest-pose angles across the hand’s midline to switch sides. This is anatomy and is independent of your coordinate system’s handedness so don’t try to express it as a rotation.

Convert frames with a rotation, never a mirror

To move from the glove’s frame into your engine’s frame, apply a single proper rotation at the boundary. Mirroring an axis flips handedness and will render a right hand as a left one. Keep the solve in the glove’s frame and rotate once at the end.

Heading is relative

Yaw is referenced to magnetic north at startup, so the hands’ heading in your scene is arbitrary until the user orients themselves. Offer a recenter action that captures the current yaw and points the hands where you want them.

Where to go next

• RGMP Viewer — a complete reference implementation of this solve, with a live 3D hand. Start here, then read its source for the concrete math.
• Web apps overview — how the browser samples connect to the driver through the bridge.
• Streaming data and the RGMP specification — the stream these sensors arrive on.