Celestine

Low-cost body tracking
using AprilTags

A motion-capture system for VR and gaming that fuses cheap IMUs with AprilTag fiducial markers — drift-free tracking at a fraction of the cost of commercial systems.

How it works

IMUs give fast, low-latency motion updates but drift over time. AprilTags give high-precision absolute pose, but only when a marker is in view. Fusing the two — IMU as the primary stream, AprilTag detections as periodic correction — keeps the tracker responsive and drift-free.

Overview

This project combines inexpensive Inertial Measurement Units (IMUs) with AprilTag fiducial detection to build a robust motion-tracking system aimed at gaming and virtual reality. The goal was to deliver near-perfect tracking accuracy without the cost or complexity of traditional motion-capture rigs.

Challenges

The main challenge was reconciling two data streams running at very different rates. AprilTag detection in OpenCV is significantly slower than the IMU loop, so the IMU drives continuous motion while AprilTag data is folded in as a correction signal — fast enough to feel responsive, accurate enough to stay drift-free.

Hardware

IMU AprilTag fusion system diagram Custom sensor board

The custom sensor board is open hardware. Schematics, BOM, and PCB files are published on OSHW-Lab.

Source

Firmware and host-side fusion code live on GitHub.