U-Net
On the prototypeResNet-18 backbone
Segments the painted lane lines, so the device knows where the lane runs and where you are inside it.
84.23% IoUIoU, validation set
Technology
How RunSight turns a camera view into spoken guidance.
The models, lane geometry and cue priorities running on the prototype, plus the performance measured so far.
USB RGB camera
Hailo-8 accelerator
Three-model perception
U-Net
YOLOv8n
Sc-Depth V3
Lane and obstacle geometry
Cue engine
Spoken guidance
A three-model perception stack from one RGB camera.
A USB camera at eye level feeds the Hailo-8 accelerator. The on-device perception stack uses three compiled models: lane-line segmentation, person detection, and monocular depth. All three work from the same RGB input, so the prototype does not need a depth sensor, LiDAR, or a second camera.
Frames are handled latest-first. If inference falls behind capture, stale frames are dropped rather than queued: it is better to say nothing than to guide you from the past.
YOLOv8n
On the prototypeNano detector
Finds people on the track ahead so the guidance can respond when someone is in the runner's path.
88.24% IoUIoU, validation set
Sc-Depth V3
On the prototypeMonocular RGB
Estimates how far away a detected person is from a single RGB frame, so the guidance engine knows whether an obstacle is close enough to matter.
Self-supervisedmonocular depth, on-device
Converting model output into guidance.
A mask and a few boxes are not guidance. They have to become geometry: where the lane runs, where you are inside it, how sharply it bends, and whether the person ahead is close enough to matter.
01
Fit the lane lines
Contours are extracted from the segmentation mask and each lane line is fitted with a quadratic curve, so a line broken by a person or a shadow is interpolated back into one continuous edge.
02
Split the track into regions
The fitted lines divide the track into five regions: far left, left, centre, right, and far right. The runner is placed in one of them, and that is what a steering cue is computed from.
03
Measure the bend
A curvature radius is computed from the centre points of the fitted lines. A large radius means the lane is straight; a small one means it turns, and tells you which way.
04
Judge the person ahead
A detected person only becomes an obstacle if they are close. The depth model estimates that distance from the same RGB frame, so the engine reacts to who is near rather than to every person in view.
From the scene to one spoken cue.
A running scene produces far more information than a person can act on at pace. The engine emits at most one instruction and holds back the rest, with a separate cooldown per priority.
Independent cooldowns by priority
Every level has its own cooldown, so a warning is never blocked by a steering cue that spoke a moment ago.
Repeated cues are limited
The same phrase is held for 3 seconds, however true it still is.
Critical cues preempt other audio
A critical cue bypasses every cooldown and preempts the audio that is already playing.
Optimizing the models for real-time inference.
At float32 the models could not hold a real-time frame rate on a Raspberry Pi. So they are compiled for the Hailo accelerator: translated into its own representation, profiled, quantized against a calibration set of real track frames, and compiled into an executable the runtime loads at boot.
After compilation for the accelerator, the prototype is 6× faster and runs at more than 10 frames per second.
The data comes from Indonesian tracks.
No public dataset shows an Indonesian running track from a runner's eye level, so we built one. Images collected at public tracks, including Patriot Candrabhaga Stadium, at eye height, with and without people ahead.
Every image is annotated twice: bounding boxes for people, and polylines for the painted lane edges. Then it is augmented to survive reality: a head that tilts, a camera that shakes, and an afternoon that is either glaring or grey.
Prototype status and next tests.
The integrated prototype works, but these items still need engineering or validation.
One multi-task model
The accelerator has no DRAM, so switching between models each frame is costly. Folding the three into one model with several heads is the way out.
Wireless audio
The device speaks through a wired earphone today. Bluetooth audio needs a second radio next to the one already used for the phone.
Model updates over the air
Right now, a new model means physical access to the device.
Field validation
Current measurements come from bench tests and track sessions with sighted testers. Guided trials with visually impaired runners have not happened yet.