This is the engineering companion to our partnership announcement. Here we'll walk through exactly how the INSAION Agent uses ReductStore to deliver edge recording for robotic fleets: continuous, high-fidelity local capture with on-demand, surgical cloud sync. Recording works out of the box and can be enabled with a single click in the device UI. When something needs closer inspection, operators sync only the relevant window to the cloud for analysis, debugging, and visualization.
If you run robots with cameras, LiDAR, or high-frequency control loops, you already know the problem: the data that matters most is too heavy to stream, and too valuable to throw away. Our answer is to put a purpose-built time series storage engine on the robot itself, and that engine is ReductStore.
Architecture at a Glance
The key property of this design: the heavy data path never touches your network until you explicitly ask for a specific time window.
Why ReductStore
When we designed edge recording, we wrote down the requirements first:
| Requirement | Why it matters on a robot | How ReductStore answers it |
|---|---|---|
| Time-series blob storage | Sensor data is timestamped binary payloads, not rows | Records keyed by nanosecond timestamps, arbitrary binary content |
| Configurable retention | Edge disks fill up; runaway storage kills robots | Native FIFO, HARD, and NONE policies for eviction, stopping, or no quota |
| Fast selective retrieval | Incidents need a 90-second window, not a full scan | Time-range queries with label-based filtering |
| High ingest throughput | Multi-camera + LiDAR rigs produce hundreds of MB/s | Written in Rust, designed for high-throughput blob ingestion at the edge |
| Simple operations | A robot is not a data center | Single binary, no external dependencies, small footprint |
ReductStore hit every requirement without us having to build retention logic, indexing, or querying ourselves. That let us focus on the robotics layer: pipelines, MCAP, and fleet orchestration.
Embedded, Not Bolted On
ReductStore doesn't run as a separate container you have to manage. The INSAION Agent ships the ReductStore binary inside its Debian package and runs it as a supervised child process:
- Lifecycle management. The agent's process manager starts ReductStore after authorization preflight, tracks its PID, and reports its state in the agent's diagnostics.
- Automatic recovery. If the store crashes, it's restarted with an exponential-backoff policy (up to 5 restarts, cooldown growing from 5s to 60s, reset after 5 minutes of stability).
- Local-only by default. It binds to
127.0.0.1:8383and authenticates with the device's own token. Your raw sensor data never leaves the machine unless you say so. - Predictable storage. All data lives under
/var/lib/insaion-agent/reductstore, so disk usage is transparent and easy to audit.
On the client side, the agent talks to the store through the official ReductStore C++ SDK (reduct-cpp), so writes and queries are type-safe and versioned with the rest of our stack.
The Data Model: MCAP Slices as Time Series Records
Robots don't produce one stream, they produce many. INSAION organizes recording into pipelines: named, configurable groups of topics. You configure them from the Rolling Buffer editor in the device UI rather than hand-writing a configuration file. Add a pipeline, give it a meaningful name such as perception or telemetry, and set the maximum duration of each MCAP slice. The editor also lets you choose included topics from the device's suggestions or type topic patterns such as /camera/* or *, with an optional excluded topics list for exceptions. Pipelines can be renamed or removed as the robot's mission changes.
Each pipeline continuously writes its messages into an in-memory MCAP file, the standard container format for ROS 2 data, complete with schemas, channels, and nanosecond timestamps, compressed with Zstandard and protected with CRC checksums. When a slice closes (every split.max_duration_s seconds, or when it hits split.max_size_bytes), it's handed to ReductStore:
- Bucket: a single
rolling-bufferbucket per device. - Entry: the pipeline name (
perception,telemetry, ...). - Record key: the slice's start timestamp, in nanoseconds since epoch.
- Labels:
recorder_uuid(which recording session produced it),duration_ns, plus any custom static labels you configure per pipeline.
That last point is what makes retrieval surgical. When the backend asks for "the perception data from recording X between T1 and T2", the agent issues a ReductStore query with a time range and a label condition, and gets back exactly the matching slices, nothing more.
No full-bucket scans. No re-processing. The store's index does the work.
The Rolling Buffer: Policies in Practice
The "rolling" in rolling buffer is pure ReductStore. In the device UI, operators first enable or disable recording, then choose the storage policy and, when applicable, the quota size in GB. There are three policies:
- FIFO keeps recording within the configured quota by automatically evicting the oldest records first. A 5 GB quota therefore behaves like a flight recorder that always keeps the newest data that fits.
- HARD enforces the quota without evicting existing records. When the quota is reached, recording stops until space is available or the configuration changes.
- NONE disables the quota limit. The size field is not applicable, so the operator gets unbounded retention with the corresponding responsibility to monitor disk usage.
New slices are appended as time-series records, and the selected policy is handled by ReductStore itself. There is no TTL bookkeeping, no cron job, and no deletion logic in our code. Quotas are configurable per device, and the agent reconciles the bucket settings on startup, so a change in the INSAION UI propagates to the edge store automatically.
With FIFO, retention is space-based rather than time-based: a quiet robot keeps days of history, while a sensor-heavy robot keeps hours, always within the same disk budget.
Surgical Cloud Sync: From Edge Store to S3
Recording locally is only half the story. When an alarm fires or an operator drags across the timeline and hits Upload, this happens:
1. Command. The backend sends an upload job over MQTT: recording ID, time range, target pipelines.
2. Query. The agent queries the local ReductStore for the matching slices (time range + recorder_uuid label) and estimates the total size.
3. Presign. For each slice, the agent requests a presigned S3 PUT URL from the INSAION backend.
4. Direct upload. The slice is streamed straight from ReductStore to S3. It never transits through our API servers, so there's no bandwidth bottleneck.
5. Complete. When all chunks for a pipeline land, the agent signals completion; the backend stitches the chunks into a final MCAP per pipeline and marks the recording as fully in the cloud.
Reliability is built in at every step:
- Retries with backoff: up to 5 attempts per chunk, exponential backoff, retryable on 408/429/5xx and network errors.
- Pause / resume / cancel: operators control in-flight jobs; state is persisted on disk (
upload_state.json), so an agent restart doesn't lose the queue. - Offline tolerance: if the robot loses connectivity, recording continues uninterrupted into the local store; uploads simply resume when the link returns.
What This Buys You in Production
- Full fidelity where it matters. Record raw sensor data at native rates locally. No downsampling compromises forced by uplink costs.
- One-click capture, on-demand insight. Start recording locally with a single action, then sync selected time windows to the cloud when you need them for analysis, debugging, or visualization.
- Cloud bills that scale with incidents, not with fleet size. You pay to store the windows you actually need, not your entire fleet's data.
- A clean separation of concerns. ReductStore owns storage, retention, and retrieval. The INSAION Agent owns robotics semantics (topics, pipelines, MCAP). The cloud owns visualization and analysis.
- Fleet-wide consistency. The same storage engine, config model, and upload pipeline runs on every device, from a single prototype to hundreds of production units.
Try It
Edge recording with the rolling buffer is available in INSAION today. Spin up the live demo, or read the device recordings documentation to configure your first pipeline. To learn more about the storage engine underneath it all, visit ReductStore.