What this dashboard is

This dashboard is the operator view into the ALT IoT pipeline: every packet a device sends over MQTT, how it gets decoded, and what the pipeline produces from it (events, log uploads, commands). It's read-mostly except for device onboarding, sending commands, and running manual test verification.

Pages & Navigation

Data Flow

How a byte on the device becomes a row you can click on this dashboard:

Every packet lands in raw_iot_packets whether it decodes or not (so DLQ debugging has the original bytes). A successful decode also writes a normalized_iot_events row with clean, typed fields — that's what most of this dashboard reads from.

Device Config & CAN signal map — why JSON, not a DBC

The Device Config card on Device Detail lets you download, as one JSON file, everything the backend uses to decode and configure that device right now. It is the same data the decoder worker reads at runtime, so the file always reflects what is actually live — not a copy that can drift.

Why we don't use a DBC file

A DBC is a standalone CAN-database format maintained outside the system. Our CAN signal map lives in the backend as a versioned JSON config instead, because the same config also drives things a DBC can't express: which CAN IDs the firmware must send in every bundle (completeness check), where each decoded signal lands in the normalized event (battery.soc etc.), and composite signals computed from several frames. Keeping one source of truth avoids maintaining a DBC and the JSON in parallel, and a DBC→JSON round trip would lose that extra information. So the JSON is the signal definition; there is no separate DBC export.

What is in the download

CAN config fields (canDecoderConfig.configJson)

What the firmware / IoT team needs to know

• Firmware only decides which CAN IDs to collect and how often; all bit offsets, scaling and formulas are backend-only and are never sent to the device or returned by any device-facing API.
• To change a signal (bit position, scale, new signal), send the change request with CAN ID, start bit, length, endian, signed, scale/offset and target field; the backend ships it as a new configVersion — the old one stays in history.
• If the download shows canDecoderConfig: null, no active config exists for that device type and CAN bundles for it will not decode.

Test Plan Verification — what & why

The IoT/firmware team verifies each build by hand against a physical device, using a fixed sheet of ~64 test cases. Most of those checks can be answered from data we already have — so instead of the tester manually watching Recent Packets and writing PASS/FAIL in a spreadsheet, this feature watches the same data and fills it in live.

Not everything can be auto-checked — cases that need comparing to an external ground truth (a BMS screen, phone GPS, CAN analyzer) or that need a raw tool (RSSI, FOTA) stay manual by design. Auto-detection only marks PASS when the signal is fully internal and objective — it will never claim something is correct that it can't actually verify.

How to use it

1. Open Test Plan, pick the device, optionally tag the firmware build, click Start Run.
2. Do your normal manual testing on the device — auto-detectable cases (marked ⚡) turn green on their own as matching data arrives, usually within 15 seconds.
3. Fill in the rest by hand — status, note, and (optionally) which packet/event proves it. For cases where it helps (SOC, speed, cell voltages/temps, GPS position, odometer, etc.) a blue Live now strip shows the device's most recent reading right on the case row, so you can compare against your reference (BMS, GPS, speedometer) without switching tabs.
4. Click Stop Run when done. Runs auto-expire after 24h if you forget.
5. Every past run stays in history per device — useful for comparing pass/fail across firmware builds.

⚡ Which cases auto-detect, and why only some

22 of the ~64 sheet cases carry the ⚡ icon. That's not a bug or a coincidence — a case only gets auto-detection when its pass signal is something this pipeline already stores as data, fully on its own, with no outside comparison needed. Most cases on the sheet aren't like that: they ask you to compare a decoded value against a phone's GPS, a BMS screen, a CAN analyzer, RSSI on a signal meter, or an FOTA tool — things the dashboard has no way to read. Those stay manual by design, forever, no matter how "visible" the underlying number is on the dashboard, because a false auto-PASS on something we can't actually verify would be worse than no automation at all. The full breakdown of exactly which cases those are, and what each one specifically needs, is in Why some cases stay manual below.

On the Test Plan run page, a case without ⚡ simply has no automatic path — you always set its status by hand. A case with ⚡ starts the run as ⚡ Watching (still PENDING, but the evaluator is actively checking it every 15 seconds) and flips itself to PASS with the AUTO tag the moment the real signal shows up in the data — you still don't have to do anything except perform the actual test action on the device. You can always override any case by hand at any time; a manual edit is never overwritten by the evaluator.

The 22 auto-detectable cases

T05–T08, T17, T29, T45 and T46 are new additions on top of the original 14 — added once we confirmed their pass signal is a plain internal state change (an event type arriving, or a flag flipping), not a threshold judgment call. Note what these 8 actually prove: that the event fired or the state changed — not that the physical trigger you applied was strong/precise enough (e.g. T05 proves a harsh-acceleration event arrived, not that your nudge was hard enough to be "harsh" by spec). Tune-the-threshold judgment calls like that stay something a human glances at.

Everything else on the sheet (raw IMU numbers, RSSI, FOTA, GPS-vs-phone comparisons, BMS comparisons, etc.) is manual-only — see the next section for the full list and exactly why each one can't be auto.

How motion events (T05 to T08) reach the dashboard

Harsh acceleration, harsh braking and the two turn cases do not have a packet type of their own. When the device detects one, it sends a full_snapshot_bundle packet (a snapshot of the whole vehicle state taken at the moment of the event) with an event block on top: a numeric event.type, severity, duration, peak acceleration and gyro on each axis, and speed before/at/after. So in Recent Packets you will see a full_snapshot_bundle row, not a row named "harsh_accel". Open it, then open Detail and look at event.type.

Why this was not working before

The first version of the evaluator looked for a packet whose type was literally harsh_accel, harsh_braking, sharp_turn or regular_turn. No device ever sends a packet with those names (nothing like it appeared in 30 days of data on any device), so these four cases stayed PENDING forever even when the device did fire the event. The evaluator now matches the numeric event.type inside full_snapshot_bundle instead.

Things to know when testing

- The event must arrive after you click Start Run; events from before the run do not count.
- Ordinary driving also produces these events (a normal turn gives type 5, a firm stop gives type 1), so a PASS means "the device reported this kind of event during the run", not "your specific nudge caused it". Check the timestamp and event.peakAccelX / event.peakGyroZ in Detail if it matters.
- Speed fields in the event block (speedBefore, speedAtTrigger, speedAfter) and headingChange currently look unpopulated by the firmware (speed never changes across the event, heading change is always 0), so do not use them to judge a test.

Why the other 42 cases stay manual

Every manual case falls into exactly one of three buckets, based on why it can't be auto-detected. This isn't "we haven't gotten to it yet" — these are structural: the dashboard genuinely doesn't have what it would need, for the reasons below. If firmware/decoder work ever changes that (e.g. a new event type starts carrying a comparable ground-truth value), the case can be revisited — but as things stand today, none of these 42 can safely auto-PASS.

1. Needs an outside reference to judge "correct" (12 cases)

The data is on the dashboard — but confirming it's right means comparing it to something outside the pipeline: a BMS screen, a phone's GPS, a known load, a known cell temperature. The dashboard only ever shows what the device itself reported. If the device's own number were wrong, the dashboard has no independent way to know — there's nothing to compare it against.

2. Bench/tool-only — never becomes pipeline data (15 cases)

These checks happen on a bench, a debugger, or a CAN analyzer, before or entirely outside of anything that gets encoded into an MQTT packet. There's no event for the evaluator to watch, because the signal never reaches the pipeline at all.

3. Partially visible — full check needs another source (15 cases)

Some of the picture is on the dashboard, but a full pass/fail call needs a second source too — a decoder debug view of raw fields, a backend log, or a serial console — that the dashboard doesn't surface.

The exact same text shown here is what you'll see on each case's own detail popup on the Test Plan page — click any case title to open it.

Test Plan Flow

Full rule table (which cases auto-detect and how) and the schema are documented in docs/13-test-plan-verification.md in the repo, for engineers extending this later.