# Telemetry You Can Trust: Lessons from Production Firmware

> Nine habits from production firmware, and the Serial Studio machinery that meets them: checksums, transforms, data tables, timestamps, and the Control Script.
>
> Telemetry · July 8, 2026 · by Alex Spataru · https://serial-studio.com/blog/telemetry-lessons-from-production-firmware

A dashboard will happily plot anything it receives. It cannot know that a sensor chip silently reset two seconds ago, that channel 5 has been repeating its last good reading since a connector worked loose, or that the number it just drew came from a converter that was mid-fault when the register was read. Whether the picture on screen means anything is decided earlier, in firmware, by code that runs next to the sensor.

Production data-acquisition systems, the kind that people trust with expensive test articles, converge on a remarkably consistent set of habits for exactly this reason. None of them require an RTOS, a heavyweight protocol stack, or more than a few hundred bytes of code. But the firmware is only half of the contract. Every one of these habits asks something of the receiving side too: honor the sentinel, apply the calibration, notice the silence, react to the reboot. This post collects nine of the habits, and for each one, the machinery Serial Studio has for holding up the host's end.

## 1. Decide validity next to the sensor

The driver that talks to the converter chip is the only code with enough context to judge whether a reading is real. By the time a value reaches a bus, a log file, or a dashboard, that context is gone. So production firmware makes validity the driver's job, with several independent checks:

- **Hardware fault flags.** Most sensor front-ends (RTD converters, thermocouple interfaces, delta-sigma ADCs) expose a fault-status register. Read it with the data, not instead of it.
- **Rail rejection.** A raw code of all zeros or full scale is usually not a measurement. With a pull-down on the data-input pin, a dead or absent chip reads as zeros, and zeros get rejected instead of becoming a plausible-looking value.
- **A plausibility band.** If the sensor is physically incapable of reading 900 °C in your setup, a 900 °C reading is a wiring or electronics problem, not data. Configure the band per channel and reject outside it.
- **A warm-up window.** This one catches a genuinely nasty failure mode. Some converters halt conversion updates while a fault condition is active. When the fault clears, the data register still holds the last pre-fault result: in range, plausible, and wrong. The fix: after any fault, reset, or configuration change, mark the channel invalid until a fresh conversion is guaranteed. Firmware that does this never reports that ghost.

One more check belongs in this list: **read your configuration back**. A brown-out or an EMI transient can reset a sensor chip to power-on defaults, and a chip on default settings often keeps producing numbers, just subtly wrong ones. Firmware can catch this by re-reading the configuration registers every cycle and comparing them against what it wrote. On a mismatch, it rewrites the configuration, rejects that cycle's data, and heals within one read period. The alternative is a dataset that drifted for an hour and nobody can say when.

The unifying principle: stale or implausible data never leaves the device, because nothing downstream can un-trust it.

But the host has a matching obligation, because validity is contagious in reverse: once one channel is known bad, everything derived from it is bad too. In Serial Studio, the clean way to express that is a quality flag in a [data table](https://serial-studio.com/help/data-tables). One dataset's transform range-checks the reading and publishes a flag to a shared register; every downstream transform branches on it:

```lua
-- Range check on the temperature probe
function transform(value)
  tableSet("runtime", "probe_ok", (value < -50 or value > 150) and 0 or 1)
  return value
end
```

```lua
-- Any derived channel that depends on the probe
function transform(value)
  if (tableGet("runtime", "probe_ok") or 0) == 0 then
    return 0  -- or a sentinel your widgets recognize
  end
  return value
end
```

One out-of-range probe now taints the heat-flux estimate, the efficiency figure, and every other channel computed from it, automatically, in one place. The driver decided the reading was suspect; the table makes sure nothing downstream forgets it. One ordering rule applies: transforms run in tree order, so the probe's dataset must sit above its dependents in the Project Editor tree. A dependent placed higher runs first and reads the *previous* frame's flag, one frame stale.

## 2. Reserve sentinel values instead of going quiet

When a channel is invalid, the tempting options are to send zero or to send nothing. Both are wrong. Zero is a legal reading for almost every physical quantity, so it silently poisons statistics and plots. Sending nothing makes "sensor broken" indistinguishable from "wire broken" and leaves the receiver guessing.

The pattern that scales is in-band sentinels: reserve a few codes at the top of the numeric range and give them fixed meanings. This is exactly what the J1939 standard does on heavy-vehicle CAN networks: a byte of `0xFF` means *not available* and `0xFE` means *error indicator*. That convention is why a J1939 tool can tell you precisely which signals a truck is not providing. A 16-bit channel loses three or four codes out of 65,536 and gains an explicit vocabulary:

- **invalid**: the driver rejected the reading (fault, implausible, warming up),
- **above range** and **below range**: the sensor is alive and working, but the value is beyond the configured measurement limit.

That last distinction matters more than it looks. An over-range channel and a faulted channel demand different responses from whoever is watching the screen: one means "the process exceeded the limit", the other means "stop trusting this channel". The 4-20 mA current-loop world learned this decades ago and codified it as NAMUR NE43: a healthy loop never drops below 3.8 mA, so anything under 3.6 mA is unambiguously a broken wire or dead transmitter, not a very small reading. A live zero is the whole point of the 4 mA offset.

Sentinels are easy to consume precisely because they are explicit. A [dataset transform](https://serial-studio.com/help/dataset-transforms) can catch them before they touch a plot, holding the last good value and publishing the channel's health instead of drawing a 65,535-count spike:

```lua
local last = 0
function transform(value)
  if value >= 0xFFFD then         -- invalid / above range / below range
    tableSet("runtime", "ch_ok", 0)
    return last                   -- hold last good value, flag the outage
  end
  tableSet("runtime", "ch_ok", 1)
  last = value
  return last
end
```

The `local last` upvalue persists between frames, so the plot holds steady through the outage while `ch_ok` records it. Now pair that flag with a **virtual dataset**: a dataset with no frame index, whose value is computed entirely by its transform. The channel's health becomes a channel of its own, plotted, recorded, and visible to the API, without the device transmitting a single extra byte.

## 3. Send integers, convert at the receiver

Production telemetry links overwhelmingly carry scaled integers, not floats. Pick a resolution and an offset so the physical range fits the integer type: tenths of a degree with a +1000 offset put −100.0 °C to +6453.5 °C in a `uint16`. Hundredths of an ohm, microamps, millivolts, all the same trick.

The reasons stack up. A `uint16` is half the wire cost of a `float32` and a quarter of a `float64`. Integers have no NaN, no infinity, and no ambiguity about representation, so the protocol document is one line: "big-endian uint16, 0.1 °C per count, offset −100 °C". Formatting cost on the microcontroller is a shift and a store instead of a software float-to-string routine, which is the same arithmetic that makes [binary protocols worth it](https://serial-studio.com/blog/binary-serial-protocols-cobs-messagepack) at high sample rates.

There is a subtler design choice hiding here: send the quantity you actually measured, not the quantity you eventually want. A resistance-thermometer board that transmits resistance, leaving the resistance-to-temperature conversion to the receiver, can have its calibration curve fixed, improved, or swapped without reflashing hardware that might be bolted inside a test rig. The device stays simple and dumb; the science lives on the host, where it is editable.

This is the arrangement Serial Studio's calibration registers exist for. Store the slope and offset once, as Constant registers in a `calibration` table, and let every affected dataset read them (the transform editor ships a **Calibration from Data Table** template that is exactly this):

```lua
function transform(value)
  local slope  = tableGet("calibration", "slope")  or 1.0
  local offset = tableGet("calibration", "offset") or 0.0
  return slope * value + offset
end
```

When the lab recalibrates the sensor, you edit two defaults in the Project Editor and every channel picks them up on the next load. No transform code changes, no firmware release, and the calibration lives in exactly one place instead of six copies of a magic number.

Derived quantities follow the same logic. The bus should not carry power when it already carries voltage and current; a virtual dataset recreates it host-side with `datasetGetFinal()`:

```lua
function transform(value)
  local v = datasetGetFinal(10)
  local i = datasetGetFinal(11)
  if v == nil or i == nil then return 0 end
  return v * i
end
```

Transforms run in project-tree order, so the one rule to respect is that voltage and current must sit above the derived power channel in the Project Editor. The [same pattern](https://serial-studio.com/blog/sensor-data-transforms) covers IMU acceleration magnitude, RMS of three phase currents, and differential pressure from two absolute channels: things you want on screen and in the export, but that would be wasted bytes on the wire.

## 4. Assume the wire lies

Everything so far has been about the number being true when it leaves the device. The trip is a separate hazard, and it is easy to forget because the failure looks like success: a corrupted frame does not announce itself, it just arrives as a slightly different number and gets plotted with a straight face. How much protection you already have depends entirely on the link:

- **[CAN](https://serial-studio.com/blog/what-is-can-bus)** validates every frame in hardware with a 15-bit CRC, and receivers refuse to acknowledge a damaged one. On a discrete CAN bus, a payload checksum is redundant for general telemetry, and disciplined CAN protocols skip it deliberately rather than pay eight bits of every frame for protection they already have. The assumption does have a ceiling: safety-rated CAN stacks (CANopen Safety, the J1939 functional-safety layers) add an application-level CRC anyway, because the hardware CRC's guarantees weaken in known corner cases. That is acceptable for telemetry and not acceptable when the data protects people or certified equipment.
- **A UART** guarantees nothing. No checksum, no delivery confirmation, at best a parity bit almost nobody enables. A byte lost to a receive overrun simply vanishes, and every byte after it shifts into its place, which is the classic recipe for a frame that parses cleanly and means something else. This is half the argument for [self-resynchronizing framing like COBS](https://serial-studio.com/blog/binary-serial-protocols-cobs-messagepack); a checksum is the other half.
- **UDP** carries a 16-bit checksum but makes no delivery promise at all: datagrams drop, duplicate, and arrive out of order. Rarely, on a quiet bench. Regularly, the day the network gets busy.

So the habit is: know what your link already guarantees, and add exactly what it does not. On serial and network paths, that means the device appends a CRC and the receiver verifies it before believing a single byte.

The receiver's half of this is one dropdown in Serial Studio. The frame extraction pipeline has a per-source checksum stage with nine algorithms (XOR-8, MOD-256, CRC-8, CRC-16 in three flavors, Fletcher-16, CRC-32, Adler-32), and a frame that fails the check is dropped before the parser ever sees it. If your protocol uses something more exotic, a [custom parser](https://serial-studio.com/help/javascript-api) can verify it and return an empty frame to reject, which is exactly what the shipped XOR-validation example does.

**A checksum catches corruption; it cannot catch absence.** A dropped frame leaves no evidence, because the frames around it are individually perfect. The cure costs one byte: a rolling counter the device increments on every frame ([MAVLink](https://mavlink.io/en/) dedicates a header byte to precisely this). The host then knows not just that data is flowing but that *all* of it is flowing, and a transform on the counter dataset turns gaps into a number you can put on a widget:

```lua
local expected
function transform(value)
  if expected ~= nil and value ~= expected then
    local delta = (value - expected) % 256
    if delta < 128 then  -- a near-wrap delta is a late or duplicate frame, not ~200 drops
      tableSet("runtime", "dropped", (tableGet("runtime", "dropped") or 0) + delta)
    end
  end
  expected = (value + 1) % 256
  return value
end
```

One honesty note before you copy that onto any source. The counter math is exact on serial links, which deliver frames in order or not at all. UDP is different: frames arrive late, duplicated, or out of order, and no simple counter arithmetic survives that perfectly. The `delta < 128` guard handles the worst of it, because a late or duplicate frame produces a delta just under the wrap point and gets ignored instead of being booked as two hundred phantom drops. A genuine reordering can still miscount by a frame or two.

So on UDP, treat the result as a health indicator, not an audit. Exact accounting on an unordered transport needs a reorder window, which is a fine exercise and a poor blog snippet.

Pair the `dropped` register with a virtual dataset and the link's loss count becomes a channel on the dashboard, recorded alongside the data it qualifies. A counter tells you about partial loss; total silence is the next habit's job.

## 5. Make silence meaningful

Bandwidth spent on frames that say nothing is bandwidth stolen from frames that matter, and on a shared bus it is stolen from every other device too. Disciplined firmware transmits by exception where it can: a data frame whose channels are all invalid is suppressed entirely, and a diagnostics frame (say, one fault byte per channel) is sent only while at least one channel is actually faulted.

This inverts the usual meaning of silence: the *absence* of the diagnostics frame is the "all healthy" signal. It also makes raw bus captures dramatically more readable, because everything on the wire is there for a reason. A slow periodic frame carrying static identity or configuration information (transmitted every few seconds, not every cycle) completes the scheme: it proves the device is alive, doubles as self-documentation on the bus, and lets a tool that connects mid-run discover what it is looking at without a round trip.

Silence-by-exception still leaves one job that only the host can do, because only the host has a clock that keeps running when the frames stop: telling *quiet* apart from *dead*. Serial Studio's [Control Script](https://serial-studio.com/help/control-script) gives every project an Arduino-style `setup()` and `loop()`, and `io.getLatestFrame()` reports the age of the newest frame directly, so a communication-loss watchdog is a few honest lines:

```js
function loop() {
  var r = io.getLatestFrame();
  if (r.ok && r.result.ageMs > 2000) {
    tableSet("runtime", "link_ok", 0);
    refreshDashboard();
    notifyCritical("Watchdog", "No frames for " + r.result.ageMs + " ms");
  }
  delay(250);
}
```

The detail that makes this work is `refreshDashboard()`. Dataset transforms normally re-run only when a frame arrives, which is precisely what does not happen during an outage: the screen would keep showing the last rendered values, frozen and green, while the link lies dead. `refreshDashboard()` re-runs every transform from the last received values and repaints, so the datasets your script just marked invalid actually change on screen. And it is view-only by design: nothing is appended to CSV or the session recordings, because a rendering refresh is not new data.

There is a pleasing symmetry in where all this runs. The device refreshes its hardware watchdog in exactly one place, the top of its main task loop, and anything that hangs gets reset. The Control Script `loop()` runs on its own watchdog-protected worker thread: a script that blocks or runs away is stopped and reported instead of freezing the dashboard. Both ends of the link assume their own code can hang, and both make that assumption survivable.

## 6. Filter before you decimate

Most acquisition firmware samples faster than it transmits: an ADC free-running at hundreds of samples per second feeding a telemetry rate of ten or twenty values per second. The naive approach, taking the most recent sample when it is time to transmit, throws information away and keeps the noise.

The right move is to average every sample across the full reporting window, on the device. A plain boxcar average is enough, and it has a lovely property: its frequency response has nulls at every multiple of one over the window length. Average over exactly 100 ms and you get nulls at 10 Hz, 20 Hz, 30 Hz... which includes both 50 Hz and 60 Hz, the two mains frequencies that couple into every long sensor cable on Earth. This is not an obscure trick; it is why bench multimeters specify integration time in "power line cycles".

This is the one lesson in the list that the host genuinely cannot rescue, which is worth being honest about. Any periodic interference at a multiple of your output rate does not just survive decimation, it *aliases to DC*: it becomes a constant offset in the transmitted values, indistinguishable from signal forever after. Serial Studio's smoothing templates (EMA, moving average, median, even a one-dimensional Kalman filter) are the right tool for taming visual jitter on a plot, but they operate on what already came through the wire. Ten lines of accumulator code in the driver beat any amount of post-processing cleverness after the damage is done. Filter on the device; polish at the receiver.

## 7. Answer "when did this happen" at the source

The previous habit was quietly a timing argument. Those boxcar nulls land on 50 and 60 Hz only if the sampling clock is honest, which is why acquisition boards run their converters from a crystal (tens of parts per million) rather than an internal RC oscillator that drifts by whole percent with temperature. Pull that thread and you reach the question every production data-acquisition system has to answer before any other: *when did this sample actually happen?* For a system trusted with expensive test articles, "when" is not metadata, it is half the measurement.

The default answer, stamping each value when it reaches the host, is the wrong one at any interesting rate. Operating systems buffer serial input, USB adapters batch transfers, and network stacks coalesce packets, so samples that occurred milliseconds apart arrive in a clump and get stamped with the same instant. The arrival time measures your transport's mood, not your experiment.

Production firmware answers "when" at the source, and there is a ladder of rigor:

- **A sample counter.** The cheapest honest answer. The device's read cycle is already crystal-timed, so a 16-bit counter that increments per reading *is* a timestamp: counter times period equals time since boot, deterministic, immune to every buffer between the ADC and the plot. The device counts, the host anchors: record one wall-clock reference when the session starts, and the counter carries the rest. Two bytes per frame buy back the entire transport's jitter.
- **Per-sample device timestamps.** When the device batches high-rate data, several samples per frame, each sample needs its own time from the device's clock, because the frame's arrival instant says nothing about the spacing inside the batch.
- **A synchronized clock.** The moment two separate boards must agree on "t = 0", counters stop being enough, and you are choosing between a shared trigger line, GPS pulse-per-second, or PTP (IEEE 1588). Nothing cheaper makes two crystals tell the same story for hours.

Alignment *across channels* hides inside the same question. A multiplexed converter reads eight channels as eight slightly different instants, which is fine for eight temperatures and disastrous for computing phase between a voltage and a current. Simultaneous-sampling ADCs exist precisely so that one frame can honestly claim one instant. Decide what your frame's timestamp means, one instant or eight, and say so in the protocol document.

On the receiving side, Serial Studio's rule is that device time is data: map the timestamp or counter as its own dataset, and any plot can use it as a [custom X axis](https://serial-studio.com/blog/plot-x-axis-time-samples-custom). A batch of accelerometer samples then lands at the device times they were recorded, not stacked on the arrival time of the frame that carried them.

One complement worth knowing: transforms also receive `info.timestampMs`, which is the *host's* monotonic clock. It is ideal for rate-limiting and for measuring deltas between frames, and it deliberately says nothing about when the sample happened. Keep the two clocks distinct: device time for the physics, host time for the plumbing. That is the whole discipline in one sentence.

## 8. Plan for the reboot

Firmware that runs unattended for weeks will eventually hit a state nobody anticipated. The production answer is not to prevent every crash; it is to make every crash short, loud, and visible in the data.

**Watchdog discipline.** Enable the independent watchdog and refresh it in exactly one place: the top of the main loop. Never refresh it from a timer interrupt, because a hung main loop with a healthy timer ISR will keep kicking the watchdog forever, which is the precise failure the watchdog exists to catch. Fault handlers should signal (a solid LED is traditional), stop refreshing, and let the reset happen.

**Report the reboot.** Nearly every microcontroller latches the cause of the last reset in a register: watchdog, brown-out, power-on, software request, reset pin. Firmware that transmits one boot frame carrying those flags plus its self-test result converts an invisible event into telemetry. Without it, a watchdog reset is a mysterious two-second gap in a plot; with it, the gap comes labeled with its own explanation, and a pattern of watchdog resets becomes visible across a whole campaign of runs.

**Trust delivery, not the queue.** Handing a frame to the transmit hardware is not the same as delivering it. A CAN controller whose bus has no other live node goes error-passive and keeps accepting frames into its FIFO while delivering none of them. The honest health check is retrospective: at the start of each transmit cycle, verify the *previous* cycle's frames actually left, and if several cycles in a row did not, recover deliberately and say so on the status LED, rather than letting a green light shine over a dead link.

The receiving side can do better than passively display any of this. Parsers and transforms in Serial Studio can call a small set of dashboard helpers, which turn device state changes into view changes. A reboot sentinel in the frame can trigger `clearPlots()`, so the traces start clean instead of drawing a dramatic falling edge from the pre-reset value. And a device that reports an operating mode can drive `setActiveWorkspace(mode)`: workspaces are dashboard tabs with their own layouts, so a flight controller that steps from *Standby* to *Calibration* to *Flight* puts the matching instruments on screen the moment the mode byte changes, without the operator touching anything.

These helpers demand the same discipline as alarms everywhere: gate them on a state *transition*, not on state. `setActiveWorkspace` called on every frame yanks the view away from the user; latched behind a `lastMode` variable, it fires exactly once per mode change, which is the difference between a dashboard that feels alive and one that fights you.

## 9. Fake the device before you build it

The highest-leverage habit costs the least: get the dashboard and the protocol agreed before the hardware exists. Protocol mistakes (a byte-order mix-up, a wrong scale factor, a forgotten sentinel) surface while they are still one-line fixes in a script instead of a firmware release. And the receiving configuration becomes an executable specification of the wire format: if the simulator and the dashboard agree, the protocol document cannot quietly drift from the code.

The external flavor is a thirty-line Python script pushing frames over UDP, or onto Serial Studio's VirtualCAN backend with the same DBC file the real bus will use. The project you build against the fake is the one you connect to the real thing; only the transport changes.

The fully in-app flavor needs no script outside Serial Studio at all, and it composes nearly everything in this post. The Control Script owns the values: `loop()` computes the simulation, writes the results into data-table registers, and calls `dashboardTick()`. Virtual datasets read the registers through their transforms and feed the widgets. The frame parser, with no real device talking, simply returns an empty array:

```js
function parse(frame) {
  return [];  // the Control Script owns the data; nothing arrives by wire
}
```

That empty return is load-bearing, not laziness. `dashboardTick()` runs the full transform pass and publishes the synthesized frame through the export pipeline, so the simulation is recorded to CSV or the session database exactly like real device data, and it seeds the frame structure before any real frame has arrived. If the parser *also* returned values, every sample would be exported twice, once per path. The rule of thumb: whoever owns the data drives the pipeline. When the script owns it, the parser returns empty and the script calls `dashboardTick()`. When a real device owns it and [the parser writes the tables itself](https://serial-studio.com/blog/frame-parsers-and-the-empty-parser), each arriving frame already flows through the pipeline and no tick is needed. Either way, each value is exported exactly once.

When the hardware finally powers up, you swap the simulation out and the wire format in, and everything downstream of the parser, the transforms, the calibration registers, the virtual datasets, the workspaces, stays exactly as you built it.

## What trustworthy telemetry has in common

Every habit here moves a decision to the place that has the context to make it: validity to the sensor driver, integrity to whatever the link cannot already guarantee, the question of *when* to a clock that was actually there, engineering units to editable calibration registers, aliveness to a host that still has a clock when the frames stop, layout changes to the data that justifies them, protocol checking to a simulation you can run today. None of it is glamorous, and all of it is cheap. The firmware habits keep bad data off the wire; the host machinery keeps good data meaningful. And when the plot finally shows something strange, you get the only property [telemetry](https://serial-studio.com/blog/what-is-telemetry) is really for: confidence that the strangeness happened in the physics, not in the pipeline. That confidence is built at both ends of the cable.