## Identifying the frame length for an unknown serial protocol

Let’s suppose you’re reverse-engineering a serial protocol. You already know the correct configuration for the serial port (baudrate etc.) and you assume the protocol is built from frames of equal length.

For simplicity we will also assume that the device sends the data without the necessity to request data from it first. If this is not the case, you can use a different and much simpler approach (just send the request character and se

The next step is to determine the the frame length of the protocol. This post not only details two variants of one of the algorithms you can use in order to do this, but also provides a ready-to-use Python script you can use for your own protocols.

## Approach 1: Autocorrelation with argmax

We will use a simple mathematical approach in order to find out what the most likely frame length will be. This is based on the assumption that frames will have a high degree of self similarity, i.e. many of the bytes in a single frame will match the corresponding bytes in the next frame.

It is not required that all bytes are the same in every frame, but if you have entirely different bytes in every frame, the approach will likely not deduce the correct frame length.

This approach is based on autocorrelation. Although it sounds complicated, it means nothing more Compare a sequence by a delayed/shifted version of itself.

This means we will perform the following steps:

• Read a set of characters from the serial device
• Correlate the set of characters with shifted versions of itself
• The framelength is the shift where the maximum similarity occurs (using np.argmax)

As similiarity score, we’ll use 1 if the bytes equal or 0 else. For specific protocols, it might be a more viable approach to introduce individual bit matching, but this will also introduce noise into the process.

For most simple protocols I’ve seen, this approach works very well for both ASCII and binary.

Plotting the correlation looks like this:

## Approach 2: Multiple-shift aware Autocorrelation

This modified algorithms works well for protocols where there is insignificant similarity between any two frames or if there is a lot of noise. For such protocol, the maximum score approach does not yield the correct result.

However, we can use the property of constant-framelength protocols that we get high matching scores by shifting a frame by an integer multiple of the (unknown) framelength. Instead of just taking one maximum peak, we multiply all the scores for the integer-multiples of any length.

While this approach doesn’t sound too complicated compared to the first one, it has more caveats and pitfalls, e.g. that there are no integer multiples within the data array for the second half of the correlation result array, and the second quarter is not very significant as there are not many multiples to multiply.

The script (see below) works around these issues by only computing the first quarter of the possible result space. Use the -n parameter in order to increase the number of characters read by the script.

After computing the multiple-shift aware correlation, we can use argmax just like in the first approach to find the best correlation. Sometimes this identifies a multiple of the frame length due to noise. You can look at the plot (use -p) and manually determine the frame length in order to find the correct frame length.

As you can see from the result, the “noise” (in between the frame-length shift matches, caused by random matches between characters) is mostly gone.

In many real usecases, this algorithm will produce a more distinct signal in the plot, but the automatically calculated frame size will not be correct as several effects tend to increase the lobe height for multiples of the frame height. Therefore, it is adviseable to have a look at the plot (-p in the script) before taking the result as granted.

## Automating the algorithm

Here’s the Python3 script to this article which works well without modification, but for some protocols you might need to adjust it to fit your needs:

#!/usr/bin/env python3
"""
ProtocolFrameLength.py

Determine the frame length of an unknown serial protocol
with constant-length frames containing similar bytes in every frame.

For an explanation, see
Identifying the frame length for an unknown serial protocol

Example usage:

## Accurate short & long delays on microcontrollers using ChibiOS

### How system ticks work

In order to understand how delays work, we’ll first need to have a look at system ticks. Although ChibiOS 3.x supports a feature called tickless mode, we’ll stick to a simple periodic tick model for simplicity reasons.

A system tick is simply a timer that interrupts the microcontroller periodically and performs some kernel management tasks. For example, with a 1 kHz system tick (systick) frequency, the program flow is interrupted every millisecond. When being interrupted, one of the things the kernel does is to check if a thread that is currently asleep needs to be woken up. In other words, if your thread has some code like this:

// [...]
chThdSleepMilliseconds(5);
// [...]

and the kernel has a 1 kHz systick frequency, the kernel will set your thread to sleep, wait for 5 system ticks (i.e. 5 ms) and then wake up the

## Using burnout current sources for Wheatstone bridge detection

Many recent high-performance ADCs like the AD7190 include a builtin so-called burnout current source that can allegedly be used to detect an open circuit in the sensor. However, most vendors don’t provide an easy explanation on how this can be done.

In this blogpost I will attempt to explain how those current sources can be useful for practical applications. For this example, we will assume the ADC has one idealized differential channel and is connected to a simple wheatstone bridge strain gauge:

## Computing the LP2980 adjust resistor using Python

The LP2980ADJ is a 50 mA LDO that be configured for an output voltage from 1.23V to 15V using a pair of resistors.

The datasheet lists a formula for the output voltage, however no easy-to-use customizable software is provided that can be used to directly compute the correct resistor in a reproducible way. Read more

## Normalizing electronics engineering value notations using Python

In electronics engineering there is a wide variety of notations for values that need to be recognized by intuitive user interfaces. Examples include:

• 1fA
• 0.1A
• 0.00001
• 1e-6
• 4,5nA
• 4,500.123 A
• 4A5
• 4k0 A

The wide variety of options, including thousands separators, comma-as-decimal-separator and suffix-as-decimal-separator, optional whitespace and scientific notations makes it difficult to normalize values without using specialized libraries. Read more

## Calculating the NCP380 Ilim resistor using Python

#### Problem

You want to calculate the correct value for the Ilim resistor for the NCP380 current limiter IC with a custom current limit. Read more

## Finding the nearest E96 resistor value in Python

### Problem

You want to find the E24/E48/E96 resistor value that is closest to a given exact value programmatically using python. Read more