Chip Hippo ← Back to site

Serial Protocol

Chip Hippo serial protocol, version 1. This page defines the bytes that travel between Chip Hippo and a board at the other end of a serial connection. You never need it to use the Arduino integration — the header and module Chip Hippo generates speak it for you — but it is the whole contract. Anything that follows this page (a sketch in another language, firmware in Rust, a test harness on a pseudo-terminal) can take the board's place.

1. Conventions

2. Transport

3. Frames

3.1 Frame format

START  TYPE  SEQ  LEN  PAYLOAD     CRC_LO  CRC_HI
0x7E   1     1    1    LEN bytes   1       1
Field Size Meaning
START 1 Always 0x7E, never escaped. Begins a frame.
TYPE 1 The frame type (§4).
SEQ 1 In OUTPUT and INBOUND: the frame's sequence number, 1–255 (§6.1); one with SEQ 0 is ignored. In ACK: the sequence number acknowledged. In every other frame: 0, which the receiver MUST ignore.
LEN 1 The number of payload bytes, counted before escaping: 0–255.
PAYLOAD LEN Defined per type (§4).
CRC 2 CRC-16/CCITT-FALSE (§3.3) of the unescaped bytes TYPE SEQ LEN PAYLOAD, little-endian.

A frame's body is everything after START, from TYPE to CRC_HI.

3.2 Size limits

Quantity Bytes
Payload 0–255
Body, unescaped: TYPE + SEQ + LEN + payload + CRC = 1 + 1 + 1 + 255 + 2 at most 260
Frame on the wire: START + every body byte escaped = 1 + 2 × 260 at most 521

Note: so a receiver never holds more than 3 + its limit + 2 bytes of a frame — 260 for the host, 12 for a device with the smallest limit. Escaping needs no buffer of its own: each escape pair is undone as it arrives.

3.3 CRC

CRC-16/CCITT-FALSE: polynomial 0x1021, initial value 0xFFFF, input and output not reflected, no final XOR. The check value, for the ASCII bytes 123456789, is 0x29B1.

uint16_t crc16(const uint8_t *p, size_t n) {
  uint16_t crc = 0xFFFF;
  while (n--) {
    crc ^= (uint16_t)(*p++) << 8;
    for (uint8_t i = 0; i < 8; i++)
      crc = (crc & 0x8000) ? (crc << 1) ^ 0x1021 : (crc << 1);
  }
  return crc;
}

3.4 Escaping

Every body byte that is 0x7E or 0x7D is sent as two bytes: 0x7D, then the byte XOR 0x20. No other byte is escaped.

Unescaped On the wire
0x7E 0x7D 0x5E
0x7D 0x7D 0x5D

Escaping applies to every body byte — TYPE, SEQ, LEN, the payload and both CRC bytes — while LEN and the CRC are computed over the unescaped bytes. So 0x7E on the wire is always the start of a frame, which is how a receiver finds the next frame after noise, a torn frame or a reset.

3.5 Receiving a frame

A receiver is either hunting (between frames) or in a frame.

  1. While hunting, it discards every byte but 0x7E, which begins a frame.
  2. In a frame, a 0x7E abandons the partial frame and begins a new one. A frame cut short this way is torn, not damaged: it is dropped without a word, and the sender's timer (§6.4) covers it. Any other byte is unescaped (§3.4) and read, in order, as TYPE, SEQ, LEN, LEN payload bytes and the two CRC bytes.
  3. The frame is damaged, and the receiver goes back to hunting, when:
    • a 0x7D is followed by anything but 0x5E or 0x5D — or 0x7E, which tears the frame instead (step 2);
    • LEN is greater than the receiver's payload limit (§3.2) — known as soon as LEN is read; or
    • the CRC does not match — known when the second CRC byte is read.
  4. A damaged frame is discarded. If the receiver is in a session (ACTIVE, §5.2 and §5.3), it MUST then send a NAK (§4.4) at once — unless the damaged frame's TYPE, as received, is HELLO, HELLO_ACK, ACK, NAK or LOG: frames never resent in answer to a NAK. A frame damaged before its TYPE arrived is NAKed. A receiver not in a session MUST NOT send a NAK: before a handshake, damage is most likely a bootloader's noise.
  5. An intact frame is dispatched by TYPE. A frame of a type the receiver does not know, or never receives (a device never receives INBOUND), MUST be ignored.

Note: a LEN damaged into a larger value is damage at once if it exceeds the receiver's payload limit (step 3). Otherwise the receiver waits for bytes that belong to the next frame, whose 0x7E then tears it (step 2), and the sender's timer (§6.4) covers the loss. A LEN damaged into a smaller value fails the CRC.

4. Frame types

Value Name Sent by SEQ Payload Delivery
0x01 HELLO host 0 7 bytes (§4.1) resent until answered (§5.4)
0x02 HELLO_ACK device 0 7 bytes (§4.1) —
0x06 ACK either the SEQ acknowledged none (§4.3) —
0x15 NAK either 0 none (§4.4) —
0x10 OUTPUT host 1–255 4 bytes (§4.2) reliable (§6)
0x11 INBOUND device 1–255 4 bytes (§4.2) reliable (§6)
0x20 LOG device 0 1–255 bytes (§4.5) not acknowledged

The values 0x00, 0x7D, 0x7E and 0xFF are never frame types. Every other unassigned value is reserved for later versions (§11).

4.1 HELLO and HELLO_ACK

The payload of both (7 bytes):

Offset Size Field
0 1 Protocol version: 1.
1 2 Session (§5.1), little-endian.
3 4 Layout signature (§7.3), little-endian.

For protocol version 1, the first three payload bytes are the version and the session. They are all a device needs to answer a HELLO, and all the host needs to judge a HELLO_ACK's version. A later version MAY change the HELLO and HELLO_ACK payloads after establishing that it is speaking to a different version (§11).

4.2 OUTPUT and INBOUND

OUTPUT carries an Output's value from the circuit to the device; INBOUND carries an Input's value from the device to the circuit. The payload of both (4 bytes):

Offset Size Field
0 1 Element index (§7.1), within this connection and direction.
1 1 Width, 1–16: the element's pin count.
2 2 Value, little-endian and right-aligned: bit 0 is the element's pin 1.

4.3 ACK

SEQ is the sequence number of the data frame acknowledged. The payload is empty, and a receiver MUST NOT examine it. When an ACK is sent, and what one means, is §6.2.

4.4 NAK

SEQ is 0 and the payload empty; a receiver MUST NOT examine either.

A NAK means: a frame arrived damaged — if you have a data frame outstanding, send it again now. It names no frame, because the damaged frame's TYPE and SEQ cannot be trusted; and it needs to name none, because each side has at most one data frame outstanding (§6).

A NAK is never acknowledged, never resent and never answered with a NAK, so it can cause no more retransmissions than the send limit allows.

4.5 LOG

The payload is 1–255 bytes of UTF-8 text, with no terminator. A newline (0x0A) ends a line; text without a trailing newline continues on the same line when the next LOG arrives.

5. Sessions

5.1 Session numbers

A session number is 16 bits, 1–65535. 0 is reserved: it is never a run's session, and a HELLO_ACK for session 0 is a device's announcement that it is in no session.

5.2 Host states

State Accepts Ignores Moves on
CLOSED — — Run opens the port → HANDSHAKE.
HANDSHAKE HELLO_ACK for its session; LOG Every other frame — including the announcement and a HELLO_ACK for any other session. Damage is not NAKed. Its session is established → ACTIVE. A mismatch, no answer in 5 s, or the port lost → FAILED.
ACTIVE INBOUND, ACK, NAK, LOG; the announcement Every other HELLO_ACK (a late copy of its own, another session's); types it never receives The announcement, a failed delivery (§6.4), or the port lost → FAILED. Stop → CLOSED.
FAILED LOG Everything else. Damage is not NAKed. The run stops and the port closes → CLOSED.

5.3 Device states

A device remembers the last session a HELLO named — 0 when its program starts.

State Accepts Ignores Moves on
WAITING HELLO Every other frame. Damage is not NAKed, and no data frame is sent. A HELLO for a new session that matches the device → ACTIVE.
ACTIVE HELLO, OUTPUT, ACK, NAK Types it never receives A HELLO for a new session → ACTIVE if it matches, else WAITING. A failed delivery → WAITING, announced.

Note: the byte stream keeps two sessions apart without a session number in every frame. Whatever a device sent in an old session precedes its HELLO_ACK for the new one, and the host accepts nothing but that HELLO_ACK until it arrives; after it, the device sends nothing that belongs to the old session. The same holds the other way round.

5.4 The handshake

  1. Open. The host opens the port. On many boards this resets the device (§2).
  2. Greet. The host chooses a session (§5.1) and sends HELLO at once, then again every 250 ms — the same session each time — until a HELLO_ACK for that session arrives, for up to 5 s.
  3. Judge. Of a HELLO_ACK for its session, the host:
    • refuses the run — version mismatch — if its version is not 1, whatever its length;
    • ignores it and keeps waiting if it is not exactly 7 bytes;
    • refuses the run — layout mismatch — if its signature is not the host's;
    • otherwise establishes the session and becomes ACTIVE.
  4. No answer in 5 s refuses the run.

A run is all-or-nothing: if any of its connections fails to open, or fails its handshake, every port the run opened is closed again.

Note: the device judges the HELLO as the host judges the HELLO_ACK (§5.3), so after a mismatch neither side is in the session. A device built for another design never runs onConnect, or sends an Input, into a run that has refused it.

6. Reliable delivery

OUTPUT and INBOUND are delivered stop-and-wait. A side has at most one data frame outstanding, and sends the next only when that one has been acknowledged or its delivery has failed.

6.1 Frame identity and sequence numbers

6.2 Acknowledgement

In a session, a receiver acknowledges every intact data frame — at the time the table below gives — with an ACK carrying its SEQ. A frame is delivered when an intact ACK with its SEQ arrives while it is outstanding. Any other ACK — another SEQ, or nothing outstanding — is stale, and MUST be ignored.

Receiver Frame The ACK is sent…
device OUTPUT after the Output's handler returns.
host INBOUND as soon as the frame arrives — even while the circuit is stalled. The value is applied at the next settle boundary (§8).

Acknowledging an OUTPUT after its handler means everything the handler sent — each INBOUND, delivered and acknowledged, and each LOG — reaches the host before the ACK. The host stalls the circuit until every OUTPUT is acknowledged, so it always sees the device's reaction before the circuit moves on. That is what makes request and response deterministic (§8).

6.3 Duplicates

Each receiver remembers the SEQ of the last data frame it acknowledged in the session — none, when the session begins.

Note: because a sender never has two frames outstanding, a receiver only ever sees the frame it last acknowledged again, or the next one. One remembered SEQ is enough to tell them apart.

6.4 Timeout, NAK and retry

6.5 Ordering

Each direction's data frames are processed in the order they were sent, since each waits for the one before. The two directions interleave freely. A side waiting for an ACK MUST keep receiving meanwhile: it acknowledges the other side's data frames, answers a HELLO and acts on a NAK as usual. LOG frames sit in the byte stream in the order they were written, among the data frames around them.

7. Element indices and the layout signature

Names never travel over the wire. Elements are identified by index, and the layout signature makes sure both ends number them alike.

7.1 Element indices

7.2 The layout string

A connection's layout is written as an ASCII string, with no byte-order mark, no NUL terminator, no newline and no spaces:

layout  = [ element *( "," element ) ]
element = kind index ":" width *( "+" width )
kind    = "O" / "I"
index   = "0" / ( %x31-39 *DIGIT )   ; decimal, no leading zeros
width   = "1" / "8" / "16"           ; one field: bit, byte or word

For example, Outputs of fields [byte, bit] and [bit], and an Input of one word, are O0:8+1,O1:1,I0:16.

7.3 The signature

The layout signature is the CRC-32 (IEEE 802.3, also known as CRC-32/ISO-HDLC) of the layout string's bytes: polynomial 0x04C11DB7, reflected (0xEDB88320), initial value 0xFFFFFFFF, final XOR 0xFFFFFFFF. It travels little-endian. The check value, for the ASCII bytes 123456789, is 0xCBF43926; the empty layout's signature is 0x00000000.

Layout string Signature
O0:8,O1:1,I0:16 0xF8ACB506
O0:8+1,O1:1,I0:16 0x7EF66B41
O0:1,I0:1+1+1+1 0x0EC32022
I0:16 0x43F1C7E9

The signature covers what decides which bits reach which parameter: the elements, their order, and each one's field boundaries, not only its total width. A [bit, byte] element reordered to [byte, bit] keeps its width of 9 but would hand every value to the wrong parameters, so it must be refused.

Everything else is the circuit's business and changes nothing on the wire: names, colours, descriptions, where the tags are planted, and each element's trigger. Renaming an element or a field changes the generated code's design hash — a second fingerprint, written into the generated file as a comment and compared by the Generate button — so the code reads as out of date, but a run still works until it is regenerated. The design hash never travels on the wire.

8. What the values mean

The protocol moves values; this is what Chip Hippo does with them, which a device has to know to answer sensibly.

9. Invariants

The rules above, reduced to what must always hold:

  1. SEQ is non-zero only in data frames and ACKs. HELLO, HELLO_ACK, NAK and LOG carry 0; a session travels in the HELLO payload; a data frame with SEQ 0 is ignored.
  2. Each side has at most one data frame outstanding.
  3. A retransmission is the same frame — the same bytes, the same SEQ — and a data frame is sent at most 3 times.
  4. A data frame is identified by session, direction and SEQ. The two directions' counters are independent.
  5. An ACK carries the SEQ of the frame it acknowledges, and only an ACK with the outstanding frame's SEQ delivers it.
  6. A receiver processes each data frame at most once: a duplicate is acknowledged again, never processed again.
  7. A device acknowledges an OUTPUT only after its handler has returned, and never one received in a session it has since left.
  8. HELLO, HELLO_ACK, ACK, NAK and LOG consume no sequence numbers and are never acknowledged.
  9. A NAK names no frame, and causes no more retransmissions than the send limit allows.
  10. LOG is never acknowledged, resent or de-duplicated, and may be sent outside a session.
  11. Session 0 is never a run's: a HELLO_ACK for session 0 is the device's announcement that it is in no session.
  12. Only a HELLO for a new session resets a device's session state.
  13. Neither side enters a session unless the versions and the layout signatures match.
  14. A failed delivery ends the session on both sides: the host stops the run; the device leaves it and announces that, which stops a host still listening, and stays offline until the next session.
  15. A device sends nothing that belongs to a session it has left — not an ACK, and not an INBOUND from a handler of that session.

10. Constants

Constant Value Meaning
PROTOCOL_VERSION 1 This protocol's version.
START 0x7E Begins a frame.
ESC 0x7D Escapes the next byte.
ESC_XOR 0x20 XORed into an escaped byte.
HEADER_BYTES 3 TYPE, SEQ, LEN.
CRC_BYTES 2 The CRC.
MAX_PAYLOAD 255 The longest payload.
MAX_FRAME_BODY 260 The longest body, unescaped.
MAX_WIRE_FRAME 521 The longest frame on the wire.
DATA_PAYLOAD 4 An OUTPUT or INBOUND payload.
HELLO_PAYLOAD 7 A HELLO or HELLO_ACK payload.
HELLO_PREFIX 3 The shortest HELLO a device answers, and HELLO_ACK a host reads: version, session.
MAX_HOST_PAYLOAD 7 The longest payload the host sends: a device's smallest payload limit.
ANNOUNCE_SESSION 0 The session of the announcement: the device is in no session.
MAX_SESSION 65535 Sessions are 1 to this.
MAX_SEQ 255 Data SEQs are 1 to this.
MAX_WIDTH 16 The widest element, in pins.
CRC16_POLY 0x1021 The frame CRC's polynomial.
CRC16_INIT 0xFFFF The frame CRC's initial value.
HELLO_INTERVAL_MS 250 How often the host sends HELLO while it waits.
HELLO_WINDOW_MS 5000 How long the host waits for HELLO_ACK.
ACK_TIMEOUT_MS 500 The ACK timer.
MAX_SENDS 3 The most times a data frame is sent.

These are protocol values, not settings: no connection overrides them. The names are those of Chip Hippo's source (§13).

11. Versions

12. Worked examples

These examples use the layout O0:8,O1:1,I0:16 — signature 0xF8ACB506, sent as 06 B5 AC F8 — and session 0x1234, sent as 34 12. Each frame is written as it is on the wire. They are test vectors: Chip Hippo's test suite checks every frame on this page against its encoder.

12.1 The handshake

The host's HELLO: version 1, session 0x1234, its signature (CRC 0x9C63).

7E  01 00 07  01 34 12 06 B5 AC F8  63 9C

The device's answer, HELLO_ACK: the same session, with its own version and signature (CRC 0x2DAC).

7E  02 00 07  01 34 12 06 B5 AC F8  AC 2D

The same device announcing its start-up: a HELLO_ACK for session 0 (CRC 0x4458).

7E  02 00 07  01 00 00 06 B5 AC F8  58 44

Escaping reaches the session as well. Session 0x7D7E is sent as 7E 7D, both of them markers, so a HELLO for it is (CRC 0x0C54):

7E  01 00 07  01 7D 5E 7D 5D 06 B5 AC F8  54 0C

12.2 An OUTPUT and its ACK

The host sends OUTPUT SEQ 1: element 0, width 8, value 0x7E — deliberately a value that needs escaping.

7E  10 01 04  00 08 7D 5E 00  E4 88

The device calls the handler for Output 0 with 0x7E. The handler returns, and the device sends ACK 1 (CRC 0x4D0D):

7E  06 01 00  0D 4D

Escaping reaches the CRC too. ACK 16 has the CRC 0x7D4F, and ACK 17 0x4E7E:

7E  06 10 00  4F 7D 5D
7E  06 11 00  7D 5E 4E

12.3 An INBOUND, a LOG and a NAK

The device sends INBOUND SEQ 1 — its own first data frame, since each direction counts separately (§6.1): element 0, width 16, value 0xBEEF (CRC 0xB788).

7E  11 01 04  00 10 EF BE  88 B7

The host acknowledges it with the same bytes as the ACK in §12.2: 7E 06 01 00 0D 4D. It is the device's frame this ACK answers, because the device receives it.

The device logs ok and a newline, unacknowledged (CRC 0x6104):

7E  20 00 03  6F 6B 0A  04 61

A NAK, from either side (CRC 0x640F):

7E  15 00 00  0F 64

12.4 A retransmission

The host sends Output 1 (width 1) the value 1, as SEQ 2. The frame is damaged on the way, and the device's ACK is later lost:

Step Direction What happens On the wire
1 host → device OUTPUT SEQ 2, first send. It arrives damaged. 7E 10 02 04 01 01 01 00 46 B6
2 device → host The damage is NAKed. 7E 15 00 00 0F 64
3 host → device Second send, at once: the same bytes. 7E 10 02 04 01 01 01 00 46 B6
4 device → host The handler runs; ACK 2. It is lost. 7E 06 02 00 5E 18
5 host → device 500 ms after step 3, the third and last send. 7E 10 02 04 01 01 01 00 46 B6
6 device → host A duplicate of the last frame acknowledged: ACK 2 again, the handler not run again. 7E 06 02 00 5E 18

Had step 6 been lost too, the host would have stopped the run 500 ms after step 5. At no point does the SEQ change, and the handler runs once.

13. Implementations

Four implementations speak this page, and each is tested against the others.

Chip Hippo's test suite checks this page's frames, constants and signatures against the code, drives each device — a generated header compiled with the host's C++ compiler, the Python module under Python and MicroPython, and the Mock — with the real host link, and feeds all of them the same bytes to show they answer alike.