All projects

FPGA Streaming Parser

The parser takes a stream of 23-byte frames from on-chip block RAM, one byte per clock cycle. A state machine finds the sync bytes, captures the fields, updates a CRC-16 as each byte arrives and flags sequence gaps. Each frame becomes a 128-bit event in a 32-entry FIFO. A separate benchmark build measures it on the physical board.

Role
Independent - design & implementation
Stack
SystemVerilog, Xilinx Artix-7 (Basys 3), Vivado, xsim, Tcl, Python
Status
Personal project · measured on a Basys 3 board

How it works

One byte per clock, one event per frame

Each frame is 23 bytes: two sync bytes, a length, a message type, a 16-bit sequence number, a timestamp, a price, a quantity, an instrument id, flags, and a CRC-16 over everything before it. A replay block streams a corpus of these frames from block RAM at one byte per cycle, and can loop it.

The parser is a state machine: hunt for the first sync byte, confirm the second, read the length, then the payload and the CRC. The CRC is updated in the same cycle each byte is accepted, so the check is ready as soon as the last byte lands. A frame with a bad CRC is still reported, with an error status, and the parser then hunts for the next sync byte.

Every frame becomes a 128-bit event carrying its status, type, sequence number, timestamp, price, instrument and the top half of the quantity. Each stage hands data on with a valid/ready handshake, so a full FIFO stalls the parser and the replay instead of dropping data.

Architecture

FPGA streaming parser architectureEverything runs on a Basys 3 board with a Xilinx Artix-7 XC7A35T on one 100 MHz clock. The board switches start and loop a BRAM replay that streams a 4 KiB corpus of 23-byte frames into parser_core at one byte per cycle. The parser's frame state machine moves through HUNT, SYNC1, HDR, PAYLOAD, CRC and EMIT, with a RESYNC state after a bad CRC, while a CRC-16 unit and a sequence check update as each byte is accepted. Each frame becomes a 128-bit event that goes through a valid and ready handshake into a 32-entry event FIFO, which a consumer drains every cycle; a full FIFO stalls the parser and the replay. perf_counters counts messages and latency for the seven-segment display and LEDs. In the benchmark build only, bench_monitor reads the replay, parser and FIFO handshakes without driving them, and a VIO and ILA connect it over JTAG to a Vivado Tcl script on the host that programs the board, runs the tests and reads the counters.Basys 3 · Artix-7 · 100 MHzSwitchesStart, loop, resetBRAM replay4 KiB corpus · 1 byte/cycleparser_coreFrame state machineHUNT → … → CRC → EMITCRC-16Per byteSequenceFlags gapsEvent FIFO32 × 128-bitConsumerDrains each cycleperf_countersCount, latency7-seg + LEDsRate, latencybench_monitorRead-only · per-message latencyVIO + ILADebug coresHostVivado TclProgram, run tests, read countersJTAG1234
Data path, with valid/ready backpressureBenchmark build only
  1. 1One byte per cycle. The parser stops taking bytes for 2 cycles per frame while it hands off the event, so a 23-byte frame takes 25 cycles.
  2. 2Checked as it streams. The CRC-16 and the sequence check update as each byte is accepted, so the status is ready when the last byte lands.
  3. 3No silent loss. A full FIFO holds the event in the parser, and the parser holds the replay. Data waits; it isn’t dropped.
  4. 4Measured in place. The monitor only reads the handshakes. The host reads its counters over JTAG.

On the board

Measured on the Basys 3

These numbers come from the physical board, not simulation. Beside the unchanged parser I added a monitor that only reads its signals. It counts over an exact number of clock cycles and pairs every event with the cycle its frame started. A Tcl script programs the board over JTAG, runs each test and reads the counters back through Vivado’s debug cores.

A 23-byte frame takes 25 cycles: 23 to read it and 2 to hand off the event. At 100 MHz that is 4 million frames a second, and each of five one-second runs counted exactly 4,000,000. The original test corpus includes padding and deliberately corrupt bytes, which brings it to 3.62 million.

Every one of more than 74 million messages took 24 cycles between the parser accepting its first sync byte and handing off its event. With the FIFO drained only once every 32 cycles, the parser stalled as designed and no message was lost.

4.00M

messages/s, back-to-back frames

240 ns

first byte to event, every message

3.62M

messages/s, original test corpus

0

messages dropped under backpressure

One message, captured on the board

One message on the board, cycle by cycleOne 23-byte frame captured cycle by cycle on the board. Cycle 0: the first sync byte FA is accepted in HUNT. Cycles 1 and 2: the second sync byte and the length. Cycles 3 to 20: the payload in PAYLOAD, carrying sequence number 118. Cycles 21 and 22: the two CRC bytes. Cycles 23 and 24: EMIT, while the next sync byte waits. Cycle 24: the event is handed off with status OK, 24 cycles or 240 ns after the first byte. Cycle 25: the event leaves the FIFO and the next frame's first byte is accepted, so frames repeat every 25 cycles.CycleByteStateField0FAHUNTsync153SYNC1sync212HDRlen302PAYLOADtype400PAYLOADseq576PAYLOADseq600PAYLOADtimestamp71EPAYLOADtimestamp884PAYLOADtimestamp9F5PAYLOADtimestamp1000PAYLOADprice1100PAYLOADprice124EPAYLOADprice1395PAYLOADprice1400PAYLOADquantity1500PAYLOADquantity1600PAYLOADquantity17D9PAYLOADquantity180BPAYLOADinstr1915PAYLOADinstr2001PAYLOADflags2175CRCCRC22A8CRCCRC23waitEMIT24waitEMITevent: seq 118, OK25FAHUNTFIFO read · next frame24 cycles = 240 ns, first byte to event25 cycles per frame = 4 million/s
Recorded cycle by cycle by the on-chip logic analyzer while the parser ran at 100 MHz. The highlighted bytes are the sequence number, 118.

Verification

Checked against simulation

Each check below was run, in simulation or on the board.

  • The instrumented design matches the original top level cycle for cycle over a million simulated cycles, so the monitor doesn’t change the parser’s behavior
  • The monitor’s counts and latency statistics match an independent scoreboard in the testbench exactly
  • Runs of 1,000,000 cycles from reset produced identical counters and latency histograms on the board and in simulation
  • A raw capture from the on-chip logic analyzer shows the same 24-cycle latency, computed from the sampled signals alone
  • The design meets timing at 100 MHz after place and route. With the measurement logic added, the parser’s own paths keep 2.6 ns of slack
  • Unit testbenches check the CRC against the standard check value, and the bad-CRC and sequence-gap paths

Decisions

Three choices that shape the design

Checking while streaming

The CRC-16 and the sequence check update as each byte is accepted, so nothing is buffered and there is no second pass. When the last CRC byte arrives, the frame’s status is decided in the next cycle.

Backpressure instead of loss

Every stage uses a valid/ready handshake. If the FIFO fills, the parser holds its event and stops taking bytes, and the replay holds its byte. Throughput drops to what the consumer can take, and nothing is lost.

Measuring without touching the parser

The benchmark logic sits beside the datapath and only reads it. It records the cycle each frame starts and matches it to that frame’s event, so latency stays correct even when events queue in the FIFO. Unmatched or overflowing entries are counted, and there were none.

The tradeoff

The format is fixed: every frame is 23 bytes and fields sit at fixed offsets. A per-type message table exists, but the parser doesn’t use it yet. To fit in 128 bits, the event keeps only the top 16 bits of the quantity and leaves out the flags. After a corrupt frame, the next good one is reported as a sequence gap rather than a resync, because the corrupt frame’s sequence number was never accepted. Input comes from block RAM, not a network port.

Technical deep dive

Frame format

Sync bytes 0xFA and 0x53, a length of 18, then type, sequence (16 bits), timestamp (32), price (signed 32), quantity (32), instrument (16), flags (8), and a CRC-16-CCITT (polynomial 0x1021, initial value 0xFFFF) over bytes 0 to 20. Multi-byte fields are big-endian.

The state machine

Seven states: HUNT, SYNC1, HDR, PAYLOAD, CRC, EMIT and RESYNC. A new first sync byte while waiting for the second restarts the frame. In EMIT the parser stops taking bytes for two cycles, one to present the event and one to complete the handshake, which is where the 25-cycle rhythm comes from.

Each event carries one status: CRC error, gap, resync or OK, in that order of priority. A gap is flagged when the sequence number isn’t one more than the last good frame’s.

The benchmark build

A VIO core gives the host a register window onto 64-bit counters: messages, bytes, stall cycles, FIFO-full cycles, latency sums and 64-bin latency histograms. An ILA captures 2,048 cycles of raw handshake signals.

Before it trusts any number, the script checks the bitstream’s hash, the device ID, and a build ID read back from the running design. Every run writes its raw register values and captures next to the report.

Board controls

Without the host, the board’s switches start and loop the replay, and the seven-segment display shows the message count, the message rate or the last latency, in hex.

Status

What works and what doesn’t yet

The parser runs on the board, and its numbers come from there. The parts below are split by what exists today.

Working

  • Frame sync, CRC check and resync after a bad frame
  • Sequence-gap detection
  • 128-bit events through a FIFO, with backpressure
  • Looping replay from block RAM at one byte per cycle
  • Automated build, programming and measurement on the board

Not built yet

  • A network or serial input (frames only come from block RAM)
  • Per-type message layouts (the table exists, but the parser doesn’t use it)
  • The full quantity and the flags in the event