Back to course

Parsing a Message Stream

Parsing shows up in these assessments far more than a computer science syllabus would predict, and for an obvious reason: every trading system starts with bytes arriving from somewhere else, in a format you did not choose and cannot change. Several major firms set parsing problems, and they are all really about the same three questions.

Where does one message end?

A stream is a sequence of bytes, not a sequence of messages. Recovering the boundaries is called framing, and there are three schemes.

Delimited
Boundary isA separator byte, often a newline
Cost to findScan for the separator
Body may containAnything except the separator
Fails whenThe payload contains the separator
Length-prefixed
Boundary isA count at the front of each message
Cost to findRead the header, then skip
Body may containAnything at all
Fails whenThe declared length is a lie
Fixed width
Boundary isKnown in advance
Cost to findArithmetic only
Body may containAnything
Fails whenA field needs to grow

Delimited is easiest to write, length-prefixed is what real protocols use.

The rest of this lesson is for subscribers

Unlock every lesson in Programming for Quantitative Developers, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextCleaning a Feed You Did Not Write

Keep reading Programming for Quantitative Developers

25 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them