What the Machine Does With Your Object

Ask a general software engineer where an object lives and you will usually get an answer about scope. Ask a candidate for a low-latency role and the expected answer is about addresses: which region of the process image the bytes are in, how many bytes there are, and how many cache lines they straddle.

Candidates have been asked to implement bool isHeap(void* addr), deciding whether an address is on the stack or the heap, and then to explain the memory layout that makes such a function possible at all. It is a good question precisely because it cannot be answered from language rules alone.

The process image

A running program has its memory divided into regions that behave completely differently.

The stack is a contiguous region per thread, typically a few megabytes, and allocation is an adjustment to the stack pointer. That is why a local array is free and a newed one is not. It is also why the stack is almost always hot in cache: you keep reusing the same few kilobytes.

An isHeap implementation exploits the layout rather than the language. Take the address of a local variable, which sits at the current low-water mark of the calling thread's stack, and compare against the address in question. Because the stack grows downwards, every live stack object sits between that local and the stack's base a few megabytes above it; the heap sits far away in the address space. The function is unportable and slightly disreputable, which is the point: the interviewer wants to see whether you know the map.

Padding is not free, and it is visible

struct Quote {
    char  side;      // 1 byte
    double price;    // 8 bytes
    int    quantity; // 4 bytes
};                   // sizeof == 24, not 13

The compiler must place each member at an address that is a multiple of its alignment, so price cannot start at offset 1. Seven bytes of padding go in after side and four go on the end so that an array of Quote keeps every element aligned. Reordering the members from widest to narrowest gives you a 16-byte struct with the same fields:

struct Quote {
    double price;    // offset 0
    int    quantity; // offset 8
    char   side;     // offset 12
};                   // sizeof == 16

A third off the size, for free. On a book holding a million quotes that is eight megabytes of memory bandwidth you no longer spend, and bandwidth is the resource that runs out first in a market data path.

The cache line is the real unit

Memory does not move between the CPU and RAM in bytes, it moves in cache lines, universally 64 bytes on the hardware these firms run. Every access pulls in the whole line. This has two consequences that interviewers probe constantly.

First, locality is throughput. Walking an array of 16-byte structs touches four of them per line. Walking a linked list of the same structs touches one per line, plus a dependent load to find the next, which the prefetcher cannot hide. The asymptotic complexity is identical and the measured time differs by an order of magnitude. This is the substance behind the standard question of why a std::vector beats a std::list for almost everything.

Second, two threads writing to the same line contend even when they touch different variables. That is false sharing, and it is the subject of its own lesson in the concurrency section.

Reading a layout question

When an interviewer asks what sizeof your type is, they are not testing arithmetic. They are checking three things in one question: that you know members are aligned rather than packed, that you know the trailing padding exists so arrays work, and that you know virtual functions add a pointer. A type with any virtual member carries a hidden pointer to its virtual table, so an otherwise-empty class with one virtual function is eight bytes rather than one, and that pointer is the first thing in the object on every mainstream implementation.

If you can say that much out loud, the follow-up is usually about what the pointer costs at the call site, which is what virtual dispatch costs, later in this section.