Multicore Systems & Multicore Synchronization

For a long time, processor performance improved mostly by making a single CPU core faster.

In the 1990s, the usual expectation was:

• wait for a newer processor;
• get a higher clock frequency;
• single-threaded programs become faster automatically.

This was the era where optimizing software often felt less urgent, because hardware speed kept improving quickly.

Around the early 2000s, this trend changed. Since roughly the mid-2000s, mainstream desktop and server processors have mostly become multicore processors rather than simply much faster single-core processors.

Deeply embedded systems may still use single-core processors because they are small, low-power, and often do not need much computation. But for desktops, laptops, and servers, multicore is now the normal case.

The important physical trend that stopped is called Dennard scaling.

The old trend was:

• transistors became smaller;
• smaller transistors also became more power-efficient;
• therefore processors could run at higher frequencies within roughly the same power budget.

When Dennard scaling stopped, transistor counts could still increase, but clock frequency could no longer keep increasing in the old way.

The key limitation is power and heat:

• a laptop cannot dissipate several kilowatts of heat;
• a server room is limited by how much heat it can remove;
• almost all consumed electrical power eventually becomes heat.

So modern CPUs cannot simply keep increasing clock frequency.

Modern chips still contain more and more transistors.

However:

• frequency has flattened;
• power has flattened because of thermal limits;
• single-thread performance still improves, but much more slowly than before;
• the number of cores has increased.

The hardware industry therefore used the extra transistors to provide more cores instead of just making one core faster.

The consequence for operating systems is important:

The old abstraction “the processor will just get faster” disappeared. The problem was pushed to software and operating systems: use many cores efficiently.

Before mainstream multicore, high-end machines already used SMP, meaning symmetric multiprocessing.

In an SMP system:

• there are multiple identical CPUs;
• each CPU has the same instruction set and capabilities;
• each CPU often has its own private cache;
• all CPUs share main memory;
• the CPUs are connected through a shared system bus or interconnect.

These systems existed in high-performance computing, servers, and specialized machines before multicore became common.

Historically, many operating systems were designed for uniprocessor machines.

In a uniprocessor system, the OS could often rely on a very strong reasoning principle:

If one thread is running on the only CPU, then no other thread is running at the same time.

This principle made many kernel designs simpler.

For example, if the kernel disabled interrupts, then it could often assume that no other kernel code would run concurrently.

This assumption no longer holds on multicore systems.

In an SMP or multicore system, each CPU/core has private caches.

This creates a problem:

• core 0 may have a cache line containing variable x;
• core 1 may also access x;
• main memory may contain an older value;
• the newest value may be inside one core’s cache.

The system needs a mechanism that makes memory appear globally shared and coherent.

This mechanism is called cache coherence.

Cache coherence gives programmers the illusion that all cores read and write one shared memory, even though the hardware actually contains many private caches.

Cache coherence does not usually track each byte individually.

Instead, it tracks cache lines.

A cache line is a block of memory managed as one unit by the cache coherence protocol.

This matters because two different variables can accidentally live in the same cache line. Then the hardware treats them as shared, even if the source code treats them as separate variables.

This leads to false sharing, discussed later.

A classic cache coherence protocol family is MOESI.

Each cache line may be in one of several states:

The exact protocol can vary, but the important idea is:

• to read, a core needs a valid copy;
• to write, a core usually needs exclusive ownership of the cache line;
• if other cores have the line, they may need to be invalidated or coordinated with.

A simplified view:

• If a core has no copy and performs a read miss, it may obtain the line in Exclusive state if no other core has it.
• If another core later reads the same line, both may move to a Shared state.
• If a core wants to write to a shared line, it must invalidate other copies first.
• If a cache line is Modified or Owned, memory may be stale, so the cache holding the dirty copy is responsible for eventually writing it back.

The lecture emphasized that these state bits are stored in the cache metadata, not in main memory.

Reason:

• main memory is too far away and too slow;
• different caches can have different local views of the same line;
• cache coherence is managed by cache controllers and the interconnect.

At a high level, multicore systems look similar to SMP systems: multiple execution units share memory.

But modern multicore systems usually have much more shared on-chip hardware.

Examples:

• shared L2 cache between a small cluster of cores;
• shared last-level cache, usually L3;
• shared interconnect;
• shared memory controllers;
• shared I/O fabric.

This means activity on one core can directly affect the performance of another core.

If two cores share an L2 cache, they can evict each other’s cache lines.

If many cores share an LLC, then heavy memory traffic from one group of cores can reduce cache capacity and bandwidth available to other cores.

A local performance problem can propagate:

1. two cores fight over shared L2 space;
2. more cache lines get evicted;
3. more requests go to the LLC;
4. LLC and interconnect traffic increases;
5. other cores suffer even if they are not directly sharing the same L2.

So performance effects are often non-local and hard to predict.

Because these effects are hard to reason about manually, modern CPUs provide performance counters.

They can measure events such as:

• L1 misses;
• L2 misses;
• LLC misses;
• cache invalidations;
• interconnect transactions;
• memory bandwidth usage.

Performance debugging on multicore systems often requires looking at these hardware counters.

NUMA means Non-Uniform Memory Access.

In a NUMA system:

• all cores may still access all memory;
• but not all memory has the same access latency;
• each NUMA node has local memory attached to a local memory controller;
• accessing remote memory requires going through an interconnect, possibly off-chip.

So the system is still a shared-memory system, but memory access time depends on where the core and the memory are located.

If a core accesses memory attached to its own NUMA node, the access is relatively fast.

If it accesses memory attached to another NUMA node, the access may be much slower.

The lecture emphasized that remote memory can be many times slower than local memory.

Therefore:

• thread placement matters;
• memory allocation placement matters;
• scheduling and memory management are no longer independent.

On a uniprocessor system, there was little reason to ask “where” memory is allocated.

On a NUMA system, this becomes crucial.

The OS should try to place:

• a thread on a core close to its data;
• related threads close to each other if they communicate heavily;
• memory on the NUMA node where it will mostly be used.

Bad placement can create unnecessary cross-node traffic.

A homogeneous multicore system has cores with the same capabilities.

A heterogeneous multicore system has different types of cores.

For example, ARM big.LITTLE systems contain:

• big cores: faster, higher power consumption;
• LITTLE cores: slower, more power-efficient.

On a smartphone, background tasks may not need a fast core.

Examples:

• checking notifications;
• updating time display;
• background synchronization;
• light system maintenance.

These can run on LITTLE cores to save energy.

Latency-sensitive or compute-heavy tasks may use big cores.

Examples:

• games;
• touch interactions;
• UI rendering;
• short bursts of computation.

Sometimes it is better to run a task on a fast core, finish quickly, and return the system to an idle low-power state.

This strategy is called race to idle.

The tradeoff is subtle:

• slow core: lower power, but runs longer;
• fast core: higher power, but finishes sooner.

The best choice depends on workload, latency requirements, and energy behavior.

With heterogeneous cores, the scheduler must decide not only which thread runs next, but also which kind of core it should run on.

It must consider:

• performance needs;
• latency sensitivity;
• energy consumption;
• thermal constraints;
• cache and NUMA locality;
• current system load.

There is no single perfect scheduling policy for all hardware and workloads.

The lecture’s main OS-level message is:

Locality is now a first-class concern.

On old uniprocessor systems, APIs often did not expose locality.

For example, simple APIs such as memory allocation, process creation, or thread scheduling often do not ask:

• which NUMA node?
• which core?
• close to which cache?
• close to which other thread?

But on multicore systems, these questions can dominate performance.

In a multicore system, cores communicate through:

• shared memory;
• cache coherence;
• interconnect traffic;
• explicit messages;
• interrupts between processors.

The interconnect has limited bandwidth.

Therefore, communication itself becomes a resource that the OS must manage or at least be aware of.

On a uniprocessor, if a thread runs, other threads are not running simultaneously.

On multicore, other threads may run at the same time and affect performance.

Co-runners can interfere through:

• shared caches;
• memory bandwidth;
• interconnect traffic;
• locks;
• TLB shootdowns;
• I/O paths.

So the performance of one thread may depend heavily on what is running on other cores.

The lecture suggests that multicore systems should often be viewed like a small distributed system.

Even though the programming model may expose shared memory, the hardware behaves more like:

• multiple local execution units;
• local caches;
• non-uniform access costs;
• communication channels;
• explicit coordination costs.

This perspective helps explain why locality and communication are so important.

In a uniprocessor kernel, disabling interrupts often gave the kernel exclusive access to kernel state.

If interrupts are disabled:

• timer interrupts cannot preempt the current kernel code;
• device interrupts cannot run handlers;
• no other thread can run on another core, because there is no other core.

So the kernel could safely modify many kernel data structures inside one critical section.

On multicore, disabling interrupts only affects the current core.

Other cores may still:

• execute user code;
• enter the kernel through syscalls;
• handle interrupts;
• modify shared kernel state.

Therefore, disabling interrupts is not enough for global synchronization.

The simplest multicore port is the Big Kernel Lock.

Idea:

• when a core enters the kernel, it acquires one global spin lock;
• while holding it, it can access arbitrary kernel state;
• when it leaves the kernel, it releases the lock.

This makes the multicore kernel behave somewhat like a uniprocessor kernel.

The Big Kernel Lock serializes kernel execution.

If many cores frequently enter the kernel, they all contend on one lock.

This is especially bad for workloads such as:

• network servers;
• file servers;
• workloads with many syscalls;
• I/O-heavy applications.

The lecture jokingly says this scales to roughly “one and a half cores”.

Kernel locking often evolves like this:

1. One big kernel lock.
2. Subsystem locks, such as:
   • file system lock;
   • VM lock;
   • process table lock.
3. Fine-grained locks protecting individual objects, such as:
   • one inode;
   • one page table;
   • one thread object;
   • one socket.

Fine-grained locking improves parallelism but creates new problems:

• more lock/unlock overhead;
• more complex correctness reasoning;
• higher risk of deadlocks;
• more subtle performance behavior.

Suppose the kernel wants to count context switches:

int32_t x;

void schedule(void) {
    ...
    atomic_inc(&x);
    ...
}

On a uniprocessor, this is cheap.

On multicore, every core updates the same variable.

To write to x, a core must obtain the cache line containing x in a writable/exclusive/modified state.

If another core recently wrote x, then the cache line may be in that core’s private cache.

The line must move between cores.

This repeated movement is called cache-line bouncing.

The problem is not just the atomic instruction.

Even if the code used a non-atomic increment:

x++;

the cache line would still need to move to the writing core.

So the core problem is shared writable cache-line ownership.

Cache-line bouncing causes:

• cache coherence traffic;
• invalidations;
• interconnect bandwidth usage;
• latency before a core can write;
• poor scaling under contention.

A variable that looks tiny in source code can become a major multicore bottleneck.

Suppose we use two counters on a dual-core system:

int32_t x;
int32_t y;

void schedule(void) {
    ...
    if (this_core() == CORE0) {
        atomic_inc(&x);
    } else if (this_core() == CORE1) {
        atomic_inc(&y);
    }
    ...
}

Logically:

• core 0 writes only x;
• core 1 writes only y.

So it looks like sharing is gone.

The variables x and y may be placed next to each other in memory.

If they are in the same cache line, the hardware treats them as one coherence unit.

Then:

• core 0 writes the cache line to update x;
• core 1 writes the same cache line to update y;
• ownership still bounces between cores.

This is called false sharing.

The variables are not logically shared, but they are physically shared at the cache-line level.

To avoid false sharing, low-level code may need:

• padding;
• alignment;
• compiler attributes;
• cache-line-aware data layout.

For example, a per-core counter may need to occupy a whole cache line by itself.

However, this creates portability problems:

• different hardware may have different cache-line sizes;
• kernels distributed for many machines must choose layouts carefully;
• sometimes a structure should be packed together, but sometimes fields should be separated.

A common scheduling principle is work conservation:

Do not leave a core idle if there is runnable work.

This seems reasonable.

But on a NUMA system, moving a thread to an idle core may hurt performance if the thread’s memory is far away.

Example:

• thread’s memory is on NUMA node 0;
• scheduler moves the thread to a core on NUMA node 1;
• now most memory accesses become remote;
• the system may generate expensive cross-node traffic.

So work conservation can conflict with locality.

Fairness is also desirable: if many threads compete for cores, each should get a reasonable share.

But fairness may move threads around in ways that destroy cache locality.

For example, producer-consumer threads may share data.

If producer and consumer run:

• on the same core, or
• on nearby cores sharing cache,

then produced data may already be in cache when the consumer reads it.

If the scheduler moves them far apart, performance can become worse.

The scheduler often does not know:

• which threads communicate heavily;
• which memory regions they share;
• which data is hot in which cache;
• whether threads in the same process actually cooperate.

Therefore, perfect locality-aware scheduling is difficult.

The lecture emphasizes that old abstractions treated scheduling as separate from memory placement, but multicore systems break this separation.

Many programs use malloc and free from many threads.

A naive allocator may have one global heap data structure.

Then many threads contend on allocator metadata, even if their application-level work is independent.

This can destroy scalability.

High-performance allocators often use:

• per-thread arenas;
• per-core arenas;
• local free lists;
• batching.

This improves locality and reduces contention.

Examples of high-performance allocators mentioned in this context include allocators such as jemalloc or other specialized scalable allocators.

A tricky case:

1. thread A allocates memory from its local arena;
2. thread A passes the object to thread B;
3. thread B frees the object.

Now thread B is freeing memory that belongs to thread A’s arena.

This reintroduces synchronization problems.

The lecture emphasizes a major lesson:

Multicore performance problems often do not respect abstraction boundaries.

An application may be parallelized at a high level, but a hidden lower-level component such as the allocator may contain shared state and become the bottleneck.

A multithreaded TCP server may do:

for (;;) {
    int c = accept(shared_listen_fd, NULL, NULL);
    handle_connection(c);
}

Expectation:

• many worker threads wait on the same listening socket;
• incoming connections should be distributed roughly evenly.

But in practice this may not happen.

The lecture mentions a Google-scale observation:

• applications accepted around 40,000 connections per second;
• the busiest accepting thread accepted about 3x as many connections as the least busy thread.

This imbalance can underutilize CPU cores.

The root cause is not that the kernel intentionally favors one thread.

The problem is that with one shared listen socket there is one shared wait queue.

Wake-up order depends implicitly on many factors:

• cache locality;
• whether a thread recently ran;
• NUMA placement;
• current core load;
• how quickly a thread reaches the critical section;
• memory hierarchy effects.

A thread that recently ran may still have hot cache state and can wake up faster.

This creates a “rich get richer” effect.

Linux introduced SO_REUSEPORT.

Normally, one port belongs to one socket.

With SO_REUSEPORT:

• multiple sockets can bind to the same TCP/UDP port;
• each worker can have its own listening socket;
• each socket can have its own wait queue;
• the kernel explicitly distributes incoming connections among sockets.

This changes the design from implicit load distribution to explicit load distribution.

It can improve balance and locality.

The lecture also notes that at very large scale, smart NICs may help distribute flows before the kernel is deeply involved.

POSIX specifies that open() returns the lowest-numbered file descriptor not currently open in the process.

On a uniprocessor, this is natural:

• scan the file descriptor table;
• find the first free slot;
• return it.

On multicore, many threads in the same process may open files concurrently.

If the kernel must always return the lowest free file descriptor, then all open() calls must synchronize on the shared file descriptor table.

This creates an artificial serialization point.

Example workload:

• many threads index many different files;
• they should be able to open files independently;
• but they all contend on the process’s file descriptor table because the exact fd number is over-specified.

The application usually does not care whether the returned fd is 7 or 512.

It only needs a handle.

The API specifies more determinism than most programs actually need.

This is harmful because it prevents more scalable implementations.

Over-specified behavior in old APIs can become a scalability bottleneck on multicore.

POSIX file descriptors refer to an open file description that contains a shared file offset.

A normal read(fd, buf, n) does two things:

1. reads from the current file offset;
2. increments the file offset by the number of bytes read.

If multiple threads read from the same file descriptor, they must synchronize on this shared offset.

This is bad for parallel workloads.

A more scalable interface is to pass the offset explicitly, as with pread.

Then different threads can read different parts of a file without sharing an implicit offset.

A POSIX process has a current working directory.

Relative path operations depend on it.

But chdir() changes the current working directory for the whole process.

In a multithreaded process, this is problematic:

• one thread may call chdir();
• another thread may open a relative path;
• the meaning of the path depends on shared mutable process state.

Again, implicit shared state becomes a problem.

A better design is to make the directory context explicit, for example with APIs like openat.

munmap() removes mappings from an address range.

The specification says that subsequent references to those pages should generate a segmentation fault.

On a single core, this is straightforward:

• update page table entries;
• invalidate local TLB entries.

On multicore, it is much more expensive.

If other cores are running threads in the same address space, they may have TLB entries for the unmapped pages.

If those TLB entries remain valid, those cores could still access the pages.

Therefore, before munmap() returns, the kernel may need to ensure that relevant TLB entries on other cores are invalidated.

This is called a TLB shootdown.

It usually involves inter-processor interrupts, which are expensive.

A scalable allocator may frequently map and unmap memory.

If every unmap forces synchronous TLB shootdowns on many cores, performance suffers badly.

Some allocators therefore avoid returning memory to the OS immediately.

They may keep memory for reuse rather than repeatedly calling munmap.

The API forces synchrony:

After the call returns, all cores must behave as if the mapping is gone.

This synchronous guarantee may be stronger than what some workloads need.

An asynchronous or batched design could sometimes be more scalable.

A classic POSIX way to create a new program is:

fork();
exec(...);

The implicit old assumption is:

Creating a copy of a process is about as cheap as creating a new one from scratch.

This was more reasonable in simpler systems.

After fork and before exec, the child process exists as a globally visible process.

Other cores may observe it.

Therefore, the kernel must create a child process that is fully consistent:

• process structures initialized;
• locks initialized;
• references and reference counts correct;
• lists updated consistently;
• address space structures valid;
• process table state coherent.

This requires synchronization.

Then exec immediately throws most of this carefully constructed state away.

Copy-on-write helps reduce the cost of copying user-space memory.

But the scalability bottleneck discussed here is mostly in kernel data structures.

Kernel structures are not simply copied lazily with user-space copy-on-write.

So fork+exec can still be expensive on multicore.

A multikernel is an OS design where each core runs something like its own local kernel instance.

Instead of one shared-memory kernel protected by many locks, the system is treated as a network of cores.

Each core has:

• local OS state;
• a CPU driver;
• local resource management;
• coordination services.

Cores coordinate through explicit message passing.

The multikernel approach treats a multicore machine as a distributed system inside one chip or one machine.

State is:

• partitioned where possible;
• replicated where useful;
• coordinated explicitly through messages.

This makes communication visible rather than hidden in shared-memory coherence traffic.

Benefits:

• strong locality;
• fewer hidden shared-memory bottlenecks;
• clearer communication structure;
• easier reasoning about inter-core coordination points.

Costs:

• more complex OS design;
• distributed protocols needed inside the OS;
• some global services become harder to implement.

Research systems such as Barrelfish explored this design direction.

In a uniprocessor kernel, disabling interrupts can protect critical sections from interruption.

On a multicore system, disabling interrupts on one core does not stop other cores.

Other cores can still:

• enter the kernel;
• handle interrupts;
• modify shared kernel data;
• acquire locks;
• access shared memory.

Therefore, interrupt masking is still useful, but it is not sufficient for multiprocessor synchronization.

Kernel synchronization often uses spin locks.

A spin lock is appropriate when:

• the critical section is very short;
• sleeping is not allowed or too expensive;
• the lock may be used in interrupt context;
• interrupts are often disabled while holding it.

But spin locks interact strongly with the cache coherence system.

This is the main topic of the lecture.

Atomic instructions combine a check and an update so that no other core can observe an invalid interleaving.

They are implemented with help from the cache coherence protocol.

Common atomic instructions include:

• test-and-set;
• fetch-and-increment;
• compare-and-swap;
• load-linked/store-conditional.

test_and_set(lock) atomically:

1. reads the old value of lock;
2. sets lock to 1;
3. returns the old value.

If the old value was 0, the caller acquired the lock.

fetch_and_increment(x) atomically:

1. reads the old value of x;
2. increments x;
3. returns the old value.

This is useful for ticket locks.

CAS(addr, old, new) atomically:

• checks whether *addr == old;
• if yes, writes new;
• reports success or failure.

CAS is central to many lock-free algorithms.

Load-linked/store-conditional works as a pair:

• ldl(addr) reads a value and marks the address;
• stc(addr, new) succeeds only if no conflicting modification happened since the load-linked.

If another core modified the location, the store-conditional fails and the algorithm retries.

A simple TAS spin lock looks like this:

typedef volatile unsigned int spin_lock_t;

void spin_lock(spin_lock_t *lock) {
    do {
        while (*lock)
            /* busy-wait */;
    } while (test_and_set(lock) == 1);

    memory_barrier();
}

void spin_unlock(spin_lock_t *lock) {
    memory_barrier();
    *lock = 0;
}

The outer loop tries to acquire the lock.

The inner loop waits while the lock appears busy.

Under heavy contention, TAS locks can cause severe load imbalance.

Reason:

• the core that releases the lock writes 0 to the lock variable;
• this gives that core exclusive ownership of the cache line;
• nearby cores may see the new value faster than remote cores;
• the same core or a nearby core may reacquire the lock repeatedly;
• remote cores keep spinning and waste time.

Thus locality affects who wins the lock.

TAS locks have no built-in fairness.

The lock tends to stay near the core or NUMA node where it was recently used.

This can be good for locality but bad for fairness.

Some cores may make progress while others spin for a long time.

Ticket locks use the same idea as taking a number in a queue.

The algorithm has:

• an arrival_counter, which gives out ticket numbers;
• a now_serving counter, which says whose turn it is.

Each arriving thread takes a ticket and waits until its number is served.

Ticket locks provide:

• FIFO access;
• fairness;
• no starvation under normal assumptions;
• predictable acquisition order.

This solves the unfairness problem of TAS locks.

All waiting cores spin on now_serving.

When the lock owner unlocks, it increments now_serving.

That write requires exclusive ownership of the cache line.

Therefore:

1. all other shared copies of now_serving are invalidated;
2. all waiting cores soon read it again;
3. the cache line gets reshared;
4. the same happens at every unlock.

This creates repeated global cache coherence traffic.

Ticket locks force FIFO order.

Under heavy contention, the lock may move from:

• one NUMA node;
• to a far-away NUMA node;
• then to another distant core;
• and so on.

But the protected data structure also has a cache footprint.

So moving lock ownership far across the machine also moves the protected data’s cache lines.

This hurts locality.

Ticket locks improve fairness but can reduce locality and scalability.

TAS locks may be unfair but can keep the lock near a local group of cores.

There is no universally best lock.

The right choice depends on workload and contention.

Ticket locks make all waiters spin on one shared location.

MCS locks avoid this by making every waiting core spin on a different memory location.

This gives local spinning.

The lock is named after Mellor-Crummey and Scott.

An MCS lock builds a linked list of waiting cores.

Each waiter has a queue node:

struct qnode {
    volatile struct qnode *next;
    volatile bool blocked;
};

The lock itself stores a pointer to the last node in the queue:

typedef struct {
    volatile struct qnode *last;
} mcs_lock_t;

Each waiter spins on its own blocked flag.

Simplified MCS lock acquisition:

void mcs_lock(struct qnode *self, mcs_lock_t *lock) {
    struct qnode *prev;

    self->next = NULL;

    prev = atomic_fetch_and_store(&lock->last, self);

    if (prev != NULL) {
        self->blocked = true;
        memory_barrier();

        prev->next = self;

        while (self->blocked)
            /* busy-wait */;
    } else {
        memory_barrier();
    }
}

Explanation:

1. The caller initializes its node.
2. It atomically swaps itself into the tail pointer.
3. If the old tail was NULL, the lock was free and the caller owns it.
4. If there was a predecessor, the caller:
   • marks itself blocked;
   • links itself after the predecessor;
   • spins on its own blocked field.

Simplified MCS unlock:

void mcs_unlock(struct qnode *self, mcs_lock_t *lock) {
    memory_barrier();

    if (self->next == NULL) {
        if (compare_and_swap(&lock->last, self, NULL))
            return;
        else
            while (self->next == NULL)
                /* busy-wait */;
    }

    self->next->blocked = false;
}

There are two cases when self->next == NULL:

1. There really is no waiter.
2. A new waiter is currently enqueuing but has not yet linked itself through prev->next.

The CAS distinguishes these cases.

If CAS succeeds:

• no one is waiting;
• the lock becomes free.

If CAS fails:

• someone is arriving;
• the unlocker waits until the successor’s node is linked;
• then it clears the successor’s blocked flag.

Only the next waiter spins on the cache line that will be modified.

When the lock owner releases the lock, it writes only to the successor’s blocked flag.

Other waiters continue spinning on their own local flags and do not need to observe the unlock yet.

This avoids the global invalidation wave caused by ticket locks.

MCS locks need a queue node per lock acquisition.

This changes the API compared with TAS locks.

Two allocation strategies:

1. Pre-allocate nodes:
   • one per processor;
   • also one per lock nesting depth if locks can be nested.
2. Allocate the node on the stack:

void foo(bar_obj_t *obj) {
    struct qnode my_node;

    mcs_lock(&my_node, &obj->lock);
    /* critical section */
    mcs_unlock(&my_node, &obj->lock);
}

Stack allocation is elegant if the critical section stays within the function.

MCS locks provide:

• FIFO fairness;
• local spinning;
• better scalability under contention;
• less global cache coherence traffic.

Costs:

• more complex implementation;
• larger per-acquisition state;
• different API;
• queue nodes must be managed correctly.

Reader-writer locks distinguish between:

• read-side critical sections;
• write-side critical sections.

Typical API:

read_lock();
/* read shared state */
read_unlock();

write_lock();
/* modify shared state */
write_unlock();

Multiple readers may enter together.

Writers require exclusive access.

RW locks make sense for skewed access patterns:

• many reads;
• few writes.

Example:

• routing table;
• mostly read by packet processing;
• updated rarely.

Readers can proceed concurrently, which improves performance compared with a simple exclusive lock.

RW locks are more complex than ordinary spin locks.

If a data structure is frequently written, or if most operations are read-modify-write, RW locks bring little benefit.

They may be slower because they must manage more cases.

So ordinary locks are still preferable for write-heavy data.

In any RW lock design, a reader must make its presence known.

Reason:

• a writer must know whether readers are currently inside;
• otherwise it might modify data while readers are using it.

This usually means the reader updates some lock state.

The key problem:

A reader becomes a writer with respect to the lock state.

The protected data may be read-only most of the time.

If no one writes it, the data can live in shared cache state on many cores with excellent locality.

But with an RW lock:

• each reader must acquire the read lock;
• acquiring the read lock writes or atomically updates lock metadata;
• this causes cache invalidations and coherence traffic.

So even a read-mostly workload can suffer because every read-side critical section writes to shared lock state.

Non-blocking synchronization tries to synchronize without mutual exclusion.

Instead of locking a critical section, the data structure is designed so concurrent operations are safe.

Benefits:

• no conventional blocking on a lock;
• no deadlock;
• often better scalability;
• sometimes no starvation, depending on guarantee.

For a single producer and single consumer, a bounded FIFO queue can be implemented as a ring buffer:

char buffer[BUF_SIZE];

int head = 0; /* next slot that can be read */
int tail = 0; /* next slot that can be written */

The producer advances tail.

The consumer advances head.

Because there is only one producer and one consumer, each index has a single writer.

bool TryProduce(char item) {
    if ((tail + 1) % BUF_SIZE == head) {
        return false; /* buffer full */
    } else {
        buffer[tail] = item;
        memory_barrier();
        tail = (tail + 1) % BUF_SIZE;
        return true;
    }
}

The full condition prevents overwriting unread data.

bool TryConsume(char *item) {
    if (tail == head) {
        return false; /* buffer empty */
    } else {
        *item = buffer[head];
        memory_barrier();
        head = (head + 1) % BUF_SIZE;
        return true;
    }
}

The empty condition is tail = head=.

There is no retry loop and no spinning.

Each operation completes in a bounded number of steps.

This works because of the restrictions:

• fixed-size buffer;
• single producer;
• single consumer.

The lecture mentions this as practical for situations such as passing data from an interrupt handler to a processing thread.

A LIFO queue is essentially a stack.

The lecture uses a linked list:

struct QElem {
    struct Item *item;
    struct QElem *next;
};

struct QElem *last = NULL;

Here last points to the top of the stack.

void AppendToTail(struct Item *item) {
    struct QElem *new_elem = malloc(sizeof(struct QElem));
    new_elem->item = item;

    do {
        new_elem->next = ldl(&last);
    } while (!stc(&last, new_elem));
}

The algorithm:

1. allocate a new element;
2. read the current top using load-linked;
3. point the new element to the old top;
4. try to publish the new element with store-conditional;
5. retry if another thread modified last.

struct QElem *RemoveTail(void) {
    struct QElem *current_tail;

    do {
        current_tail = ldl(&last);

        if (current_tail == NULL)
            return NULL;
    } while (!stc(&last, current_tail->next));

    return current_tail;
}

The algorithm:

1. read the current top;
2. if the stack is empty, return NULL;
3. try to update last to the next element;
4. retry if another thread modified last.

If stc fails, that means another thread modified the shared pointer.

So at least one thread has made progress.

The current thread may retry, but globally the data structure progresses.

A subtle problem appears after popping an element.

It is not safe to immediately do:

struct QElem *x = RemoveTail();
if (x != NULL) {
    struct Item *v = x->item;
    free(x);
}

Reason:

• another thread may have loaded a pointer to x;
• it may later access x->next;
• if x was already freed, this is a use-after-free.

This is one of the central difficulties of lock-free programming.

In languages with garbage collection, such as Java or C#, this problem is easier.

A node is not reclaimed while another thread may still reference it.

This is why many lock-free data structures are easier to use in garbage-collected languages.

In a kernel, there is usually no full garbage collector.

So memory reclamation becomes the programmer’s problem.

The lecture presents a general lock-free pattern for arbitrary objects.

The goal is to make readers very cheap.

There is a shared pointer:

struct any_object *current_version;

Readers simply read the current version.

Writers create and publish a new version.

void do_update(void) {
    struct any_object *cpy = alloc_object();

    do {
        struct any_object *old = current_version;

        memcpy(cpy, old, sizeof(*old));

        cpy->some_field = ...;

    } while (!CAS(&current_version, old, cpy));
}

The writer:

1. allocates a new object;
2. reads the current pointer;
3. copies the old object;
4. modifies the copy;
5. tries to atomically publish the copy;
6. retries if another writer published first.

Readers do not take a lock.

They just:

1. dereference current_version exactly once;
2. use the local pointer they obtained.

No reader needs to update shared lock state.

This avoids the RW-lock problem where readers become writers with respect to lock metadata.

This approach is good when:

• reads are frequent;
• writes are rare;
• objects are not too expensive to copy.

Costs include:

• copying overhead on writes;
• retry overhead under writer contention;
• memory reclamation problems;
• possible ABA problem.

The universal object pattern assumes:

If the pointer value is the same, then the version is the same.

Equivalently:

Different logical versions must have different addresses.

But memory allocators may reuse addresses.

This can break CAS-based algorithms.

Suppose current_version initially points to object A.

Thread 1 reads A and stalls.

Meanwhile:

1. thread 2 replaces A with B;
2. A is freed;
3. another allocation reuses address A;
4. current_version again points to address A, but it is logically a different version.

Thread 1 resumes and performs CAS:

• it checks whether current_version is still A;
• the address is indeed A;
• CAS succeeds;
• but the object changed from A to B to A in the meantime.

The thread failed to notice the intermediate change.

This is the ABA problem.

In a counter example:

• original A has counter 123;
• another thread updates to B with counter 124;
• another update reuses A with counter 125;
• a stalled thread may publish a stale update based on old A;
• the counter can go backwards or lose updates.

So pointer equality alone is not enough to identify object versions if addresses can be reused.

The lecture uses ABA to motivate RCU, but common general techniques include:

• version counters or tagged pointers;
• avoiding address reuse while old references may exist;
• hazard pointers;
• epoch-based reclamation;
• garbage collection;
• RCU-style delayed reclamation.

The central idea is: do not reuse memory too early.

RCU stands for Read-Copy-Update.

Its goals are:

• extremely cheap read-side critical sections;
• better reader scalability than RW locks;
• safe memory reclamation;
• avoidance of ABA-style address reuse problems through delayed freeing;
• use execution history to decide when old versions are safe to reclaim.

RCU is similar to the reader-friendly universal object pattern:

1. A shared pointer indicates the current version.
2. Readers dereference the current-version pointer exactly once.
3. Readers do not make their presence known.
4. Writers make a copy.
5. Writers update the copy.
6. Writers publish the copy.
7. Old versions are not freed immediately.
8. Deallocation is delayed until the system knows no reader can still be using the old version.

The main challenge is:

When is it safe to free the old version?

A quiescent state is a point where a processor is definitely not using any RCU-protected resource.

In other words, the processor is not inside a read-side critical section that could still hold a pointer to the old version.

A grace period ends when every processor has passed through a quiescent state.

After a grace period ends, any object replaced before the start of that grace period can be safely freed.

Reason:

• any reader that could have seen the old pointer must have finished;
• every processor has reached a state where it cannot still be using the old version.

Suppose:

• initially current_version points to old;
• a writer publishes new;
• some readers may still hold pointers to old;
• new readers will see new.

The old version cannot be freed immediately.

The OS waits until all CPUs pass through quiescent states.

After that, no reader can still hold old.

Then old can be reclaimed.

A simple kernel implementation uses non-preemptive readers.

Read-side critical section:

• disable preemption, or otherwise ensure the reader cannot be context-switched.

Then:

• a context switch is a quiescent state;
• because a CPU cannot context-switch while inside a non-preemptive RCU read-side critical section.

Therefore:

A grace period ends after every processor has context-switched at least once.

This makes read-side critical sections essentially free:

• no lock acquisition;
• no shared counter update;
• no cache-line bouncing on read lock metadata.

The cost is shifted to writers and reclamation latency.

Spin locks interact with:

• cache coherence;
• NUMA topology;
• cache-line ownership;
• interconnect bandwidth;
• locality.

Therefore, spin lock performance can be non-obvious.

Benchmarking is important.

TAS locks:

• simple;
• locality-friendly in some cases;
• unfair under contention.

Ticket locks:

• fair;
• FIFO;
• can scale poorly due to global spinning and poor locality.

MCS locks:

• fair;
• local spinning;
• better under contention;
• more complex and require queue nodes.

RW locks are useful for read-mostly data.

But every reader must make its presence known.

This usually writes shared lock state.

Thus RW locks can scale poorly on large machines because they turn readers into writers with respect to the lock metadata.

Non-blocking algorithms avoid ordinary locking.

But they often introduce other problems:

• retry loops;
• object copying;
• ABA;
• memory reclamation;
• delayed freeing;
• subtle correctness requirements.

A common pattern is atomic publish:

1. build a new version;
2. publish it atomically;
3. ensure old versions are reclaimed safely.

RCU gives extremely cheap reads.

But it pays through:

• copy/update overhead on writers;
• delayed reclamation;
• more complex lifetime management;
• grace-period tracking.

The final lesson is:

There is no universally best synchronization mechanism. Multicore synchronization is a set of tradeoffs among fairness, locality, read cost, write cost, copying overhead, and reclamation latency.