# Mutexes and Semaphores on Linux, C++, x86, ARM, and DPDK

*Published: 2026-07-26*

A practical implementation guide covering portable C++ synchronization, Linux futexes, Linux-kernel locking, architecture-specific atomic behavior, and DPDK data-plane synchronization.

**Combined Markdown edition:** July 26, 2026

> **Scope:** The examples are intended for learning, prototyping, and architecture discussions. For production application code, prefer standard-library, POSIX, Linux-kernel, or DPDK primitives unless a custom implementation is genuinely required.
>
> **Version note:** DPDK APIs and build options vary by release. Verify names and availability against the headers and documentation installed with your target DPDK version.

> **TOC mode:** This edition uses only plain Markdown headings and links. No inline HTML anchors are required. The webpage renderer must generate IDs for headings.

## Contents

1. [Synchronization concepts](#synchronization-concepts)
2. [Portable Cpp implementation](#portable-cpp-implementation)
3. [Linux futex foundation](#linux-futex-foundation)
4. [Futex based mutex](#futex-based-mutex)
5. [Futex based counting semaphore](#futex-based-counting-semaphore)
6. [Using the futex primitives](#using-the-futex-primitives)
7. [Memory ordering rationale](#memory-ordering-rationale)
8. [X86 and ARM instruction mapping](#x86-and-arm-instruction-mapping)
9. [Production limitations of the educational futex code](#production-limitations-of-the-educational-futex-code)
10. [Linux kernel implementation](#linux-kernel-implementation)
11. [DPDK synchronization model](#dpdk-synchronization-model)
12. [Minimal DPDK spin lock example](#minimal-dpdk-spin-lock-example)
13. [Cpp wrappers for DPDK locks](#cpp-wrappers-for-dpdk-locks)
14. [DPDK polling counting semaphore](#dpdk-polling-counting-semaphore)
15. [Using the DPDK primitives](#using-the-dpdk-primitives)
16. [Runnable DPDK EAL lcore example](#runnable-dpdk-eal-lcore-example)
17. [Building and running the DPDK example](#building-and-running-the-dpdk-example)
18. [DPDK behavior on X86 and ARM](#dpdk-behavior-on-x86-and-arm)
19. [Choosing the appropriate primitive](#choosing-the-appropriate-primitive)
20. [Comparison matrix](#comparison-matrix)
21. [References](#references)

---

## Synchronization concepts

### Mutex

A mutex protects a critical section so that only one execution context owns the lock at a time.

Important properties:

- A mutex normally has ownership semantics.
- The thread or lcore that locks it should unlock it.
- It is appropriate for protecting shared data structures and invariants.
- A sleeping mutex blocks or parks a waiter; a spin mutex consumes CPU while waiting.

### Counting semaphore

A counting semaphore represents a number of permits. Acquiring consumes one permit; releasing returns one or more permits.

Important properties:

- A semaphore generally does not have ownership semantics.
- A different thread may release a permit.
- It is appropriate for bounded concurrency, resource pools, and producer/consumer coordination.
- A counting semaphore initialized to one can resemble a binary lock, but it is not a mutex because ownership and error semantics differ.

### Sleeping versus polling synchronization

| Behavior | Sleeping primitive | Polling primitive |
|---|---|---|
| Waiter action | Parks in the OS or scheduler | Repeatedly checks shared state |
| CPU consumption while waiting | Low | Potentially one full CPU |
| Typical use | General-purpose threads, control plane, long waits | Dedicated data-plane cores, very short waits |
| Examples | `std::mutex`, futex-backed locks, POSIX semaphores | `rte_spinlock_t`, ticket locks, custom DPDK polling semaphore |

---

## Portable Cpp implementation

For application code, use `std::mutex` and, with C++20 or later, `std::counting_semaphore`. The same source can be compiled for Linux/x86-64 and Linux/AArch64; the standard library and compiler select the platform implementation.

```cpp
#include <mutex>
#include <semaphore>

std::mutex data_mutex;

// At most four operations may execute concurrently.
std::counting_semaphore<64> available_slots{4};

int shared_value = 0;

void worker()
{
    available_slots.acquire();

    {
        std::lock_guard<std::mutex> guard(data_mutex);
        ++shared_value;
    }

    available_slots.release();
}
```

Build on x86-64:

```bash
g++ -std=c++20 -O2 -pthread example.cpp -o example
```

Cross-compile for AArch64:

```bash
aarch64-linux-gnu-g++ -std=c++20 -O2 -pthread example.cpp -o example-arm64
```

### Why this is normally the preferred implementation

- The standard library supplies the correct architecture-specific atomic operations.
- The implementation can use Linux futexes or another optimized runtime mechanism.
- It handles details such as spurious wake-ups, scheduling, ABI compatibility, and library integration.
- `std::lock_guard` provides exception-safe unlocking through RAII.

---

## Linux futex foundation

The following is Linux **user-space** code, not Linux-kernel code.

A futex is based on a 32-bit userspace word. The uncontended path executes with ordinary atomic instructions. A thread invokes `FUTEX_WAIT` only when it may need to sleep, and an unlock or semaphore release invokes `FUTEX_WAKE` when sleepers may need to be resumed.

The futex wait operation compares the word with an expected value before blocking. If the value has already changed, the kernel returns without sleeping. This compare-and-block behavior is what prevents a wake-up from being lost between a userspace check and entry into the kernel.

Futex waits can return because of a wake operation, a value mismatch, a signal, or a spurious event. The userspace condition must therefore always be rechecked in a loop.

### Shared futex support code

```cpp
#include <atomic>
#include <cerrno>
#include <climits>
#include <cstdint>
#include <cstdlib>

#include <linux/futex.h>
#include <sys/syscall.h>
#include <unistd.h>

/*
 * A futex is always a 32-bit word on Linux, including 64-bit systems.
 *
 * atomic_ref gives C++ atomic semantics to the word while leaving the
 * underlying object suitable for the futex syscall.
 */
struct FutexWord {
    static_assert(
        std::atomic_ref<std::uint32_t>::is_always_lock_free,
        "This implementation requires lock-free 32-bit atomics");

    explicit constexpr FutexWord(std::uint32_t initial = 0) noexcept
        : value(initial)
    {
    }

    alignas(std::atomic_ref<std::uint32_t>::required_alignment)
        std::uint32_t value;

    [[nodiscard]]
    std::atomic_ref<std::uint32_t> atomic() noexcept
    {
        return std::atomic_ref<std::uint32_t>(value);
    }
};

namespace detail {

int futex_wait(FutexWord& word, std::uint32_t expected) noexcept
{
    return static_cast<int>(
        ::syscall(
            SYS_futex,
            &word.value,
            FUTEX_WAIT_PRIVATE,
            expected,
            nullptr,
            nullptr,
            0));
}

int futex_wake(FutexWord& word, int number_to_wake) noexcept
{
    return static_cast<int>(
        ::syscall(
            SYS_futex,
            &word.value,
            FUTEX_WAKE_PRIVATE,
            number_to_wake,
            nullptr,
            nullptr,
            0));
}

/*
 * Return when:
 *
 * - another thread wakes us;
 * - the futex value changed before we entered the kernel; or
 * - a spurious wake-up occurred.
 *
 * The caller must always recheck its userspace condition.
 */
void wait_while_equal(
    FutexWord& word,
    std::uint32_t expected) noexcept
{
    for (;;) {
        const int result = futex_wait(word, expected);

        if (result == 0) {
            return;
        }

        if (errno == EAGAIN) {
            // Value no longer equals expected.
            return;
        }

        if (errno == EINTR) {
            // Interrupted by a signal; retry the wait operation.
            continue;
        }

        // EFAULT, EINVAL, and similar errors indicate a programming error.
        std::abort();
    }
}

void wake(FutexWord& word, int number_to_wake) noexcept
{
    if (futex_wake(word, number_to_wake) < 0) {
        std::abort();
    }
}

} // namespace detail
```

`FUTEX_WAIT_PRIVATE` and `FUTEX_WAKE_PRIVATE` are suitable only when all participants are threads in the same process. For interprocess synchronization, place the futex word in shared memory and use the corresponding non-private futex operations.

---

## Futex based mutex

The mutex uses this state machine:

```text
0 = unlocked
1 = locked, with no known sleeping waiters
2 = locked, with possible sleeping waiters
```

```cpp
class FutexMutex {
public:
    FutexMutex() = default;

    FutexMutex(const FutexMutex&) = delete;
    FutexMutex& operator=(const FutexMutex&) = delete;

    void lock() noexcept
    {
        auto state = state_.atomic();

        /*
         * Fast path:
         *
         * Try to change 0 -> 1 entirely in userspace.
         */
        std::uint32_t expected = 0;

        if (state.compare_exchange_strong(
                expected,
                1,
                std::memory_order_acquire,
                std::memory_order_relaxed)) {
            return;
        }

        /*
         * Slow path:
         *
         * Set state to 2, indicating that a sleeping waiter may exist.
         */
        for (;;) {
            if (expected != 2) {
                expected =
                    state.exchange(2, std::memory_order_acquire);
            }

            /*
             * exchange() returned 0, so this thread changed 0 -> 2
             * and acquired the mutex.
             */
            if (expected == 0) {
                return;
            }

            /*
             * Sleep only while the state still equals 2.
             *
             * If an unlock happened just before this syscall, the
             * kernel sees a value other than 2 and returns EAGAIN
             * rather than putting us to sleep.
             */
            detail::wait_while_equal(state_, 2);

            /*
             * Try to acquire the mutex while keeping it in the
             * contended state. Keeping state == 2 means that the
             * next unlock will wake another waiter.
             */
            expected = 0;

            if (state.compare_exchange_strong(
                    expected,
                    2,
                    std::memory_order_acquire,
                    std::memory_order_relaxed)) {
                return;
            }
        }
    }

    [[nodiscard]]
    bool try_lock() noexcept
    {
        auto state = state_.atomic();
        std::uint32_t expected = 0;

        return state.compare_exchange_strong(
            expected,
            1,
            std::memory_order_acquire,
            std::memory_order_relaxed);
    }

    void unlock() noexcept
    {
        auto state = state_.atomic();

        /*
         * Uncontended case:
         *
         *     1 -> 0
         *
         * No syscall is needed.
         *
         * Contended case:
         *
         *     2 -> 1 -> 0
         *
         * Then wake one possible waiter.
         */
        if (state.fetch_sub(1, std::memory_order_release) == 1) {
            return;
        }

        state.store(0, std::memory_order_release);
        detail::wake(state_, 1);
    }

private:
    FutexWord state_{0};
};
```

### Mutex operation summary

1. Attempt an atomic `0 -> 1` transition.
2. If successful, enter the critical section without a syscall.
3. If the mutex is busy, mark it contended with state `2`.
4. Call `FUTEX_WAIT` while state remains `2`.
5. After waking, retry acquisition.
6. On unlock, use the fast userspace path when state was `1`.
7. When state indicates contention, publish state `0` and wake one waiter.

---

## Futex based counting semaphore

This version stores the number of available permits in the futex word.

```cpp
class FutexSemaphore {
public:
    explicit FutexSemaphore(std::uint32_t initial_count) noexcept
        : count_(initial_count)
    {
    }

    FutexSemaphore(const FutexSemaphore&) = delete;
    FutexSemaphore& operator=(const FutexSemaphore&) = delete;

    void acquire() noexcept
    {
        auto count = count_.atomic();

        std::uint32_t current =
            count.load(std::memory_order_relaxed);

        for (;;) {
            /*
             * Try to consume one available permit.
             */
            while (current != 0) {
                if (count.compare_exchange_weak(
                        current,
                        current - 1,
                        std::memory_order_acquire,
                        std::memory_order_relaxed)) {
                    return;
                }

                /*
                 * On CAS failure, current is automatically replaced
                 * with the value that was actually observed.
                 */
            }

            /*
             * There are no permits. Sleep while count is still zero.
             */
            detail::wait_while_equal(count_, 0);

            /*
             * Recheck after every wake-up.
             */
            current = count.load(std::memory_order_relaxed);
        }
    }

    [[nodiscard]]
    bool try_acquire() noexcept
    {
        auto count = count_.atomic();

        std::uint32_t current =
            count.load(std::memory_order_relaxed);

        while (current != 0) {
            if (count.compare_exchange_weak(
                    current,
                    current - 1,
                    std::memory_order_acquire,
                    std::memory_order_relaxed)) {
                return true;
            }
        }

        return false;
    }

    /*
     * The caller must ensure that adding update does not overflow
     * the 32-bit permit count.
     */
    void release(std::uint32_t update = 1) noexcept
    {
        if (update == 0) {
            return;
        }

        if (update > static_cast<std::uint32_t>(INT_MAX)) {
            std::abort();
        }

        count_.atomic().fetch_add(
            update,
            std::memory_order_release);

        /*
         * Correctness-first implementation: wake at most one waiter
         * for each newly added permit.
         *
         * This deliberately makes a syscall on every release.
         * Production implementations generally track waiter state
         * to avoid a syscall when no thread is asleep.
         */
        detail::wake(count_, static_cast<int>(update));
    }

private:
    FutexWord count_;
};
```

### Semaphore operation summary

1. Read the permit count.
2. If the count is nonzero, atomically decrement it.
3. If the count is zero, wait with `FUTEX_WAIT` while it remains zero.
4. Recheck the count after every return from the syscall.
5. A release atomically increments the count and wakes up to the number of newly available permits.

---

## Using the futex primitives

```cpp
FutexMutex mutex;
FutexSemaphore slots{4};

int shared_value = 0;

void operation()
{
    slots.acquire();

    mutex.lock();
    ++shared_value;
    mutex.unlock();

    slots.release();
}
```

A safer C++ interface can add a lock-guard-compatible wrapper or use `std::lock_guard<FutexMutex>` directly because `FutexMutex` supplies `lock()` and `unlock()`:

```cpp
#include <mutex>

FutexMutex mutex;
int shared_value = 0;

void guarded_operation()
{
    std::lock_guard<FutexMutex> guard(mutex);
    ++shared_value;
}
```

---

## Memory ordering rationale

### Mutex acquisition

Successful mutex acquisition uses `std::memory_order_acquire`.

Acquire semantics prevent reads and writes in the protected critical section from being reordered before the successful lock operation.

### Mutex release

Unlock uses `std::memory_order_release`.

Release semantics publish writes performed in the critical section before another execution context successfully acquires the mutex.

### Semaphore acquisition and release

- A successful permit decrement uses acquire semantics.
- Adding permits uses release semantics.
- Relaxed loads and failed compare-and-exchange operations do not themselves establish synchronization; the successful acquire operation does.

### Why x86 and ARM differ internally

The C++ memory model is architecture-independent, but the generated instructions differ:

- x86 has a relatively strong hardware memory model, so a release store can normally be an ordinary aligned store.
- ARM has a weaker memory model, so the compiler uses acquire/release instructions or explicit barriers where necessary.

The source-level memory orders must still be correct even when the x86 assembly appears to require fewer barriers.

---

## X86 and ARM instruction mapping

The application source does not need separate x86 and ARM implementations. The compiler lowers C++ atomic operations to the appropriate target instructions.

### Representative mapping

| C++ atomic operation | x86-64 | AArch64 without LSE | AArch64 with LSE |
|---|---|---|---|
| Acquire compare-and-exchange | `lock cmpxchg` | `ldaxr` / `stxr` retry loop | `casa` |
| Acquire exchange | `xchg` | `ldaxr` / `stxr` retry loop | `swpa` |
| Release store | Ordinary `mov` store | `stlr` | `stlr` |
| Release fetch-add | `lock xadd` | `ldxr` / `stlxr` retry loop | `ldaddl` |

The exact output depends on compiler version, optimization level, architecture flags, surrounding code, and whether the target guarantees Large System Extensions (LSE).

### x86-64 mutex fast path

A representative `0 -> 1` compare-and-exchange can look like this:

```asm
xor     eax, eax              ; expected = 0
mov     edx, 1                ; desired = 1
lock cmpxchg dword ptr [rdi], edx
sete    al                    ; return whether comparison succeeded
```

A release unlock can be an ordinary store:

```asm
mov     dword ptr [rdi], 0
```

The locked read-modify-write instruction provides the atomicity and ordering needed for acquisition. x86's memory model permits a plain aligned store for the release operation.

### Baseline AArch64 compare-and-exchange

Without LSE, a compiler normally uses an exclusive load/store retry loop:

```asm
retry:
    ldaxr   w1, [x0]          // load-exclusive with acquire semantics
    cbnz    w1, failed

    mov     w2, #1
    stxr    w3, w2, [x0]      // store only if exclusive monitor is valid
    cbnz    w3, retry

    mov     w0, #1
    ret

failed:
    clrex
    mov     w0, #0
    ret
```

Release unlock:

```asm
stlr    wzr, [x0]             // store-release zero
ret
```

### AArch64 with LSE

With ARMv8.1-A LSE, the lock operation can use a hardware compare-and-swap instruction:

```asm
mov     w1, #0
mov     w2, #1
casa    w1, w2, [x0]          // compare-and-swap, acquire semantics
```

The exact operand behavior and register allocation depend on compiler output, but the important distinction is that LSE supplies single-instruction atomic read-modify-write operations instead of a software retry loop built from exclusive accesses.

### ARMv7

A 32-bit ARMv7 implementation generally uses `LDREX` and `STREX` retry loops together with appropriate `DMB` barriers because ARMv7 does not provide AArch64's `LDAXR` and `STLR` forms.

Conceptually:

```asm
retry:
    ldrex   r1, [r0]
    cmp     r1, #0
    bne     failed

    mov     r2, #1
    strex   r3, r2, [r0]
    cmp     r3, #0
    bne     retry

    dmb
    mov     r0, #1
    bx      lr

failed:
    clrex
    mov     r0, #0
    bx      lr
```

This is conceptual assembly; compiler-generated sequences and barrier placement depend on the operation and requested memory order.

---

## Production limitations of the educational futex code

The futex implementations demonstrate the essential algorithms but omit production features:

- No FIFO fairness.
- No priority inheritance.
- No robust-mutex or owner-death recovery.
- No timed waits.
- No cancellation handling.
- No adaptive spinning before sleeping.
- No mutex-owner tracking.
- No recursive-lock detection.
- No permit maximum in the futex semaphore.
- The semaphore performs a wake syscall on every nonzero release.
- `FUTEX_WAIT_PRIVATE` restricts the examples to one process.
- No lifecycle protection against destruction while another thread is waiting.

`FUTEX_WAKE` does not guarantee FIFO selection of waiters. Neither the educational mutex nor the semaphore should be assumed to be fair.

For production user-space code, prefer one of the following:

```text
std::mutex
std::counting_semaphore
pthread_mutex_t
sem_t
```

For special requirements, consider the appropriate established API:

- Priority-inheritance pthread mutexes or PI futex operations.
- Robust pthread mutexes for owner-death detection.
- Process-shared pthread synchronization objects for interprocess use.
- Condition variables for predicate-based sleeping waits.

---

## Linux kernel implementation

Inside the Linux kernel, do not invoke futex syscalls. Use the kernel locking APIs.

```c
#include <linux/mutex.h>
#include <linux/semaphore.h>

static DEFINE_MUTEX(data_lock);
static struct semaphore available_slots;

static int initialize_component(void)
{
    sema_init(&available_slots, 4);
    return 0;
}

static void do_work(void)
{
    down(&available_slots);

    mutex_lock(&data_lock);
    /* Access protected kernel data. */
    mutex_unlock(&data_lock);

    up(&available_slots);
}
```

### Kernel mutex

A Linux-kernel mutex has owner semantics and is intended for mutual exclusion. Its implementation can combine:

- An atomic fast path.
- Optional optimistic spinning.
- A sleeping wait queue under contention.

### Kernel semaphore

A Linux-kernel semaphore is a counting primitive without mutex ownership semantics.

### Context restriction

Kernel mutexes and semaphores are sleeping locks. Do not acquire them in interrupt context, with preemption constraints that prohibit sleeping, or while holding a spin lock that requires an atomic context.

In non-sleepable kernel contexts, the relevant primitive is normally a spin lock or another context-specific synchronization mechanism.

---

## DPDK synchronization model

DPDK applications commonly dedicate logical cores to poll-mode processing. For very short data-plane critical sections, DPDK provides busy-wait synchronization primitives such as:

- `rte_spinlock_t`
- `rte_ticketlock_t`
- `rte_mcslock_t`

A DPDK spin lock is not a sleeping Linux mutex. A waiting lcore continues polling until the lock becomes available.

### DPDK and semaphores

The DPDK API is centered on data-plane polling and does not necessarily provide a general-purpose semaphore abstraction equivalent to `std::counting_semaphore` in every release. The implementation below is therefore an application-level polling counting semaphore built from DPDK atomic and wait/pause facilities.

Use such a polling semaphore only when:

- The participating lcores are intentionally dedicated to polling.
- Permit hold times are short and bounded.
- Oversubscription is avoided.
- Burning CPU while waiting is acceptable.

Use a sleeping OS primitive for control-plane threads, long waits, blocking I/O, or oversubscribed systems.

---

## Minimal DPDK spin lock example

```c
#include <rte_spinlock.h>

static rte_spinlock_t data_lock = RTE_SPINLOCK_INITIALIZER;
static unsigned int shared_value;

static void update_shared_value(void)
{
    rte_spinlock_lock(&data_lock);

    ++shared_value;

    rte_spinlock_unlock(&data_lock);
}
```

The uncontended lock operation is an architecture-appropriate atomic acquisition. Under contention, the waiter repeatedly observes the lock and executes a pause or wait hint. Unlock publishes the protected writes with release semantics.

---

## Cpp wrappers for DPDK locks

The following wrappers satisfy the C++ `BasicLockable` interface and can be used with `std::lock_guard`.

```cpp
// dpdk_sync.hpp
#pragma once

#include <cassert>
#include <cstdint>

#include <rte_common.h>
#include <rte_pause.h>
#include <rte_spinlock.h>
#include <rte_stdatomic.h>
#include <rte_ticketlock.h>

/*
 * Fast, non-FIFO DPDK spin mutex.
 *
 * Suitable for:
 *   - very short critical sections;
 *   - dedicated EAL lcores;
 *   - low or moderate contention.
 *
 * It does not put the calling thread to sleep.
 */
class alignas(RTE_CACHE_LINE_SIZE) DpdkSpinMutex final {
public:
    DpdkSpinMutex() noexcept
    {
        rte_spinlock_init(&lock_);
    }

    DpdkSpinMutex(const DpdkSpinMutex&) = delete;
    DpdkSpinMutex& operator=(const DpdkSpinMutex&) = delete;

    void lock() noexcept
    {
        rte_spinlock_lock(&lock_);
    }

    [[nodiscard]]
    bool try_lock() noexcept
    {
        return rte_spinlock_trylock(&lock_) != 0;
    }

    void unlock() noexcept
    {
        rte_spinlock_unlock(&lock_);
    }

private:
    rte_spinlock_t lock_;
};

/*
 * Fairer DPDK mutex.
 *
 * Waiting lcores receive tickets and acquire the lock in ticket order.
 * This can reduce starvation but may cost more under contention.
 */
class alignas(RTE_CACHE_LINE_SIZE) DpdkTicketMutex final {
public:
    DpdkTicketMutex() noexcept
    {
        rte_ticketlock_init(&lock_);
    }

    DpdkTicketMutex(const DpdkTicketMutex&) = delete;
    DpdkTicketMutex& operator=(const DpdkTicketMutex&) = delete;

    void lock() noexcept
    {
        rte_ticketlock_lock(&lock_);
    }

    [[nodiscard]]
    bool try_lock() noexcept
    {
        return rte_ticketlock_trylock(&lock_) != 0;
    }

    void unlock() noexcept
    {
        rte_ticketlock_unlock(&lock_);
    }

private:
    rte_ticketlock_t lock_;
};
```

### Using a wrapper with `std::lock_guard`

```cpp
#include <mutex>

DpdkSpinMutex mutex;
int shared_value = 0;

void update()
{
    std::lock_guard<DpdkSpinMutex> guard(mutex);
    ++shared_value;
}
```

### Cache-line alignment

Aligning lock objects to `RTE_CACHE_LINE_SIZE` can reduce false sharing when adjacent data is frequently modified by different lcores. Alignment alone does not fix contention on the lock's own cache line; it prevents unrelated objects from sharing that line.

### Optional MCS lock for heavy contention

An MCS lock gives each waiter its own node and makes it spin on local state, reducing shared-cache-line bouncing. The node must remain alive for the complete lock hold, and each concurrent acquirer needs a distinct node.

```cpp
#include <rte_mcslock.h>

static rte_mcslock_t* global_lock = nullptr;

void critical_section()
{
    // Each thread or lcore must use its own node.
    rte_mcslock_t local_node{};

    rte_mcslock_lock(&global_lock, &local_node);

    /* Very short protected work. */

    rte_mcslock_unlock(&global_lock, &local_node);
}
```

---

## DPDK polling counting semaphore

This semaphore stores the number of available permits in a DPDK atomic variable.

```cpp
// Add this to dpdk_sync.hpp

class alignas(RTE_CACHE_LINE_SIZE) DpdkCountingSemaphore final {
public:
    /*
     * Preconditions:
     *
     *   maximum > 0
     *   initial <= maximum
     */
    DpdkCountingSemaphore(
        std::uint32_t initial,
        std::uint32_t maximum) noexcept
        : maximum_(maximum)
    {
        assert(maximum_ > 0);
        assert(initial <= maximum_);

        rte_atomic_store_explicit(
            &count_,
            initial,
            rte_memory_order_relaxed);
    }

    DpdkCountingSemaphore(const DpdkCountingSemaphore&) = delete;
    DpdkCountingSemaphore&
    operator=(const DpdkCountingSemaphore&) = delete;

    /*
     * Consume one permit.
     *
     * This is a polling operation. It does not invoke futex(), sleep(),
     * or any other Linux blocking syscall.
     */
    void acquire() noexcept
    {
        for (;;) {
            if (try_acquire()) {
                return;
            }

            /*
             * Wait until count != 0.
             *
             * The mask covers the entire 32-bit value. The wait uses a
             * relaxed load because the successful CAS below performs
             * the actual acquire operation.
             */
            RTE_WAIT_UNTIL_MASKED(
                &count_,
                UINT32_MAX,
                !=,
                0u,
                rte_memory_order_relaxed);
        }
    }

    /*
     * Try to consume one permit without waiting.
     */
    [[nodiscard]]
    bool try_acquire() noexcept
    {
        std::uint32_t observed =
            rte_atomic_load_explicit(
                &count_,
                rte_memory_order_relaxed);

        while (observed != 0) {
            if (rte_atomic_compare_exchange_weak_explicit(
                    &count_,
                    &observed,
                    observed - 1,
                    rte_memory_order_acquire,
                    rte_memory_order_relaxed)) {
                return true;
            }

            /*
             * On CAS failure, observed is updated with the value that
             * is currently stored in count_.
             */
        }

        return false;
    }

    /*
     * Return one or more permits.
     *
     * Returns false if the update would exceed maximum_. Such a failure
     * normally means the caller released more permits than it acquired.
     */
    [[nodiscard]]
    bool release(std::uint32_t update = 1) noexcept
    {
        if (update == 0) {
            return true;
        }

        if (update > maximum_) {
            return false;
        }

        std::uint32_t observed =
            rte_atomic_load_explicit(
                &count_,
                rte_memory_order_relaxed);

        for (;;) {
            if (observed > maximum_ - update) {
                return false;
            }

            if (rte_atomic_compare_exchange_weak_explicit(
                    &count_,
                    &observed,
                    observed + update,
                    rte_memory_order_release,
                    rte_memory_order_relaxed)) {
                return true;
            }
        }
    }

    /*
     * Diagnostic snapshot only. The result can become stale immediately.
     */
    [[nodiscard]]
    std::uint32_t available() const noexcept
    {
        return rte_atomic_load_explicit(
            &count_,
            rte_memory_order_relaxed);
    }

    [[nodiscard]]
    std::uint32_t maximum() const noexcept
    {
        return maximum_;
    }

private:
    RTE_ATOMIC(std::uint32_t) count_;
    const std::uint32_t maximum_;
};
```

### Why the loop is required

Multiple lcores can observe the same nonzero count. Only one can successfully perform a particular decrement. The others receive an updated observed value from the failed compare-and-exchange and retry.

### Why the wait is still polling

`RTE_WAIT_UNTIL_MASKED` repeatedly tests an atomic condition while invoking the target architecture's pause or wait mechanism. It does not turn this abstraction into a Linux sleeping semaphore.

### Fairness

The semaphore does not provide FIFO fairness. A lcore that begins retrying at the right moment can acquire a newly released permit before an older waiter.

---

## Using the DPDK primitives

```cpp
#include "dpdk_sync.hpp"

#include <cstdlib>
#include <mutex>

DpdkSpinMutex counter_mutex;

// Allow at most four lcores into the bounded operation.
DpdkCountingSemaphore operation_slots{4, 4};

std::uint64_t shared_counter = 0;

void process_operation()
{
    operation_slots.acquire();

    /*
     * Only four lcores can be in this overall operation concurrently.
     * The mutex separately protects the shared counter.
     */
    {
        std::lock_guard<DpdkSpinMutex> guard(counter_mutex);
        ++shared_counter;
    }

    if (!operation_slots.release()) {
        /*
         * A false return indicates an unbalanced release or another
         * programming error.
         */
        std::abort();
    }
}
```

### RAII permit guard

A permit guard makes release exception-safe and reduces the chance of an unbalanced acquire/release pair.

```cpp
class DpdkSemaphorePermit final {
public:
    explicit DpdkSemaphorePermit(DpdkCountingSemaphore& semaphore) noexcept
        : semaphore_(&semaphore)
    {
        semaphore_->acquire();
    }

    DpdkSemaphorePermit(const DpdkSemaphorePermit&) = delete;
    DpdkSemaphorePermit& operator=(const DpdkSemaphorePermit&) = delete;

    ~DpdkSemaphorePermit()
    {
        if (semaphore_ != nullptr) {
            const bool released = semaphore_->release();
            if (!released) {
                std::abort();
            }
        }
    }

    void release_early() noexcept
    {
        if (semaphore_ == nullptr) {
            return;
        }

        const bool released = semaphore_->release();
        if (!released) {
            std::abort();
        }

        semaphore_ = nullptr;
    }

private:
    DpdkCountingSemaphore* semaphore_;
};
```

Usage:

```cpp
void process_with_raii()
{
    DpdkSemaphorePermit permit(operation_slots);

    std::lock_guard<DpdkSpinMutex> guard(counter_mutex);
    ++shared_counter;
}
```

---

## Runnable DPDK EAL lcore example

```cpp
// dpdk_sync.cpp

#include "dpdk_sync.hpp"

#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <mutex>

#include <rte_debug.h>
#include <rte_eal.h>
#include <rte_launch.h>
#include <rte_lcore.h>

namespace {

constexpr std::uint32_t iterations_per_lcore = 100'000;

/*
 * Only two lcores may hold a semaphore permit concurrently.
 */
DpdkCountingSemaphore slots{2, 2};

/*
 * The counter itself remains protected by a mutex.
 */
DpdkSpinMutex counter_mutex;
std::uint64_t counter = 0;

int worker_main(void*)
{
    for (std::uint32_t i = 0; i < iterations_per_lcore; ++i) {
        slots.acquire();

        {
            std::lock_guard<DpdkSpinMutex> guard(counter_mutex);
            ++counter;
        }

        if (!slots.release()) {
            std::abort();
        }
    }

    return 0;
}

} // namespace

int main(int argc, char** argv)
{
    const int eal_result = rte_eal_init(argc, argv);

    if (eal_result < 0) {
        rte_exit(EXIT_FAILURE, "Failed to initialize DPDK EAL\n");
    }

    unsigned int lcore_id;

    /*
     * rte_eal_remote_launch() is called from the main lcore and launches
     * worker_main() on every enabled worker lcore.
     */
    RTE_LCORE_FOREACH_WORKER(lcore_id) {
        const int result =
            rte_eal_remote_launch(worker_main, nullptr, lcore_id);

        if (result != 0) {
            rte_exit(
                EXIT_FAILURE,
                "Failed to launch worker on lcore %u\n",
                lcore_id);
        }
    }

    /*
     * Include the main lcore in the test.
     */
    worker_main(nullptr);

    RTE_LCORE_FOREACH_WORKER(lcore_id) {
        const int worker_result = rte_eal_wait_lcore(lcore_id);

        if (worker_result < 0) {
            rte_exit(
                EXIT_FAILURE,
                "Worker on lcore %u failed\n",
                lcore_id);
        }
    }

    const std::uint64_t expected =
        static_cast<std::uint64_t>(rte_lcore_count()) *
        iterations_per_lcore;

    std::printf(
        "counter:  %llu\n"
        "expected: %llu\n",
        static_cast<unsigned long long>(counter),
        static_cast<unsigned long long>(expected));

    const int cleanup_result = rte_eal_cleanup();

    return cleanup_result == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
}
```

This example:

1. Initializes the DPDK Environment Abstraction Layer.
2. Launches a worker function on every enabled worker lcore.
3. Runs the same function on the main lcore.
4. Limits concurrent entry through the semaphore.
5. Protects the shared counter with a DPDK spin mutex.
6. Waits for all worker lcores.
7. Verifies the final count.

---

## Building and running the DPDK example

DPDK installations normally expose compiler and linker flags through `libdpdk.pc`.

```bash
g++ -O3 -std=gnu++17 \
    $(pkg-config --cflags libdpdk) \
    dpdk_sync.cpp \
    -o dpdk_sync \
    $(pkg-config --libs libdpdk)
```

GNU C++ mode may be needed because some DPDK headers use GNU extensions.

A CPU-only test that does not access a NIC can commonly be launched as:

```bash
./dpdk_sync -l 0-3 --no-pci --no-huge
```

The exact EAL options depend on the DPDK version, platform, privileges, memory configuration, and application requirements.

### Cross-compilation

The application source is architecture-neutral. For an AArch64 target:

1. Use an AArch64 cross-compiler or native AArch64 build environment.
2. Build or install DPDK for the same target and ABI.
3. Ensure `pkg-config` resolves the target DPDK metadata rather than host libraries.
4. Link the application against the target DPDK build.

---

## DPDK behavior on X86 and ARM

### x86 pause behavior

On x86, DPDK's pause helper maps to the processor pause hint, commonly `_mm_pause()` in C/C++ and `pause` in assembly.

Conceptually:

```asm
retry:
    lock cmpxchg [lock], desired
    je   acquired

wait:
    pause
    cmp  dword ptr [lock], 0
    jne  wait
    jmp  retry
```

The pause hint can improve behavior of tight spin loops by reducing pipeline penalties and resource pressure. It does not put the thread to sleep.

### AArch64 pause behavior

A normal AArch64 pause helper can emit `yield`:

```asm
retry:
    ldaxr   w1, [x0]
    cbnz    w1, wait

    mov     w2, #1
    stxr    w3, w2, [x0]
    cbnz    w3, retry
    ret

wait:
    yield
    b       retry
```

Some DPDK/ARM configurations can use event-based waiting facilities such as `WFE` in appropriate wait helpers:

```asm
wait:
    wfe
    b       retry
```

`WFE` can reduce power consumption relative to an aggressive load loop, but the exact behavior depends on platform support, DPDK configuration, exclusive-monitor behavior, and the wait helper being used.

### ARM LSE

On systems where ARM Large System Extensions are enabled for compilation, atomic operations can use instructions such as `CAS`, `SWP`, and `LDADD`. Without LSE, they are generally implemented using exclusive load/store loops.

### Application-level implication

No architecture-specific lock code is normally required in the DPDK application. DPDK headers and compiler atomics select the appropriate:

- Atomic read-modify-write instructions.
- Memory barriers or acquire/release instruction forms.
- Pause, yield, or optional event-wait behavior.

---

## Choosing the appropriate primitive

### Use `std::mutex` when

- The code is ordinary C++ application or control-plane code.
- Wait times may be nontrivial.
- Threads can be descheduled.
- Portability and maintainability are priorities.

### Use `std::counting_semaphore` when

- The program needs bounded concurrency or a permit pool.
- C++20 is available.
- A standard sleeping/waking implementation is appropriate.

### Use a futex-based custom primitive when

- You are studying low-level synchronization.
- You need a specialized userspace ABI or state machine.
- You can justify and test the substantial correctness burden.

For production, a mature library implementation is usually safer.

### Use a Linux-kernel mutex or semaphore when

- The code runs inside the Linux kernel.
- Sleeping is permitted in the current context.
- Mutex ownership or semaphore counting semantics match the requirement.

### Use `rte_spinlock_t` when

- The critical section is extremely short.
- The holder cannot block or be descheduled for a long time.
- Participating lcores are dedicated and not oversubscribed.
- Minimal lock overhead matters more than fairness.

### Use `rte_ticketlock_t` when

- FIFO-style acquisition ordering is useful.
- Reduced starvation is more important than the smallest lock footprint.
- The contention level remains suitable for a single shared ticket state.

### Use `rte_mcslock_t` when

- Many lcores can contend for the same lock.
- Cache-line bouncing on a centralized lock becomes expensive.
- Per-waiter queue nodes can be managed correctly.

MCS waiters spin primarily on local state, which can scale better under heavy contention.

### Use the custom DPDK polling semaphore when

- The wait is intentionally polling.
- Permit hold times are short and predictable.
- Dedicated lcores are available.
- Non-FIFO semantics are acceptable.

### Do not use a polling primitive when

- The lock holder may perform disk, network, file, or blocking system calls.
- The holder may sleep or be preempted for long periods.
- The process is oversubscribed.
- A waiter may remain blocked for milliseconds or longer.
- Energy consumption is important and event-based waiting is unavailable.

---

## Comparison matrix

| Primitive | Wait behavior | Ownership | Fairness | Cross-process | Best fit |
|---|---|---|---|---|---|
| `std::mutex` | Usually sleeping/adaptive | Yes | Implementation-defined | No, by itself | General C++ mutual exclusion |
| `std::counting_semaphore` | Usually sleeping/adaptive | No | Implementation-defined | No, by itself | General C++ bounded concurrency |
| Educational futex mutex | Sleeping under contention | Intended yes, not tracked | No guarantee | Private version: no | Learning/custom Linux userspace |
| Educational futex semaphore | Sleeping under contention | No | No guarantee | Private version: no | Learning/custom Linux userspace |
| Kernel `struct mutex` | Kernel sleeping lock | Yes | Kernel implementation policy | Kernel internal | Kernel mutual exclusion |
| Kernel `struct semaphore` | Kernel sleeping lock | No | Kernel implementation policy | Kernel internal | Kernel permit counting |
| `rte_spinlock_t` | Busy wait | By convention | Not FIFO | Shared-memory dependent | Very short DPDK critical sections |
| `rte_ticketlock_t` | Busy wait | By convention | Ticket order | Shared-memory dependent | Fairer DPDK locking |
| `rte_mcslock_t` | Busy wait on queue node | By convention | Queue-based | Shared-memory dependent | Highly contended DPDK locking |
| Custom DPDK semaphore | Busy wait | No | No guarantee | Shared-memory dependent | Short bounded data-plane operations |

### Decision rule

A useful default decision sequence is:

1. Use a lock-free or ownership-partitioned design when it simplifies the data plane.
2. Otherwise use the highest-level standard or platform primitive that satisfies the requirement.
3. Use a spin-based DPDK primitive only for short, bounded, data-plane waits.
4. Use custom atomic/futex implementations only after defining invariants, lifecycle rules, fairness expectations, failure behavior, and a serious stress-testing strategy.

---

## References

### C++ synchronization and memory model

- [C++ draft: mutex requirements](https://eel.is/c++draft/thread.mutex.requirements)
- [C++ draft: multithreading and memory model](https://eel.is/c++draft/intro.multithread)

### Linux futexes

- [Linux manual: futex(2)](https://man7.org/linux/man-pages/man2/futex.2.html)
- [Linux manual: FUTEX_WAKE](https://man7.org/linux/man-pages/man2/FUTEX_WAKE.2const.html)

### Linux-kernel locking

- [Linux kernel documentation: mutex design](https://docs.kernel.org/locking/mutex-design.html)

### ARM synchronization

- [Arm instruction reference: LDAXR](https://developer.arm.com/documentation/ddi0596/latest/Base-Instructions/LDAXR--Load-Acquire-Exclusive-Register-)
- [Arm synchronization primitives: LDREX and STREX](https://developer.arm.com/documentation/dht0008/latest/arm-synchronization-primitives/exclusive-accesses/ldrex-and-strex)

### DPDK APIs and guides

- [DPDK API: spin locks](https://doc.dpdk.org/api/rte__spinlock_8h.html)
- [DPDK API: ticket locks](https://doc.dpdk.org/api/rte__ticketlock_8h.html)
- [DPDK API: MCS locks](https://doc.dpdk.org/api/rte__mcslock_8h.html)
- [DPDK API: launch functions](https://doc.dpdk.org/api/rte__launch_8h.html)
- [DPDK Programmer's Guide: thread safety](https://doc.dpdk.org/guides/prog_guide/thread_safety.html)
- [DPDK Linux Getting Started Guide: building applications](https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html)
- [DPDK API index](https://doc.dpdk.org/api/)

---

## Closing guidance

For ordinary Linux C++ software, start with `std::mutex` and `std::counting_semaphore`. For Linux-kernel code, use the kernel locking primitives appropriate to the execution context. For DPDK data-plane code, first try to avoid shared writable state through per-lcore ownership, queues, rings, or batching; when a shared lock is unavoidable, keep the protected section short and select the lock according to contention and fairness requirements.