Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Atomics

Thousands of invocations run at once, and sooner or later two of them want to update the same place. A counter, a histogram bin, a “who got here first” flag. A plain bins[b] += 1 is a read, an add and a write, and another invocation can do its own read between yours and your write. Both add one, the bin goes up by one, and nobody is told.

An atomic does the read, the change and the write as one step that nothing can get between:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn count_values(values: &[u32], bins: &mut [AtomicU32]) {
    let index = global_id().x;
    if index < values.len() {
        bins[values[index] % 16u32].fetch_add(1u32);
    }
}

The types are AtomicU32 and AtomicI32, and the methods are named after the ones on Rust’s own atomics, without the ordering argument, since a GPU gives you one ordering and no choice about it.

Where they live

An atomic is something to update, so it goes where updates happen: in a &mut buffer, or in workgroup memory.

use unipute::kernel;
#[kernel(workgroup_size(256))]
fn count_locally(values: &[u32], bins: &mut [AtomicU32]) {
    #[workgroup]
    let local_bins: [AtomicU32; 16];

    let index = global_id().x;
    if index < values.len() {
        local_bins[values[index] % 16u32].fetch_add(1u32);
    }
    workgroup_barrier();

    if local_index() < 16u32 {
        bins[local_index()].fetch_add(local_bins[local_index()].load());
    }
}

This is the usual shape. An atomic in workgroup memory is cheap, an atomic in a buffer is not, so each workgroup counts into its own bins and adds the whole set into the buffer once. Two hundred and fifty six adds on the chip, sixteen in memory.

A uniform cannot be an atomic, since nothing writes to a uniform, and neither can a &[AtomicU32] buffer: a buffer of atomics nobody updates is a buffer of u32 with extra steps, and the macro says so. A local cannot be one either, and nor can a nested function’s parameter, because an atomic is a place that several invocations share rather than a value one of them holds. Pass a helper what .load() gave you.

On the host an atomic is a plain integer. A &mut [AtomicU32] is uploaded as a Vec<u32> and read back the same way, and nothing in BINDINGS changes.

The operations

Each takes a value of the atomic’s type and gives back what the atomic held before the change.

MethodDoes
fetch_add(v)adds v
fetch_sub(v)subtracts v
fetch_min(v)keeps the smaller of the two
fetch_max(v)keeps the larger
fetch_and(v), fetch_or(v), fetch_xor(v)bitwise
swap(v)replaces the value with v

Use the old value or not, as you like. As a statement on its own, the value is dropped and the shader does a little less work:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn append(items: &[u32], output: &mut [u32], count: &mut [AtomicU32]) {
    let index = global_id().x;
    if index < items.len() && items[index] > 100u32 {
        // The old value is where this item goes, and nobody else gets it.
        let slot = count[0].fetch_add(1u32);
        output[slot] = items[index];
    }
}

That is stream compaction: every invocation that has something to say claims the next free slot, and the slots come out dense.

load() reads the atomic, and store(v) writes it. Both are single atomic accesses, so a value read with load is one another invocation wrote whole, never half of one. Reading an atomic without load, or assigning to it, is an error that tells you which method to use instead. So is +=.

compare_exchange

compare_exchange(current, new) stores new only if the atomic still holds current, and says whether it did:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn claim_slots(slots: &mut [AtomicU32]) {
    let me = global_id().x + 1u32;
    let mut slot = me % slots.len();
    let mut claimed = false;
    while !claimed {
        claimed = slots[slot].compare_exchange(0u32, me);
        if !claimed {
            slot = (slot + 1u32) % slots.len();
        }
    }
}

It answers true or false rather than handing back the old value as Rust’s does, because on some hardware the exchange is allowed to fail even when the values matched. The old value alone would not tell you that. Write the loop and let it try again.

It is the one atomic method that is not allowed as a statement on its own. An exchange that may not have happened is worth checking, so use the result.

What atomics do not do

They do not order anything else. fetch_add on a counter and a plain store to another buffer are two separate things, and another invocation may see them in either order. When one workgroup’s atomics have to be visible to the rest of it before the next step, that is what the barrier is for, and storage_barrier() is the same for a buffer.

They are integers only. There is no AtomicF32, since most hardware has no such thing, and no atomic on a Vec or on a struct’s field yet. What is not built yet keeps the list.