Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Unipute lets you write a GPU compute kernel as an ordinary looking Rust function. When your crate compiles, the shader is generated and pasted back into your code as a constant.

use unipute::{Kernel, WgslKernel, kernel};

#[kernel(workgroup_size(64))]
fn double(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index] * 2.0;
    }
}

fn main() {
    // This string already exists by the time the program starts.
    println!("{}", double::WGSL);
}

That is the whole idea. You write Rust, you get WGSL, SPIR-V, MSL, HLSL or GLSL, and none of it happens while your program is running.

What Unipute is not

It is not a graphics library. Unipute never creates a device, a queue, a buffer or a pipeline. It hands you shader source or a buffer of SPIR-V words and stops there. You keep using wgpu, ash, Metal, OpenGL or whatever else you already picked, and Unipute sits upstream of that choice.

This sounds like a limitation and is mostly the opposite. Because Unipute never touches an API, it never locks you into one, and adding a target later does not mean rewriting your kernels.

Why generate at compile time

Most ways of getting a shader onto a GPU hand a string to a driver at run time. If the shader is wrong, you find out when the program runs, and the error points into a string rather than into your code.

Unipute runs the whole pipeline during compilation instead, so a broken kernel is a compile error in your own crate:

error: `frobnicate` is not a built-in Unipute provides
 --> src/main.rs:5:17
  |
5 |     output[0] = frobnicate(1.0);
  |                 ^^^^^^^^^^

You also get the layout information as data, so the host side of your code can read the binding numbers from the kernel rather than you keeping two lists in sync by hand.

How the pieces fit

There are three stages, and they do not know about each other:

your Rust function  ->  Unipute IR  ->  shader text or SPIR-V

The middle stage is a small, backend agnostic description of what the kernel does. The front end turns Rust into it, and a back end turns it into a language. Neither end depends on the other, which is what makes it possible to add a language, or eventually a whole different kind of target like CUDA, without touching kernels that already work.

How it works goes through this in more detail, once the rest of the book has given it some context.

Where to go next

If you want to try it, start with Installing and then Your first kernel.

If you want to know what a kernel is allowed to contain, the chapters under “Writing kernels” are the reference, and Control flow is probably the one with the most surprises in it.

If you are wondering whether Unipute can do the thing you need, What is not built yet is an honest list.

A note on maturity

This is version 0.2. Compute kernels work end to end across five shader languages, and that part is tested. Graphics stages, backends other than naga, and bindings for languages other than Rust are designed for and not built. The roadmap chapter says what each one would take.

Installing

Add Unipute to your crate:

[dependencies]
unipute = "0.2"

That gives you WGSL. Every other language is a feature, and you can turn on as many as you like:

[dependencies]
unipute = { version = "0.2", features = ["spv", "msl"] }
FeatureLanguage
wgslWebGPU Shading Language, on by default
spvSPIR-V
mslMetal Shading Language
hlslHigh Level Shading Language
glslOpenGL Shading Language

Choosing targets explains what each one gives you and how to pick.

What gets pulled in

Unipute depends on naga, the shader translator from the wgpu project, and on syn and quote for reading Rust syntax. It does not depend on wgpu itself, or on any other graphics library.

Both naga and the macro crate are compiled during a clean build, which takes a little while the first time. If you build Unipute often, a compilation cache is worth having:

cargo install sccache

Then add this to ~/.cargo/config.toml:

[build]
rustc-wrapper = "sccache"

Build time

Because the shaders are generated while your crate compiles, kernels cost compile time rather than run time. A handful of kernels is not noticeable. If you have hundreds, turning off the targets you are not using will help, since each enabled target runs its own writer over every kernel.

Rust version

Unipute uses the 2024 edition. Any reasonably current stable toolchain works, and there is nothing nightly only in it.

You may notice that Unipute’s own repository pins nightly. That is a build speed choice for people working on Unipute, not something that reaches you. It turns on the cranelift codegen backend for dev builds, which is still unstable, and it lives in that repository’s cargo config rather than in the published manifest so that it stays there. Every part of Unipute you touch, the #[kernel] macro, the traits, the layout types and the runtime API, is stable Rust. CI builds a crate against Unipute on stable on every change to keep that true.

Checking it worked

Put this in src/main.rs:

use unipute::{Kernel, kernel};

#[kernel(workgroup_size(64))]
fn nothing_much(output: &mut [f32]) {
    output[0] = 1.0;
}

fn main() {
    println!("{} with workgroup {:?}", nothing_much::NAME, nothing_much::WORKGROUP_SIZE);
}
cargo run
nothing_much with workgroup [64, 1, 1]

If that runs, you are set up. On to your first real kernel.

Your first kernel

We are going to write a kernel that scales every number in a buffer, look at what it turns into, and then take it apart a piece at a time.

The kernel

use unipute::{Kernel, WgslKernel, kernel};

#[kernel(workgroup_size(64))]
fn scale(input: &[f32], output: &mut [f32], factor: &f32) {
    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    output[index] = input[index] * factor;
}

fn main() {
    println!("{}", scale::WGSL);
}

Run it and you get this:

@group(0) @binding(0)
var<storage> input: array<f32>;
@group(0) @binding(1)
var<storage, read_write> output: array<f32>;
@group(0) @binding(2)
var<uniform> factor: f32;

@compute @workgroup_size(64, 1, 1)
fn scale(@builtin(global_invocation_id) global_id: vec3<u32>) {
    var index: u32;

    index = global_id.x;
    let _e6 = index;
    if (_e6 >= arrayLength((&input))) {
        return;
    }
    let _e9 = index;
    let _e11 = index;
    let _e13 = input[_e11];
    let _e14 = factor;
    output[_e9] = (_e13 * _e14);
}

Nothing about that shader was written by hand, and nothing about it was produced while the program was running. It was already a string constant before main started.

Reading it line by line

#[kernel(workgroup_size(64))]

This is the only required option. A workgroup is the batch of invocations the GPU schedules together, and 64 is a reasonable default for one dimensional work. You can give up to three dimensions:

use unipute::kernel;
#[kernel(workgroup_size(8, 8))]
fn over_a_grid(output: &mut [f32]) {
    let cell = global_id();
    output[cell.y * 8u32 + cell.x] = 1.0;
}

Dimensions you leave out become 1, so workgroup_size(64) means [64, 1, 1].

The parameters

fn scale(input: &[f32], output: &mut [f32], factor: &f32) {

The type of each parameter decides how it is bound:

  • &[f32] became a read only storage buffer
  • &mut [f32] became a read and write storage buffer
  • &f32 became a uniform

You did not write any binding numbers. Parameters get consecutive bindings in group 0, in the order you declared them, which is usually what you want. Parameters and bindings covers how to say otherwise.

global_id().x

global_id() is where this invocation sits in the whole dispatch. It is a three component vector, and .x picks the first component. A kernel runs once per invocation, so this is how each copy of the kernel knows which element of the buffer belongs to it.

The bounds check

if index >= input.len() {
    return;
}

You dispatch whole workgroups, so if the buffer has 100 elements and the workgroup is 64 wide you end up dispatching 128 invocations. The 28 extra ones have to be told to do nothing. Forgetting this check is the single most common GPU bug, and it is worth getting into the habit.

input.len() works because Unipute knows input is a runtime sized buffer. It becomes arrayLength in the shader, so the length comes from however much data the host actually bound.

The work

output[index] = input[index] * factor;

factor is a uniform holding a single f32, so using it by name reads the value. There is no dereference, because there are no pointers to speak of inside a kernel.

What else the macro gave you

The function is gone, replaced by a type with the same name. Along with the shader, that type carries the information the host side needs:

use unipute::{Kernel, kernel};
#[kernel(workgroup_size(64))]
fn scale(input: &[f32], output: &mut [f32], factor: &f32) {
    let index = global_id().x;
    if index >= input.len() { return; }
    output[index] = input[index] * factor;
}
fn main() {
    assert_eq!(scale::NAME, "scale");
    assert_eq!(scale::WORKGROUP_SIZE, [64, 1, 1]);

    for binding in scale::BINDINGS {
        println!(
            "group {} binding {}: {} ({:?})",
            binding.group, binding.binding, binding.name, binding.access
        );
    }
}
group 0 binding 0: input (Read)
group 0 binding 1: output (ReadWrite)
group 0 binding 2: factor (Uniform)

This matters more than it looks. Building a bind group by hand means keeping the numbers in your host code in step with the numbers in your shader, and nothing checks that for you. Here there is one source of truth, and it is the kernel.

Breaking it on purpose

Try changing the multiply to something Unipute does not have:

output[index] = wibble(input[index]);
error: `wibble` is not a built-in Unipute provides
 --> src/main.rs:8:21
  |
8 |     output[index] = wibble(input[index]);
  |                     ^^^^^^

The error points at your source, at the right span, during compilation. That is the payoff for doing the work at compile time.

Where to go next

Parameters and bindings

A kernel’s parameters are the buffers the host binds before dispatching it. The type you write decides what kind of binding you get.

You writeYou getKernel can
&[T]read only storage bufferread
&mut [T]read and write storage bufferread and write
&Tuniformread
Tuniformread

T is a scalar, a vector, or a struct of your own with #[derive(Layout)] on it, which Your own structs covers.

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn blend(
    source: &[f32],      // read only buffer
    target: &mut [f32],  // buffer we write into
    amount: &f32,        // a single number, the same for every invocation
) {
    let index = global_id().x;
    if index < source.len() {
        target[index] = mix(target[index], source[index], amount);
    }
}

Storage buffers

A slice parameter is a storage buffer. Its length is not fixed at compile time, which is why .len() works and why the host decides how much data to bind.

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn sum_lengths(a: &[f32], b: &[f32], output: &mut [u32]) {
    output[0] = a.len() + b.len();
}

.len() gives a u32, not a usize, because there is no usize on a GPU. It counts elements, not bytes.

Take a buffer by &mut only when you write to it. A read only buffer tells the driver more about what you are doing, and some APIs place the two in different descriptor types.

Uniforms

A parameter that is not a slice is a uniform: one value, the same for every invocation in the dispatch. Use one for anything the host decides per dispatch, such as a scale factor, a time value or a grid size.

use unipute::kernel;
#[kernel(workgroup_size(8, 8))]
fn fill_grid(output: &mut [f32], width: &u32) {
    let cell = global_id();
    output[cell.y * width + cell.x] = 1.0;
}

Using a uniform by name reads its value. There is no dereference step, and *factor is not something a kernel can say.

You cannot write to a uniform. Trying gives you:

error: `width` is a uniform, it cannot be written to

A uniform also cannot be a slice. If you need a variable number of values, use a &[T] storage buffer instead.

Binding numbers

Parameters get consecutive bindings in group 0, in the order you wrote them. The kernel in the first example ends up as:

group 0 binding 0: source (Read)
group 0 binding 1: target (ReadWrite)
group 0 binding 2: amount (Uniform)

Most of the time that is what you want, and you never think about it.

When it is not, say so per parameter:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn split_across_groups(
    #[binding(group = 0, index = 0)] camera: &f32,
    #[binding(group = 1, index = 1)] input: &[f32],
    #[binding(group = 1, index = 2)] output: &mut [f32],
) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index] * camera;
    }
}

This is worth doing when some bindings change per frame and others do not, since most APIs let you rebind one group without touching the others. Putting the per frame data in its own group means you swap only that group.

Either part can be left out. #[binding(group = 1)] keeps the automatic index and only moves the group.

Group 1 above carries on counting from 1 rather than starting again at 0. That is for GLSL, which has no groups and so needs every binding number to be unique across all of them. Nothing else cares, but if the glsl feature is ever going to be on, numbering that way from the start saves an edit later. Choosing targets has the details.

Two parameters landing on the same slot is an error, caught at compile time:

error: `input` and `output` both use group 0 binding 0, set one with #[binding(...)]

Reading the layout from the host

Whatever the layout ends up being, the kernel will tell you:

use unipute::{Kernel, kernel};
#[kernel(workgroup_size(64))]
fn blend(source: &[f32], target: &mut [f32], amount: &f32) {
    let index = global_id().x;
    if index < source.len() { target[index] = mix(target[index], source[index], amount); }
}
fn main() {
for binding in blend::BINDINGS {
    println!(
        "group {} binding {}: {} ({:?})",
        binding.group, binding.binding, binding.name, binding.access
    );
}
}

Build your bind group layout from this rather than from a list you maintain by hand. If you change the kernel’s parameters, the host side follows automatically instead of going quietly out of sync.

Running a kernel shows this against a real API.

What a kernel cannot take

  • No self. A kernel is a free function.
  • No generics.
  • No return type. Results go into a &mut parameter. Trying to return gives you a message saying exactly that.
  • No &mut T for a single value. Write &mut [T] and use index 0, or think about whether you wanted a storage buffer all along.

Types

Kernels use a small set of types. This chapter is the whole list.

Scalars

TypeNotes
f32the usual floating point type
u32unsigned, and what indices and lengths are
i32signed
boolconditions only, not something you put in a buffer

There is no f64, no u64, no usize and no u8. GPUs either do not have them or make you ask for an extension, and a version 0.2 that quietly picks for you would be worse than one that says no.

usize is the one people reach for out of habit. Lengths and indices are u32 here:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn count(input: &[f32], output: &mut [u32]) {
    let total = input.len();   // u32, not usize
    let index = global_id().x; // u32 as well
    if index == 0u32 {
        output[0] = total;
    }
}

Vectors

Vec2<T>, Vec3<T> and Vec4<T>, built with vec2, vec3 and vec4:

use unipute::kernel;
#[kernel(workgroup_size(8, 8))]
fn gradient(output: &mut [f32], width: &u32) {
    let cell = global_id();
    let color = vec3(cell.x as f32, cell.y as f32, 0.5);
    output[cell.y * width + cell.x] = length(color);
}

Note the capital letter in the type and the lower case in the constructor, which follows how Rust names types against functions.

Components are .x, .y, .z and .w, in that order. Reading past the end of the vector is a compile error, so .w on a Vec3 will tell you off rather than producing something surprising.

Swizzles

Two to four components at once make a new vector, in whatever order you name them. color.xy is a Vec2, color.zyx is the first three reversed, and color.xxx repeats one. You can assign through a swizzle too, which writes just the components it names:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn step(bodies: &mut [Vec4<f32>], dt: &f32) {
    let index = global_id().x;
    if index < bodies.len() {
        // Position in `xyz`, mass in `w`. Only the position moves.
        let mut body = bodies[index];
        body.xyz += vec3(0.0f32, -9.8, 0.0) * dt;
        bodies[index] = body;
    }
}

The value is worked out in full before anything is written, so v.xy = v.yx swaps the two rather than copying one over the other. A swizzle you assign to cannot name a component twice, since v.xx = ... would have two values for one place.

The letters are x, y, z and w only. Some shader languages also take rgba for colours, but Unipute keeps one set of names.

Vectors can be stored in buffers:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn normalise_all(points: &mut [Vec3<f32>]) {
    let index = global_id().x;
    if index < points.len() {
        points[index] = normalize(points[index]);
    }
}

Keep in mind that a Vec3 takes the space of a Vec4 in a buffer. That is not Unipute being wasteful, it is how every shader language lays out three component vectors, and your host side allocation has to agree.

Casts

as converts between scalar types:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn to_float(input: &[u32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index] as f32;
    }
}

This is a value conversion, the same as in Rust. You can only cast to a scalar, so there is no as Vec3<f32>.

How literals get their type

An unsuffixed integer takes its type from whatever it is used with:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn literals(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;

    // `4` becomes a u32, because `index` is one.
    if index < 4 {
        // `2.0` is an f32, since that is the only float type there is.
        output[index] = input[index] * 2.0;
    }
}

This is the same idea as Rust’s inference, done in a much simpler way: an untyped literal on one side of an operator picks up the type of the other side. It handles the cases that come up in practice.

When there is no other side to learn from, an integer literal defaults to i32, exactly as in Rust. In a kernel that is usually not what you meant, so say which one you want:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn explicit(output: &mut [u32]) {
    let counter = 0u32;        // suffix
    let other: u32 = 0;        // or an annotation
    output[0] = counter + other;
}

If Unipute genuinely cannot work out a type it says so rather than guessing:

error: the type of this value is unclear, annotate it as in `let x: f32 = ...`

Structs

A struct you define, with #[derive(Layout)] on it, can be a buffer element, a uniform, a local, or a parameter of a nested function. Its fields are read with .name. Your own structs is the chapter on them, and on the one thing to know about their layout.

Atomics

AtomicU32 and AtomicI32 are integers that many invocations can update at once without losing an update. They live in &mut buffers and in workgroup memory, and are reached through .load(), .store() and the fetch_ methods rather than read and assigned like a number. Atomics is the chapter on them.

Arrays with a fixed length

[T; N] is the type of workgroup memory, and that is the only place the macro accepts it. A local cannot be an array yet, and a buffer is always a slice, since its length is the host’s decision.

What is not here yet

Matrices, textures and samplers. All of them are on the list, and What is not built yet says roughly what each one needs. If you hit one of these, that chapter is the honest answer rather than a workaround.

Your own structs

A buffer does not have to hold plain numbers. It can hold a struct you define, so that a particle is one record with a position, a mass and a velocity rather than three buffers that happen to line up.

use unipute::{Layout, kernel};
#[derive(Layout, Clone, Copy)]
#[repr(C)]
struct Particle {
    position: [f32; 3],
    mass: f32,
}

#[kernel(workgroup_size(64))]
fn heavier(particles: &mut [Particle]) {
    let index = global_id().x;
    if index < particles.len() {
        particles[index].mass = particles[index].mass * 2.0;
    }
}

The same struct is used on the host to fill the buffer and in the kernel to read it. That is the point of it, and it is also the danger: the two sides have to agree about where every byte is. #[derive(Layout)] is what makes sure they do.

Declaring one

Three things are required, and the derive checks all of them.

#[repr(C)]. Without it, Rust is free to reorder fields, and then nothing about the struct’s memory is knowable. The derive refuses a struct without it.

Named fields. A kernel reads a field by name, so a tuple struct has nothing to read by.

Field types the GPU has. A field is an f32, a u32 or an i32, or an array of two, three or four of them. Inside the kernel the array is a vector: [f32; 3] on the host is Vec3<f32> in the kernel, and particle.position.x works as you would expect. There is no Vec3 type on the host, and writing one in a struct gets you a message saying to use the array.

bool is not allowed, since the host and the GPU do not agree on how big one is. Store a u32 and compare it with zero.

A struct inside a struct, or an array of more than four elements, is not supported yet. What is not built yet has the details.

Using one

Anywhere a scalar or a vector can go, a struct can go too.

A &[Particle] parameter is a read only buffer of them, &mut [Particle] a buffer the kernel writes, and &Settings a uniform holding one:

use unipute::{Layout, kernel};
#[derive(Layout, Clone, Copy)]
#[repr(C)]
struct Particle { position: [f32; 3], mass: f32 }
#[derive(Layout, Clone, Copy)]
#[repr(C)]
struct Settings {
    gravity: [f32; 3],
    dt: f32,
}

#[kernel(workgroup_size(64))]
fn fall(particles: &mut [Particle], settings: &Settings) {
    let index = global_id().x;
    if index >= particles.len() {
        return;
    }
    let particle = particles[index];
    particles[index] = Particle {
        position: particle.position + settings.gravity * settings.dt,
        mass: particle.mass,
    };
}

Reading particles[index] copies the whole struct into a local. Reading particles[index].mass loads that one field and nothing else, which is what you want in a kernel that only looks at one.

Assign to a field in place with particles[index].mass = ..., or replace the whole element with a struct literal. A literal names every field, in any order, and there is no .. to fill in the rest, since a shader has no idea what a default would be. Field shorthand works, so Particle { position, mass } is fine when the locals have those names.

A nested function can take a struct and return one:

use unipute::{Layout, kernel};
#[derive(Layout, Clone, Copy)]
#[repr(C)]
struct Particle { position: [f32; 3], mass: f32 }
#[kernel(workgroup_size(64))]
fn merge(particles: &mut [Particle]) {
    fn combined(a: Particle, b: Particle) -> Particle {
        Particle {
            position: (a.position + b.position) * 0.5,
            mass: a.mass + b.mass,
        }
    }

    let index = global_id().x;
    if index + 1u32 < particles.len() {
        particles[index] = combined(particles[index], particles[index + 1u32]);
    }
}

The struct can be declared before or after the kernels that use it, and in another module, as long as it is in the same crate. use it the way you would use any type.

Where the bytes go

Shader languages lay a struct out by rules that are almost, but not quite, what #[repr(C)] does. Every field starts at a multiple of its alignment:

FieldSizeAligns to
f32, u32, i3244
[T; 2]88
[T; 3]1216
[T; 4]1616

Then the whole struct is rounded up to a multiple of its largest alignment.

The one to watch is [T; 3]. It takes twelve bytes but has to start at a multiple of sixteen, and Rust knows nothing about that: to Rust it is three floats, aligned to four. So Particle above works, with position at byte 0 and mass fitting in the four bytes after it, and this does not:

use unipute::Layout;
#[derive(Layout)]
#[repr(C)]
struct Weighted {
    mass: f32,
    position: [f32; 3],
}

Rust puts position at byte 4. The GPU puts it at byte 16. Rather than let the kernel read the wrong twelve bytes, the derive stops:

error: `position` sits at byte 4 on the CPU and byte 16 on the GPU, add `_pad: [u8; 12]` before it

A field whose name starts with an underscore is padding. Its type is [u8; N], the kernel never sees it, and it exists to push the next field to where the GPU expects it. Do what the message says and the derive is satisfied on that field, and then possibly tells you about the end of the struct:

error: `Weighted` is 28 bytes on the CPU and 32 on the GPU, add `_pad2: [u8; 4]` as its last field

The GPU rounds the struct up to 32 because its largest field aligns to 16. One more padding field and the two agree:

use unipute::Layout;
#[derive(Layout)]
#[repr(C)]
struct Weighted {
    mass: f32,
    _pad: [u8; 12],
    position: [f32; 3],
    _pad2: [u8; 4],
}

Or reorder the fields so that the vector comes first and the scalar fills the gap after it, which needs no padding at all. When you have a choice, that is the better layout, and it is why the examples in this book put the vector first.

The derive always says exactly what to add, so there is no need to work any of this out by hand. It also leaves a check behind that compares its answer against what rustc actually did, so if the two ever disagreed the build would fail rather than the kernel silently reading garbage.

Uploading one

Unipute stops at the shader, so getting a Vec<Particle> into a buffer is the host’s job, the same as for a Vec<f32>. Because the struct is #[repr(C)] with no hidden padding, a crate such as zerocopy can turn a slice of them into bytes with a derive and no unsafe:

#[derive(Layout, Clone, Copy, IntoBytes, FromBytes, Immutable, KnownLayout)]
#[repr(C)]
struct Particle {
    position: [f32; 3],
    mass: f32,
}

let buffer = gpu.storage(&particles); // takes anything IntoBytes

That is what the particles example does, and the wgpu code behind it is in examples/host/mod.rs. A uniform buffer holding a struct needs the usual sixteen byte size that uniforms have on every API, and the example’s helper pads to that.

Particle::ty() hands the struct back as IR, offsets included, if a host wants to check a layout or build a descriptor from it.

What is not here yet

  • A struct from another crate. The kernel macro learns a struct’s fields through a macro the derive leaves next to the struct, and that macro is only visible inside the crate that defined it. Define the struct in the crate with the kernels.
  • Nested structs and arrays as fields.
  • Structs as workgroup memory are not tested yet.

Control flow

Kernels use ordinary Rust control flow. This chapter covers what is supported and the few places where a kernel behaves differently from a function on the CPU.

Branches

if and else work as you would expect, including else if chains:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn classify(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index >= input.len() {
        return;
    }

    let value = input[index];
    if value < 0.0 {
        output[index] = -1.0;
    } else if value > 0.0 {
        output[index] = 1.0;
    } else {
        output[index] = 0.0;
    }
}

if is a statement here, not an expression. This does not work:

let sign = if value < 0.0 { -1.0 } else { 1.0 };

Assign in both branches instead, or reach for mix and step from Built-ins and functions, which is what shader code usually does anyway.

There is no match and no if let.

Loops

while, loop and for over a range all work:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn loops(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    let mut total = 0.0;

    // A counted loop over a range.
    for offset in 0..4u32 {
        let sample = index + offset;
        if sample < input.len() {
            total += input[sample];
        }
    }

    // The same thing spelled out.
    let mut i = 0u32;
    while i < 4u32 {
        i += 1u32;
    }

    output[index] = total;
}

for only iterates over a range. There are no iterators on a GPU, so for value in buffer is not something that can be translated. Loop over the indices and read the buffer inside.

Both a..b and a..=b work. break and continue do what they do in Rust, and using either outside a loop is a compile error rather than something you find out about later.

A note on divergence

This is the one place where GPU thinking differs from CPU thinking, and it is worth knowing even though Unipute does not make you do anything about it.

Invocations in a workgroup run in lockstep. When they take different branches, the hardware runs both sides and masks off the invocations that should not be affected. A branch that different invocations resolve differently therefore costs roughly the sum of both sides rather than the cheaper one.

This does not change what your kernel means, only how fast it goes. It is why shader code often prefers mix and step over a short if, and why a bounds check at the top of a kernel is cheap while a long branch in the middle of a hot loop might not be.

Returning early

return with no value leaves the kernel:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn guarded(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    output[index] = input[index];
}

return never takes a value, because a kernel produces nothing. Results go into a &mut parameter.

Variables

let introduces a local, and mut is accepted but not required, since a kernel local is a shader var and writable either way:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn locals(output: &mut [f32]) {
    let base = 1.0;
    let mut total = base;
    total += 2.0;
    output[0] = total;
}

Locals are scoped to their block, like in Rust, and shadowing works.

All the compound assignments are available: +=, -=, *=, /=, %=, &=, |=, ^=, <<= and >>=.

Operators

Arithmetic (+ - * / %), comparison (== != < <= > >=), logical (&& || !), bitwise (& | ^) and shifts (<< >>).

Whether && and || short circuit depends on which target you generate. WGSL says they do. The SPIR-V that naga produces evaluates both sides. Since Unipute’s whole point is that one kernel feeds every target, do not write a kernel that depends on the answer:

// Do not do this. Whether the index is evaluated depends on the target.
if index < input.len() && input[index] > 0.0 { }

Nest the checks instead:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn safe(input: &[f32], output: &mut [f32]) {
let index = global_id().x;
if index < input.len() {
    if input[index] > 0.0 {
        output[index] = 1.0;
    }
}
}

Barriers

When invocations in a workgroup need to see each other’s writes, put a barrier between the writing and the reading:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn with_barrier(data: &mut [f32]) {
    let index = global_id().x;
    data[index] = data[index] * 2.0;

    storage_barrier();

    if index > 0u32 {
        data[index] = data[index] + data[index - 1u32];
    }
}

workgroup_barrier() covers workgroup memory, which is what most barriers are for, and storage_barrier() covers storage buffers.

Every invocation in the workgroup has to reach the same barrier. Putting one inside an if that only some invocations take is undefined behaviour on most hardware, and an early return before a barrier is the same mistake in a different shape. Unipute does not check this for you yet, so keep barriers at the top level of the kernel body, or in a loop that every invocation runs the same number of times.

Quick reference

WorksDoes not
if / else if / elsematch, if let
while, looplabelled loops
for i in a..b, a..=bfor x in slice
break, continuebreak with a value
return;return value;
let, shadowing, mutlet else, patterns
all the usual operatorsrelying on && to short circuit

Built-ins and functions

This is everything Unipute provides. A kernel can also call functions you declare inside it, which Your own functions covers.

Knowing where you are

A kernel runs once per invocation. These tell an invocation which one it is.

CallTypeMeaning
global_id()Vec3<u32>position in the whole dispatch
local_id()Vec3<u32>position inside this workgroup
local_index()u32local_id flattened to one number
workgroup_id()Vec3<u32>which workgroup this is
num_workgroups()Vec3<u32>how many workgroups were dispatched

global_id() is the one you want most of the time. For one dimensional work it is the index into your buffer:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn one_dimensional(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index];
    }
}

For two dimensional work, use two components and a width uniform to flatten them:

use unipute::kernel;
#[kernel(workgroup_size(8, 8))]
fn two_dimensional(output: &mut [f32], width: &u32, height: &u32) {
    let cell = global_id();
    if cell.x >= width {
        return;
    }
    if cell.y >= height {
        return;
    }
    output[cell.y * width + cell.x] = 1.0;
}

Notice the bounds check on both axes. A workgroup of 8 by 8 over an image that is not a multiple of 8 in each direction will run invocations past the edge.

The relationship between them is worth keeping in mind:

global_id = workgroup_id * workgroup_size + local_id

local_index() is what you use to index into workgroup memory, since it runs from zero to one less than the workgroup size, and workgroup_id() is where a result for the whole group goes. Everywhere else, global_id() covers almost everything.

Barriers

workgroup_barrier() and storage_barrier(), both taking no arguments. The first is what makes a write to workgroup memory visible to the rest of the group. See Control flow for when you need them and the rule about keeping them out of branches.

Buffer length

.len() on a slice parameter gives its length in elements, as a u32:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn lengths(input: &[f32], output: &mut [u32]) {
    output[0] = input.len();
}

It only works on a parameter, not on an arbitrary expression, and it is the only method a kernel can call.

Building vectors

vec2(x, y), vec3(x, y, z) and vec4(x, y, z, w). All the components have to agree on a type:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn make_vectors(output: &mut [f32]) {
    let a = vec3(1.0, 0.0, 0.0);
    let b = vec3(0.0, 1.0, 0.0);
    output[0] = dot(a, b);
}

Numeric functions

All of these are available. Most take and return whatever type you give them, so abs works on a scalar or a vector.

ArgumentsFunctions
oneabs, sign, sqrt, inverse_sqrt, exp, log, floor, ceil, round, sin, cos, tan, length, normalize
twomin, max, pow, step, dot, cross
threeclamp, mix, fma

The ones worth explaining

mix(a, b, t) blends between a and b. At t = 0 you get a and at t = 1 you get b.

step(edge, x) gives 0 when x < edge and 1 otherwise. Together with mix it replaces a lot of small branches:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn branchless(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    let value = input[index];

    // The same as: if value < 0.5 { 0.0 } else { 1.0 }
    output[index] = step(0.5, value);
}

clamp(x, low, high) keeps x between the two bounds.

fma(a, b, c) is a * b + c done in one step, which is both faster and more accurate than writing it out.

dot and length collapse a vector down to a single number. cross takes two three component vectors and gives another one.

What you cannot call

A free function declared outside the kernel, including another #[kernel]. The macro only sees the function it is attached to, so a helper has to be declared inside the kernel body. See Your own functions.

Your own functions

A kernel can declare fn items inside its body and call them, the same way a nested function works in ordinary Rust:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn tonemap(input: &[f32], output: &mut [f32]) {
    fn compress(value: f32) -> f32 {
        value / (value + 1.0)
    }

    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    output[index] = compress(input[index]);
}

That comes out as two functions in the generated shader, with compress written above the entry point that calls it.

They capture nothing

This is the rule worth reading twice. A nested fn in Rust cannot see the variables around it, and the same is true here, only more so: a nested function cannot reach the kernel’s buffers, its workgroup memory or the hardware built-ins either.

#[kernel(workgroup_size(64))]
fn wrong(input: &[f32], output: &mut [f32]) {
    fn first() -> f32 {
        input[0] // error: `input` is not a parameter or a local variable
    }

    output[0] = first();
}

The reason is not a limitation of Unipute. Shader languages hand bindings and built-ins to the entry point, and a function called from it has no way to ask for them. So a nested function takes what it needs as a parameter:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn right(input: &[f32], output: &mut [f32]) {
    fn scale(value: f32, by: f32) -> f32 {
        value * by
    }

    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    output[index] = scale(input[index], 2.0);
}

Calling global_id() inside a nested function is rejected for the same reason. Read it in the kernel body and pass it down.

Parameters and results

A parameter is a scalar or a vector, taken by value. Slices cannot be passed, which follows from the rule above: a buffer belongs to the entry point.

A result is a scalar or a vector too, or nothing at all. The body ends with a value or a return, as in Rust:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn shades(output: &mut [f32]) {
    fn luminance(color: Vec3<f32>) -> f32 {
        // A trailing expression is the return value.
        dot(color, vec3(0.2126, 0.7152, 0.0722))
    }

    fn clamp_up(value: f32) -> f32 {
        if value < 0.0 {
            return 0.0;
        }
        value
    }

    output[global_id().x] = clamp_up(luminance(vec3(0.5, 0.25, 0.75)));
}

One thing Rust allows that a kernel does not is a block used as a value, so if a { b } else { c } cannot be the result of a function. Write a return in each branch instead.

A function that returns nothing is called as a statement:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn with_a_void_helper(output: &mut [f32]) {
    fn fill(_value: f32) {
        // Nothing useful, but it shows the shape.
    }

    fill(1.0);
    output[0] = 1.0;
}

Order does not matter

Declare them in whichever order reads best. Unipute works out which function calls which and writes them out so that every callee comes before its callers, which is what a shader language needs since none of them have forward declarations.

The one thing that follows from this is that a function cannot call itself, directly or through another function. No shader language supports recursion, so a cycle is a compile error naming the function that closes it.

Where they go

At the top level of the kernel body. A fn inside an if or a loop is rejected, since it would be the only thing in a kernel whose meaning depended on where it appeared.

Two nested functions in the same kernel cannot share a name, and neither can take the name of a built-in such as abs or vec3.

Functions are not shared between kernels. Two kernels that want the same helper each declare it. Sharing one across kernels needs a different surface than #[kernel] offers, and is not built yet.

Workgroup memory

Every invocation has its own locals, and every invocation in the dispatch shares the buffers. Workgroup memory sits between the two: one copy per workgroup, shared by the invocations in that workgroup and by nobody else. It lives on the chip rather than out in the card’s memory, so it is fast, and it is where the invocations of a group leave things for each other.

Declaring it

A let marked #[workgroup], at the top level of the kernel body, with a type and no value:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn block_sum(input: &[f32], output: &mut [f32]) {
    #[workgroup]
    let tile: [f32; 64];

    let lane = local_index();
    tile[lane] = input[global_id().x];
    workgroup_barrier();

    if lane == 0u32 {
        let mut total = 0.0;
        for i in 0..tile.len() {
            total += tile[i];
        }
        output[workgroup_id().x] = total;
    }
}

The type is a scalar, a vector, or a fixed length array of either. [f32; 64] is the usual shape: one slot per invocation, indexed by local_index(). An array of arrays such as [[f32; 8]; 8] works too, for a tile with two dimensions, and is indexed as tile[y][x]. It can also be an AtomicU32 or an array of them, for a counter or a histogram the whole group updates at once, which Atomics covers.

There is no value to give it, because there is no single invocation to give it. Every invocation writes its own part, and a barrier makes the parts visible to the rest. Every target Unipute generates zeroes it before the workgroup starts, so a slot nobody has written reads as zero, but a kernel that depends on that is usually a kernel with a missing write.

Index a shared array like a buffer, and use .len() on it the same way. The length is written in the kernel, so .len() here is a plain number rather than a question to the driver. A shared scalar or vector is read and assigned by name:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn subtract_first(input: &[f32], output: &mut [f32]) {
    #[workgroup]
    let first: f32;

    let index = global_id().x;
    if local_index() == 0u32 {
        first = input[index];
    }
    workgroup_barrier();
    if index < input.len() {
        output[index] = input[index] - first;
    }
}

That is a broadcast: one invocation reads a value, and after the barrier the whole group has it.

The barrier

A write to workgroup memory is not seen by the other invocations until workgroup_barrier() has run. The shape is always the same: write, barrier, read. Between one round of writing and reading and the next, there is another barrier.

Every invocation in the workgroup has to reach every barrier. That is easy to break by accident with the bounds check most kernels start with:

let index = global_id().x;
if index >= input.len() {
    return;             // this invocation never reaches the barrier below
}
tile[lane] = input[index];
workgroup_barrier();    // the rest of the group waits for it

The last workgroup of a dispatch is usually partly past the end of the input, and the invocations that leave early leave the others waiting at a barrier that never completes. Guard the reads and writes instead, and let every invocation fall through to the barrier:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn block_sum(input: &[f32], output: &mut [f32]) {
    #[workgroup]
    let tile: [f32; 64];

    let index = global_id().x;
    let lane = local_index();
    if index < input.len() {
        tile[lane] = input[index];
    } else {
        tile[lane] = 0.0;
    }
    workgroup_barrier();

    // Fold the top half of the tile onto the bottom half, then the top half
    // of what is left, until element 0 holds the whole sum.
    let mut stride = 32u32;
    while stride > 0u32 {
        if lane < stride {
            tile[lane] += tile[lane + stride];
        }
        workgroup_barrier();
        stride /= 2u32;
    }

    if lane == 0u32 {
        output[workgroup_id().x] = tile[0];
    }
}

This is a reduction, and it is most of what workgroup memory is for. Six rounds of adding replace sixty three, and every round reads memory on the chip rather than the buffer. The barrier inside the loop is fine, because every invocation runs the loop the same number of times. The if around the addition is fine too, since there is no barrier inside it.

The rules

  • Top level of the kernel body. A declaration inside an if or a loop is rejected. The memory exists for the whole kernel whether or not the line runs, so putting it inside a block would suggest something that is not so.
  • Kernel body only. A nested fn cannot declare workgroup memory or reach the kernel’s, for the same reason it cannot reach a buffer: the shader language hands both to the entry point alone. See Your own functions.
  • Not in the layout. The host never binds it, so it is not in BINDINGS. Adding workgroup memory to a kernel changes nothing on the host side.
  • One name once. Two declarations with the same name is an error, where a let would shadow. Two tiles both called tile are two allocations, and that is more likely a mistake than a choice.
  • A fixed size. The length is a number written in the kernel, not a const from outside, since nothing from outside the kernel is visible in it.

How much of it a workgroup can have depends on the hardware. Sixteen kilobytes is safe everywhere, and most desktop GPUs allow thirty two or more. Unipute does not check the total, so a kernel asking for more than the device has fails when the host creates the pipeline rather than at compile time.

Atomics

Thousands of invocations run at once, and sooner or later two of them want to update the same place. A counter, a histogram bin, a “who got here first” flag. A plain bins[b] += 1 is a read, an add and a write, and another invocation can do its own read between yours and your write. Both add one, the bin goes up by one, and nobody is told.

An atomic does the read, the change and the write as one step that nothing can get between:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn count_values(values: &[u32], bins: &mut [AtomicU32]) {
    let index = global_id().x;
    if index < values.len() {
        bins[values[index] % 16u32].fetch_add(1u32);
    }
}

The types are AtomicU32 and AtomicI32, and the methods are named after the ones on Rust’s own atomics, without the ordering argument, since a GPU gives you one ordering and no choice about it.

Where they live

An atomic is something to update, so it goes where updates happen: in a &mut buffer, or in workgroup memory.

use unipute::kernel;
#[kernel(workgroup_size(256))]
fn count_locally(values: &[u32], bins: &mut [AtomicU32]) {
    #[workgroup]
    let local_bins: [AtomicU32; 16];

    let index = global_id().x;
    if index < values.len() {
        local_bins[values[index] % 16u32].fetch_add(1u32);
    }
    workgroup_barrier();

    if local_index() < 16u32 {
        bins[local_index()].fetch_add(local_bins[local_index()].load());
    }
}

This is the usual shape. An atomic in workgroup memory is cheap, an atomic in a buffer is not, so each workgroup counts into its own bins and adds the whole set into the buffer once. Two hundred and fifty six adds on the chip, sixteen in memory.

A uniform cannot be an atomic, since nothing writes to a uniform, and neither can a &[AtomicU32] buffer: a buffer of atomics nobody updates is a buffer of u32 with extra steps, and the macro says so. A local cannot be one either, and nor can a nested function’s parameter, because an atomic is a place that several invocations share rather than a value one of them holds. Pass a helper what .load() gave you.

On the host an atomic is a plain integer. A &mut [AtomicU32] is uploaded as a Vec<u32> and read back the same way, and nothing in BINDINGS changes.

The operations

Each takes a value of the atomic’s type and gives back what the atomic held before the change.

MethodDoes
fetch_add(v)adds v
fetch_sub(v)subtracts v
fetch_min(v)keeps the smaller of the two
fetch_max(v)keeps the larger
fetch_and(v), fetch_or(v), fetch_xor(v)bitwise
swap(v)replaces the value with v

Use the old value or not, as you like. As a statement on its own, the value is dropped and the shader does a little less work:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn append(items: &[u32], output: &mut [u32], count: &mut [AtomicU32]) {
    let index = global_id().x;
    if index < items.len() && items[index] > 100u32 {
        // The old value is where this item goes, and nobody else gets it.
        let slot = count[0].fetch_add(1u32);
        output[slot] = items[index];
    }
}

That is stream compaction: every invocation that has something to say claims the next free slot, and the slots come out dense.

load() reads the atomic, and store(v) writes it. Both are single atomic accesses, so a value read with load is one another invocation wrote whole, never half of one. Reading an atomic without load, or assigning to it, is an error that tells you which method to use instead. So is +=.

compare_exchange

compare_exchange(current, new) stores new only if the atomic still holds current, and says whether it did:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn claim_slots(slots: &mut [AtomicU32]) {
    let me = global_id().x + 1u32;
    let mut slot = me % slots.len();
    let mut claimed = false;
    while !claimed {
        claimed = slots[slot].compare_exchange(0u32, me);
        if !claimed {
            slot = (slot + 1u32) % slots.len();
        }
    }
}

It answers true or false rather than handing back the old value as Rust’s does, because on some hardware the exchange is allowed to fail even when the values matched. The old value alone would not tell you that. Write the loop and let it try again.

It is the one atomic method that is not allowed as a statement on its own. An exchange that may not have happened is worth checking, so use the result.

What atomics do not do

They do not order anything else. fetch_add on a counter and a plain store to another buffer are two separate things, and another invocation may see them in either order. When one workgroup’s atomics have to be visible to the rest of it before the next step, that is what the barrier is for, and storage_barrier() is the same for a buffer.

They are integers only. There is no AtomicF32, since most hardware has no such thing, and no atomic on a Vec or on a struct’s field yet. What is not built yet keeps the list.

Choosing targets

Each shader language is a cargo feature. Turning one on adds a constant to every kernel in your crate.

FeatureTraitConstantFor
wgslWgslKernelWGSL: &strwgpu, WebGPU, Dawn
spvSpirvKernelSPIRV: &[u32]Vulkan, OpenCL, and as an input to other tools
mslMslKernelMSL: &strMetal, so Apple platforms
hlslHlslKernelHLSL: &strDirect3D 12
glslGlslKernelGLSL: &strOpenGL and OpenGL ES

wgsl is on by default. Turn it off with default-features = false if you do not want it.

[dependencies]
unipute = { version = "0.2", features = ["spv", "msl"] }

Getting at the constant

Import the trait for the target you want, then read the constant off the kernel type:

use unipute::{WgslKernel, kernel};

#[kernel(workgroup_size(64))]
fn double(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index] * 2.0;
    }
}

fn main() {
    let source: &str = double::WGSL;
    println!("{} bytes of WGSL", source.len());
}

Forgetting the import gives you a “cannot find” error, which is the usual Rust trait import problem rather than anything specific to Unipute.

Picking more than one

There is no cost to your program in enabling several. The strings all exist as constants and the ones you never read get dropped by the linker. What you pay is compile time, since each target runs its own writer over every kernel.

Supporting several backends usually looks like this:

#[cfg(target_vendor = "apple")]
fn shader() -> &'static str {
    use unipute::MslKernel;
    double::MSL
}

#[cfg(not(target_vendor = "apple"))]
fn shader() -> &'static str {
    use unipute::WgslKernel;
    double::WGSL
}

If you do not know the target until the program runs, see Generating at run time.

Notes per target

WGSL

The best supported path, and the one the tests exercise most. If you are using wgpu, this is what you want, and you can hand the string straight to create_shader_module.

SPIR-V

A &[u32], not bytes. Vulkan wants words, so this is the right shape already. If you need bytes, for writing to a file say, convert with to_le_bytes.

The first word is the SPIR-V magic number 0x07230203, which is a quick way to check you have a valid module.

MSL

Metal has no bind groups, so naga assigns buffer slots itself rather than using the group and binding numbers. Read the slots from Metal’s reflection rather than assuming they match BINDINGS. This is the one target where the layout information does not carry across directly.

HLSL

Targets Shader Model 6 by default. Groups and bindings become register assignments.

GLSL

Targets OpenGL ES 3.10, which is the first version with compute shaders, and which a desktop OpenGL 4.3 or later context also accepts when it has ARB_ES3_1_compatibility. For a particular desktop version, the runtime feature gives you naga_backend::write::glsl_with, which takes naga’s version type.

GLSL has no bind groups. Each resource is written with its binding number as its layout(binding = ...) and the group is dropped, so an OpenGL host binds a buffer at binding.binding and ignores binding.group. The cost is one namespace where the kernel had several: once glsl is on, two resources in different groups cannot share a binding number, and the macro reports the pair rather than writing a shader that binds them on top of each other. Give a second group’s parameters indices that carry on from the first, as in #[binding(group = 1, index = 2)].

One more thing if the output is going to wgpu rather than to OpenGL: wgpu reads GLSL through naga, and naga’s reader only takes desktop profiles 4.40 and up. Hand it glsl_with output for one of those rather than the GLSL constant. The GPU tests do exactly that.

Seeing all of them

The repository has an example that prints every enabled target for one kernel:

cargo run --example emit_shaders --features "spv,msl,hlsl,glsl"

That is the fastest way to see what a kernel actually becomes, and a good sanity check when something is not behaving. Examples lists the others that come with the repository.

Running a kernel

Unipute stops at the shader. Getting it onto a GPU is your graphics API’s job. This chapter shows how the two meet.

The code here is sketched against wgpu to show the shape of it. The full version, which the examples and the GPU tests run through, is examples/host/mod.rs in the repository. Graphics APIs change their signatures often, so if the two ever disagree, the file is the one that compiles. The Unipute side, which is the part this book can promise, is the constants.

What a kernel gives the host

Three things:

use unipute::{Kernel, WgslKernel, kernel};
#[kernel(workgroup_size(64))]
fn scale(input: &[f32], output: &mut [f32], factor: &f32) {
    let index = global_id().x;
    if index >= input.len() { return; }
    output[index] = input[index] * factor;
}
fn main() {
scale::WGSL;            // the shader
scale::NAME;            // the entry point name inside it
scale::WORKGROUP_SIZE;  // how many invocations one group runs
scale::BINDINGS;        // where each buffer goes
}

Those four cover everything the host has to know.

Creating the pipeline

let module = device.create_shader_module(wgpu::ShaderModuleDescriptor {
    label: Some(scale::NAME),
    source: wgpu::ShaderSource::Wgsl(scale::WGSL.into()),
});

let pipeline = device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
    label: Some(scale::NAME),
    layout: None,
    module: &module,
    entry_point: Some(scale::NAME),
    compilation_options: Default::default(),
    cache: None,
});

Using scale::NAME rather than the string "scale" means that renaming the kernel, or setting name = "..." on the attribute, does not leave the host side pointing at an entry point that no longer exists.

Building the bind group layout

This is where BINDINGS earns its place. Instead of writing the layout by hand and keeping it in step with the kernel, build it from the kernel:

use unipute::Access;

let entries: Vec<_> = scale::BINDINGS
    .iter()
    .map(|binding| wgpu::BindGroupLayoutEntry {
        binding: binding.binding,
        visibility: wgpu::ShaderStages::COMPUTE,
        ty: wgpu::BindingType::Buffer {
            ty: match binding.access {
                Access::Uniform => wgpu::BufferBindingType::Uniform,
                Access::Read => wgpu::BufferBindingType::Storage { read_only: true },
                Access::ReadWrite => wgpu::BufferBindingType::Storage { read_only: false },
            },
            has_dynamic_offset: false,
            min_binding_size: None,
        },
        count: None,
    })
    .collect();

Add a parameter to the kernel and this follows along. That is the difference between a layout that is derived and one that is duplicated.

binding.group matters once you use more than one group. Group the entries by it and create one layout per group.

Working out the dispatch size

WORKGROUP_SIZE tells you how many invocations one workgroup runs, so the number of workgroups is your element count divided by that, rounded up:

use unipute::{Kernel, kernel};
#[kernel(workgroup_size(64))]
fn scale(input: &[f32], output: &mut [f32], factor: &f32) {
    let index = global_id().x;
    if index >= input.len() { return; }
    output[index] = input[index] * factor;
}
/// Number of workgroups needed to cover `elements` items.
fn groups_for(elements: u32, workgroup: u32) -> u32 {
    elements.div_ceil(workgroup)
}

fn main() {
    let elements = 100u32;
    let groups = groups_for(elements, scale::WORKGROUP_SIZE[0]);
    assert_eq!(groups, 2); // 2 groups of 64 covers 100
}

Rounding up is why your kernel needs the bounds check. Two groups of 64 is 128 invocations for 100 elements, and the last 28 have to notice and stop. See Your first kernel if that check is not yet a reflex.

The image_blur example does this for two dimensions and prints the numbers, including how many invocations a 1920x1080 picture wastes. See Examples.

Then dispatch:

pass.set_pipeline(&pipeline);
pass.set_bind_group(0, &bind_group, &[]);
pass.dispatch_workgroups(groups, 1, 1);

For two dimensional work, use WORKGROUP_SIZE[1] for the second axis the same way.

Other APIs

The pieces are the same everywhere, only the spelling changes.

Vulkan. Enable spv and hand SPIRV to vkCreateShaderModule as words. BINDINGS maps onto descriptor set layout bindings, with group as the descriptor set number.

Metal. Enable msl and compile MSL with newLibraryWithSource. Metal assigns its own buffer slots rather than using group and binding numbers, so read them from Metal’s reflection instead of BINDINGS.

Direct3D 12. Enable hlsl and compile HLSL with DXC. Bindings appear as register assignments.

OpenGL. Enable glsl and compile GLSL as a compute shader. Bindings appear as layout(binding = ...), and since GLSL has no groups the binding number is the whole address. Choosing targets has the rule that follows from that.

Compiling shaders ahead of time

Nothing stops you writing the generated shaders out to files as part of a build step, since they are just constants:

std::fs::write("shaders/scale.wgsl", scale::WGSL)?;

This is useful for shipping precompiled pipelines, or for feeding SPIR-V through an optimiser before it reaches the driver.

Generating at run time

Everything so far happened during compilation. Sometimes that is not what you want.

When you need this

  • A tool that takes a language on the command line and prints the shader
  • An editor or asset pipeline that shows several targets side by side
  • A program that picks a backend after asking the machine what it supports

In all three the target is not known until the program runs, so a constant is the wrong shape.

Turning it on

[dependencies]
unipute = { version = "0.2", features = ["runtime", "spv", "msl"] }

The runtime feature pulls in the naga backend as a normal dependency rather than only as a compile time one. Which languages it can produce still depends on the target features, so enable the ones you want.

Using it

Every kernel can hand back its IR, and the IR is what the backend takes:

#[cfg(feature = "runtime")]
fn demo() {
use unipute::{Kernel, Target, kernel};

#[kernel(workgroup_size(64))]
fn double(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() {
        output[index] = input[index] * 2.0;
    }
}

let ir = double::ir();
let wgsl = unipute::compile_text(&ir, Target::Wgsl).unwrap();
assert!(wgsl.contains("@compute"));
}

compile_text covers the text languages. SPIR-V is not text, so it has its own function:

let words: Vec<u32> = unipute::compile_spirv(&double::ir())?;

Picking a target by name

Target::from_name turns a string into a target, which is what a command line flag needs:

#[cfg(feature = "runtime")]
fn demo() {
use unipute::Target;

let wanted = "msl";
match Target::from_name(wanted) {
    Some(target) => println!("generating {target}"),
    None => println!("no target called {wanted}"),
}
}

Knowing what this build can do

There are two different questions, and they have two different answers.

Target::is_implemented asks whether Unipute has a backend for it at all. It is false for Ptx, which is reserved and not written yet.

enabled_targets asks what this particular binary can actually produce, which depends on the features it was built with:

#[cfg(feature = "runtime")]
fn demo() {
for target in unipute::enabled_targets() {
    println!("can generate {target}");
}
}

Use enabled_targets when reporting to a user, since it is the honest answer about the program in front of them. Asking for a target that exists but was not compiled in gives a clear error rather than a panic:

target msl is not available, enable the "msl" feature

It is the same pipeline

The compile time path and the run time path are the same code. The only difference is when it runs. The test suite checks this by generating a kernel both ways and comparing the strings, so if the two ever drift apart, that test fails.

You can rely on that. Prototyping through the run time path and then switching to constants will not change your shaders. The retarget example takes a target name on the command line and ends by making that comparison in front of you. See Examples.

Inspecting a kernel without the runtime feature

Kernel::ir() does not need the runtime feature. Only translating does. So you can always look at what a kernel contains:

use unipute::{Kernel, kernel};
#[kernel(workgroup_size(64))]
fn double(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() { output[index] = input[index] * 2.0; }
}
fn main() {
let ir = double::ir();
println!("{} resources, {} statements", ir.resources.len(), ir.body.len());
}

This is useful for writing your own tooling, and it is the same door a future backend would come in through. The inspect_ir example walks a whole kernel this way, printing its body back out and counting what is in it.

Examples

The repository has a set of runnable examples in examples/. Each one is about a different part of the job, so between them they cover most of what this book describes.

Five of them run their kernels on a GPU and check the answer. Unipute stops at the shader, so the wgpu code that takes it from there lives in examples/host/mod.rs, shared by those examples and by the GPU tests. It is the Running a kernel chapter written out in full, and it is not part of the library: wgpu is a dev dependency of the repository, not of Unipute. An example that needs a device says so and exits if the machine has none.

The other three never touch a device. They print what a kernel becomes.

emit_shaders

One small kernel, every target it was built with.

cargo run --example emit_shaders --features "spv,msl,hlsl,glsl"

The fastest way to see what a kernel actually becomes, and a good first check when a shader is not behaving.

image_blur

A two dimensional kernel, run over a picture you can see.

cargo run --example image_blur

A 3x3 blur over a flat image buffer, with taps clamped to the edge. The host works out the workgroup count for a 40x12 picture, including how many invocations land outside it and return early, runs the blur, prints the picture before and after as characters, and checks the result against the same filter written as two plain loops. See Running a kernel for the dispatch arithmetic in prose.

prefix_sum

An algorithm that does not fit in one dispatch.

cargo run --example prefix_sum

A prefix sum needs every element to know the sum of the ones before it, which no single pass can do. This is three kernels and fourteen dispatches, ping ponging between two buffers, with the three results compared against a running total on the CPU at the end. It also shows what a second bind group is for: the step size is the only thing that changes between passes, so it is bound on its own and group 0 is bound once. The step takes binding index 2 rather than 0, which is the GLSL rule at work.

nbody

Vector types from one end to the other.

cargo run --example nbody

A direct n-body simulation, run for twenty steps. Buffers of Vec4<f32>, a helper that takes vectors and returns one, swizzles to pick the position out of each body, and dot and inverse_sqrt doing the work. The first step is checked against the same maths on the CPU, and the total momentum is printed as the simulation goes, since every pull has an equal and opposite one and the physics says it should not change. Packing position and mass into one Vec4 is the sort of thing that matters in a kernel that reads every body once per invocation.

particles

Structs in buffers, from the host to the kernel and back.

cargo run --example particles

A particle system where each particle is one #[repr(C)] struct with #[derive(Layout)] on it, held in a single buffer, and the settings for a step are another struct bound as a uniform. The kernel reads fields through an index, passes a whole particle to a helper and writes the one it gets back. The two structs between them show the one layout rule worth knowing: a three component vector wants a scalar after it, and where that is not possible a padding field makes up the difference. The first step is checked against the same maths on the CPU, and the host counts the particles still alive as the steps go by.

histogram

Atomics, in workgroup memory and in a buffer.

cargo run --example histogram

A brightness histogram of an image, where thousands of pixels want to add one to the same sixteen bins at the same time. Each workgroup counts into its own [AtomicU32; 16] with fetch_add, and after the barrier sixteen lanes add the workgroup’s bins into the buffer, which every other workgroup is adding into as well. The host prints the bins as bars and checks each against a count made on the CPU, which comes out exact, because nothing was lost on the way.

inspect_ir

Reading a kernel back as data.

cargo run --example inspect_ir

A Mandelbrot kernel, printed back out of its own IR and then counted: statements, branches, loop nesting, which built-ins and which maths functions it reaches for. It is a walk over Kernel::ir(), which is the same door a new back end, a linter or an editor tooltip would come in through. Nothing here needs a target feature, since nothing here generates a shader.

retarget

Choosing a target while the program runs.

cargo run --example retarget --features "runtime,msl,spv" -- msl spirv

A small command line tool over the runtime feature. Run it with no arguments and it prints what this build can produce, which is not the same question as what Unipute can produce. At the end it regenerates WGSL at run time and compares it to the constant the macro baked in, which is the claim Generating at run time makes, checked in front of you.

How it works

You do not need this chapter to use Unipute. It is here because knowing the shape of the thing makes its error messages and its limits make more sense.

Three stages

your Rust function
        |
        |  the #[kernel] macro reads it
        v
   Unipute IR          a small description of what the kernel does
        |
        |  a backend reads that
        v
WGSL, SPIR-V, MSL, HLSL, GLSL

The middle stage is the whole trick. It is a plain data description of a kernel: which buffers there are, which variables, and what the body does. It knows nothing about Rust and nothing about any shader language.

That has a practical consequence. The front end and the backend never talk to each other, so either can be replaced. A backend for a completely different kind of target, or a front end that reads a different language, both plug into the same middle without disturbing the other side.

What the macro does

When the compiler reaches your #[kernel] function, the macro runs. In order:

  1. Reads the attribute. Workgroup size, and a name if you set one.
  2. Reads the signature. Each parameter becomes a buffer, with its access worked out from whether it is &[T], &mut [T] or neither, and its binding from its position or its #[binding(...)].
  3. Reads the body. Statements and expressions become IR, with names resolved against a scope stack as it goes. This is where an unknown function or an unsupported statement is caught, which is why those errors point at the exact line.
  4. Runs the backend, during your compilation. For each target feature you enabled, the IR is lowered to naga IR, validated, and written out.
  5. Emits the replacement. Your function is gone, replaced by a type of the same name carrying the shaders and the layout, plus a method that rebuilds the IR.

Step 4 is the unusual one. The shader compiler runs inside the proc macro, so the finished shader is a string literal in your crate by the time your code is type checked.

How a kernel learns about a struct

A proc macro sees the one item it is attached to. When a kernel names a struct, the fields are on a different item, and the macro cannot go and read it. So #[derive(Layout)] does the reading, and leaves a small macro_rules! next to the struct, under the struct’s own name, holding the fields.

When #[kernel] meets a name it does not know, it does not expand to a kernel at all. It expands to a call of that macro, handing along the whole function. The macro adds the struct’s fields and calls back into Unipute, which either asks about the next unknown name the same way or, once it has them all, runs the five steps above. Every token of your function is passed through untouched, so errors still point at the line you wrote.

Two things follow. A struct and the kernels using it have to be in the same crate, since the macro the derive leaves behind is not something a crate can re-export. And a type that has no derive gets two errors: one that the macro under its name could not be found, and one from the Layout trait saying what to do about it.

Why your function disappears

A kernel is not a function your CPU can call. It describes work for a different processor with a different memory model. Leaving a callable Rust function behind would be offering something that cannot work.

So the macro replaces it with a type of the same name. scale stops being a function and becomes a type carrying scale::WGSL, scale::BINDINGS and the rest. Running kernels on the CPU, which would be genuinely useful for testing, is on the roadmap and not here yet.

Where validation happens

Three different things can reject a kernel, at three different points, and the message tells you which.

The macro, reading your body. Unsupported syntax, an unknown function, a name that is not in scope, two parameters on the same binding. These point at your source with a span, and are the ones you will see most.

Lowering, building naga IR. Things that are structurally wrong, like a zero workgroup dimension or a uniform declared as a slice. Still a compile error against your kernel.

Naga’s validator. If a module Unipute built gets this far and is rejected, that means Unipute built something wrong. The message says the module was rejected and passes naga’s own text through. If you see one of these it is a bug worth reporting, not something to work around.

Why naga

Naga is the shader translator from the wgpu project. It already knows how to write five languages and, importantly, how to validate a module before writing. Reimplementing that would have been a lot of work to arrive somewhere worse.

Unipute does not expose naga in its public API, though, and unipute-ir does not depend on it. That is deliberate. A target that naga has no concept of, such as PTX, would be a sibling of the naga backend rather than something bolted onto it.

Why the generated shaders look like that

If you have read the output, you will have noticed a lot of this:

let _e9 = index;
let _e11 = index;
let _e13 = input[_e11];

Those temporaries are naga’s doing, not Unipute’s. Naga keeps expressions in a flat list where each one is named, and its writers print that list literally rather than trying to reconstruct nested expressions.

It is noisier than handwritten code and completely equivalent. Every shader compiler downstream folds it away immediately. If you are reading the output to check your kernel, read past the temporaries and look at the stores and the control flow.

What the crates are

CrateJob
uniputewhat you depend on: the macro, the traits, the layout types
unipute-irthe IR. No dependencies at all
unipute-macrosthe Rust front end
unipute-nagathe naga backend

unipute-ir having no dependencies is load bearing rather than tidiness. It is what lets a new front end or backend be added without inheriting a shader compiler it has no use for.

For the level of detail a contributor needs, including exactly where a new backend or a graphics stage would attach, see the architecture notes.

When something goes wrong

Unipute tries to tell you what to do rather than only what went wrong. This chapter covers the messages that need more explanation than fits in one line, grouped by what you were probably trying to do.

Writing the kernel signature

a kernel needs a workgroup size, write #[kernel(workgroup_size(64))]

The attribute has one required option. Even a kernel that does one item of work per invocation needs to say how many invocations a group runs.

a kernel returns nothing, write results into a &mut parameter

Kernels produce nothing. There is nowhere for a return value to go, because there is no caller waiting for one. Take a &mut [T] and write into it.

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn total(input: &[f32], output: &mut [f32]) {
    // not `-> f32`
    output[0] = input[0] + input[1];
}

a &mut parameter must be a slice, write &mut [f32]

&mut f32 is not a thing a kernel can have. A single mutable value has no meaning when thousands of invocations run at once. Use &mut [f32] and index it, even if you only ever use index 0.

a kernel cannot be generic

Shaders are concrete. If you want the same kernel over f32 and u32, write two kernels, or generate them with a macro of your own.

Writing the body

X is not a built-in Unipute provides

You called something that is not on the list. A kernel cannot call your other Rust functions, only the built-ins in Built-ins and functions.

If the function you want is a normal numeric one that is missing, that is worth reporting. If you wanted to call your own function, see What is not built yet.

X is not a kernel parameter or a local variable

The name is not in scope. Kernels cannot see anything outside themselves, so constants, statics and other items from your crate are not available. Pass the value in as a uniform.

const SCALE: f32 = 2.0;

#[kernel(workgroup_size(64))]
fn nope(output: &mut [f32]) {
    output[0] = SCALE; // SCALE is not visible in here
}
use unipute::kernel;
#[kernel(workgroup_size(64))]
fn yes(output: &mut [f32], scale: &f32) {
    output[0] = scale; // pass it in instead
}

this statement has no effect, a kernel statement must assign, branch, loop or call a built-in

Usually an expression written where a statement was expected, often because if was being used as an expression:

let sign = if value < 0.0 { -1.0 } else { 1.0 };

Assign in both branches instead, or use step and mix.

the type of this value is unclear, annotate it as in let x: f32 = ...

Type inference here is deliberately simple, and it ran out of information. Add a suffix or an annotation:

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn annotated(output: &mut [u32]) {
    let counter = 0u32;
    let other: u32 = 1;
    output[0] = counter + other;
}

X is a buffer, assign to one element as in X[i] = ...

You assigned to a whole buffer. Buffers are not values that can be copied wholesale, so assign per element inside a loop or per invocation.

X is a uniform, it cannot be written to

Uniforms are read only. If you need to write, take the parameter as &mut [T].

workgroup memory is declared at the top level of the kernel body, not inside a block

#[workgroup] let goes at the top of the kernel body, next to any nested fn items, not inside an if or a loop. It exists for the whole kernel either way, so the top level is the only place that says so. A nested fn cannot declare it either, for the reason Your own functions gives. See Workgroup memory.

Indices and types

X cannot be indexed

Indexing works on slice parameters. Indexing a scalar or a vector is not allowed, and to read a vector component use .x through .w rather than [0].

usize errors, or a length not matching

Lengths and indices are u32, not usize. There is no usize on a GPU. If you are comparing a .len() against something, make sure both sides are u32.

cannot find macro X in this scope, followed by X cannot be used in a kernel

The kernel names a type Unipute does not know. If X is a struct of yours, put #[derive(unipute::Layout)] on it, and make sure it is in the same crate as the kernel. If it is anything else, the type list is short on purpose: see Types for what is there. f64, u64, u8 and usize are all deliberately absent.

The first of the two errors is how the kernel macro asks a struct for its fields, which How it works explains. The second one is the useful message.

X sits at byte 4 on the CPU and byte 16 on the GPU, add _pad: [u8; 12] before it

Rust and the GPU lay the struct out differently, almost always because a three component vector follows a scalar. Add the padding field the message names, or move the vector before the scalar. Your own structs has the rules.

this is an atomic, read it with .load() or update it with one of the fetch_ methods

An AtomicU32 or AtomicI32 is not read or assigned like a number. x.load() reads it, x.store(v) writes it, x.fetch_add(v) and the rest update it in one step. The same goes for +=, which has its own message naming the method. Atomics has the list.

X needs #[repr(C)], so that its fields sit where the kernel expects them

Add it. Without it Rust may reorder the fields, and then nothing about where they are can be relied on.

Bindings

X and Y both use group 0 binding 0, set one with #[binding(...)]

Two parameters landed on the same slot, which can only happen if you set at least one by hand. Give them different index values, or different group values.

Bindings not matching what the host expects

Read them from the kernel rather than writing them out twice:

use unipute::{Kernel, kernel};
#[kernel(workgroup_size(64))]
fn scale(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index < input.len() { output[index] = input[index]; }
}
fn main() {
for binding in scale::BINDINGS {
    println!("{} -> group {} binding {}", binding.name, binding.group, binding.binding);
}
}

On Metal this does not apply, because Metal assigns its own buffer slots. See Choosing targets.

Targets

cannot find WGSL in this scope, or similar

Import the trait:

use unipute::WgslKernel; // for WGSL
use unipute::SpirvKernel; // for SPIRV

target msl is not available, enable the "msl" feature

The target exists but this build was not compiled with it. Add the feature.

the naga back end does not support the vertex stage yet

Graphics stages are reserved in the IR and not implemented. Compute only for now.

Things that are not your fault

naga rejected the generated module: ...

This one is different from all the others. It means Unipute built a module that naga considered invalid, which is a bug in Unipute rather than in your kernel. The naga message is passed through unchanged so it can go in a report.

Worth reporting with the kernel that caused it. There is no workaround to look for, and you have not done anything wrong.

Runtime problems Unipute cannot catch

These compile fine and go wrong on the GPU.

Reading past the end of a buffer. There is no bounds checking in a kernel. Guard with .len():

use unipute::kernel;
#[kernel(workgroup_size(64))]
fn guarded(input: &[f32], output: &mut [f32]) {
    let index = global_id().x;
    if index >= input.len() {
        return;
    }
    output[index] = input[index];
}

Forgetting that dispatch rounds up. 100 elements with a workgroup of 64 is 128 invocations. The check above is what stops the extra 28.

A barrier inside a branch. Every invocation in a workgroup has to reach the same barrier. Unipute does not check this yet. Keep barriers at the top level of the body.

An early return before a barrier. The same mistake in another shape. The invocations past the end of the input leave, and the rest of the group waits at the barrier for invocations that are never coming. Guard the work with an if instead and let every invocation reach the barrier. Workgroup memory shows the pattern.

A Vec3 taking four slots. Three component vectors are padded to the size of four in a buffer. Your host side allocation has to agree, or everything after the first element reads from the wrong place.

Races between invocations. Two invocations writing the same element is undefined. If invocations need to see each other’s writes, a barrier is required, and even then only within one workgroup.

Still stuck

Look at the shader. It is the fastest way to find out what your kernel actually became:

cargo run --example emit_shaders

Or in your own code, print YourKernel::WGSL. Read past naga’s _e12 temporaries and look at the stores and the control flow. See How it works for why the output reads that way.

What is not built yet

This is version 0.2. Rather than let you find the edges by walking into them, here is the list.

What does work

Compute kernels, end to end, across WGSL, SPIR-V, MSL, HLSL and GLSL. Scalars, vectors and their swizzles, your own structs, storage buffers and uniforms, the control flow in Control flow, the functions in Built-ins and functions, the nested functions in Your own functions, workgroup memory and barriers, atomics, and binding layout you can read from the host.

That part is tested and is what the rest of this book describes.

Sharing a function between kernels

A helper declared inside one kernel belongs to that kernel. Two kernels wanting the same helper each declare their own copy.

Fixing this needs a surface #[kernel] does not have, because a macro attached to one function cannot read another one. The likely shape is a module level macro wrapping several kernels and their shared helpers at once.

The rest of structs

A struct’s fields are scalars and vectors. A struct holding another struct, or an array, is refused by the derive. The IR and the naga back end can already represent both, so the missing piece is the derive working out the host layout for them, and the uniform rules, which put stricter alignment on a nested struct than a storage buffer does.

A struct has to be in the same crate as the kernels that name it. The kernel macro learns a struct’s fields through a macro_rules! the derive leaves next to it, and that is the one kind of item Rust will not let a crate re-export by path. A struct from a dependency would need another way of handing the fields over, which is the same question the serialised IR under Other languages answers.

The rest of atomics

An atomic is a u32 or an i32 in a buffer or in workgroup memory. Three things are missing around that.

An atomic as a struct field. The IR and the back end allow it. What is not worked out is the host side, where the field would be a u32 to one reader and an AtomicU32 to another, and how the derive should spell that.

Float atomics. Naga has atomicAdd on an f32 behind a capability that only some hardware offers. Turning it on would mean a kernel that compiles for one device and not another, which needs a story about capabilities first.

An old value from compare_exchange. It answers whether the exchange happened. The old value is there in the IR, but a target may fail the exchange with the values matching, so handing the old value back on its own would invite a loop that never ends. The two together are Rust’s Result<T, T>, which the kernel language has no way to spell yet.

Matrices

No matrix types. Multiply vectors by hand, or pass the elements separately.

Textures and samplers

Not available. These mostly matter once graphics stages exist, so the two are likely to arrive together.

Graphics stages

Compute only. Stage::Vertex and Stage::Fragment exist in the IR and are rejected with a clear message rather than a confusing one, which is the extent of the support today.

This is the biggest single piece of work, because it needs location based bindings, return types on kernels, the graphics built-ins and matrices. The architecture notes break it down.

Backends other than naga

Naga gives five languages. Anything naga does not target, meaning CUDA and PTX in particular, needs its own backend.

Target::Ptx is reserved and the Backend trait is deliberately vague about what a backend produces, so one can be added without disturbing the naga path. CUDA’s memory model does not have bind groups, so that mapping would live in the new backend rather than in the IR.

Other languages

Two different things people mean by this.

Calling Unipute from C or Zig, to generate shaders from a host program written in something other than Rust. That needs a C ABI layer over the IR. Because the IR is plain data with no borrowed lifetimes, this is an opaque pointer API and not much more.

Writing kernels in C or Zig. That is a new front end rather than a binding layer. It produces IR the same way the Rust macro does, and everything downstream is unchanged. The likely first step is a serialised form of the IR so a front end in any language can emit it as data.

Neither exists. The IR being dependency free is the groundwork for both.

Running kernels on the CPU

Your #[kernel] function stops being callable from Rust. Being able to run a kernel on the CPU, for testing without a GPU, would be genuinely useful and is not there.

How to read this list

Everything here is a gap rather than a decision against. The one with the clearest path is matrices, since naga already has them and a matrix that stays inside a kernel does not touch the host. One in a buffer does, because the host has to lay out its columns the way the shader reads them.

If one of these is blocking you, saying so is useful. It is easier to prioritise against a real kernel someone is trying to write than against a guess.

Architecture notes

The rest of this book is for people using Unipute. This page is for people working on it, and goes into more detail than a user needs.

It is the same file as docs/ARCHITECTURE.md in the repository.

Architecture

This describes how Unipute is put together and where the pieces that are not built yet will attach. It is written for anyone working on Unipute itself.

The shape of it

front end            IR                 back end           output
---------            --                 --------           ------
#[kernel] macro  ->  unipute-ir     ->  unipute-naga   ->  WGSL, SPIR-V,
(unipute-macros)     Kernel             lower + write      MSL, HLSL, GLSL

Three properties follow from that shape, and every change should preserve them.

The IR has no dependencies. unipute-ir is plain data: structs, enums, String and Vec. It pulls in nothing, not a shader compiler and not a graphics library. That is what lets a new front end or back end be added without dragging the rest of the project along.

Neither end knows about the other. The macro never mentions naga. The naga back end never mentions Rust syntax. They meet at Kernel and nowhere else.

Nothing touches a device. No crate here creates an instance, a queue or a buffer. Output is text or a word buffer, and the caller decides what to do with it. This is what “graphics API agnostic” means in practice, and it is why the layout information in BindingInfo names groups and bindings rather than any particular API’s descriptor types.

Crates

CrateDepends onPurpose
unipute-irnothingKernel, Function, Type, Expr, Stmt, the Backend trait, Target
unipute-macrossyn, quote, unipute-ir, unipute-nagathe #[kernel] attribute
unipute-naganaga, unipute-irlowering to naga IR and writing shaders
uniputethe three abovethe crate users depend on

unipute-macros depends on unipute-naga because the whole point is that the back end runs during compilation. Cargo builds a proc macro for the host, so the two copies of unipute-naga in a build (host and target) have their features resolved separately. That is why the facade’s target features name both, as in spv = ["unipute-macros/spv", "unipute-naga?/spv"].

How a kernel becomes a shader

  1. unipute_macros::kernel parses the attribute and the function with syn.
  2. parse::kernel walks the signature into Resources and the body into Stmts, resolving names against a scope stack. Untyped integer literals take their type from the other side of the operator they appear in. Nested fn items are read first, since a call has to resolve against a signature whichever order the two were written in. A #[workgroup] let becomes a Shared entry on the kernel rather than a local, and produces no statement.
  3. unipute_naga::lower builds a naga::Module. Two naga rules shape it: expressions must appear before their users, and most expressions have to sit inside a Statement::Emit range while a specific few must not. The ones that must not, which are literals, globals, locals and function arguments, are all created before the first emit range opens and looked up from a cache afterwards. A call’s result is in that group too, but it cannot be made up front, so a call closes the open range and opens a new one instead. Helpers are lowered before the entry point. Workgroup memory is a global variable in naga’s workgroup address space, declared next to the resources and looked up the same way, with no binding.
  4. unipute_naga::validate runs naga’s validator. A failure here means a bug in Unipute, not in the user’s kernel, so the message is passed through as it is.
  5. Each enabled target’s writer runs, and generate::kernel_type pastes the result into the caller’s crate as a constant.

The function is replaced by a type of the same name implementing Kernel, plus one trait per enabled target. Keeping the shader on a trait rather than an inherent constant means a caller can be generic over kernels.

Extension points

Four things are designed for and not implemented. Each one already has a place to attach.

Other back ends, such as CUDA and PTX

Attaches to: unipute_ir::Backend and unipute_ir::Target.

Backend is deliberately vague about output. It has an associated Output type, so a back end can produce a String of PTX, a word buffer, or machine code, and callers written against the trait do not care which.

Target::Ptx is already in the enum and Target::is_implemented already returns false for it, so the rest of the project can talk about a target before the generator for it exists.

What a PTX back end needs:

  • A new crate, unipute-ptx, depending on unipute-ir and nothing from naga.
  • impl Backend for PtxBackend, mirroring NagaBackend.
  • A feature on the facade and on unipute-macros, plus a #[cfg] block in generate::compile_targets and a PtxKernel trait.

What it does not need: any change to unipute-ir or unipute-macros beyond that feature. If a PTX back end turns out to need an IR change, that is a signal the IR has grown a naga assumption, and the fix belongs in the IR rather than in the new back end.

The one real gap is that CUDA’s memory model does not line up with bind groups. Resource names a group and a binding, which a PTX back end would have to flatten into kernel parameters. That mapping belongs in the back end, not in the IR.

Graphics stages

Attaches to: unipute_ir::Stage.

Stage already has Vertex and Fragment alongside Compute, and it is #[non_exhaustive] so more can be added. Stage::is_implemented returns false for both, and lower checks it first, so a graphics kernel fails with a clear message today rather than a confusing one.

What graphics support needs:

  • Binding::Location in the IR, since a vertex shader passes values to a fragment shader by location rather than by built-in. This is the one change that does touch the IR.
  • A return type on Kernel, since a vertex shader produces a position and a fragment shader produces a colour. Today check_signature rejects any return type outright.
  • The graphics built-ins: Position, VertexIndex, InstanceIndex, FragDepth and so on. ir::BuiltIn is a flat enum, so these are additions rather than restructuring.
  • Matrix types in ir::Type, which is the other likely IR change.
  • #[kernel(vertex)] and #[kernel(fragment)] in the attribute parser, in place of the workgroup_size that compute requires.

Naga handles all of this already, so unipute-naga is mostly a matter of mapping the new variants across.

Bindings for C, C++ and other languages

Attaches to: unipute-ir, from the other side.

There are two different things people mean by bindings, and they need different work.

Calling Unipute from C. A host program written in C or Zig wants to hand Unipute a kernel and get a shader back. That needs an unipute-ffi crate with extern "C" functions over Kernel, built as a cdylib and staticlib, plus a generated header. The natural surface is narrow: build a kernel, add resources and statements, compile to a target, free the result. Because the IR is plain data with no borrowed lifetimes inside it, this is an opaque pointer API and not much more.

Writing kernels in another language. A Zig or C++ programmer wants to write the kernel itself in their own language. That is a new front end, not an FFI layer. It produces a Kernel the same way unipute-macros does, and from there the pipeline is identical. Nothing in unipute-ir or unipute-naga needs to change, which is the whole reason the IR has no dependencies.

The likely shape is a serialised form of Kernel, so a front end written in any language can emit it as data and hand it to a small Rust driver. That means adding optional serde support to unipute-ir, which is the one piece of preparation worth doing before anyone starts.

More of the Rust language in kernels

The front end covers a useful subset and rejects the rest with a message. The pieces most likely to be wanted next, in rough order of value:

  • Structs nested in structs, and arrays as struct fields. The IR and the back end take both already; the derive does not, see Structs.
  • Atomics as struct fields, for the same reason.
  • Textures and samplers, which mostly matter once graphics stages land.

Calling other functions is done, as nested fn items inside a kernel body. See Functions below for the shape of it and what is left. User defined structs are done too, and Structs is about how a macro on one item came to know about another. So are atomics, see Atomics.

Atomics

Type::Atomic(Scalar) is a u32 or i32 that invocations update through Expr::Atomic, Expr::AtomicCompareExchange and Stmt::Atomic. Reading one is an ordinary Expr::Index or Expr::Shared of its place and writing one is an ordinary Stmt::Store; naga’s writers see the pointer’s type and spell both as the atomic load and store of each language. The IR keeps atomics as a type rather than a flag on the operation because that is what every target does: the declaration says the memory is atomic, and the operations follow.

In naga an atomic operation is a statement that binds its result to an Expression::AtomicResult, the same arrangement as a call, so the lowering is the same dance: close the emit range, push the statement, open a new one. Stmt::Atomic pushes it with no result, except for an exchange, where naga insists on one. A compare and exchange binds naga’s predeclared two member struct, which Module::generate_predeclared_type makes and the writers know by name, and only the exchanged member is handed on. The old value would not say whether a target that is allowed to fail spuriously actually swapped.

The front end treats an atomic as a place and never a value. Scope::expr refuses an atomic typed result, so a bare read is an error pointing at .load(), and only a method receiver or the left side of an assignment goes through place_or_value, which allows it. Assignment and compound assignment to an atomic are caught there too, each naming the method to use instead. Uniforms, read only buffers, locals and helper parameters refuse the type outright, and the back end checks the buffer case again.

Structs

Type::Struct carries a StructType: a name, members with explicit byte offsets, and a size. The offsets are redundant in the sense that StructType::new computes them from the member types by the rules in Type::alignment, and that redundancy is deliberate. A host reading the IR should not need to know the rules, and a back end should not trust a front end to have followed them, so unipute-naga recomputes the layout and refuses a struct whose offsets differ. Naga’s writers lay a struct out by their own reckoning and only the SPIR-V one reads offsets back, so an unusual layout would come out five different ways rather than one wrong way.

The rules are the ones WGSL, SPIR-V, MSL and HLSL share for a storage buffer: scalars 4 bytes aligned to 4, vec2 8 aligned to 8, vec3 12 aligned to 16, vec4 16 aligned to 16, each member at the next multiple of its alignment, the struct rounded up to its largest. For a uniform the same rules hold as long as no member is itself a struct or an array, which is why the derive stops there for now: a nested struct in a uniform needs 16 byte alignment that a storage buffer does not, and one StructType cannot carry both.

How #[kernel] learns the fields. A proc macro sees the item it is on and nothing else. #[derive(Layout)] therefore leaves a macro_rules! next to the struct, holding its fields, and re-exports it under the struct’s own name with pub(crate) use. A macro and a type live in different namespaces, so the two do not collide, and a use of the struct brings the macro along. When #[kernel] finds a type name it does not know, it expands to a call of that macro, passing the names still unknown, the definitions gathered so far, the attribute and the whole function. The macro adds its struct and calls unipute::__kernel_with_layouts, which repeats until nothing is pending and then expands the kernel for real. crates/unipute-macros/src/chain.rs is the whole of it, and layout.rs is the derive.

Two consequences are worth knowing. A macro_rules! without #[macro_export] cannot be pub used out of its crate, and one with it lands at the crate root where a macro expanded macro cannot be reached by path (the macro_expanded_macro_exports_accessed_by_absolute_paths lint), so a struct and the kernels naming it have to share a crate. And a name that is not a Layout struct fails as “cannot find macro”, which is why the kernel macro also emits a Layout trait bound on every unknown name: the trait’s on_unimplemented message is the one that says what to do.

Host layout. The derive computes the #[repr(C)] layout from the same syntax, compares the two, and reports the first field that differs, with the padding field that would fix it. It emits offset_of! and size_of assertions as well, so a mistake in that computation would fail the build rather than pass silently. Vectors are spelled as arrays on the host, [f32; 3] for Vec3<f32>, because a host type with the GPU’s alignment would need a size Rust cannot give it: a vec3 is 12 bytes aligned to 16, and in Rust size is always a multiple of alignment.

Functions

A kernel body can declare fn items and call them. Three things about the design are worth knowing before changing any of it.

A helper captures nothing. ir::Function has parameters, locals and a body, and no access to resources, workgroup memory or built-ins. That is not a simplification: shader languages hand bindings and built-ins to the entry point, and a function called from it cannot ask for them. It also happens to match what a nested fn means in Rust, so the rule needs no explaining to someone reading a kernel.

Kernel::functions is ordered, callees first. No shader language has forward declarations, so a callee has to be written before its callers. The order is an invariant of the IR rather than something each back end works out, and the front end is what establishes it. parse::function::order reads the call graph off the syntax with a syn::visit::Visit before any body is parsed, which is also where a cycle is caught. Recursion cannot be represented in the IR at all, which is right, since no target supports it.

Naga has no call expression. Statement::Call binds its result to an Expression::CallResult, and that result is one of the expressions naga requires to sit outside an emit range. So lowering a call in the middle of an expression means closing the current range, pushing the call statement, and opening a new one. That is why lower_expr carries a &mut Block and a &mut Emitter rather than just returning a handle.

What is not done: sharing a helper between two kernels. #[kernel] is an attribute on one function and cannot see its siblings, so this needs a different surface, most likely a module level macro wrapping several kernels and their shared helpers together. Nothing in the IR would have to change for it.

Testing

  • crates/unipute-naga/tests/lowering.rs builds IR by hand and checks the generated shaders. This is where lowering bugs surface.
  • tests/kernels.rs goes from Rust source through the macro to shader text for every target, and checks that the compile time and run time paths agree.
  • tests/ui/ pins the error messages, since those are most of what a macro’s users experience. Run with TRYBUILD=overwrite to refresh them after changing a message on purpose.
  • tests/gpu.rs runs kernels on whatever adapter wgpu finds and compares the results with the same computation on the CPU. It is the only place a kernel that is well formed and wrong gets caught. It feeds wgpu the WGSL, SPIR-V and GLSL the crate writes, and the naga module straight out of lower, so each of those paths is exercised on a device rather than only through the validator. A machine with no adapter skips it with a note. CI installs a software Vulkan driver on Linux and sets UNIPUTE_REQUIRE_GPU there, so a missing adapter is a failure rather than a silent skip. The wgpu glue is examples/host/mod.rs, shared with the examples and outside the library.

When adding a back end or a stage, the first test to write is a lowering test, because it fails in a readable way. The macro tests are for the front end.

Toolchain

rust-toolchain.toml pins nightly, and .cargo/config.toml builds the dev profile with the cranelift codegen backend. That backend is the only reason for the pin. It compiles unoptimised binaries faster than LLVM, which is worth having on a workspace that rebuilds naga and a proc macro, and the cargo keys that select it are unstable.

It is a build speed setting and a constraint at the same time, so treat it as a rule rather than a detail:

  • No nightly language or library features anywhere. Not in the public API, not in the macro, not in tests or examples. Someone on stable has to be able to depend on Unipute and to work on it.
  • The cranelift keys stay in .cargo/config.toml. They belonged in Cargo.toml once, which meant the manifest carried cargo-features and stable cargo could not parse it, so every project depending on Unipute broke. A cargo config only applies to commands run inside this repository, so a dependent never sees it.
  • Nightly is not a capability to build on. If something seems to need a nightly feature, that is a design problem to solve rather than a feature to reach for.

rustup installs the cranelift component from the toolchain file, so a fresh checkout builds as is. To turn cranelift off, comment out the [unstable] and [profile.dev] sections of .cargo/config.toml. While they are there a stable cargo cannot read that file, so commenting them out is also what lets you run cargo +stable test locally.

CI runs the pinned nightly for testing and linting, and a separate job builds a crate against Unipute on stable from outside the checkout, where neither the pin nor the cargo config applies. Anything nightly only that reaches the manifest or the code fails there.