Observe / Harden · Linux userspace / kernel ABI

io_uring ring and request lifetimes

Examines how shared SQ/CQ rings, submission entries, kernel requests, completion entries, user buffers, and fd lifetimes overlap in asynchronous I/O.

Series
36 / 38
Build
cc -std=c17 -Wall -Wextra -O2 uring_read.c -luring -o uring_read
Run
./uring_read /etc/hostname
Kernel
Linux 6.18.37 LTS

Can a submitted buffer be reused as soon as io_uring_enter returns?

io_uring setup creates submission and completion ring memory shared by the kernel and userspace. Userspace fills an SQE, publishes the tail, and notifies the kernel with enter; after the kernel request executes, the kernel writes a CQE.

A submit syscall returning does not mean the operation is complete. The operation may refer to a user buffer and file until its CQE is consumed, so track per-request ownership and cancellation results.

Structure diagram

Figure 1. Shared SQ/CQ rings and in-flight requests

Submission Queue

head 2tail 6
0free1free2READ#413WRITE#424TIMEOUT#435NOP#446free7free

Kernel in-flight

head 0tail 3
0req#41 · buffer A1req#42 · buffer B2req#43 · timer3worker/driver

Completion Queue

head 1tail 4
0seen1CQE#422CQE#413CQE#434free5free6free7free

An SQE slot can be reused after the kernel consumes it, but the user buffer must remain valid until CQE completion. The two lifetimes differ.

Call path

Figure 2. From userspace code to observable results
io_uring_setup ring fd + mmap offsets
fill SQE opcode/fd/buffer
publish SQ tail memory ordering
kernel request asynchronous/synchronous execution
CQE consume res/user_data

Draw the SQE-slot lifetime, request lifetime, user-buffer lifetime, and CQE lifetime on one overlapping timeline. Slot reuse and buffer reuse have different completion conditions.

Figure 3. Major points along the kernel-internal path
io_submit_sqes SQE copy/validate
io_init_req io_kiocb
issue file operation
io_req_complete finalize result
io_cqring_fill_event CQ publish

This diagram is not for memorizing function names. Follow which return value, file descriptor, memory mapping, or wait queue is passed to the next stage.

Linux 6.18.37 LTS source locations

Go beyond the glibc function to the files where the syscall implementation meets kernel objects. Each link points to the original file at the same tag.

FileFunction / structureWhat to inspect
io_uring/io_uring.c io_uring_setup(), io_uring_enter() Create ring context and enter submission/completion paths
io_uring/io_uring.c io_submit_sqes(), io_init_req() Convert an SQE into a kernel request
io_uring/io_uring.c io_req_complete_post(), io_cqring_event_overflow() Publish a CQE and handle overflow

Complete runnable example

The code below is not pseudocode with explanatory lines omitted. It is a minimal example that can be built and run as a file.

Buildcc -std=c17 -Wall -Wextra -O2 uring_read.c -luring -o uring_read
01#define _DEFAULT_SOURCE
02#include <fcntl.h>
03#include <liburing.h>
04#include <stdio.h>
05#include <string.h>
06#include <unistd.h>
07
08int main(int argc, char **argv)
09{
10    if (argc != 2)
11        return 2;
12    int fd = open(argv[1], O_RDONLY | O_CLOEXEC);
13    struct io_uring ring;
14    if (fd < 0 || io_uring_queue_init(8, &ring, 0) < 0)
15        return 1;
16
17    char buffer[256];
18    struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
19    io_uring_prep_read(sqe, fd, buffer, sizeof(buffer), 0);
20    io_uring_sqe_set_data64(sqe, 0x1001);
21    if (io_uring_submit(&ring) < 0)
22        return 1;
23
24    struct io_uring_cqe *cqe;
25    if (io_uring_wait_cqe(&ring, &cqe) < 0)
26        return 1;
27    int result = cqe->res;
28    if (result > 0)
29        write(STDOUT_FILENO, buffer, (size_t)result);
30    else if (result < 0)
31        fprintf(stderr, "read: %s\n", strerror(-result));
32    io_uring_cqe_seen(&ring, cqe);
33    io_uring_queue_exit(&ring);
34    close(fd);
35    return result < 0;
36}

Run results

The example was run on x86_64 Linux.

Execution environment

Built and run on 2026-11-04 on x86_64 Debian (Linux 6.12.94, gcc 14.2.0, glibc 2.41, -std=c17). This differs from the article baseline (ARM64, Linux 6.18.37 LTS), so PIDs, inodes, and addresses differ from run to run.

Before the fix, the code failed to compile under -std=c17 with 19 errors because sigset_t used by liburing.h was undeclared. The code above adds #define _DEFAULT_SOURCE as its first line; it builds without warnings and reads /etc/hostname.

$ cc -std=c17 -Wall -Wextra -O2 uring_read.c -luring -o uring_read   # before the fix
/usr/include/liburing.h:176:24: error: unknown type name ‘sigset_t’; did you mean ‘size_t’?
$ ./uring_read /etc/hostname   # after adding #define _DEFAULT_SOURCE
host
[exit code: 0]
Terminal capture of the uring_read compile errors before the fix, the fixed run, and strace
Image prepared from build, run, strace, and experiment records. Hostnames, paths, and account names are anonymized; omitted output is marked in the image.

ARM64 execution result

The example was also run in an ARM64 emulator (QEMU, Cortex-A53).

This example additionally used liburing 2.9.

ARM64 commands and captured output for io-uring-lifecycle
ARM64 QEMU execution record
Output as text

This image displays the saved execution log.

arm64-qemu
Verification harness exit status: 0
Execution environment

The same C example was cross-compiled for ARM64 and executed on 2026-11-04 in QEMU virt with an emulated Cortex-A53 (TCG, 2 vCPUs, 512 MiB). The guest ran Linux 6.18.58-0-virt, not the article baseline 6.18.37. GCC 15.2.0 and glibc 2.43 were used with static linking; the guest shell and utilities came from Alpine 3.23.7.

The guest ran as root, used loopback only, and had no host disk mounts. This checks the example's observed behavior, not physical hardware timing or every kernel-internal path.

The source above is unchanged. Cross-compiled on the x86_64 host and executed inside the ARM64 guest; static linking was added for the isolated guest. UAPI build headers were 7.0.0, which does not change the running kernel version.

aarch64-linux-gnu-gcc-15 -static -std=c17 -Wall -Wextra -O2 uring_read.c -luring -o uring_read

/examples/uring_read /etc/hostname

Commands and captured output · Environment and hashes · Full guest boot and run log

Code notes

Source line 14io_uring_queue_init(8

Prepares 8-entry SQ/CQ rings, a ring fd, and shared mappings. Inspect setup results for the actual CQ size and features.

Source line 17char buffer[256]

The stack frame and buffer must remain valid until read completion. Returning from the function or reusing them for another request first creates a data race.

Source line 20io_uring_sqe_set_data64

Stores an opaque key that connects the completion to an application request object. Storing only an fd number does not solve reuse.

Source line 27int result = cqe->res

A CQE result is not -1/errno like libc; it is either a successful byte count or negative errno. Pass -result to strerror.

Source line 32io_uring_cqe_seen

Advances the CQ head, returning that completion slot to the ring. Copy the result and user_data first.

Detailed behavior

01

Asynchrony depends on opcode and file type

Not every operation always runs on a separate worker or hardware-async path. Inline completion, task work, and io-wq offload can be mixed.

Measure completion latency, worker saturation, and context switches rather than only syscall reduction.

02

Registered resources change pinning costs

Fixed files and registered buffers can reduce per-request fd lookup and pinning, but extend registration-table and page-pin lifetimes. Account for in-flight requests during update/unregister.

Long-term pinned pages can affect reclaim and migration.

03

Confirm cancellation through completions too

A successful cancel SQE means the target operation was found and the cancellation request applied; handle ordering of both the target CQE and cancel CQE. A request that already completed can produce a result such as ENOENT.

For linked timeouts and multishot operations, check flags that allow one user_data to produce several CQEs.

Objects and lifetimes

ObjectCreation and releaseValues to inspect
io_ring_ctxCreated by queue_init/setup and released by queue_exit/ring-fd closeSQ/CQ, task refs, worker
SQE/io_kiocbBegins in a written SQ slot, becomes a kernel request, and remains in flight until completionopcode, user_data, buffer/fd refs
CQECreated when completion is published and returned to the ring by cqe_seenres, flags, user_data

Failure conditions and common misconceptions

Observed symptomLikely causesHow to verify
Buffer contents are corruptedThe buffer was reused or its stack frame returned before completionInspect the request owner and CQE timeline
CQ overflow/stallCompletions are not consumed fast enoughInspect CQ depth, the overflow flag, and seen calls
Errors are interpreted incorrectlycqe->res was handled like errnoApply the negative-errno rule

Verify it yourself

  1. Submit reads at several offsets and use user_data to record that completion order can differ from submission order.
  2. Compare syscalls, throughput, tail latency, and pinned memory before and after registering files/buffers.
  3. Add a linked timeout and async cancel, then tabulate combinations of target CQEs with cancel/timeout CQEs.
Run./uring_read /etc/hostname
Tracestrace -e trace=io_uring_setup,io_uring_enter,io_uring_register,mmap,munmap ./uring_read /etc/hostname

Primary sources