Linux RT Setup

HORUS handles real-time automatically — no manual setup required. rate=1000 just works on any Linux. This page covers optional configuration for users who need the lowest possible jitter.

Quick Setup (Recommended)

# Check current RT status
horus setup-rt --check

# Install RT kernel + configure system (interactive)
sudo horus setup-rt

# Reboot, then verify
sudo reboot
horus setup-rt --check

That's it. horus setup-rt detects your distro, installs the RT kernel package, configures memory lock limits, and suggests CPU isolation. See Real-Time Tuning for what each setting does.

When To Use This

Your robot has force control, balance, or high-bandwidth servo loops that need ±20μs jitter
You are using .require_rt() on the scheduler
You see "Operation not permitted" or "cannot set SCHED_FIFO" errors at startup
horus doctor shows "Standard kernel" in the Real-Time section

Skip this if you are prototyping, running in simulation, or doing position control. The default scheduler works without RT configuration, and .prefer_rt() degrades gracefully when RT is unavailable.

Manual Setup

If horus setup-rt doesn't support your distro, or you prefer manual control:

Check Current RT Capabilities

# HORUS built-in check (recommended)
horus setup-rt --check

# Or manually:
uname -v | grep -i preempt   # Check for PREEMPT_RT
ulimit -r                     # RT priority limit (0 = no RT)
chrt -f 1 echo "RT works"    # Test SCHED_FIFO

If ulimit -r returns 0 or chrt fails with "Operation not permitted", follow the sections below.

Grant RT Permissions

Edit /etc/security/limits.conf to allow your user (or group) to use RT scheduling:

# Add to /etc/security/limits.conf
# Replace 'robotics' with your username or group (@groupname for groups)
robotics  soft  rtprio  99
robotics  hard  rtprio  99
robotics  soft  memlock  unlimited
robotics  hard  memlock  unlimited

Log out and back in for changes to take effect. Verify with ulimit -r — it should now return 99.

Install PREEMPT_RT Kernel

A standard kernel uses PREEMPT_VOLUNTARY or PREEMPT_DYNAMIC, which gives millisecond-scale worst-case latency. PREEMPT_RT brings that down to microseconds.

Ubuntu / Debian

sudo apt install linux-image-rt-amd64    # Debian
sudo apt install linux-lowlatency        # Ubuntu (close to RT)
# For full PREEMPT_RT on Ubuntu:
sudo apt install linux-image-realtime    # Ubuntu Pro / 24.04+

Reboot and select the RT kernel from GRUB. Verify:

uname -v
# Should contain "PREEMPT_RT" or "PREEMPT RT"

From Source (Any Distro)

Download the PREEMPT_RT patch from kernel.org/pub/linux/kernel/projects/rt, apply it to a matching kernel version, and build with CONFIG_PREEMPT_RT=y.

CPU Isolation

Isolate cores from the Linux scheduler so only your RT threads run on them. This eliminates scheduling jitter from other processes.

Add isolcpus to your kernel command line in /etc/default/grub:

# Isolate cores 2 and 3 for RT use
GRUB_CMDLINE_LINUX="isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3"

Then sudo update-grub && sudo reboot. Verify with:

cat /sys/devices/system/cpu/isolated
# Should output: 2-3

Pin Horus nodes to isolated cores:

// simplified
let mut scheduler = Scheduler::new()
    .require_rt()
    .cores(&[2, 3])
    .tick_rate(1000_u64.hz());

Grant CAP_SYS_NICE

As an alternative to limits.conf, you can grant the RT capability directly to a binary:

sudo setcap cap_sys_nice=eip ./target/release/my_robot

This lets that specific binary use RT scheduling without root or limits.conf changes. Useful for deployment where you do not want blanket RT permissions.

Verify the Setup

# Run a program with SCHED_FIFO at priority 50
chrt -f 50 ./target/release/my_robot

# Check that it is actually running with RT scheduling
ps -eo pid,cls,rtprio,comm | grep my_robot
# Should show "FF" (FIFO) and priority 50

Horus RT Integration

`.prefer_rt()` vs `.require_rt()` (Rust) / `rt=True` (Python)

// simplified
// Prefer RT: use RT if available, fall back to normal scheduling
let mut scheduler = Scheduler::new()
    .prefer_rt()
    .tick_rate(500_u64.hz());

// Require RT: panic at startup if RT is not available
let mut scheduler = Scheduler::new()
    .require_rt()
    .tick_rate(1000_u64.hz());

Use .prefer_rt() / rt=True during development. Use .require_rt() (Rust) in production when timing guarantees matter.

Checking Degradations

After building the scheduler, inspect whether RT was successfully acquired:

// simplified
let scheduler = Scheduler::new()
    .prefer_rt()
    .tick_rate(500_u64.hz());

// After running, check status (includes any degradations)
println!("{}", scheduler.status());

If RT was requested but unavailable, a degradation entry will explain why (missing permissions, no PREEMPT_RT, etc).

Troubleshooting

"Operation not permitted" / "cannot set SCHED_FIFO"

Check ulimit -r — must be > 0
Check that limits.conf changes are applied (requires re-login)
Try setcap cap_sys_nice=eip on the binary
If running in Docker: add --cap-add SYS_NICE to docker run

RT works but latency is high

Verify PREEMPT_RT kernel: uname -v | grep PREEMPT_RT
Check for isolated CPUs: cat /sys/devices/system/cpu/isolated
Disable CPU frequency scaling: cpupower frequency-set -g performance
Disable SMT/hyperthreading in BIOS for dedicated RT cores

Platform-Specific Notes

NVIDIA Jetson

Jetson runs a custom L4T kernel. PREEMPT_RT patches are available from NVIDIA for Jetson Orin and later. Apply them when building the kernel with the Jetson Linux BSP. isolcpus works — isolate the performance cores (typically 4-7 on Orin).

Raspberry Pi

Use the linux-image-rt package from the Raspberry Pi OS repo, or apply the PREEMPT_RT patch to the rpi-6.x.y kernel branch. The Pi 4/5 have 4 cores — isolating cores 2-3 for RT while leaving 0-1 for the OS works well. Set arm_freq in config.txt to a fixed value to avoid frequency scaling jitter.

Design Decisions

Why .prefer_rt() and .require_rt() instead of always using RT?

RT scheduling requires kernel support and permissions that are not always available -- especially during development, CI, or in containers. .prefer_rt() lets you develop on any Linux system and only enforce RT in production. .require_rt() fails fast at startup so you know immediately if your deployment target is misconfigured.

Why CPU isolation with isolcpus instead of just thread affinity?

Thread affinity (.core(N)) pins your node to a specific core, but other processes can still be scheduled on that core. isolcpus removes the core from the Linux scheduler entirely, so only your pinned threads run there. This eliminates scheduling jitter from kernel threads, interrupts, and other userspace processes.

Why CAP_SYS_NICE instead of running as root?

Running a robot as root is a security risk. setcap cap_sys_nice=eip grants RT scheduling to a specific binary without root access. This follows the principle of least privilege -- the binary can set thread priorities but cannot modify the filesystem or network configuration.

Trade-offs

Gain	Cost
PREEMPT_RT gives microsecond worst-case latency	Must install and maintain a separate kernel
CPU isolation eliminates scheduling jitter	Isolated cores are unavailable for other processes
`setcap` avoids running as root	Must re-apply after recompiling the binary
`.prefer_rt()` degrades gracefully	Timing guarantees silently degrade if RT is unavailable

Common Errors

Symptom	Cause	Fix
"Operation not permitted" on startup	Missing RT permissions	Add `rtprio 99` to `/etc/security/limits.conf` and re-login
`ulimit -r` still returns 0 after editing limits.conf	Did not log out and back in	Log out completely (not just close terminal) and log back in
RT works but latency spikes above 1ms	No PREEMPT_RT kernel	Install `linux-image-rt-amd64` (Debian) or `linux-image-realtime` (Ubuntu)
Latency spikes on specific cores	CPU frequency scaling or SMT interference	Set `cpupower frequency-set -g performance` and disable hyperthreading in BIOS
`.require_rt()` panics in Docker	Missing `CAP_SYS_NICE` capability	Add `--cap-add SYS_NICE` to `docker run`
`setcap` has no effect	Binary was recompiled after `setcap`	Re-run `setcap` after every `cargo build --release`
Isolated cores show 0% usage in `htop`	Expected -- `isolcpus` removes them from the general scheduler	Use `htop` with per-thread view to see your pinned RT threads

RT Readiness Report

Run horus doctor --rt to get a complete assessment of your system's real-time capabilities. The report includes a live jitter benchmark, IPC latency measurement, and actionable recommendations.

horus doctor --rt

Example output:

╔══════════════════════════════════════════════════════════════╗
║               HORUS RT READINESS REPORT                     ║
║               Grade: STANDARD ★★☆                           ║
╠══════════════════════════════════════════════════════════════╣
║  SYSTEM                                                     ║
║    Kernel:         Linux 6.1.0-rt7                          ║
║    PREEMPT_RT:     ✗                                        ║
║    SCHED_FIFO:     ✓                                        ║
║    Memory lock:    ✓                                        ║
║    CPUs:           8 total, 2 isolated                      ║
╠══════════════════════════════════════════════════════════════╣
║  JITTER BENCHMARK @ 1kHz (2999 samples)                     ║
║    P99:       12.3 μs                                       ║
║    Max:       45.7 μs                                       ║
║    Rate:     999.8 Hz (target: 1000 Hz)                     ║
╠══════════════════════════════════════════════════════════════╣
║  IPC BENCHMARK                                              ║
║    Latency:       136 ns per message                        ║
║    Throughput: 7362626 msg/sec                              ║
╠══════════════════════════════════════════════════════════════╣
║  RECOMMENDATIONS                                            ║
║    → Install PREEMPT_RT for sub-20μs jitter                 ║
╚══════════════════════════════════════════════════════════════╝

Grades:

Grade	Meaning	Requirements
Production ★★★	Safety-critical deployment	PREEMPT_RT + SCHED_FIFO + mlockall + P99 jitter <50μs
Standard ★★☆	Most robotics applications	SCHED_FIFO + P99 jitter <500μs
Development ★☆☆	Prototyping only	No RT capabilities or high jitter

Run this command on every target machine before deploying. The recommendations tell you exactly what to fix.

From code:

use horus_core::scheduling::rt_report::RtReport;
use std::time::Duration;

let report = RtReport::generate(Duration::from_secs(5));
report.print();
assert!(report.is_production_ready());

What HORUS Auto-Configures

After horus setup-rt, when your code uses .core() or .prefer_rt(), the RT executor automatically:

Locks CPU governor to performance on pinned cores — prevents frequency scaling jitter (10-100us spikes)
Moves hardware interrupts off RT cores — prevents IRQ latency (50-500us spikes)
Attempts SCHED_DEADLINE when .deadline_scheduler() is used — kernel-guaranteed CPU bandwidth
Falls back gracefully to SCHED_FIFO, then normal scheduling, if any feature is unavailable

You don't need to configure these manually — they happen automatically when the permissions are available.

SCHED_DEADLINE (Advanced)

For nodes that need kernel-guaranteed CPU bandwidth (not just priority):

scheduler.add(motor_ctrl)
    .rate(1000_u64.hz())
    .budget(500_u64.us())
    .deadline_scheduler()    // kernel EDF scheduling
    .build()?;

The kernel guarantees 500us of CPU every 1ms. If the system can't honor this (CPU overcommitted), it rejects the request and HORUS falls back to SCHED_FIFO.

Requires PREEMPT_RT kernel (installed by horus setup-rt) and CAP_SYS_NICE.