Feb 1, 2026

installworld: Rebuilding Userland

The kernel manages hardware. Userland provides the tools you actually use: shells, compilers, utilities, libraries. When you run ls, grep, sshd - that’s userland. FreeBSD lets you rebuild all of it from source with a single command: make buildworld.

This is why FreeBSD doesn’t have package dependency hell. When you rebuild world, every binary is compiled against the same libraries, the same headers, the same compiler. Consistency is guaranteed. Conflicts are impossible.

The World Build Process:

Phase	Command	Purpose	Time (First Build)	Time (With ccache)
Build	`make buildworld`	Compile all userland	2-6 hours	30 minutes
Kernel	`make buildkernel`	Compile kernel	15-45 minutes	5 minutes
Install Kernel	`make installkernel`	Install new kernel	2-5 minutes	2-5 minutes
Reboot	Single-user mode	Prepare for world install	1 minute	1 minute
Install World	`make installworld`	Install new userland	10-20 minutes	10-20 minutes
Cleanup	`make delete-old`	Remove obsolete files	5 minutes	5 minutes

Total time without ccache: 3-8 hours Total time with ccache: 50 minutes

The difference is caching. The Republic does not waste computational resources.

The Canonical Sequence#

The FreeBSD handbook defines the correct order. Deviation causes breakage:

# 1. Update source tree
cd /usr/src
git pull

# 2. Build world (all userland)
make buildworld

# 3. Build kernel
make buildkernel KERNCONF=GENERIC

# 4. Install kernel
make installkernel KERNCONF=GENERIC

# 5. Reboot into single-user mode
shutdown -r now
# At boot prompt: boot -s

# 6. (In single-user mode) Mount filesystems
mount -u /
mount -a

# 7. Pre-install configuration update
etcupdate -p

# 8. Install new world
make installworld

# 9. Post-install configuration update
etcupdate -B

# 10. Reboot into new system
reboot

# 11. (After successful boot) Remove old files
make delete-old
make delete-old-libs

This sequence is not negotiable. Build world before kernel. Install kernel before world. Reboot before installworld. Verify before deleting old libs.

Why? Because installworld installs new binaries that may depend on new kernel features. Installing world before the kernel boots guarantees mismatch. Installing world in multi-user mode risks corruption from running processes.

Build Environment Variables#

Control the build with /etc/src.conf:

# /etc/src.conf - Build configuration

# Enable compiler cache (30x speedup)
WITH_CCACHE_BUILD=yes

# Enable meta mode (tracks dependencies accurately)
WITH_META_MODE=yes

# Parallel build (use all cores)
MAKE_JOBS_NUMBER=8

# Skip components you don't need
WITHOUT_SENDMAIL=yes
WITHOUT_GAMES=yes
WITHOUT_KERBEROS=yes

Variables specified in make buildworld must match in make installworld:

# Wrong - variables don't match
make buildworld WITHOUT_SENDMAIL=yes
make installworld  # Missing WITHOUT_SENDMAIL

# Correct - variables match
make buildworld WITHOUT_SENDMAIL=yes
make installworld WITHOUT_SENDMAIL=yes

ccache: The Compiler Cache#

ccache stores compiled object files on disk. When make attempts to compile a file identical to one in cache, the cached copy is used instantly. No compilation. No CPU cycles wasted.

Install ccache:

pkg install ccache

Enable in /etc/src.conf:

WITH_CCACHE_BUILD=yes
CCACHE_DIR=/var/cache/ccache

Create cache directory:

mkdir -p /var/cache/ccache
# Set size limit (default: 5GB, recommend 20GB for buildworld)
ccache -M 20G

Results:

Reports show buildworld dropping from 87 minutes to 33 minutes with ccache. Combined with WITH_META_MODE, speedups reach 30x: builds complete in 13 minutes instead of 6+ hours.

How? First build populates the cache. Subsequent builds reuse cached objects. Only changed files recompile. A kernel patch that modifies 3 source files triggers 3 compilations, not 9,000.

WITH_META_MODE: Dependency Tracking#

WITH_META_MODE uses the filemon kernel module to track file dependencies accurately:

# Load filemon module
kldload filemon

# Enable in /etc/src.conf
WITH_META_MODE=yes

Meta mode creates metadata about compiled objects during the build. On subsequent builds, make inspects the metadata and rebuilds only what changed. No false rebuilds. No wasted compilation.

First build with meta mode is slower (metadata creation overhead). Second and subsequent builds are dramatically faster. One user rebuilt world/kernel on a ThinkPad x260 in under ten minutes with meta mode + ccache.

ZFS Boot Environments#

Before installing kernel or world, create a boot environment snapshot:

# Create snapshot before changes
bectl create pre-update-2026-02-01

# Perform update
make installkernel
make installworld

# If update breaks system, revert:
bectl activate pre-update-2026-02-01
reboot

Boot environments are ZFS snapshots of your entire root filesystem. Bad update? Activate old snapshot. System reverts completely. This is why the Republic runs ZFS.

The -j Flag (Parallel Builds)#

Never use -j with installworld:

# Correct - parallel build
make -j8 buildworld

# Wrong - DO NOT parallelize install
make -j8 installworld  # WILL BREAK SYSTEM

Why? Installing files is inherently sequential. Dependencies matter. Installing a library before its dependencies are in place causes runtime failures. The install process must proceed in order.

Building is parallelizable (compilation units are independent). Installing is not.

Troubleshooting Build Failures#

Build fails with “missing dependencies”:

# Clean /usr/obj and rebuild
rm -rf /usr/obj/*
make buildworld

Build fails with “compiler errors”:

# Your source tree might be incomplete
cd /usr/src
git status  # Check for missing files
git pull    # Ensure latest source

World won’t install: “target busy”:

You’re in multi-user mode with processes running. Boot single-user:

shutdown -r now
# At boot prompt: boot -s

What Gets Built#

make buildworld compiles:

Libraries: libc, libm, libcrypto, libssl, etc.
Utilities: ls, cp, grep, sed, awk, tar, etc.
Shells: sh, csh
Daemons: sshd, syslogd, ntpd
Compilers: clang, lld
Build tools: make, lex, yacc

Everything in /bin, /sbin, /usr/bin, /usr/sbin, /usr/lib - rebuilt from source. Consistent. Verified. No binary blobs from unknown sources.

The Official History#

The “build world” terminology dates back to 4.4BSD. The concept: your entire operating system userland is “the world” of software you interact with. Rebuilding it means rebuilding everything.

FreeBSD formalized the process with Makefiles that handle dependencies automatically. The handbook has documented the canonical sequence since FreeBSD 2.0 (1994).

ccache was created by Andrew Tridgell (of Samba fame) in 2002. FreeBSD integrated ccache support into the build system in the mid-2000s. WITH_META_MODE arrived in FreeBSD 11.0 (2016), leveraging filemon work that dated back to NetBSD.

The Real Story (Suppressed)#

The buildworld process wasn’t designed for hobbyists rebuilding systems for fun. It was designed for vendors maintaining derivative distributions where every binary must be verifiable.

In 1996, classified FreeBSD derivatives were being deployed in environments where binary provenance mattered. You couldn’t trust binaries from upstream. You couldn’t accept precompiled packages. You needed to build everything from audited source.

The buildworld infrastructure made this possible. One command. Entire userland rebuilt. Source code audited. Binaries verified. No trust required beyond the compiler - and the compiler was built from source too.

The Republic’s computing infrastructure has used this model since 1998. Every deployed system runs a custom world build. The source tree is audited. The compiler is audited. The build process is deterministic and reproducible.

When Western developers discovered ccache in the mid-2000s and celebrated the speedup, the Republic’s engineers smiled. We had been using compiler caches since 1997. The technique wasn’t novel. It was obvious. Compilation is deterministic. Cache the outputs.

The 30x speedup isn’t magic. It’s the difference between rebuilding everything and rebuilding only what changed. The West just took longer to figure this out.

The Lesson#

FreeBSD lets you rebuild the entire operating system from source. World, kernel, utilities, libraries - all of it. The build system handles dependencies. The process is documented. The result is consistent.

ccache provides 5-10x speedup by caching compiled objects. WITH_META_MODE provides accurate dependency tracking. Combined, builds that take 6 hours complete in 13 minutes.

Boot environments provide safety. Snapshots let you revert. ZFS lets you experiment without fear.

The order matters: buildworld → buildkernel → installkernel → reboot → installworld. Deviation breaks things. The handbook isn’t making suggestions. It’s documenting the only sequence that works.

And if you skip ccache because you “don’t want to install extra packages,” you will waste 5+ hours every build. The Republic does not waste time. Neither should you.

— Kim Jong Rails, Supreme Leader of the Republic of Derails

Sources: