Chapter 11: Kernel

Introduction

The chapter introduces a project to build a minimal operating system in Rust and frames it as relevant to programming without an OS.

11.1 A fledgling operating system (FledgeOS)

Introduces FledgeOS and explains the basic responsibilities of an OS kernel.

• OS kernel responsibilities: Describe an OS kernel’s roles in interacting with hardware, managing memory, and coordinating work, typically through processes and threads.
  Why it matters: These roles explain what the FledgeOS project is beginning to build.

Key terms: OS kernel, hardware, memory management, processes, threads

11.1.1 Setting up a development environment for developing an OS kernel

This section explains why building FledgeOS requires a specially prepared development environment and identifies the role of each required tool.

• Kernel build environment challenge: Explain why building FledgeOS requires more than the current Rust environment, and describe how the nightly toolchain and rust-src help prepare it.
  Why it matters: This explains why kernel development needs a different toolchain setup from an ordinary Rust project.
• Boot images and virtual machines: Describe what bootimage produces and what QEMU provides when developing FledgeOS.
  Why it matters: These tools provide a bootable output and a way to run an operating system during development.
• Supporting build utilities: Describe the distinct purposes of cargo-binutils and llvm-tools-preview in the FledgeOS setup.
  Why it matters: The names are easier to remember when each is tied to its specific job in the build environment.

Key terms: nightly toolchain, rust-src, core Rust language, QEMU, virtual machine monitors, bootimage crate, boot image, cargo-binutils, llvm-tools-preview, LLVM

11.1.2 Verifying the development environment

This subsection gives checks for confirming that the tools needed to develop FledgeOS are installed and accessible.

• Development environment verification: Describe how to check that QEMU, cargo-binutils, bootimage, and the LLVM tools are available before continuing with FledgeOS.
  Why it matters: Checking the tools now can prevent frustrating setup problems later.

Key terms: qemu-system-x86_64, cargo-binutils, bootimage, llvm-tools-preview, rustc

11.2 Fledgeos-0: Getting something working

The section introduces FledgeOS as short code that may take patience to understand because it exposes concepts normally hidden by an operating system, then invites the reader to see it run.

11.2.1 First boot

This section shows what the first FledgeOS boot looks like and explains the build-and-boot steps that make it possible.

• First Boot Build Chain: Trace how a target definition, Rust’s core library, the kernel, and a bootloader come together to run FledgeOS in a virtual environment. Explain that bootimage automates these steps.
  Why it matters: The visible boot depends on several components working together, not just on compiling the kernel.
• First Boot Success Signal: Describe the visible result of the first FledgeOS boot and what the section counts as success at this stage.
  Why it matters: The simple display is evidence that the first boot worked, not a claim that FledgeOS is already a full-featured OS.

Key terms: target platform, compiler target, core, bootloader, virtual environment, bootimage crate, fledgeos-0, first boot

11.2.2 Compilation instructions

This section gives the file setup steps for building the fledgeos-0 project by hand and points readers to the published source.

• Manual project file setup: Describe the recommended file setup for creating fledgeos-0 by hand: update Cargo.toml, create fledge.json and .cargo/config.toml, and replace the main source file using the cited listings.
  Why it matters: These steps identify which project files must be prepared before working with the kernel source.

Key terms: Cargo.toml, fledge.json, .cargo/config.toml, src/main.rs

11.2.3 Source code listings

This section explains the build configuration and source-code differences that let a small Rust kernel boot and write to the screen without an operating system.

• Custom kernel build configuration: Explain why FledgeOS adds fledge.json and .cargo/config.toml to a Cargo project.
  Why it matters: A kernel needs a build target and core library suited to the machine it will boot on.
• Bootimage project directives: Describe what the bootimage metadata in Cargo.toml changes about building and running the kernel.
  Why it matters: The usual Cargo commands need special instructions to produce and run a bootable kernel.
• Bare-metal runtime constraints: Explain why the kernel excludes the standard library and why its central functions never return.
  Why it matters: Kernel code cannot assume the operating-system services and return path available to ordinary applications.
• Kernel boot entry conventions: Explain the roles of no_main, no_mangle, and extern "C" in making the kernel's _start function bootable.
  Why it matters: The bootloader needs a predictable entry point rather than an ordinary application startup.
• Direct frame buffer output: Describe how the first kernel changes what appears on screen by writing to memory.
  Why it matters: This explains how the kernel can show a result before it has ordinary output facilities.
• Unstable compiler intrinsics: Explain why using core_intrinsics requires an explicit feature attribute and the nightly compiler.
  Why it matters: The compiler toolchain choice follows from the kernel's use of an API that can change.

Key terms: fledge.json, .cargo/config.toml, compiler target, std::core, [package .metadata.bootimage], bootimage, build-command, run_command, QEMU, Never type (!)

11.2.4 Panic handling

This section explains why FledgeOS needs its own panic handler, contrasts two handler choices, and identifies the information available from PanicInfo.

• Manual panic handling requirement: Explain why FledgeOS must provide panic handling when it uses #[no_std].
  Why it matters: This explains why even a minimal kernel needs explicit panic-handling code.
• Infinite loop handler tradeoff: Describe the infinite loop alternative to intrinsics::abort() and its disadvantage.
  Why it matters: A simple handler choice has a concrete cost when an error occurs.
• PanicInfo location information: State what PanicInfo tells a panic handler about a panic’s origin.
  Why it matters: The location information can make a later, more informative panic handler possible.

Key terms: #[no_std], panic handling, Rust runtime, intrinsics::abort(), infinite loop, panic handler, PanicInfo, filename, line number

11.2.5 Writing to the screen with VGA-compatible text mode

This section explains how VGA-compatible text mode maps screen cells to memory and how FledgeOS writes to that memory with raw pointers.

• VGA text frame buffer: Describe how boot-time setup makes an 80×25 text display available by mapping its cells to a fixed area of memory.
  Why it matters: It explains why writing to a particular memory region can produce text onscreen.
• Frame buffer pointer writes: Explain how offset() and write_volatile() help write values to the VGA frame buffer.
  Why it matters: The code must reach the intended buffer address and actually perform the write that changes the display.
• Manual pointer arithmetic tradeoff: Distinguish manually incrementing and dereferencing a pointer from using offset() to reach a frame-buffer location.
  Why it matters: The comparison shows why the more explicit pointer style is preferable when writing directly to memory.

Key terms: VGA-compatible text mode, frame buffer, 80x25 grid, offset(), write_volatile(), volatile write, dereference operator, pointer arithmetic

11.2.6 _start(): The main() function for FledgeOS

Explains why FledgeOS uses a bootloader and _start as its entry point instead of a conventional main() function.

• Kernel startup without main: Explain why an OS kernel cannot rely on the usual main() startup and exit conventions, and what starts it instead.
  Why it matters: It clarifies why kernel startup differs from launching an ordinary program.
• FledgeOS entry point responsibilities: Describe how the linker identifies _start and contrast the usual jobs of _start() with what FledgeOS keeps in that function.
  Why it matters: It connects the kernel’s source-code entry point to the way the program is linked and run.

Key terms: main(), bootloader, OS kernel, linker, _start, _exit()

11.3 fledgeos-1: Avoiding a busy loop

11.3.1 Being power conscious by interacting with the CPU directly

This section explains how FledgeOS avoids wasting power in its main loop by issuing the CPU’s halt instruction.

• Power-conscious CPU halting: Explain why FledgeOS uses the hlt instruction in _start() and what causes the CPU to resume.
  Why it matters: Halting an idle CPU avoids spending power on a loop that accomplishes nothing.

Key terms: hlt, interrupt, _start(), x86_64 crate

11.3.2 fledgeos-1 source code

This subsection identifies what changes in fledgeos-1, how to compile it, and the crate used to add assembly instructions.

• FledgeOS Version and Build: Describe how fledgeos-1 differs from fledgeos-0 and how to compile it using the earlier instructions.
  Why it matters: You can work with this version without treating it as an entirely new project.
• Assembly Instruction Support: State what the x86_64 crate provided for this version of the code.
  Why it matters: It identifies the tool the code used for assembly instructions without requiring you to explain an approach reserved for later.

Key terms: fledgeos-0, fledgeos-1, src/main.rs, x86_64 crate, assembly instructions

11.4 fledgeos-2: Custom exception handling

11.4.1 Handling exceptions properly, almost

The section explains how a personality function participates in exception handling and why FledgeOS uses an empty one.

• Stack unwinding and personality functions: Explain how unwinding traverses stack frames after an exception and what a personality function decides at each frame.
  Why it matters: This explains what the handler FledgeOS must provide is normally meant to do.
• FledgeOS minimal exception handler: Explain why FledgeOS uses an empty personality function and what that means for exceptions.
  Why it matters: The empty function fulfills the need to define a handler without pretending FledgeOS can recover from exceptions.
• Language items without std: Describe what a Rust language item is and why removing the standard library means implementing some functionality yourself.
  Why it matters: It explains why kernel code must provide pieces normally supplied by Rust libraries.

Key terms: stack frame, stack unwinding, personality function, catching an exception, FledgeOS, minimal handler, language item, #[no_std], standard library

11.4.2 fledgeos-2 source code

This subsection identifies the fledgeos-2 source file and explains how to compile the project using earlier instructions.

• FledgeOS-2 compilation instructions: Describe how to compile fledgeos-2 by reusing the instructions in section 11.2.1 and replacing references to fledgeos-0 with fledgeos-2.
  Why it matters: You can build this version without needing a separate set of compilation steps.

Key terms: fledgeos-2, fledgeos-0

11.5 fledgeos-3: Text output

The section introduces screen text output as a way for FledgeOS to report panics.

• Purpose of screen text output: Explain why FledgeOS needs to write text to the screen: it gives the system a way to report a panic.
  Why it matters: Text output lets the kernel communicate an error instead of leaving a panic unexplained.

Key terms: panic, screen, frame buffer

11.5.1 Writing colored text to the screen

The section introduces an enum for VGA text-mode colors, explaining why it groups the color constants and must follow the VGA standard.

• VGA color enum: Explain why the VGA text-mode color constants are represented as an enum and what determines their numeric values.
  Why it matters: The color type keeps related values together while matching the values expected by VGA text mode.

Key terms: enum, const values, type safety, VGA-compatible text mode color palette, bit patterns

11.5.2 Controlling the in-memory representation of enums

This subsection explains how to prescribe an enum’s integer representation for external requirements and why doing so limits compiler flexibility.

• Prescribed enum representation: Explain why an external system may require a specific enum representation and how the repr attribute lets Rust code prescribe an integer type and variant discriminants.
  Why it matters: A prescribed representation lets an enum match the data format an external system expects.
• Representation flexibility tradeoff: Describe the disadvantage of prescribing an enum representation and the space optimization the section uses as an example.
  Why it matters: Choosing a fixed representation can give up space savings Rust could otherwise make.

Key terms: repr attribute, discriminant, u8, VGA-compatible text mode, prescribed representation, space optimizations, single variant

11.5.3 Why use enums?

The section explains why an enum is preferable to an u8 for representing colors, even though numeric constants could have the same in-memory representation.

• Enum guard for colors: Explain why the section favors an enum over an u8 for colors, despite the possibility of using numeric constants with the same in-memory representation.
  Why it matters: The choice of type helps prevent invalid values, not just determine how colors look in memory.

Key terms: enum, u8, illegal value, numeric constants

11.5.4 Creating a type that can print to the VGA frame buffer

This section introduces Cursor as the interface between the kernel’s code and the VGA frame buffer.

• Cursor frame buffer role: Explain that Cursor is the type used to print to the screen by handling raw memory manipulation and converting the code’s Color type to what VGA expects.
  Why it matters: It identifies which type bridges the printing code and the VGA display memory.

Key terms: Cursor, Color, VGA frame buffer

11.5.5 Printing to the screen

This subsection shows how to print to the screen with Cursor by setting its position and calling Cursor.print() with a reference.

• Cursor screen printing sequence: Describe how this section uses Cursor to print to the screen: set its position, then send a reference to Cursor.print().
  Why it matters: This is the sequence the section uses when adding screen output to _start().

Key terms: Cursor, Cursor.print(), _start()

11.5.6 fledgeos-3 source code

This section points to the complete fledgeos-3 source file and says to compile it using the earlier instructions with the project name changed.

11.6 fledgeos-4: Custom panic handling

11.6.1 Implementing a panic handler that reports the error to the user

The section introduces associating core::fmt::Write with a panic handler to display a message when a panic occurs.

• Panic error reporting: Explain that associating core::fmt::Write with a panic handler lets it display a message reporting where a panic occurs, which is especially useful in embedded development.
  Why it matters: A visible panic message helps someone running Rust on a microcontroller identify where a panic occurred.

Key terms: panic handler, core::fmt::Write, embedded development, microcontrollers

11.6.2 Reimplementing panic() by making use of core::fmt::Write

This subsection explains how FledgeOS’s panic handler clears the screen and then uses core::write! to report an error.

• Two-stage panic output: Describe the two stages of panic(): it clears the screen, then uses core::write! to report an error. Explain that the macro’s first argument is a destination object, cursor, that implements core::fmt::Write.
  Why it matters: This explains how the panic handler produces a visible error report.

Key terms: panic(), core::write!, cursor, core::fmt::Write

11.6.3 Implementing core::fmt::Write

This subsection explains how Cursor implements core::fmt::Write by supplying write_str() and reusing its existing print() method.

• Cursor write_str implementation: Explain why implementing write_str() is enough to implement core::fmt::Write here, and describe how Cursor's implementation passes text to print().
  Why it matters: It shows how the existing printing method can satisfy Rust's formatting trait without implementing every trait method separately.

Key terms: core::fmt::Write, write_str(), print(), &str, &[u8], to_bytes()

11.6.4 fledge-4 source code

Points to the FledgeOS-4 panic-handling source code and directs readers to reuse the earlier compilation instructions with the project name changed.

11.7 Summary