- In early boot code, we transition from the
Hypervisorprivilege level (EL2in AArch64) to theKernel(EL1) privilege level.
- Introduction
- Scope of this tutorial
- Checking for EL2 in the entrypoint
- Transition preparation
- Returning from an exception that never happened
- Test it
- Diff to previous
Application-grade CPUs have so-called privilege levels, which have different purposes:
| Typically used for | AArch64 | RISC-V | x86 |
|---|---|---|---|
| Userspace applications | EL0 | U/VU | Ring 3 |
| OS Kernel | EL1 | S/VS | Ring 0 |
| Hypervisor | EL2 | HS | Ring -1 |
| Low-Level Firmware | EL3 | M |
EL in AArch64 stands for Exception Level. If you want more information regarding the other
architectures, please have a look at the following links:
At this point, I strongly recommend that you glimpse over Chapter 3 of the Programmer’s Guide for
ARMv8-A before you continue. It gives a concise overview about the topic.
By default, the Raspberry will always start executing in EL2. Since we are writing a traditional
Kernel, we have to transition into the more appropriate EL1.
First of all, we need to ensure that we actually execute in EL2 before we can call respective code
to transition to EL1. Therefore, we add a new checkt to the top of boot.s, which parks the CPU
core should it not be in EL2.
// Only proceed if the core executes in EL2. Park it otherwise.
mrs x0, CurrentEL
cmp x0, {CONST_CURRENTEL_EL2}
b.ne .L_parking_loop
Afterwards, we continue with preparing the EL2 -> EL1 transition by calling
prepare_el2_to_el1_transition() in boot.rs:
#[no_mangle]
pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! {
prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr);
// Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1.
asm::eret()
}Since EL2 is more privileged than EL1, it has control over various processor features and can
allow or disallow EL1 code to use them. One such example is access to timer and counter registers.
We are already using them since tutorial 07, so of course we want to keep them.
Therefore we set the respective flags in the Counter-timer Hypervisor Control register and
additionally set the virtual offset to zero so that we get the real physical value everytime:
// Enable timer counter registers for EL1.
CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET);
// No offset for reading the counters.
CNTVOFF_EL2.set(0);Next, we configure the Hypervisor Configuration Register such that EL1 runs in AArch64 mode,
and not in AArch32, which would also be possible.
// Set EL1 execution state to AArch64.
HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64);There is actually only one way to transition from a higher EL to a lower EL, which is by way of executing the ERET instruction.
This instruction will copy the contents of the Saved Program Status Register - EL2 to Current Program Status Register - EL1 and jump to the instruction address that is stored in the Exception
Link Register - EL2.
This is basically the reverse of what is happening when an exception is taken. You'll learn about that in an upcoming tutorial.
// Set up a simulated exception return.
//
// First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a
// stack pointer.
SPSR_EL2.write(
SPSR_EL2::D::Masked
+ SPSR_EL2::A::Masked
+ SPSR_EL2::I::Masked
+ SPSR_EL2::F::Masked
+ SPSR_EL2::M::EL1h,
);
// Second, let the link register point to kernel_init().
ELR_EL2.set(crate::kernel_init as *const () as u64);
// Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there
// are no plans to ever return to EL2, just re-use the same stack.
SP_EL1.set(phys_boot_core_stack_end_exclusive_addr);As you can see, we are populating ELR_EL2 with the address of the kernel_init() function that we
earlier used to call directly from the entrypoint. Finally, we set the stack pointer for SP_EL1.
You might have noticed that the stack's address was supplied as a function argument. As you might
remember, in _start() in boot.s, we are already setting up the stack for EL2. Since there
are no plans to ever return to EL2, we can just re-use the same stack for EL1, so its address is
forwarded using function arguments.
Lastly, back in _start_rust() a call to ERET is made:
#[no_mangle]
pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! {
prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr);
// Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1.
asm::eret()
}In main.rs, we print the current privilege level and additionally inspect if the mask bits in
SPSR_EL2 made it to EL1 as well:
$ make chainboot
[...]
Minipush 1.0
[MP] ⏳ Waiting for /dev/ttyUSB0
[MP] ✅ Serial connected
[MP] 🔌 Please power the target now
__ __ _ _ _ _
| \/ (_)_ _ (_) | ___ __ _ __| |
| |\/| | | ' \| | |__/ _ \/ _` / _` |
|_| |_|_|_||_|_|____\___/\__,_\__,_|
Raspberry Pi 3
[ML] Requesting binary
[MP] ⏩ Pushing 14 KiB =========================================🦀 100% 0 KiB/s Time: 00:00:00
[ML] Loaded! Executing the payload now
[ 0.162546] mingo version 0.9.0
[ 0.162745] Booting on: Raspberry Pi 3
[ 0.163201] Current privilege level: EL1
[ 0.163677] Exception handling state:
[ 0.164122] Debug: Masked
[ 0.164511] SError: Masked
[ 0.164901] IRQ: Masked
[ 0.165291] FIQ: Masked
[ 0.165681] Architectural timer resolution: 52 ns
[ 0.166255] Drivers loaded:
[ 0.166592] 1. BCM PL011 UART
[ 0.167014] 2. BCM GPIO
[ 0.167371] Timer test, spinning for 1 second
[ 1.167904] Echoing input nowdiff -uNr 08_hw_debug_JTAG/Cargo.toml 09_privilege_level/Cargo.toml
--- 08_hw_debug_JTAG/Cargo.toml
+++ 09_privilege_level/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "mingo"
-version = "0.8.0"
+version = "0.9.0"
authors = ["Andre Richter <[email protected]>"]
edition = "2021"
diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs 09_privilege_level/src/_arch/aarch64/cpu/boot.rs
--- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.rs
+++ 09_privilege_level/src/_arch/aarch64/cpu/boot.rs
@@ -11,22 +11,73 @@
//!
//! crate::cpu::boot::arch_boot
+use aarch64_cpu::{asm, registers::*};
use core::arch::global_asm;
+use tock_registers::interfaces::Writeable;
// Assembly counterpart to this file.
global_asm!(
include_str!("boot.s"),
+ CONST_CURRENTEL_EL2 = const 0x8,
CONST_CORE_ID_MASK = const 0b11
);
//--------------------------------------------------------------------------------------------------
+// Private Code
+//--------------------------------------------------------------------------------------------------
+
+/// Prepares the transition from EL2 to EL1.
+///
+/// # Safety
+///
+/// - The `bss` section is not initialized yet. The code must not use or reference it in any way.
+/// - The HW state of EL1 must be prepared in a sound way.
+#[inline(always)]
+unsafe fn prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr: u64) {
+ // Enable timer counter registers for EL1.
+ CNTHCTL_EL2.write(CNTHCTL_EL2::EL1PCEN::SET + CNTHCTL_EL2::EL1PCTEN::SET);
+
+ // No offset for reading the counters.
+ CNTVOFF_EL2.set(0);
+
+ // Set EL1 execution state to AArch64.
+ HCR_EL2.write(HCR_EL2::RW::EL1IsAarch64);
+
+ // Set up a simulated exception return.
+ //
+ // First, fake a saved program status where all interrupts were masked and SP_EL1 was used as a
+ // stack pointer.
+ SPSR_EL2.write(
+ SPSR_EL2::D::Masked
+ + SPSR_EL2::A::Masked
+ + SPSR_EL2::I::Masked
+ + SPSR_EL2::F::Masked
+ + SPSR_EL2::M::EL1h,
+ );
+
+ // Second, let the link register point to kernel_init().
+ ELR_EL2.set(crate::kernel_init as *const () as u64);
+
+ // Set up SP_EL1 (stack pointer), which will be used by EL1 once we "return" to it. Since there
+ // are no plans to ever return to EL2, just re-use the same stack.
+ SP_EL1.set(phys_boot_core_stack_end_exclusive_addr);
+}
+
+//--------------------------------------------------------------------------------------------------
// Public Code
//--------------------------------------------------------------------------------------------------
/// The Rust entry of the `kernel` binary.
///
/// The function is called from the assembly `_start` function.
+///
+/// # Safety
+///
+/// - Exception return from EL2 must must continue execution in EL1 with `kernel_init()`.
#[no_mangle]
-pub unsafe fn _start_rust() -> ! {
- crate::kernel_init()
+pub unsafe extern "C" fn _start_rust(phys_boot_core_stack_end_exclusive_addr: u64) -> ! {
+ prepare_el2_to_el1_transition(phys_boot_core_stack_end_exclusive_addr);
+
+ // Use `eret` to "return" to EL1. This results in execution of kernel_init() in EL1.
+ asm::eret()
}
diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s 09_privilege_level/src/_arch/aarch64/cpu/boot.s
--- 08_hw_debug_JTAG/src/_arch/aarch64/cpu/boot.s
+++ 09_privilege_level/src/_arch/aarch64/cpu/boot.s
@@ -27,11 +27,16 @@
// fn _start()
//------------------------------------------------------------------------------
_start:
+ // Only proceed if the core executes in EL2. Park it otherwise.
+ mrs x0, CurrentEL
+ cmp x0, {CONST_CURRENTEL_EL2}
+ b.ne .L_parking_loop
+
// Only proceed on the boot core. Park it otherwise.
- mrs x0, MPIDR_EL1
- and x0, x0, {CONST_CORE_ID_MASK}
- ldr x1, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs
- cmp x0, x1
+ mrs x1, MPIDR_EL1
+ and x1, x1, {CONST_CORE_ID_MASK}
+ ldr x2, BOOT_CORE_ID // provided by bsp/__board_name__/cpu.rs
+ cmp x1, x2
b.ne .L_parking_loop
// If execution reaches here, it is the boot core.
@@ -48,7 +53,7 @@
// Prepare the jump to Rust code.
.L_prepare_rust:
- // Set the stack pointer.
+ // Set the stack pointer. This ensures that any code in EL2 that needs the stack will work.
ADR_REL x0, __boot_core_stack_end_exclusive
mov sp, x0
@@ -60,7 +65,7 @@
b.eq .L_parking_loop
str w2, [x1]
- // Jump to Rust code.
+ // Jump to Rust code. x0 holds the function argument provided to _start_rust().
b _start_rust
// Infinitely wait for events (aka "park the core").
diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs
--- 08_hw_debug_JTAG/src/_arch/aarch64/exception/asynchronous.rs
+++ 09_privilege_level/src/_arch/aarch64/exception/asynchronous.rs
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+//
+// Copyright (c) 2018-2023 Andre Richter <[email protected]>
+
+//! Architectural asynchronous exception handling.
+//!
+//! # Orientation
+//!
+//! Since arch modules are imported into generic modules using the path attribute, the path of this
+//! file is:
+//!
+//! crate::exception::asynchronous::arch_asynchronous
+
+use aarch64_cpu::registers::*;
+use tock_registers::interfaces::Readable;
+
+//--------------------------------------------------------------------------------------------------
+// Private Definitions
+//--------------------------------------------------------------------------------------------------
+
+trait DaifField {
+ fn daif_field() -> tock_registers::fields::Field<u64, DAIF::Register>;
+}
+
+struct Debug;
+struct SError;
+struct IRQ;
+struct FIQ;
+
+//--------------------------------------------------------------------------------------------------
+// Private Code
+//--------------------------------------------------------------------------------------------------
+
+impl DaifField for Debug {
+ fn daif_field() -> tock_registers::fields::Field<u64, DAIF::Register> {
+ DAIF::D
+ }
+}
+
+impl DaifField for SError {
+ fn daif_field() -> tock_registers::fields::Field<u64, DAIF::Register> {
+ DAIF::A
+ }
+}
+
+impl DaifField for IRQ {
+ fn daif_field() -> tock_registers::fields::Field<u64, DAIF::Register> {
+ DAIF::I
+ }
+}
+
+impl DaifField for FIQ {
+ fn daif_field() -> tock_registers::fields::Field<u64, DAIF::Register> {
+ DAIF::F
+ }
+}
+
+fn is_masked<T>() -> bool
+where
+ T: DaifField,
+{
+ DAIF.is_set(T::daif_field())
+}
+
+//--------------------------------------------------------------------------------------------------
+// Public Code
+//--------------------------------------------------------------------------------------------------
+
+/// Print the AArch64 exceptions status.
+#[rustfmt::skip]
+pub fn print_state() {
+ use crate::info;
+
+ let to_mask_str = |x| -> _ {
+ if x { "Masked" } else { "Unmasked" }
+ };
+
+ info!(" Debug: {}", to_mask_str(is_masked::<Debug>()));
+ info!(" SError: {}", to_mask_str(is_masked::<SError>()));
+ info!(" IRQ: {}", to_mask_str(is_masked::<IRQ>()));
+ info!(" FIQ: {}", to_mask_str(is_masked::<FIQ>()));
+}
diff -uNr 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs 09_privilege_level/src/_arch/aarch64/exception.rs
--- 08_hw_debug_JTAG/src/_arch/aarch64/exception.rs
+++ 09_privilege_level/src/_arch/aarch64/exception.rs
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+//
+// Copyright (c) 2018-2023 Andre Richter <[email protected]>
+
+//! Architectural synchronous and asynchronous exception handling.
+//!
+//! # Orientation
+//!
+//! Since arch modules are imported into generic modules using the path attribute, the path of this
+//! file is:
+//!
+//! crate::exception::arch_exception
+
+use aarch64_cpu::registers::*;
+use tock_registers::interfaces::Readable;
+
+//--------------------------------------------------------------------------------------------------
+// Public Code
+//--------------------------------------------------------------------------------------------------
+use crate::exception::PrivilegeLevel;
+
+/// The processing element's current privilege level.
+pub fn current_privilege_level() -> (PrivilegeLevel, &'static str) {
+ let el = CurrentEL.read_as_enum(CurrentEL::EL);
+ match el {
+ Some(CurrentEL::EL::Value::EL2) => (PrivilegeLevel::Hypervisor, "EL2"),
+ Some(CurrentEL::EL::Value::EL1) => (PrivilegeLevel::Kernel, "EL1"),
+ Some(CurrentEL::EL::Value::EL0) => (PrivilegeLevel::User, "EL0"),
+ _ => (PrivilegeLevel::Unknown, "Unknown"),
+ }
+}
diff -uNr 08_hw_debug_JTAG/src/exception/asynchronous.rs 09_privilege_level/src/exception/asynchronous.rs
--- 08_hw_debug_JTAG/src/exception/asynchronous.rs
+++ 09_privilege_level/src/exception/asynchronous.rs
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+//
+// Copyright (c) 2020-2023 Andre Richter <[email protected]>
+
+//! Asynchronous exception handling.
+
+#[cfg(target_arch = "aarch64")]
+#[path = "../_arch/aarch64/exception/asynchronous.rs"]
+mod arch_asynchronous;
+
+//--------------------------------------------------------------------------------------------------
+// Architectural Public Reexports
+//--------------------------------------------------------------------------------------------------
+pub use arch_asynchronous::print_state;
diff -uNr 08_hw_debug_JTAG/src/exception.rs 09_privilege_level/src/exception.rs
--- 08_hw_debug_JTAG/src/exception.rs
+++ 09_privilege_level/src/exception.rs
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: MIT OR Apache-2.0
+//
+// Copyright (c) 2020-2023 Andre Richter <[email protected]>
+
+//! Synchronous and asynchronous exception handling.
+
+#[cfg(target_arch = "aarch64")]
+#[path = "_arch/aarch64/exception.rs"]
+mod arch_exception;
+
+pub mod asynchronous;
+
+//--------------------------------------------------------------------------------------------------
+// Architectural Public Reexports
+//--------------------------------------------------------------------------------------------------
+pub use arch_exception::current_privilege_level;
+
+//--------------------------------------------------------------------------------------------------
+// Public Definitions
+//--------------------------------------------------------------------------------------------------
+
+/// Kernel privilege levels.
+#[allow(missing_docs)]
+#[derive(Eq, PartialEq)]
+pub enum PrivilegeLevel {
+ User,
+ Kernel,
+ Hypervisor,
+ Unknown,
+}
diff -uNr 08_hw_debug_JTAG/src/main.rs 09_privilege_level/src/main.rs
--- 08_hw_debug_JTAG/src/main.rs
+++ 09_privilege_level/src/main.rs
@@ -121,6 +121,7 @@
mod console;
mod cpu;
mod driver;
+mod exception;
mod panic_wait;
mod print;
mod synchronization;
@@ -148,6 +149,7 @@
/// The main function running after the early init.
fn kernel_main() -> ! {
+ use console::console;
use core::time::Duration;
info!(
@@ -157,6 +159,12 @@
);
info!("Booting on: {}", bsp::board_name());
+ let (_, privilege_level) = exception::current_privilege_level();
+ info!("Current privilege level: {}", privilege_level);
+
+ info!("Exception handling state:");
+ exception::asynchronous::print_state();
+
info!(
"Architectural timer resolution: {} ns",
time::time_manager().resolution().as_nanos()
@@ -165,11 +173,15 @@
info!("Drivers loaded:");
driver::driver_manager().enumerate();
- // Test a failing timer case.
- time::time_manager().spin_for(Duration::from_nanos(1));
+ info!("Timer test, spinning for 1 second");
+ time::time_manager().spin_for(Duration::from_secs(1));
+
+ info!("Echoing input now");
+ // Discard any spurious received characters before going into echo mode.
+ console().clear_rx();
loop {
- info!("Spinning for 1 second");
- time::time_manager().spin_for(Duration::from_secs(1));
+ let c = console().read_char();
+ console().write_char(c);
}
}
diff -uNr 08_hw_debug_JTAG/tests/boot_test_string.rb 09_privilege_level/tests/boot_test_string.rb
--- 08_hw_debug_JTAG/tests/boot_test_string.rb
+++ 09_privilege_level/tests/boot_test_string.rb
@@ -1,3 +1,3 @@
# frozen_string_literal: true
-EXPECTED_PRINT = 'Spinning for 1 second'
+EXPECTED_PRINT = 'Echoing input now'