Asynchronous Programming in Rust: Learn asynchronous programming by building working examples of futures, green threads, and runtimes

Asynchronous Programming in Rust

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Publishing Product Manager: Samriddhi Murarka

Group Product Manager: Kunal Sawant

Senior Editor: Kinnari Chohan

Technical Editor: Rajdeep Chakraborty

Copy Editor: Safis Editing

Project Coordinator: Manisha Singh

Indexer: Rekha Nair

Production Designer: Joshua Misquitta

Marketing DevRel Coordinator: Sonia Chauhan

First published: February 2024

Production reference: 2020224

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80512-813-7

www.packtpub.com

To my family—my brother, my parents, and especially my beloved wife and fantastic children that make every day an absolute joy.

– Carl Fredrik Samson

Contributors

About the author

Carl Fredrik Samson is a popular technology writer and has been active in the Rust community since 2018. He has an MSc in Business Administration where he specialized in strategy and finance. When not writing, he’s a father of two children and a CEO of a company with 300 employees. He’s been interested in different kinds of technologies his whole life and his programming experience ranges from programming against old IBM mainframes to modern cloud computing, using everything from assembly to Visual Basic for Applications. He has contributed to several open source projects including the official documentation for asynchronous Rust.

I want to thank the Rust community for being so constructive, positive and welcoming. This book would not have happened had it not been for all the positive and insightful interaction with the community. A special thanks goes to the implementors of all the libraries that underpins the async ecosystem today like mio, Tokio, and async-std.

I also want to thank my editor, Kinnari, who has been extraordinarily patient and helpful during the process of writing this book.

About the reviewer

Evgeni Pirianov is an experienced Senior Software Engineer with a deep expertise in Backend Technologies, Web3 an Blockchain. Evgeni has graduated with a degree in Engineering from Imperial College, London and has worked for a few years developing non-linear solvers in C++ . Ever since, he has been at the forefront of architecturing, designing, and implementing decentralized applications in the fields of Defi and Metaverse. Evgeni’s passion for Rust is unsurpassed and he is a true believer of its bright future and wide range of applications.

Yage Hu is a software engineer specializing in systems programming and computer architecture. He has cut code in companies such as Uber, Amazon, and Meta and is currently conducting systems research with WebAssembly and Rust. Yage and his wife have just welcomed their first child, Maxine.

Table of Contents

Preface

Part 1: Asynchronous Programming Fundamentals

1

Concurrency and Asynchronous Programming: a Detailed Overview

Technical requirements

An evolutionary journey of multitasking

Non-preemptive multitasking

Preemptive multitasking

Hyper-threading

Multicore processors

Do you really write synchronous code?

Concurrency versus parallelism

The mental model I use

Let’s draw some parallels to process economics

Concurrency and its relation to I/O

What about threads provided by the operating system?

Choosing the right reference frame

Asynchronous versus concurrent

The role of the operating system

Concurrency from the operating system’s perspective

Teaming up with the operating system

Communicating with the operating system

The CPU and the operating system

Down the rabbit hole

How does the CPU prevent us from accessing memory we’re not supposed to access?

But can’t we just change the page table in the CPU?

Interrupts, firmware, and I/O

A simplified overview

Interrupts

Firmware

Summary

2

How Programming Languages Model Asynchronous Program Flow

Definitions

Threads

Threads provided by the operating system

Creating new threads takes time

Each thread has its own stack

Context switching

Scheduling

The advantage of decoupling asynchronous operations from OS threads

Example

Fibers and green threads

Each stack has a fixed space

Context switching

Scheduling

FFI

Callback based approaches

Coroutines: promises and futures

Coroutines and async/await

Summary

3

Understanding OS-Backed Event Queues, System Calls, and Cross-Platform Abstractions

Technical requirements

Running the Linux examples

Why use an OS-backed event queue?

Blocking I/O

Non-blocking I/O

Event queuing via epoll/kqueue and IOCP

Readiness-based event queues

Completion-based event queues

epoll, kqueue, and IOCP

Cross-platform event queues

System calls, FFI, and cross-platform abstractions

The lowest level of abstraction

The next level of abstraction

The highest level of abstraction

Summary

Part 2: Event Queues and Green Threads

4

Create Your Own Event Queue

Technical requirements

Design and introduction to epoll

Is all I/O blocking?

The ffi module

Bitflags and bitmasks

Level-triggered versus edge-triggered events

The Poll module

The main program

Summary

5

Creating Our Own Fibers

Technical requirements

How to use the repository alongside the book

Background information

Instruction sets, hardware architectures, and ABIs

The System V ABI for x86-64

A quick introduction to Assembly language

An example we can build upon

Setting up our project

An introduction to Rust inline assembly macro

Running our example

The stack

What does the stack look like?

Stack sizes

Implementing our own fibers

Implementing the runtime

Guard, skip, and switch functions

Finishing thoughts

Summary

Part 3: Futures and async/await in Rust

6

Futures in Rust

What is a future?

Leaf futures

Non-leaf futures

A mental model of an async runtime

What the Rust language and standard library take care of

I/O vs CPU-intensive tasks

Summary

7

Coroutines and async/await

Technical requirements

Introduction to stackless coroutines

An example of hand-written coroutines

Futures module

HTTP module

Do all futures have to be lazy?

Creating coroutines

async/await

coroutine/wait

corofy—the coroutine preprocessor

b-async-await—an example of a coroutine/wait transformation

c-async-await—concurrent futures

Final thoughts

Summary

8

Runtimes, Wakers, and the Reactor-Executor Pattern

Technical requirements

Introduction to runtimes and why we need them

Reactors and executors

Improving our base example

Design

Changing the current implementation

Creating a proper runtime

Step 1 – Improving our runtime design by adding a Reactor and a Waker

Creating a Waker

Changing the Future definition

Step 2 – Implementing a proper Executor

Step 3 – Implementing a proper Reactor

Experimenting with our new runtime

An example using concurrency

Running multiple futures concurrently and in parallel

Summary

9

Coroutines, Self-Referential Structs, and Pinning

Technical requirements

Improving our example 1 – variables

Setting up the base example

Improving our base example

Improving our example 2 – references

Improving our example 3 – this is… not… good…

Discovering self-referential structs

What is a move?

Pinning in Rust

Pinning in theory

Definitions

Pinning to the heap

Pinning to the stack

Pin projections and structural pinning

Improving our example 4 – pinning to the rescue

future.rs

http.rs

Main.rs

executor.rs

Summary

10

Creating Your Own Runtime

Technical requirements

Setting up our example

main.rs

future.rs

http.rs

executor.rs

reactor.rs

Experimenting with our runtime

Challenges with asynchronous Rust

Explicit versus implicit reactor instantiation

Ergonomics versus efficiency and flexibility

Common traits that everyone agrees about

Async drop

The future of asynchronous Rust

Summary

Epilogue

Index

Other Books You May Enjoy

Preface

The content in this book was initially written as a series of shorter books for programmers wanting to learn asynchronous programming from the ground up using Rust. I found the existing material I came upon at the time to be in equal parts frustrating, enlightening, and confusing, so I wanted to do something about that.

Those shorter books became popular, so when I got the chance to write everything a second time, improve the parts that I was happy with, and completely rewrite everything else and put it in a single, coherent book, I just had to do it. The result is right in front of you.

People start programming for a variety of different reasons. Scientists start programming to model problems and perform calculations. Business experts create programs that solve specific problems that help their businesses. Some people start programming as a hobby or in their spare time. Common to these programmers is that they learn programming from the top down.

Most of the time, this is perfectly fine, but on the topic of asynchronous programming in general, and Rust in particular, there is a clear advantage to learning about the topic from first principles, and this book aims to provide a means to do just that.

Asynchronous programming is a way to write programs where you divide your program into tasks that can be stopped and resumed at specific points. This, in turn, allows a language runtime, or a library, to drive and schedule these tasks so their progress interleaves.

Asynchronous programming will, by its very nature, affect the entire program flow, and it’s very invasive. It rewrites, reorders, and schedules the program you write in a way that’s not always obvious to you as a programmer.

Most programming languages try to make asynchronous programming so easy that you don’t really have to understand how it works just to be productive in it.

You can get quite productive writing asynchronous Rust without really knowing how it works as well, but Rust is more explicit and surfaces more complexity to the programmer than most other languages. You will have a much easier time handling this complexity if you get a deep understanding of asynchronous programming in general and what really happens when you write asynchronous Rust.

Another huge upside is that learning from first principles results in knowledge that is applicable way beyond Rust, and it will, in turn, make it easier to pick up asynchronous programming in other languages as well. I would even go so far as to say that most of this knowledge will be useful even in your day-to-day programming. At least, that’s how it’s been for me.

I want this book to feel like you’re joining me on a journey, where we build our knowledge topic by topic and learn by creating examples and experiments along the way. I don’t want this book to feel like a lecturer simply telling you how everything works.

This book is created for people who are curious by nature, the kind of programmers who want to understand the systems they use, and who like creating small and big experiments as a way to explore and learn.

Who this book is for

This book is for developers with some prior programming experience who want to learn asynchronous programming from the ground up so they can be proficient in async Rust and be able to participate in technical discussions on the subject. The book is perfect for those who like writing working examples they can pick apart, expand, and experiment with.
There are two kinds of personas that I feel this book is especially relevant to:

Developers coming from higher-level languages with a garbage collector, interpreter, or runtime, such as C#, Java, JavaScript, Python, Ruby, Swift, or Go. Programmers who have extensive experience with asynchronous programming in any of these languages but want to learn it from the ground up and programmers with no experience with asynchronous programming should both find this book equally useful.

Developers with experience in languages such as C or C++ that have limited experience with asynchronous programming.

What this book covers

Chapter 1, Concurrency and Asynchronous Programming: A Detailed Overview, provides a short history leading up to the type of asynchronous programming we use today. We give several important definitions and provide a mental model that explains what kind of problems asynchronous programming really solves, and how concurrency differs from parallelism. We also cover the importance of choosing the correct reference frame when discussing asynchronous program flow, and we go through several important and fundamental concepts about CPUs, operating systems, hardware, interrupts, and I/O.

Chapter 2, How Programming Languages Model Asynchronous Program Flow, narrows the scope from the previous chapter and focuses on the different ways programming languages deal with asynchronous programming. It starts by giving several important definitions before explaining stackful and stackless coroutines, OS threads, green threads, fibers, callbacks, promises, futures, and async/await.

Chapter 3, Understanding OS-Backed Event Queues, System Calls, and Cross-Platform Abstractions, explains what epoll, kqueue, and IOCP are and how they differ. It prepares us for the next chapters by giving an introduction to syscalls, FFI, and cross-platform abstractions.

Chapter 4, Create Your Own Event Queue, is the chapter where you create your own event queue that mimics the API of mio (the popular Rust library that underpins much of the current async ecosystem). The example will center around epoll and go into quite a bit of detail on how it works.

Chapter 5, Creating Our Own Fibers, walks through an example where we create our own kind of stackful coroutines called fibers. They’re the same kind of green threads that Go uses and show one of the most widespread and popular alternatives to the type of abstraction Rust uses with futures and async/await today. Rust used this kind of abstraction in its early days before it reached 1.0, so it’s also a part of Rust’s history. This chapter will also cover quite a few general programming concepts, such as stacks, assembly, Application Binary Interfaces (ABIs), and instruction set architecture (ISAs), that are useful beyond the context of asynchronous programming as well.

Chapter 6, Futures in Rust, gives a short introduction and overview of futures, runtimes, and asynchronous programming in Rust.

Chapter 7, Coroutines and async/await, is a chapter where you write your own coroutines that are simplified versions of the ones created by async/await in Rust today. We’ll write a few of them by hand and introduce a new syntax that allows us to programmatically rewrite what look like regular functions into the coroutines we wrote by hand.

Chapter 8, Runtimes, Wakers, and the Reactor-Executor Pattern, introduces runtimes and runtime design. By iterating on the example we created in Chapter 7, we’ll create a runtime for our coroutines that we’ll gradually improve. We’ll also do some experiments with our runtime once it’s done to better understand how it works.

Chapter 9, Coroutines, Self-Referential Structs, and Pinning, is the chapter where we introduce self-referential structs and pinning in Rust. By improving our coroutines further, we’ll experience first-hand why we need something such as Pin, and how it helps us solve the problems we encounter.

Chapter 10, Create Your Own Runtime, is the chapter where we finally put all the pieces together. We’ll improve the same example from the previous chapters further so we can run Rust futures, which will allow us to use the full power of async/await and asynchronous Rust. We’ll also do a few experiments that show some of the difficulties with asynchronous Rust and how we can best solve them.

To get the most out of this book

You should have some prior programming experience and, preferably, some knowledge about Rust. Reading the free, and excellent, introductory book The Rust Programming Language (https://doc.rust-lang.org/book/) should give you more than enough knowledge about Rust to follow along since any advanced topics will be explained step by step.

The ideal way to read this book is to have the book and a code editor open side by side. You should also have the accompanying repository available so you can refer to that if you encounter any issues.

You need Rust installed. If you haven’t already, follow the instructions here: https://www.rust-lang.org/tools/install.

Some examples will require you to use Windows Subsystem for Linux (WSL) on Windows. If you’re following along on a Windows machine, I recommend that you enable WSL (https://learn.microsoft.com/en-us/windows/wsl/install) now and install Rust by following the instructions for installing Rust on WSL here: https://www.rust-lang.org/tools/install.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

The accompanying repository is organized in the following fashion:

Code that belongs to a specific chapter is in that chapter’s folder (e.g., ch01).

Each example is organized as a separate crate.

The letters in front of the example names indicate in what order the different examples are presented in the book. For example, the a-runtime example comes before the b-reactor-executor example. This way, they will be ordered chronologically (at least by default on most systems).

Some examples have a version postfixed with -bonus. These versions will be mentioned in the book text and often contain a specific variant of the example that might be interesting to check out but is not important to the topic at hand.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Asynchronous-Programming-in-Rust. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: So, now we have created our own async runtime that uses Rust’s Futures, Waker, Context, and async/await.

A block of code is set as follows:

pub trait Future {

    type Output;

    fn poll(&mut self) -> PollState;

}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

struct Coroutine0 {

stack: Stack0,

    state: State0,

}

Any command-line input or output is written as follows:

$ cargo run

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read Asynchronous Programming in Rust, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781805128137

2. Submit your proof of purchase

3. That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Asynchronous Programming Fundamentals

In this part, you’ll receive a thorough introduction to concurrency and asynchronous programming. We’ll also explore various techniques that programming languages employ to model asynchrony, examining the most popular ones and covering some of the pros and cons associated with each. Finally, we’ll explain the concept of OS-backed event queues, such as epoll, kqueue, and IOCP, detailing how system calls are used to interact with the operating system and addressing the challenges encountered in creating cross-platform abstractions like mio. This section comprises the following chapters:

Chapter 1, Concurrency and Asynchronous Programming: A Detailed Overview

Chapter 2, How Programming Languages Model Asynchronous Program Flow

Chapter 3, Understanding OS-Backed Event Queues, System Calls and Cross Platform Abstractions

1

Concurrency and Asynchronous Programming: a Detailed Overview

Asynchronous programming is one of those topics many programmers find confusing. You come to the point when you think you’ve got it, only to later realize that the rabbit hole is much deeper than you thought. If you participate in discussions, listen to enough talks, and read about the topic on the internet, you’ll probably also come across statements that seem to contradict each other. At least, this describes how I felt when I first was introduced to the subject.

The cause of this confusion is often a lack of context, or authors assuming a specific context without explicitly stating so, combined with terms surrounding concurrency and asynchronous programming that are rather poorly defined.

In this chapter, we’ll be covering a lot of ground, and we’ll divide the content into the following main topics:

Async history

Concurrency and parallelism

The operating system and the CPU

Interrupts, firmware, and I/O

This chapter is general in nature. It doesn’t specifically focus on Rust, or any specific programming language for that matter, but it’s the kind of background information we need to go through so we know that everyone is on the same page going forward. The upside is that this will be useful no matter what programming language you use. In my eyes, that fact also makes this one of the most interesting chapters in this book.

There’s not a lot of code in this chapter, so we’re off to a soft start. It’s a good time to make a cup of tea, relax, and get comfortable, as we’re about start this journey together.

Technical requirements

All examples will be written in Rust, and you have two alternatives for running the examples:

Write and run the examples we’ll write on the Rust playground

Install Rust on your machine and run the examples locally (recommended)

The ideal way to read this chapter is to clone the accompanying repository (https://github.com/PacktPublishing/Asynchronous-Programming-in-Rust/tree/main/ch01/a-assembly-dereference) and open the ch01 folder and keep it open while you read the book. There, you’ll find all the examples we write in this chapter and even some extra information that you might find interesting as well. You can of course also go back to the repository later if you don’t have that accessible right now.

An evolutionary journey of multitasking

In the beginning, computers had one CPU that executed a set of instructions written by a programmer one by one. No operating system (OS), no scheduling, no threads, no multitasking. This was how computers worked for a long time. We’re talking back when a program was assembled in a deck of punched cards, and you got in big trouble if you were so unfortunate that you dropped the deck onto the floor.

There were operating systems being researched very early and when personal computing started to grow in the 80s, operating systems such as DOS were the standard on most consumer PCs.

These operating systems usually yielded control of the entire CPU to the program currently executing, and it was up to the programmer to make things work and implement any kind of multitasking for their program. This worked fine, but as interactive UIs using a mouse and windowed operating systems became the norm, this model simply couldn’t work anymore.

Non-preemptive multitasking

Non-preemptive multitasking was the first method used to be able to keep a UI interactive (and running background processes).

This kind of multitasking put the responsibility of letting the OS run other tasks, such as responding to input from the mouse or running a background task, in the hands of the programmer.

Typically, the programmer yielded control to the OS.

Besides offloading a huge responsibility to every programmer writing a program for your platform, this method was naturally error-prone. A small mistake in a program’s code could halt or crash the entire system.

Note

Another popular term for what we call non-preemptive multitasking is cooperative multitasking. Windows 3.1 used cooperative multitasking and required programmers to yield control to the OS by using specific system calls. One badly-behaving application could thereby halt the entire system.

Preemptive multitasking

While non-preemptive multitasking sounded like a good idea, it turned out to create serious problems as well. Letting every program and programmer out there be responsible for having a responsive UI in an operating system can ultimately lead to a bad user experience, since every bug out there could halt the