Modern C++

Programming
1. Introduction

Federico Busato
2024-03-29
Table of Contents

1 A Little History of C/C++ Programming Language

2 Areas of Application and Popularity
3 C++ Philosophy
4 C++ Weaknesses
C++ Alternatives
Why Switching to a New Language is Hard?

5 The Course

1/56
About Motivation 1/5

“When recruiting research assistants, I look at grades as the last indi-

cator. I find that imagination, ambition, initiative, curiosity, drive,
are far better predictors of someone who will do useful work with me. Of
course, these characteristics are themselves correlated with high grades,
but there is something to be said about a student who decides that a
given course is a waste of time and that he works on a side project in-
stead.
Breakthroughs don’t happen in regular scheduled classes, they happen
in side projects. We want people who complete the work they were as-
signed, but we also need people who can reflect critically on what
is genuinely important”
Daniel Lemire, Prof. at the University of Quebec
2/56
About Motivation 2/5

Academic excellence is not a strong predictor

of career excellence

“Across industries, research shows that the correlation between grades

and job performance is modest in the first year after college and trivial
within a handful of years...
Academic grades rarely assess qualities like creativity, leadership and team-
work skills, or social, emotional and political intelligence. Yes, straight-A
students master cramming information and regurgitating it on exams.
But career success is rarely about finding the right solution to a
problem — it’s more about finding the right problem to solve...”

3/56
About Motivation 3/5

“Getting straight A’s requires conformity. Having an influential

career demands originality.
This might explain why Steve Jobs finished high school with a 2.65
G.P.A., J.K. Rowling graduated from the University of Exeter with
roughly a C average, and the Rev. Dr. Martin Luther King Jr. got only
one A in his four years at Morehouse

If your goal is to graduate without a blemish on your transcript, you

end up taking easier classes and staying within your comfort zone. If
you’re willing to tolerate the occasional B...You gain experience coping
with failures and setbacks, which builds resilience”

4/56
About Motivation 4/5

“Straight-A students also miss out socially. More time studying in

the library means less time to start lifelong friendships, join new clubs or
volunteer...Looking back, I don’t wish my grades had been higher. If I
could do it over again, I’d study less”

Adam Grant, the New York Times

5/56
www.nytimes.com/2018/12/08/opinion/college-gpa-career-success.html
About Motivation 5/5

“Got a 2.4 GPA my first semester in college. Thought maybe I wasn’t

cut out for engineering. Today I’ve landing two spacecraft on Mars, and
designing one for the moon.
STEM is hard for everyone. Grades ultimately aren’t what matters.
Curiosity and persistence matter”

Ben Cichy, Chief Software Engineer,

NASA Mars Science Laboratory

twitter.com/bencichy/status/1197752802929364992?s=20 6/56
About Programming 1/2

“And programming computers was so fascinating. You create your

own little universe, and then it does what you tell it to do”
Vint Cerf, TCP/IP co-inventor and Turing Award

“Most good programmers do programming not because they expect to

get paid or get adulation by the public, but because it is fun to program”
Linus Torvalds, principal developer of the Linux kernel

“You might not think that programmers are artists, but programming
is an extremely creative profession. It’s logic-based creativity”
John Romero, co-founder of id Software
7/56
About Programming 2/2

Creativity Programming is extremely creative. The ability to perceive the problem in

a novel way, provide new and original solutions. Creativity allows
recognizing and generating alternatives

Form of Art Art is the expression of human creative skills. Every programmer has his
own style. Codes and algorithms show elegance and beauty in the same
way as painting or music

Learn Programming gives the opportunity to learn new things every day,
improve own skills and knowledge

Challenge Programming is a challenge. A challenge against yourself, the problem,

and the environment
8/56
Knowledge-Experience Relation

9/56
Learning

“In software development, learning is not a big part of the job.

It is the job.”
Woody Zuill

10/56
A Little History of
C/C++
Programming
Language
The Assembly Programming Language

main:
.Lfunc_begin0:
push rbp
A long time ago, in a galaxy far, .Lcfi0:
.Lcfi1:
far away....there was Assembly mov rbp, rsp
.Lcfi2:
• Extremely simple instructions sub rsp, 16
movabs rdi, .L.str

• Requires lots of code to do simple tasks

.Ltmp0:
mov al, 0
call printf
• Can express anything your computer can do xor ecx, ecx
mov dword ptr [rbp - 4], eax
• Hard to read, write mov eax, ecx
add rsp, 16

• ...redundant, boring programming, bugs pro-

pop rbp
ret

liferation .Ltmp1:
.Lfunc_end0:
.L.str:
.asciz "Hello World\n"
11/56
A Little History of C 1/3

In the 1969 Dennis M. Ritchie and Ken Thompson (AT&T, Bell Labs) worked on
developing an operating system for a large computer that could be used by a thousand
users. The new operating system was called UNIX

The whole system was still written in assembly code. Besides assembler and Fortran,
UNIX also had an interpreter for the programming language B. A high-level language
like B made it possible to write many pages of code task in just a few lines of code. In
this way the code could be produced much faster than in assembly

A drawback of the B language was that it did not know data-types (everything was
expressed in machine words). Another functionality that the B language did not provide
was the use of “structures”. The lack of these things formed the reason for Dennis
M. Ritchie to develop the programming language C. In 1988 they delivered the final
standard definition ANSI C 12/56
A Little History of C 2/3

Dennis M. Ritchie and Ken Thompson

#include "stdio.h"

int main() {
printf("Hello World\n");
} 13/56
A Little History of C 3/3

Areas of Application:
• UNIX operating system
• Computer games
• Due to their power and ease of use, C were used in the programming of the
special effects for Star Wars

14/56
Star Wars - The Empire Strikes Back
A Little History of C++ 1/3

The C++ programming language (originally named “C with Classes”) was devised
by Bjarne Stroustrup also an employee from Bell Labs (AT&T). Stroustrup started
working on C with Classes in 1979. (The ++ is C language operator)
The first commercial release of the C++ language was in October 1985

15/56
A Little History of C++ 2/3

The roots of C++

16/56
“The Evolution of C++Past, Present, and Future”, B. Stroustrup, CppCon16
A Little History of C++ 3/3

17/56
About Evolution

“If you’re teaching today what you were teaching five

years ago, either the field is dead or you are”
Noam Chomsky

18/56
Areas of Application
and Popularity
Most Popular Programming Languages
(IEEE Spectrum - 2022)

19/56
Interactive: The Top Programming Languages 2022
Most Popular Programming Languages (TIOBE - December. 2022)

20/56
www.tiobe.com/tiobe-index/
Most Popular Programming Languages (Redmonk - June, 2022)

21/56
redmonk.com
Why C++ is so Popular? 1/2

There may be more than 200 billion lines

of C/C++ code globally

• Performance is the defining aspect of C++. No other programming

language provides the performance-critical facilities of C++

• Provide the programmer control over every aspect of performance

• Leave no room for a lower level language

Total number of lines of all code in use? 22/56

Why C++ is so Popular? 2/2

• Ubiquity. C++ can run from a low-power embedded device to large-scale

supercomputers

• Multi-Paradigm. Allow writing efficient code without losing high-level

abstraction

• Allow writing low-level code. Drivers, kernels, assembly (asm), etc.

• Ecosystem. Many support tools such as debuggers, memory checkers,

coverage, static analysis, profiling, etc.

• Maturity. C++ has a 40 years history. Many software problems have been
already addressed and developing practices have been investigated 23/56
Areas of Application 1/2

• Operating systems: Windows, Android, OS X, Linux

• Compilers: LLVM, Swift compiler

• Artificial Intelligence: TensorFlow, Caffe, Microsoft Cognitive Toolkit

• Image Editing: Adobe Premier, Photoshop, Illustrator

• Web browser: Firefox, Chrome, etc. + WebAssembly

• High-Performance Computing: drug developing and testing, large scale climate

models, physic simulations

• Embedded systems: IoT, network devices (e.g. GSM), automotive

• Google and Microsoft use C++ for web indexing 24/56

Areas of Application 2/2

• Scientific Computing: CERN/NASA*, SETI@home, Folding@home

• Database: MySQL, ScyllaDB

• Video Games: Unreal Engine, Unity

• Entertainment: Movie rendering (see Interstellar black hole rendering),

virtual reality

• Finance: electronic trading systems (Goldman, JPMorgan, Deutsche Bank)**

... and many more

* The flight code of the NASA Mars drone for the Perseverance Mission, as well as the Webb
telescope software, are mostly written in C++ github.com/nasa/fprime, James Webb Space
25/56
Telescope’s Full Deployment
Why C++ is so Important?

The End of Historical Performance Scaling

26/56
An Important Example... (AI Evolution)

27/56
Performance 1/4

N-BODY SIMULATION
P R O G R A M M I N G L A N G UA G E S P E R F O R M A N C E C O M PA R I S O N
900
780
800
700 660
Execution TIme (S)

600
500
400 360
300
300
200
100 21.47 21.96 22.1 26.61
8.23
0
C++ GO SWIFT JAVA Node.js PHP Ruby Perl Python3

Programming Language

28/56
Performance 2/4

29/56
A programming language benchmark
Performance 3/4

30/56
”A New Golden Age for Computer Architecture“, J. L. Heneessey, D. A. Patterson, 2019
Performance 4/4

Hello World
Language Execution Time
C (on my machine) 0.7 ms
C 2 ms
Go 4 ms
Crystal 8 ms
Shell 10 ms
Python 78 ms
Node 110 ms
Ruby 150 ms
jRuby 1.4 s

31/56
Time to "hello world" on my machine
Performance/Expressiveness Trade-off

Mandelbrot Static Instructions per Line

1,000,000
INSTRUCTIONS PER LINE

100,000

10,000

1,000

100

10

1
Assembly C C++ Java JS Python
32/56
Memory Usage

Memory usage comparison of the

Neighbor-Joining and global alignment programs

A comparison of common programming languages used in bioinformatics (BMC

33/56
Informatic)
Energy Efficiency

34/56
Energy Efficiency across Programming Languages
CO2 Production

35/56
The Ecological Impact of High-performance Computing in Astrophysics, Nature
C++ Philosophy
C++ Philosophy - Performance 1/3

Do not sacrifice performance except as a last resort

Zero Overhead Principle (zero-cost abstraction)

“it basically says if you have an abstraction it should not
cost anything compared to write the equivalent code at lower
level”
“so I have say a matrix multiply it should be written in a
such a way that you could not drop to the C level of abstrac-
tion and use arrays and pointers and such and run faster”
Bjarne Stroustrup
36/56
C++ Philosophy - Type Safety 2/3

Enforce safety at compile time whenever possible

Statically Typed Language

“The C++ compiler provides type safety and catches
many bugs at compile time instead of run time (a critical
consideration for many commercial applications.)”
www.python.org/doc/FAQ.html

• The type annotation makes the code more readable

• Promote compiler optimizations and runtime efficiency
• Allow users to define their own type system 37/56
C++ Philosophy 3/3

• Programming model: compartmentalization, only add

features if they solve an actual problem, and allow full control

• Predictable runtime (under constraints): no garbage

collector, no dynamic type system → real-time systems

• Low resources: low memory and energy consumption →

restricted hardware platforms

• Well suited for static analysis → safety critical software

• Portability → Modern C++ standards are highly portable 38/56

Who is C++ for?

“C++ is for people who want to use hardware very well

and manage the complexity of doing that through abstrac-
tion”
Bjarne Stroustrup

“a language like C++ is not for everybody. It is gener-

ated via sharp and effective tool for professional basically and
definitely for people who aim at some kind of precision”
Bjarne Stroustrup
39/56
Suggested Introduction Video

40/56
C++ Weaknesses
Why C++ is so Difficult? 1/2

... and why teaching C++ as first programming language is a bad idea?
C++ is the hardest language from students to master
• More languages in one
- Standard C/C++ programming
- Preprocessor
- Object-Oriented features
- Templates and Meta-Programming
• Huge set of features
• Worry about memory management
• Low-level implementation details: pointer arithmetic, structure, padding,
undefined behavior, etc.
• Frustrating: compiler/runtime errors (e.g. seg. fault) 41/56
Why C++ is so Difficult? 2/2

“C makes it easy to shoot yourself in the foot; C++ makes it harder,

but when you do, it blows your whole leg off”
Bjarne Stroustrup, Creator of the C++ language

“The problem with using C++...is that there’s already a strong ten-
dency in the language to require you to know everything before you can
do anything”
Larry Wall, Creator of the Perl language

“Despite having 20 years of experience with C++, when I compile a

non-trivial chunk of code for the first time without any error or warning,
I am suspicious. It is not, usually, a good sign”
Daniel Lemire, Prof. at the University of Quebec 42/56
C++ Weaknesses 1/2

Backward-compatibility

‘‘Dangerous defaults and constructs, often originating from C, cannot be removed

or altered”

“Despite the hard work of the committee, newer features sometimes have flaws
that only became obvious after extensive user experience, which cannot then be
fixed”

“C++ practice has put an ever-increasing cognitive burden on the developer for
what I feel has been very little gain in productivity or expressiveness and at a huge cost
to code clarity”
43/56
C++ Weaknesses 2/2

C++ critics and replacements:

• Epochs: a backward-compatible language evolution mechanism

• Goals and priorities for C++

• Carbon Language

• Circle C++ Compiler

• Cppfront: Can C++ be 10x simpler & safer ... ?

44/56
C++ Alternatives: Rust

Rust (1.0, 2015) has been Stack Overflow’s most loved language for eight years in a
row. Rust focuses on performance and zero-abstraction overhead as C++. It is
designed to prevent many vulnerabilities that affect C++, especially memory bugs,
enforcing constraints at compile type. In addition, it promotes cross-platform
compatibility

“First-time contributors to Rust projects are about 70 times less likely to

introduce vulnerabilities than first-time contributors to C++ projects”
Tracey et al. 1

1
Grading on a Curve: How Rust can Facilitate New Contributors while Decreasing
Vulnerabilities
• CISA, NSA: The Case for Memory Safe Roadmap
• Octoverse: The Fastest Growing Languages
45/56
• Secure by Design: Google’s Perspective on Memory Safety
C++ Alternatives: Zig

Zig (2016) is a minimal open-source programming language that can be intended as

replacement of C. Zig supports compile time generics, reflection and evaluation,
cross-compiling, and manual memory management. It is made to be fully interoperable
with C and also includes a C/C++ compiler.

Zig Programming Language 46/56

Why Switching to a New Language is Hard? 1/3

• No perfect language. There are always newer ’shining’ languages

• Alignment. Force all developers to switch to the new language

• Interoperability. Hundreds of billion lines of existing code. Must interoperate

with C and C++ code imposing serious design constraints

• Ecosystem. Lack of tools and libraries developed in the last four decades

• Time and Cost. Converting a codebase of 10 million lines: 500 developers, 5

years, $1,400,000,0001

1 47/56
Bjarne Stroustrup: Delivering Safe C++
Why Switching to a New Language is Hard? 2/3

•Performance overhead of safe programming languages

• Towards Understanding the Runtime Performance of Rust

• How much does Rust’s bounds checking actually cost?
• How to avoid bounds checks in Rust (without unsafe!)
48/56
• Is coding in Rust as bad as in C++?
Why Switching to a New Language is Hard? 3/3

49/56
Language Complexity

Every second spent trying to understand the

language is one not spent understanding the
problem

50/56
The Course
The Course

51/56
The Course

Don’t forget: The right name of the course should be

“Introduction to Modern C++ Programming”

For many topics in the course, there are more than one book devoted to present the
concepts in detail

52/56
The Course

The primary goal of the course is to drive who has previous experience with
C/C++ and object-oriented programming to a proficiency level of (C++)
programming
• Proficiency : know what you are doing and the related implications
• Understand what problems/issues address a given language feature
• Learn engineering practices (e.g. code conventions, tools) and hardware/software
techniques (e.g. semantic, optimizations) that are not strictly related to C++
What the course is not:
• A theoretical course on programming
• A high-level concept description

What the course is:

• A practical course, prefer examples to long descriptions
53/56
• A “quite” advanced C++ programming language course
The Course

Organization:
• 22 lectures
• ∼1,500 slides
• C++03 / C++11 / C++14 / C++17 / C++20 / (C++23) / (C++26)

Roadmap:
• Review C concepts in C++ (built-in types, memory management, preprocessing,
etc.)
• Introduce object-oriented and template concepts
• Present how to organize the code and the main conventions
• C++ tool goals and usage (debugger, static analysis, etc.)

54/56
Who I Am

Federico Busato, Ph.D.

federico-busato.github.io
• Senior Software Engineer at Nvidia,
CUDA Mathematical Libraries

• Lead engineer of the Sparse Linear Algebra group

• Research/Work interests:
- Linear Algebra
- Graph Algorithms
- Parallel/High-Performance Computing
- Code Optimization

NOT a C++ expert/“guru”, still learning 55/56

A Little Bit about My Work

Our projects:

cuSPARSE GPU-accelerated sparse linear algebra library (matrix-matrix

multiplication, triangular solver, etc.), part of the CUDA Toolkit (8M
downloads every year)

cuSPARSELt Specialized library for sparse matrix-matrix multiplication that exploits the
most advanced GPU features such as Sparse Tensor Cores

NVPL Sparse CPU-accelerated (ARM) sparse linear algebra library

Top500 HPCG NVIDIA Supercomputing benchmark that performs a fixed number of

multigrid preconditioned (using a symmetric Gauss-Seidel smoother)
conjugate gradient (PCG) iterations 56/56
“What I cannot create,
I do not understand”

Richard P.
Feynman

56/56
 “The only way to learn a new pro-
gramming language is by writing pro-
grams in it”
Dennis Ritchie
Creator of the C programming language

56/56