Introduction

David Beazley thought that “JavaScript versus Data Science” would be a better title for this book. While that one word sums up how many people view the language, we hope we can convince you that modern JavaScript is usable as well as useful. Scientists and engineers are who we were thinking of when we wrote this book but we hope that these lessons will also help librarians, digital humanists, and everyone else who uses computing in their research.

We will cover:

• Core features of modern JavaScript

• Programming with callbacks and promises

• Creating objects and classes

• Writing HTML and CSS

• Creating interactive pages with React

• Building data services

• Testing

• Data visualization

• Combining everything to create a three-tier web application

Unlike most introductions to JavaScript, these lessons present an even mix of browser programming and server programming. We give each topic only shallow coverage; if you want to know more, there are many other free tutorials you can dive into once you’ve mastered the basics, some of which are both up-to-date and well designed.

Who You Are

Every lesson should aim to meet the needs of specific learners . The three people described below define the intended audience for this one.

Bhadra

received a BSc in microbiology five years ago, and has worked since then for a biotech firm with labs in four countries. She did a statistics class using R as an undergrad, then learned some more R and some Unix shell scripting in a Software Carpentry workshop, but has no other training as a programmer. Bhadra’s team is developing tools to detect structural similarities between proteins. They would like to build a browser interface to their tools so that people can test different algorithms on various datasets. This book will show Bhadra how to build, test, and deploy that interface.

Efraim

did fieldwork for the Ministry of Natural Resources for thirty-one years. He learned Visual Basic so that he could write Excel macros, then mastered C in order to maintain the control software for some second-hand remote sensing equipment. Efraim recently retired, and is now an active member of several citizen science projects. This book will show him how to create a service to share those projects’ data with the world, and how to build a web-based administrative interface for it.

Sumi

is completing a PhD in 19th Century history. As part of her research, she is transcribing and cataloging the records of several dozen Japanese-American midwives. She has been creating and customizing WordPress sites for several years, and has picked up bits and pieces of JavaScript while doing so. Sumi is about to start looking for a job, and wants to create an interactive website to showcase her research. This book will fill in some of the gaps in her knowledge and show her how to take advantage of JavaScript’s more modern features.

These prototypical users:

• can write two-page programs that use lists, loops, conditionals, and functions,

• can run commands in the Unix shell to navigate the filesystem and create and delete directories and files, and

• have reliable access to the Internet.

Who We Are

Maya Gans is a freelance data scientist and front-end developer by way of quantitative biology. She has 4 years of experience programming in R, and her passion for data visualization brought her to the weird world of JavaScript. When she isn’t debugging or blogging about code, she’s somewhere remote climbing large mountains. She dedicates this book to her fellow self-taught programmers who were told they weren’t good enough but are too driven and excited to care.

Toby Hodges is a bioinformatician turned community coordinator, working on the Bio-IT Project at EMBL. He teaches a lot of courses in computing, organizes a lot of community-building events, listens to a lot of punk rock, and occasionally still finds time to write code and ride his bike. Toby would like to thank his wife for her support and patience while he swore about how annoying JavaScript is to debug.

Greg Wilson has worked for 35 years in both industry and academia, and is the author or editor of several books on computing and two for children. He co-founded Software Carpentry, a non-profit organization that teaches basic computing skills to researchers, and is now part of the education team at RStudio. Greg would like to thank everyone at Rangle who was so patient with him when he was learning JavaScript.

Setting Up

You can find the examples for each chapter in the src directory in our GitHub repository. Each sub-folder contains the code and data needed to follow along with the text.

The exercises at the end of each chapter include new information that you will need later in the book, and are therefore not optional. You can do the first few online, using a service like RunKit, which gives you an interactive JavaScript playground in your browser. For larger things, and for chapters starting with the one on creating dynamic web pages (Chapter 7), you should download and install the latest Long-term Support (LTS) versions of Node and NPM.

Node is an open source implementation of JavaScript that includes a command-line interpreter like those for languages such as Python and R. The command node on its own starts a read-evaluate-print loop (REPL) that executes commands as they are typed in and displays their output. The command node filename.js reads and runs the commands in filename.js; we will see in Chapter 6 how to run JavaScript in a browser.

npm is the Node Package Manager, a command-line tool for finding, installing, updating, building, and executing JavaScript libraries. The command npm install --global library-name (without a .js extension) installs a library globally so that all projects can use it, while npm install --save library-name installs the library locally (i.e., in the current project folder). Local installation is usually a better idea, since it isolates projects from one another.

Contributing

Contributions of all kinds are welcome, from errata and minor improvements to entirely new sections and chapters: please submit an issue or pull request to our GitHub repository. Everyone whose work is incorporated will be acknowledged; please note that all contributors are required to abide by our Code of Conduct (Appendix 18). Please note that we use Simplified English rather than Traditional English, i.e., American rather than British spelling and grammar. We encourage translations; if you would like to take this on, please email us.

If you wish to report errata or suggest improvements to wording, please include the chapter name in the first line of the body of your report (e.g., Testing Data Analysis).

Acknowledgments

We are grateful as always to Shashi Kumar for help with the LaTeX. We are also grateful for fixes from:

• Andrew Bates

• Stephan Druskat

• Chiranjeev Gupta

• Nicolai Karcher

• Eric Leung

• Peter Munro

• Leonardo Uieda

Exercises

Setting Up

Install Node on your computer, then run the commands node --version and npm --version to see which versions you have.

Key Points

• Modern JavaScript is a good tool for building desktop and web-based applications.

• This course is for people who know what loops and functions are, but have never used JavaScript or built web applications.

• Node is a command-line interpreter for JavaScript, which can be used interactively or to run scripts in files.

• NPM is the Node Package Manager, which can be used to find, install, update, build, and execute JavaScript libraries.

Basic Features

This lesson introduces the core features of JavaScript, including how to run programs, the language’s basic data types, arrays and objects, loops, conditionals, functions, and modules. All of these concepts should be familiar if you have programmed before.

Hello, World

Use your favorite text editor to put the following line in a file called hello.js:

console.log('hello, world')

console is a built-in module that provides basic printing services (among other things). As in many languages, we use the dotted notation X.Y to get part Y of thing X—in this case, to get console’s log function. Character strings like ’hello, world’ can be written with either single quotes or double quotes, so long as the quotation marks match, and semi-colons at the ends of statements are now (mostly) optional.

To run a program, type node program_name.js at the command line. (We will preface shell commands with $ to make them easier to spot.)

$ node src/basics/hello.js

hello, world

Basic Data Types

JavaScript has the usual datatypes, though unlike C, Python, and many other languages, there is no separate type for integers: it stores all numbers as 64-bit floating-point values, which is accurate up to about 15 decimal digits. We can check this using typeof, which returns a string. (Note: typeof is an operator, not a function: we apply it to something by typing a space followed by the name of the thing we’d like to check the type of, e.g., typeof dress as opposed to typeof(dress).) We use it alongside const below, which itself is helpful when we want to give a name to a constant value:

const aNumber = 123.45
console.log('the type of', aNumber, 'is', typeof aNumber)

the type of 123.45 is number

const anInteger = 123
console.log('the type of', anInteger, 'is', typeof anInteger)

the type of 123 is number

We have already met strings, which may contain any Unicode character:

const aString = 'some text'
console.log('the type of', aString, 'is', typeof aString)

the type of some text is string

Functions are also a type of data, a fact whose implications we will explore in Chapter 3:

console.log('the type of', console.log, 'is', typeof console.log)

the type of function () { [native code] } is function

Rather than showing the other basic types one by one, we will put three values in a list and loop over it:

const otherValues = [true, undefined, null]
for (let value of otherValues) {
  console.log('the type of', value, 'is', typeof value)
}

the type of true is boolean
the type of undefined is undefined
the type of null is object

As the example above shows, we create an array of values to loop through called otherValues. We initiate our loop with the word for. Within the parentheses, let creates a variable called value to iterate over each element within otherValues, and value is the changing array value of otherValues. Finally, within the curly braces we perform our desired operation on every value.

Note that we use let rather than the older var and of rather than in: the latter returns the indexes of the collection (e.g., 0, 1, 2), which has some traps for the unwary (Appendix 23.2). Note also that indexing starts from 0 rather than 1, and that indentation is optional and for readability purposes only. This may be different from the language that you’re used to.

Constants versus Variables

You should make things constants unless they really need to be variables because it’s easier for both people and computers to keep track of things that are defined once and never change.

After all this, the types themselves are somewhat anticlimactic. JavaScript’s boolean type can be either true or false, though we will see below that other things can be treated as Booleans. undefined means “hasn’t been given a value”, while null means “has a value, which is nothing”.

Control Flow

We have already seen for loops and flat arrays, so let’s have a look at nested arrays and conditionals. We start with arrays that contain other arrays, which are usually processed by nested loops:

const nested = [['northwest', 'northeast'],
                ['southwest', 'southeast']]
for (let outer of nested) {
  for (let inner of outer) {
    console.log(inner)
  }
}

northwest
northeast
southwest
southeast

The inner loop runs a complete cycle of iterations for each iteration of the outer loop. Each value assigned to the variable outer is a pair, so each value assigned to inner is one of the two strings from that pair (Figure 2.1).

A JavaScript program can also make choices: it executes the body of an if statement if and only if the condition is true. Each if can have an else, whose body is only executed if the condition isn’t true:

const values = [0, 1, '', 'text', undefined, null, [], [2, 3]]
for (let element of values) {
  if (element) {
    console.log(element, 'of type', typeof element, 'is truthy')
  } else {
    console.log(element, 'of type', typeof element, 'is falsy')
  }
}

0 of type number is falsy
1 of type number is truthy
 of type string is falsy
text of type string is truthy
undefined of type undefined is falsy
null of type object is falsy
 of type object is truthy
2,3 of type object is truthy

This example shows that arrays are heterogeneous, i.e., that they can contain values of many different types. It also shows that JavaScript has some rather odd ideas about the nature of truth. 0 is falsy, while all other numbers are truthy; similarly, the empty string is falsy and all other strings are truthy. undefined and null are both falsy, as most programmers would expect.

But as the last two lines of output show, an empty array is truthy, which is different from its treatment in most programming languages. The argument made by JavaScript’s advocates is that an empty array is there, it just happens to be empty, but this behavior is still a common cause of bugs. When testing an array, check that Array.length is zero. (Note that this is a property, not a method, i.e., it should be treated as a variable, not called like a function.)

Safety Tip

Always use === (triple equals) and !== when testing for equality and inequality in JavaScript. == and != do type conversion, which can produce some ugly surprises (Section 23.1).

Formatting Strings

Rather than printing multiple strings and expressions, we can interpolate values into a back-quoted string. (We have to use back quotes because this feature was added to JavaScript long after the language was first created.) As the example below shows, the value to be interpolated is put in ${...}, and can be any valid JavaScript expression, including a function or method call.

for (let color of ['red', 'green', 'blue']) {
  const message = `color is ${color}`
  console.log(message, `and capitalized is ${color.toUpperCase()}`)
}

color is red and capitalized is RED
color is green and capitalized is GREEN
color is blue and capitalized is BLUE

This allows us to succinctly add variables to a string instead of:

const message = "color is" + color + "and capitalized is " + color.toUpperCase()

Objects

An object in JavaScript is a collection of key-value pairs, and is equivalent in simple cases to what Python would call a dictionary. It’s common to visualize an object as a two-column table with the keys in one column and the values in another (Figure 2.2). The keys must be strings; the values can be anything. We can create an object by putting key-value pairs in curly brackets; there must be a colon between the key and the value, and pairs must be separated by commas like the elements of arrays:

const creature = {
  'order': 'Primates',
  'family': 'Callitrichidae',
  'genus': 'Callithrix',
  'species': 'Jacchus'
}

console.log(`creature is ${creature}`)
console.log(`creature.genus is ${creature.genus}`)
for (let key in creature) {
  console.log(`creature[${key}] is ${creature[key]}`)
}

creature is [object Object]
creature.genus is Callithrix
creature[order] is Primates
creature[family] is Callitrichidae
creature[genus] is Callithrix
creature[species] is Jacchus

The type of an object is always object. We can get the value associated with a key using object[key], but if the key has a simple name, we can use object.key instead. Note that the square bracket form can be used with variables for keys, but the dotted notation cannot: i.e., creature.genus is the same as creature[’genus’], but the assignment g = ’genus’ followed by creature.g does not work.

Because string keys are so common, and because programmers use simple names so often, JavaScript allows us to create objects without quoting the names of the keys:

const creature = {
  order: 'Primates',
  family: 'Callitrichidae',
  genus: 'Callithrix',
  species: 'Jacchus'
}

[object Object] is not particularly useful output when we want to see what an object contains. To get a more helpful string representation, use JSON.stringify(object):

console.log(JSON.stringify(creature))

{"order":"Primates","family":"Callitrichidae",
 "genus":"Callithrix","species":"Jacchus"}

Here, “JSON” stands for “JavaScript Object Notation”; we will learn more about it in Chapter 11.

Functions

Functions make it possible for mere mortals to understand programs by allowing us to think about them one piece at a time. Here is a function that finds the lowest and highest values in an array:

function limits (values) {
  if (!values.length) {
    return [undefined, undefined]
  }
  let low = values[0]
  let high = values[0]
  for (let v of values) {
    if (v < low) low = v
    if (v > high) high = v
  }
  return [low, high]
}

Its definition consists of the keyword function, its name, a parenthesized list of parameters (which might be empty), and its body.

The body of the function begins with a test of the thing, referred to inside the function as values, provided as an argument to the function. If values has no length—i.e., it does not consist of multiple entries—the function returns [undefined,undefined]. (We will address the rationale behind this behavior in the exercises.)

  if (!values.length) {
    return [undefined, undefined]
  }

If that initial check finds that values does have a length—i.e., !values.length returns false—the rest of the function is run. This involves first initializing two variables, low and high, with their values set as equal to the first item in values.

  let low = values[0]
  let high = values[0]

In the next stage of the function, all of the values are iterated over and low and high are assigned a new value, equal to that of the next item, if that value is lower than low or higher than high respectively.

  for (let v of values) {
    if (v < low) low = v
    if (v > high) high = v
  }

Once all of the items in values have been examined, the values of low and high are the minimum and maximum of values. These are returned as a pair inside an array.

  return [low, high]

Note that we can use return to explicitly return a value at any time; if nothing is returned, the function’s result is undefined.

One oddity of JavaScript is that almost anything can be compared to almost anything else. Here are a few tests that demonstrate this:

const allTests = [
  [],
  [9],
  [3, 30, 300],
  ['apple', 'Grapefruit', 'banana'],
  [3, 'apple', ['sub-array']]
]
for (let test of allTests) {
  console.log(`limits of ${test} are ${limits(test)}`)
}

limits of  are ,
limits of 9 are 9,9
limits of 3,30,300 are 3,300
limits of apple,Grapefruit,banana are Grapefruit,banana
limits of 3,apple,sub-array are 3,3

Programmers generally don’t write functions this way any longer, since it interacts in odd ways with other features of the language; Section 23.3 explains why and how in more detail. Instead, most programmers now write fat arrow functions consisting of a parameter list, the => symbol, and a body. Fat arrow functions don’t have names, so the function must be assigned that to a constant or variable for later use:

const limits = (values) => {
  if (!values.length) {
    return [undefined, undefined]
  }
  let low = values[0]
  let high = values[0]
  for (let v of values) {
    if (v < low) low = v
    if (v > high) high = v
  }
  return [low, high]
}

No matter how functions are defined, each one is a scope, which means its parameters and any variables created inside it are local to the function. We will discuss scope in more detail in Chapter 3.

Stuck in the Past

Why did JavaScript introduce another syntax rather than fixing the behavior of those defined with function? The twin answers are that changes would break legacy programs that rely on the old behavior, and that the language’s developers wanted to make it really easy to define little functions. Here and elsewhere, we will see how a language’s history and use shape its evolution.

Modules

As our programs grow larger, we will want to put code in multiple files. The unavoidable bad news is that JavaScript has several module systems: Node still uses one called CommonJS, but is converting to the modern standard called ES6, so what we use on the command line is different from what we use in the browser (for now).

Ee Ess

JavaScript’s official name is ECMAScript, though only people who use the word “datum” in everyday conversation ever call it that. Successive versions of the language are therefore known as ES5, ES6, and so on, except when they’re referred to as (for example) ES2018.

Since we’re going to build command-line programs before doing anything in the browser, we will introduce Node’s module system first (Figure 2.3). We start by putting this code in a file called utilities.js:

DEFAULT_BOUND = 3

const clip = (values, bound = DEFAULT_BOUND) => {
  let result = []
  for (let v of values) {
    if (v <= bound) {
      result.push(v)
    }
  }
  return result
}

module.exports = {
  clip: clip
}

The function definition is straightforward; as you may have guessed, bound = DEFAULT_BOUND sets a default value for that parameter so that clip can be called with just an array. You may also have guessed that Array.push appends a value to the end of an array; if you didn’t, well, now you know.

What’s more important is assigning an object to module.exports. Only those things named in this object are visible to the outside world, so DEFAULT_BOUND won’t be. Remember, keys that are simple names don’t have to be quoted, so clip: clip means “associate a reference to the function clip with the string key "clip".

To use our newly-defined module we must require it. For example, we can put this in application.js:

const utilities = require('./utilities')

const data = [-1, 5, 3, 0, 10]
console.log(`clip(${data}) -> ${utilities.clip(data)}`)
console.log(`clip(${data}, 5) -> ${utilities.clip(data, 5)}`)

clip(-1,5,3,0,10) -> -1,3,0
clip(-1,5,3,0,10, 5) -> -1,5,3,0

require returns the object that was assigned to module.exports, so if we have assigned its result to a variable called utilities, we must then call our function as utilities.clip. We use a relative path starting with ./ or ../ to import local files; paths that start with names are taken from installed Node libraries.

Exercises

Typeof

What kind of thing is typeof? Is it an expression? A function? Something else? (You might notice that typeof typeof is syntactically invalid. In such circumstances, an Internet search engine is your friend, as is the Mozilla Developer Network JavaScript reference.)

Fill in the Blanks

Answer these questions about the program below:

1. What does Array.push do?

2. How does a while loop work?

3. What does += do?

4. What does Array.reverse do, and what does it return?

let current = 0
let table = []
while (current < 5) {
  const entry = `square of ${current} is ${current * current}`
  table.push(entry)
  current += 1
}
table.reverse()
for (let line of table) {
  console.log(line)
}

square of 4 is 16
square of 3 is 9
square of 2 is 4
square of 1 is 1
square of 0 is 0

What Is Truth?

Write a function called isTruthy that returns true for everything that JavaScript considers truthy, and false for everything it considers falsy except empty arrays: isTruthy should return false for those.

The Shape of Things to Come

We wrote the example function, limits, above to return [undefined,undefined] if a variable with no length is fed into it. What is the advantage of doing this as opposed to returning undefined only?

Combining Different Types

What does NaN represent? What output would you expect from the code below? Try running it and see whether the results match your expectations. What are the implications of this behavior when working with real-world data?

const first = [3, 7, 8, 9, 1]
console.log(`aggregating ${first}`)
let total = 0
for (let d of first) {
  total += d
}
console.log(total)

const second = [0, 3, -1, NaN, 8]
console.log(`aggregating ${second}`)
total = 0
for (let d of second) {
  total += d
}
console.log(total)

What Does This Do?

Explain what is happening in the assignment statement that creates the constant creature.

const genus = 'Callithrix'
const species = 'Jacchus'
const creature = {genus, species}
console.log(creature)

{ genus: 'Callithrix', species: 'Jacchus' }

Destructuring Assignment

When this short program runs:

const creature = {
  genus: 'Callithrix',
  species: 'Jacchus'
}
const {genus, species} = creature
console.log(`genus is ${genus}`)
console.log(`species is ${species}`)

it produces the output:

genus is Callithrix
species is Jacchus

but when this program runs:

const creature = {
  first: 'Callithrix',
  second: 'Jacchus'
}
const {genus, species} = creature
console.log(`genus is ${genus}`)
console.log(`species is ${species}`)

it produces:

genus is undefined
species is undefined

1. What is the difference between these two programs?

2. How does destructuring assignment work in general?

3. How can we use this technique to rewrite the require statement in src/basics/application.js so that clip can be called directly as clip(...) rather than as utilities.clip(...)?

Return To Me, For My Heart Wants You Only

What output would you see in the console if you ran this code?

const verbose_sum = (first, second) => {
  console.log(`Going to add ${first} to ${second}`)
  let total = first + second
  return total
  console.log(`Finished summing`)
}

var result = verbose_sum(3, 6)
console.log(result)

1. Going to add ${first} to ${second}
9

2. Going to add 3 to 6
9
Finished summing

3. Going to add 3 to 6
9

4. Going to add 3 to 6
36

Key Points

• Use console.log to print messages.

• Use dotted notation X.Y to get part Y of object X.

• Basic data types are Booleans, numbers, and character strings.

• Arrays store multiple values in order.

• The special values null and undefined mean ‘no value’ and ‘does not exist’.

• Define constants with const and variables with let.

• typeof returns the type of a value.

• for (let variable of collection) {...} iterates through the values in an array.

• if (condition) {...} else {...} conditionally executes some code.

• false, 0, the empty string, null, and undefined are false; everything else is true.

• Use back quotes and ${...} to interpolate values into strings.

• An object is a collection of name/value pairs written in {...}.

• object[key] or object.key gets a value from an object.

• Functions are objects that can be assigned to variables, stored in lists, etc.

• function name(...parameters...) {...body...} is the old way to define a function.

• name = (...parameters...) => {...body...} is the new way to define a function.

• Use return inside a function body to return a value at any point.

• Use modules to divide code between multiple files for re-use.

• Assign to module.exports to specify what a module exports.

• require(...path...) imports a module.

• Paths beginning with ‘.’ or ‘/’ are imported locally, but paths without ‘.’ or ‘/’ look in the library.

Callbacks

JavaScript relies heavily on callback functions: Instead of a function giving us a result immediately, we give it another function that tells it what to do next. Many other languages use them as well, but JavaScript is often the first place that programmers with data science backgrounds encounter them. In order to understand how they work and how to use them, we must first understand what actually happens when functions are defined and called.

The Call Stack

When JavaScript parses the expression let name = "text", it allocates a block of memory big enough for four characters and stores a reference to that block of characters in the variable name. We can show this by drawing a memory diagram like the one in Figure 3.1.

When we write:

oneMore = (x) => {
  return x + 1
}

JavaScript allocates a block of memory big enough to store several instructions, translates the text of the function into instructions, and stores a reference to those instructions in the variable oneMore (Figure 3.2).

The only difference between these two cases is what’s on the other end of the reference: four characters or a bunch of instructions that add one to a number. This means that we can assign the function to another variable, just as we would assign a number:

// assign the function oneMore to the variable anotherName 
const anotherName = oneMore

// instead of calling the function oneMore we can call the function anotherName
console.log(anotherName(5))

6

Doing this does not call the function: as Figure 3.3 shows, it creates a second name, or alias, that refers to the same block of instructions.

As explained in Chapter 2, when JavaScript calls a function it assigns the arguments in the call to the function’s parameters. In order for this to be safe, we need to ensure that there are no name collisions, i.e., that if there is a variable called something and one of the function’s parameters is also called something, the function will use the right one. The way every modern language implements this is to use a call stack. Instead of putting all our variables in one big table, we have one table for global variables and one extra table for each function call. This means that if we assign 100 to x, call oneMore(2 * x + 1), and look at memory in the middle of that call, we will see what’s in Figure 3.4.

Functions of Functions

The call stack allows us to write and call functions without worrying about whether we’re accidentally going to refer to the wrong variable. And since functions are just another kind of data, we can pass one function into another. For example, we can write a function called doTwice that calls some other function two times:

const doTwice = (action) => {
  action()
  action()
}

const hello = () => {
  console.log('hello')
}

doTwice(hello)

hello
hello

Again, this is clearer when we look at the state of memory while doTwice is running (Figure 3.5).

This becomes more useful when the function or functions passed in have parameters of their own. For example, the function pipeline passes a value to one function, then takes that function’s result and passes it to a second, and returns the final result:

const pipeline = (initial, first, second) => {
  return second(first(initial))
}

Let’s use this to combine a function that trims blanks off the starts and ends of strings and another function that uses a regular expression (Appendix 24) to replace spaces with dots:

const trim = (text) => { return text.trim() }
const dot = (text) => { return text.replace(/ /g, '.') }

const original = '  this example uses text  '

const trimThenDot = pipeline(original, trim, dot)
console.log(`trim then dot: |${trimThenDot}|`)

trim then dot: |this.example.uses.text|

During the call to temp = first(initial), but before a value has been returned to be assigned to temp, memory looks like Figure 3.6.

Reversing the order of the functions changes the result:

const dotThenTrim = pipeline(original, dot, trim)
console.log(`dot then trim: |${dotThenTrim}|`)

dot then trim: |..this.example.uses.text..|

We can make a more general pipeline by passing an array of functions:

const pipeline = (initial, operations) => {
  let current = initial
  for (let op of operations) {
    current = op(current)
  }
  return current
}

Going through this line by line:

• The function pipeline takes an initial input value and array of functions. Each of those functions must take one value as an input and produce one value as output.

• We initialize a variable called current to hold the current value. We have to use let for this rather than const because we want to update it after each step of the pipeline runs.

• We then call each of the functions in the pipeline in turn, passing in the current value and storing the result to be passed into the next function.

• The final result is whatever came out of the last step in the pipeline.

We don’t actually have to create a separate variable for the current value; since parameters are always variables rather than constants, we could just overwrite the parameter initial over and over again like this:

const pipeline = (current, operations) => {
  for (let op of operations) {
    current = op(current)
  }
  return current
}

Let’s add a function double to our suite of text manglers:

const double = (text) => { return text + text }

and then try it out:

const original = ' some text '
const final = pipeline(original, [double, trim, dot])
console.log(`|${original}| -> |${final}|`)

| some text | -> |some.text..some.text|

The order of operations is:

1. current is assigned ’ some text ’ (with spaces around each word).

2. It is then assigned double(’ some text ’), or ’ some text  some text ’.

3. That value is then passed to trim, so ’some text  some text’ (without leading or trailing spaces) is assigned to current.

4. Finally, the current value is passed to dot and the result some.text..some.text. is returned.

Anonymous Functions

Remember the function oneMore? We can pass it a value that we have calculated on the fly:

oneMore = (x) => {
  return x + 1
}

console.log(oneMore(3 * 2))

7

Behind the scenes, JavaScript allocates a nameless temporary variable to hold the value of 3 * 2, then passes a reference to that temporary variable into oneMore. We can do the same thing with functions, i.e., create one on the fly without giving it a name as we’re passing it into some other function. For example, suppose that instead of pushing one value through a pipeline of functions, we want to call a function once for each value in an array:

const transform = (values, operation) => {
  let result = []
  for (let v of values) {
    result.push(operation(v))
  }
  return result
}

const data = ['one', 'two', 'three']
const upper = transform(data, (x) => { return x.toUpperCase() })
console.log(`upper: ${upper}`)

upper: ONE,TWO,THREE

Taking the first letter of a word is so simple that it’s hardly worth giving the function a name, so let’s move the definition into the call to transform:

const first = transform(data, (x) => { return x[0] })
console.log(`first: ${first}`)

first: o,t,t

A function that is defined where it is used and isn’t assigned a name is called an anonymous function. Most callback functions in JavaScript are written this way: the function with the given parameters and body (in this case, x and return x[0]) is passed directly to something else (in this case, transform) to be called.

Functional Programming

Functional programming is a style of programming that relies heavily on higher-order functions like pipeline that take other functions as parameters. In addition, functional programming expects that functions won’t modify data in place, but will instead create new data from old. For example, a true believer in functional programming would be saddened by this:

// Create test array
const test = [1,2,3]

const impure = (values) => {
  for (let i in values) {
    values[i] += 1
  }
}

// Run function
impure(test)

// Original array has been modified
console.log(`test: ${test}`)

test: 2,3,4

and would politely but firmly suggest that it be rewritten like this:

const test = [1,2,3]

const pure = (values) => {
  result = []
  for (let v of values) {
    result.push(v + 1)
  }
  return result
}

// Rather than modify test, we create a new array for the results
const newArray = pure(test)
console.log(`newArray: ${newArray}$`)
console.log(`test: ${test}`)

newArray: 2,3,4
test: 1,2,3

JavaScript arrays provide several methods to support functional programming. For example, Array.some returns true if any element in an array passes a test, while Array.every returns true if all elements in an array pass a test. Here’s how they work:

const data = ['this', 'is', 'a', 'test']
console.log('some longer than 3:',
            data.some((x) => { return x.length > 3 }))
console.log('all longer than 3:',
            data.every((x) => { return x.length > 3 }))

some longer than 3: true
all longer than 3: false

Array.filter creates a new array containing only values that pass a test:

const data = ['this', 'is', 'a', 'test']
console.log('those longer than 3:',
            data.filter((x) => { return x.length > 3 }))

those longer than 3: [ 'this', 'test' ]

So do all of the elements with more than 3 characters start with a ‘t’?

const data = ['this', 'is', 'a', 'test']
const result = data
               .filter((x) => { return x.length > 3 })
               .every((x) => { return x[0] === 't' })
console.log(`all longer than 3 start with t: ${result}`)

all longer than 3 start with t: true

Array.map creates a new array by calling a function for each element of an existing array:

const data = ['this', 'is', 'a', 'test']
console.log('shortened', data.map((x) => { return x.slice(0, 2) }))

shortened [ 'th', 'is', 'a', 'te' ]

Finally, Array.reduce reduces an array to a single value using a combining function and a starting value. The combining function must take two values, which are the current running total and the next value from the array; if the array is empty, Array.reduce returns the starting value.

An example will make this clearer. To start, let’s create an acronym using a loop:

const data = ['this', 'is', 'a', 'test']

let acronym = ''
for (let value of data) {
  acronym = acronym + value[0]
}

console.log(`acronym of ${data} is ${acronym}`)

acronym of this,is,a,test is tiat

The three key elements in the short program above are the input data, the initial value of the variable acronym, and the way that variable is updated. When Array.reduce is used, the array is the data and the initial value and update function are passed as parameters:

const data = ['this', 'is', 'a', 'test']

const concatFirst = (accumulator, nextValue) => {
  return accumulator + nextValue[0]
}
let acronym = data.reduce(concatFirst, '')
console.log(`acronym of ${data} is ${acronym}`)

acronym of this,is,a,test is tiat

As elsewhere, we can define the function where we use it:

const data = ['this', 'is', 'a', 'test']

acronym = data.reduce((accum, next) => {
  return accum + next[0]
}, '')
console.log('all in one step:', acronym)

all in one step: tiat

The indentation of the anonymous function defined inside reduce may look a little odd, but this is the style the JavaScript community has settled on.

Closures

The last tool we need to introduce is an extremely useful side-effect of the way memory is handled. The easiest way to explain it is by example. We have already defined a function called pipeline that chains any number of other functions together:

const pipeline = (initial, operations) => {
  let current = initial
  for (let op of operations) {
    current = op(current)
  }
  return current
}

However, pipeline only works if each function in the array operations has a single parameter. If we want to be able to add 1, add 2, and so on, we have to write separate functions, which is annoying.

A better option is to write a function that creates the function we want:

const adder = (increment) => {
  const f = (value) => {
    return value + increment
  }
  return f
}

const add_1 = adder(1)
const add_2 = adder(2)
console.log(`add_1(100) is ${add_1(100)}, add_2(100) is ${add_2(100)}`)

add_1(100) is 101, add_2(100) is 102

The best way to understand what’s going on is to draw a step-by-step memory diagram. In step 1, we call adder(1) (Figure 3.7). adder creates a new function that includes a reference to that 1 we just passed in (Figure 3.8). In step 3, adder returns that function, which is assigned to add_1 (Figure 3.9). Crucially, the function that add_1 refers to still has a reference to the value 1, even though that value isn’t referred to any longer by anyone else.

In steps 4–6, we repeat these three steps to create another function that has a reference to the value 2, and assign that function to add_2 (Figure 3.10).

When we now call add_1 or add_2, they add the value passed in and the value they’ve kept a reference to.

This trick of capturing a reference to a value inside something else is called a closure. It works because JavaScript holds on to values as long as anything, anywhere, still refers to them. Closures solve our pipeline problem by letting us define little functions on the fly and give them extra data to work with:

const result = pipeline(100, [adder(1), adder(2)])
console.log(`adding 1 and 2 to 100 -> ${result}`)

adding 1 and 2 to 100 -> 103

Again, adder(1) and adder(2) do not add anything to anything: they define new (unnamed) functions that add 1 and 2 respectively when called.

Programmers often go one step further and define little functions like this inline:

const result = pipeline(100, [(x) => x + 1, (x) => x + 2])
console.log(`adding 1 and 2 to 100 -> ${result}`)

adding 1 and 2 to 100 -> 103

As this example shows, if the body of a function is a single expression, it doesn’t have to be enclosed in {...} and return doesn’t need to be used.

Exercises

Side Effects With forEach

JavaScript arrays have a method called forEach, which calls a callback function once for each element of the array. Unlike map, forEach does not save the values returned by these calls or return an array of results. The full syntax is:

someArray.forEach((value, location, container) => {
  // 'value' is the value in 'someArray'
  // 'location' is the index of that value
  // 'container' is the containing array (in this case, 'someArray')
})

If you only need the value, you can provide a callback that only takes one parameter; if you only need the value and its location, you can provide a callback that takes two. Use this to write a function doubleInPlace that doubles all the values in an array in place:

const vals = [1, 2, 3]
doubleInPlace(vals)
console.log(`vals after change: ${vals}`)

vals after change: 2,4,6

Annotating Data

Given an array of objects representing observations of wild animals:

data = [
  {'date': '1977-7-16', 'sex': 'M', 'species': 'NL'},
  {'date': '1977-7-16', 'sex': 'M', 'species': 'NL'},
  {'date': '1977-7-16', 'sex': 'F', 'species': 'DM'},
  {'date': '1977-7-16', 'sex': 'M', 'species': 'DM'},
  {'date': '1977-7-16', 'sex': 'M', 'species': 'DM'},
  {'date': '1977-7-16', 'sex': 'M', 'species': 'PF'},
  {'date': '1977-7-16', 'sex': 'F', 'species': 'PE'},
  {'date': '1977-7-16', 'sex': 'M', 'species': 'DM'}
]

write a function that returns a new array of objects like this:

newData = [
  {'seq': 3, 'year': '1977', 'sex': 'F', 'species': 'DM'},
  {'seq': 7, 'year': '1977', 'sex': 'F', 'species': 'PE'}
]

without using any loops. The changes are:

• The date field is replaced with just the year.

• Only observations of female animals are retained.

• The retained records are given sequence numbers to relate them back to the original data. (These sequence numbers are 1-based rather than 0-based.)

You will probably want to use Array.reduce to generate the sequence numbers.

Key Points

• JavaScript stores the instructions making up a function in memory like any other object.

• Function objects can be assigned to variables, put in lists, passed as arguments to other functions, etc.

• Functions can be defined in place without ever being given a name.

• A callback function is one that is passed in to another function for it to execute at a particular moment.

• Functional programming uses higher-order functions on immutable data.

• Array.some is true if any element in an array passes a test, while Array.every is true if they all do.

• Array.filter selects elements of an array that pass a test.

• Array.map creates a new array by transforming values in an existing one.

• Array.reduce reduces an array to a single value.

• A closure is a set of variables captured during the definition of a function.

Objects and Classes

Making new code use old code is easy: just load the libraries you want and write calls to the functions you need. Making old code use new code without rewriting it is trickier, but object-oriented programming (OOP) can help.

Doing It By Hand

As we saw in Chapter 2, an object in JavaScript is a set of key-value pairs. Since functions are just another kind of data, an object’s values can be functions, so data can carry around functions that work on it. For example, we can create an object to represent a square:

const square = {
  name: 'square',
  size: 5,
  area: (it) => { return it.size * it.size },
  perimeter: (it) => { return 4 * it.size }
}

and then pass the object itself into each of its own functions:

const a = square.area(square)
console.log(`area of square is ${a}`)

area of square is 25

This is clumsy—we’ll often forget to pass the object into its functions—but it allows us to handle many different kinds of things in the same way. For example, we can create another object to represent a circle:

const circle = {
  name: 'circle',
  radius: 3,
  area: (it) => { return Math.PI * it.radius * it.radius },
  perimeter: (it) => { return 2 * Math.PI * it.radius }
}

and then put all of these different objects in an array and operate on them in the same way without knowing precisely what kind of object we’re dealing with:

const show_all = (shapes) => {
  for (let s of shapes) {
    const a = s.area(s)
    const p = s.perimeter(s)
    console.log(`${s.name}: area ${a} perimeter ${p}`)
  }
}

show_all([square, circle])

square: area 25 perimeter 20
circle: area 28.274333882308138 perimeter 18.84955592153876

As long as we only use the value name and the functions area and perimeter we don’t need to know what kind of shape we have. This is called polymorphism, and it allows us to add new shapes without changing the code in our loop. In other words, it allows old code (in this case, the function show_all) to use new code (the new object rectangle):

const rectangle = {
  name: 'rectangle',
  width: 2,
  height: 3,
  area: (it) => { return it.width * it.height },
  perimeter: (it) => { return 2 * (it.width + it.height) }
}

show_all([square, circle, rectangle])

square: area 25 perimeter 20
circle: area 28.274333882308138 perimeter 18.84955592153876
rectangle: area 6 perimeter 10

Classes

Building every object by hand and calling thing.function(thing) is clumsy. JavaScript solved these problems using prototypes, which turned out to be almost as clumsy as our hand-rolled solution (Section 23.3). Most object-oriented languages use classes instead (Figure 4.1); these were added to JavaScript in ES6, and we will use them instead of prototypes throughout. Here’s how we create a class that defines the properties of a square, without actually creating any specific squares:

class Square {
  constructor (size) {
    this.name = 'square'
    this.size = size
  }
  area () { return this.size * this.size }
  perimeter () { return 4 * this.size }
}

(Class names are written in CamelCase by convention.) We can then create a specific square by using the class’s name as if it were a function:

const sq = new Square(3)
console.log(`sq name ${sq.name} and area ${sq.area()}`)

sq name square and area 9

new ClassName(...) creates a new blank object and marks it to show that it is an instance of ClassName. Objects need to know what class they belong to so that they can find their methods, which are the functions defined within the class that operate on instances of it. (We’ll explain how JavaScript marks objects in Section 23.3; it’s more complicated than it needs to be in order to be compatible with older versions of the language.)

Before new hands the newly-created object back to the program, it calls the specially-named constructor to initialize the object’s state. Inside the constructor and other methods, the object being operated on is referred to by the pronoun this. For example, if the constructor assigns an array to this.values, then every instance of that class gets its own array.

Everything Old is New Again

Methods are defined within a class using classic function syntax rather than the fat arrows we have been using. The inconsistency is unfortunate but this way of defining methods is what the current version of Node prefers; we will explore this topic further in Chapter 8.

There are a lot of new terms packed into the preceding three paragraphs, so let’s retrace the execution of:

const sq = new Square(3)
console.log(`sq name ${sq.name} and area ${sq.area()}`)

1. new creates a blank object, then looks up the definition of the class Square and calls the constructor defined inside it with the value 3.

2. Inside that call, this refers to the newly-created blank object. The constructor adds two keys to the object: name, which has the value ’square’, and size, which has the value 3.

3. Once the constructor finishes, the newly-created object is assigned to the variable sq.

4. In order to create a string to pass to console.log, the program has to look up the value of sq.name, which works like looking up any other key in an object.

5. The program also has to call sq.area. During that call, JavaScript temporarily assigns sq to the variable this so that the area method can look up the value of size in the object it’s being called for.

6. Finally, the program fills in the template string and calls console.log.

JavaScript classes support polymorphism: if two or more classes have some methods with the same names that take the same parameters and return the same kinds of values, other code can use objects of those classes interchangeably. For example, here’s a class-based rewrite of our shapes code:

class Circle {
  constructor (radius) {
    this.name = 'circle'
    this.radius = radius
  }
  area () { return Math.PI * this.radius * this.radius }
  perimeter () { return 2 * Math.PI * this.radius }
}

class Rectangle {
  constructor (width, height) {
    this.name = 'rectangle'
    this.width = width
    this.height = height
  }
  area () { return this.width * this.height }
  perimeter () { return 2 * (this.width + this.height) }
}

const everything = [
  new Square(3.5),
  new Circle(2.5),
  new Rectangle(1.5, 0.5)
]
for (let thing of everything) {
  const a = thing.area(thing)
  const p = thing.perimeter(thing)
  console.log(`${thing.name}: area ${a} perimeter ${p}`)
}

square: area 12.25 perimeter 14
circle: area 19.634954084936208 perimeter 15.707963267948966
rectangle: area 0.75 perimeter 4

Inheritance

We can build new classes from old ones by adding or overriding methods. To show this, we’ll start by defining a person:

class Person {
  constructor (name) {
    this.name = name
  }

  greeting (formal) {
    if (formal) {
      return `Hello, my name is ${this.name}`
    } else {
      return `Hi, I'm ${this.name}`
    }
  }

  farewell () {
    return `Goodbye`
  }
}

This class allows us to create an instance with a formal greeting:

const parent = new Person('Hakim')
console.log(`parent: ${parent.greeting(true)} - ${parent.farewell()}`)

parent: Hello, my name is Hakim - Goodbye

We can now extend Person to create a new class Scientist, in which case we say that Scientist inherits from Person, or that Person is a parent class of Scientist and Scientist is a child class of Person.

class Scientist extends Person {
  constructor (name, area) {
    super(name)
    this.area = area
  }

  greeting (formal) {
    return `${super.greeting(formal)}. Let's talk about ${this.area}...`
  }
}

This tells us that a Scientist is a Person who:

• Has an area of specialization as well as a name.

• Says hello in a slightly longer way

• Says goodbye in the same way as a Person (since Scientist doesn’t define its own farewell method)

The word super is used in two ways here:

• In the constructor for Scientist, super(...) calls up to the constructor of the parent class Person so that it can do whatever initialization it does before Scientist does its own initialization. This saves us from duplicating steps.

• Inside greeting, the expression super.greeting(formal) means “call the parent class’s greeting method for this object”. This allows methods defined in child classes to add to or modify the behavior of methods defined in parent classes, again without duplicating code.

Let’s try it out:

const child = new Scientist('Bhadra', 'microbiology')
console.log(`child: ${child.greeting(false)} - ${child.farewell()}`)

child: Hi, I'm Bhadra. Let's talk about microbiology... - Goodbye

Figure 4.2 shows what memory looks like after these classes have been defined and the objects parent and child have been created. It looks complex at first, but allows us to see how JavaScript finds the right method when child.farewell() is called:

• It looks in the object child to see if there’s a function there with the right name.

• There isn’t, so it follows child’s link to its class Scientist to see if a function is there.

• There isn’t, so it follows the link from Scientist to the parent class Person and finds the function it’s looking for.

Exercises

Delays

Define a class called Delay whose call method always returns the value given in the previous call:

const example = new Delay('a')
for (let value of ['b', 'c', 'd']) {
  console.log(value, '->', example.call(value))
}

b -> a
c -> b
d -> c

A class like Delay is sometimes called stateful, since it remembers its state from call to call.

Filtering

Define a class called Filter whose call method returns null if its input matches one of the values given to its constructor, or the input as output otherwise:

const example = new Filter('a', 'e', 'i', 'o', 'u')
for (let value of ['a', 'b', 'c', 'd', 'e']) {
  console.log(value, '->', example.call(value))
}

a -> null
b -> b
c -> c
d -> d
e -> null

A class like Filter is sometimes called stateless, since it does not remember its state from call to call.

Pipelines

Define a class called Pipeline whose constructor takes one or more objects with a single-parameter call method, and whose own call method passes a value through each of them in turn. If any of the components’ call methods returns null, Pipeline stops immediately and returns null.

const example = new Pipeline(new Filter('a', 'e', 'i', 'o', 'u'),
                             new Delay('a'))
for (let value of ['a' ,'b', 'c', 'd', 'e']) {
  console.log(value, '->', example.call(value))
}

a -> null
b -> a
c -> b
d -> c
e -> null

Active Expressions

Consider this class:

class Active {
  constructor (name, transform) {
    this.name = name
    this.transform = transform
    this.subscribers = []
  }

  subscribe (someone) {
    this.subscribers.push(someone)
  }

  update (input) {
    console.log(this.name, 'got', input)
    const output = this.transform(input)
    for (let s of this.subscribers) {
      s.update(output)
    }
  }
}

and this program that uses it:

const start = new Active('start', (x) => Math.min(x, 10))
const left = new Active('left', (x) => 2 * x)
const right = new Active('right', (x) => x + 1)
const final = new Active('final', (x) => x)
start.subscribe(left)
start.subscribe(right)
left.subscribe(final)
right.subscribe(final)

start.update(123)

1. Trace what happens when the last line of the program is called.

2. Modify Active so that it calls transform if that function was provided, or a method Active.transform if a transformation function wasn’t provided.

3. Create a new class Delay whose transform method always returns the previous value. (Its constructor will need to take an initial value as a parameter.)

This pattern is called observer/observable.

Key Points

• Create classes to define combinations of data and behavior.

• Use the class’s constructor to initialize objects.

• this refers to the current object.

• Use polymorphism to express common behavior patterns.

• Extend existing classes to create new ones-sometimes.

• Override methods to change or extend their behavior.

HTML and CSS

HTML is the standard way to represent documents for presentation in web browsers, and CSS is the standard way to describe how it should look. Both are more complicated than they should have been, but in order to create web applications, we need to understand a little of both.

Formatting

An HTML document contains elements and text (and possibly other things that we will ignore for now). Elements are shown using tags: an opening tag <tagname> shows where the element begins, and a corresponding closing tag </tagname> (with a leading slash) shows where it ends. If there’s nothing between the two, we can write <tagname/> (with a trailing slash).

A document’s elements must form a tree (Figure 5.1), i.e., they must be strictly nested. This means that if Y starts inside X, Y must end before X ends, so <X>…<Y>…</Y></X> is legal, but <X>…<Y>…</X></Y> is not. Finally, every document should have a single root element that encloses everything else, although browsers aren’t strict about enforcing this. In fact, most browsers are pretty relaxed about enforcing any kind of rules at all, since most people don’t obey them anyway.

Text

The text in an HTML page is normal printable text. However, since < and > are used to show where tags start and end, we must use escape sequences to represent them, just as we use \" to represented a literal double-quote character inside a double-quoted string in JavaScript. In HTML, escape sequences are written &name;, i.e., an ampersand, the name of the character, and a semi-colon. A few common escape sequences are shown in Table 5.1.

The first two are self-explanatory, and & is needed so that we can write a literal ampersand (just as \\ is needed in JavaScript strings so that we can write a literal backslash). ©, ±, and µ are usually not needed any longer, since most editors will allow us to put non-ASCII characters directly into documents these days, but occasionally we will run into older or stricter systems.

Pages

An HTML page should have:

• a single html element that encloses everything else

• a single head element that contains information about the page

• a single body element that contains the content to be displayed.

It doesn’t matter whether or how we indent the tags showing these elements and the content they contain, but laying them out on separate lines and indenting to show nesting helps human readers. Well-written pages also use comments, just like code: these start with <!-- and end with -->. Unfortunately, comments cannot be nested, i.e., if you comment out a section of a page that already contains a comment, the results are unpredictable.

Here’s an empty HTML page with the structure described above:

<html>
  <head>
    <!-- description of page goes here -->
  </head>
  <body>
    <!-- content of page goes here -->
  </body>
</html>

Nothing shows up if we open this in a browser, so let’s add a little content:

<html>
  <head>
    <title>This text is displayed in the browser bar</title>
  </head>
  <body>
    <h1>Displayed Content Starts Here</h1>
    <p>
      This course introduces core features of <em>JavaScript</em>
      and shows where and how to use them.
    </p>
    <!-- The word "JavaScript" is in italics (emphasis) in the preceding paragraph. -->
  </body>
</html>

• The title element inside head gives the page a title. This is displayed in the browser bar when the page is open, but is not displayed as part of the page itself.

• The h1 element is a level-1 heading; we can use h2, h3, and so on to create sub-headings.

• The p element is a paragraph.

• Inside a heading or a paragraph, we can use em to emphasize text. We can also use strong to make text stronger. Tags like these are better than tags like i (for italics) or b (for bold) because they signal intention rather than forcing a particular layout. Someone who is visually impaired, or someone using a small-screen device, may want emphasis of various kinds displayed in different ways.

Attributes

Elements can be customized by giving them attributes, which are written as name="value" pairs inside the element’s opening tag. For example:

<h1 align="center">A Centered Heading</h1>

centers the h1 heading on the page, while:

<p class="disclaimer">This planet provided as-is.</p>

marks this paragraph as a disclaimer. That doesn’t mean anything special to HTML, but as we’ll see later, we can define styles based on the class attributes of elements.

An attribute’s name may appear at most once in any element, just like a key can only appear once in any JavaScript object, so <p align="left" align="right">…</p> is illegal. If we want to give an attribute multiple values—for example, if we want an element to have several classes—we put all the values in one string. Unfortunately, as the example below shows, HTML is inconsistent about whether values should be separated by spaces or semi-colons:

<p class="disclaimer optional" style="color: blue; font-size: 200%;">

However they are separated, values are supposed to be quoted, but in practice we can often get away with name=value. And for Boolean attributes whose values are just true or false, we can even sometimes just get away with name on its own.

Lists

Headings and paragraphs are all very well, but data scientists need more. To create an unordered (bulleted) list, we use a ul element, and wrap each item inside the list in li. To create an ordered (numbered) list, we use ol instead of ul, but still use li for the list items.

<ul>
  <li>first</li>
  <li>second</li>
  <li>third</li>
</ul>

• first

• second

• third

<ol>
  <li>first</li>
  <li>second</li>
  <li>third</li>
</ol>

1. first

2. second

3. third

Lists can be nested by putting the inner list’s ul or ol inside one of the outer list’s li elements:

<ol>
  <li>Major A
    <ol>
      <li>minor p</li>
      <li>minor q</li>
    </ol>
  </li>
  <li>Major B
    <ol>
      <li>minor r</li>
      <li>minor s</li>
    </ol>
  </li>
</ol>

1. Major A

   1. minor p

   2. minor q

2. Major B

   1. minor r

   2. minor s

Tables

Lists are a great way to get started, but if we really want to impress people with our data science skills, we need tables. Unsurprisingly, we use the table element to create these. Each row is a tr (for “table row”), and within rows, column items are shown with td (for “table data”) or th (for “table heading”).

<table>
  <tr> <th>Alkali</th>   <th>Noble Gas</th> </tr>
  <tr> <td>Hydrogen</td> <td>Helium</td>    </tr>
  <tr> <td>Lithium</td>  <td>Neon</td>      </tr>
  <tr> <td>Sodium</td>   <td>Argon</td>     </tr>
</table>

Do not use tables to create multi-column layouts: there’s a better way.

Links

Links to other pages are what make HTML hypertext (Figure 5.2). Confusingly, the element used to show a link is called a. The text inside the element is displayed and (usually) highlighted for clicking. Its href attribute specifies what the link is pointing at; both local filenames and URLs are supported. Oh, and we can use <br/> to force a line break in text (with a trailing slash inside the tag, since the br element doesn’t contain any content):

<a href="https://nodejs.org/">Node.js</a>
<br/>
<a href="https://facebook.github.io/react/">React</a>
<br/>
<a href="../index.html">home page (relative path)</a>

This appears as:

Node.js
React
home page (relative path)

with the usual clickability.

Images

Images can be stored inside HTML pages in two ways: by using SVG (which we will discuss in Chapter 8) or by encoding the image as text and including that text in the body of the page, which is clever, but makes the source of the pages very hard to read.

It is far more common to store each image in a separate file and refer to that file using an img element (which also allows us to use the image in many places without copying it). The src attribute of the img tag specifies where to find the file; as with the href attribute of an a element, this can be either a URL or a local path. Every img should also include a title attribute (whose purpose is self-explanatory) and an alt attribute with some descriptive text to aid accessibility and search engines. (Again, we have wrapped and broken lines so that they will display nicely in the printed version.)

    <img src="./assets/logo.png" title="Book Logo"
         alt="Displays the book logo using a local path" />
    <img src="https://js4ds.org/assets/logo.png"
         title="Book Logo"
         alt="Display the book logo using a URL" />

Two things to note here are:

1. Since img elements don’t contain any text, they are often written with the trailing-slash notation. However, they are also often written improperly as <img src="..."> without any slashes at all. Browsers will understand this, but some software packages will complain.

2. If an image file is referred to using a path rather than a URL, that path can be either relative or absolute. If it’s a relative path, it’s interpreted starting from where the web page is located; if it’s an absolute path, it’s interpreted relative to wherever the web browser thinks the root directory of the filesystem is. As we will see in Chapter 12, this can change from one installation to the next, so you should always try to use relative paths, except where you can’t. It’s all very confusing…

Cascading Style Sheets

When HTML first appeared, people styled elements by setting their attributes:

<html>
  <body>
    <h1 align="center">Heading is Centered</h1>
    <p>
      <b>Text</b> can be highlighted
      or <font color="coral">colorized</font>.
    </p>
  </body>
</html>

Many still do, but a better way is to use Cascading Style Sheets (CSS). These allow us to define a style once and use it many times, which makes it much easier to maintain consistency. (We were going to say “…and keep pages readable”, but given how complex CSS can be, that’s not a claim we feel we can make.) Here’s a page that uses CSS instead of direct styling:

<html>
  <head>
    <link rel="stylesheet" href="simple-style.css" />
  </head>
  <body>
    <h1 class="title">Heading is Centered</h1>
    <p>
      <span class="keyword">Text</span> can be highlighted
      or <span class="highlight">colorized</span>.
    </p>
  </body>
</html>

The head contains a link to an external style sheet stored in the same directory as the page itself; we could use a URL here instead of a relative path, but the link element must have the rel="stylesheet" attribute. Inside the page, we then set the class attribute of each element we want to style.

The file simple-style.css looks like this:

h1.title {
  text-align: center;
}
span.keyword {
  font-weight: bold;
}
.highlight {
  color: coral;
}

Each entry has the form tag.class followed by a group of properties inside curly braces, and each property is a key-value pair. We can omit the class and just write (for example):

p {
  font-style: italic;
}

in which case the style applies to everything with that tag. If we do this, we can override general rules with specific ones: the style for a disclaimer paragraph is defined by p with overrides defined by p.disclaimer. We can also omit the tag and simply use .class, in which case every element with that class has that style.

As suggested by the earlier discussion of separators, elements may have multiple values for class, as in <span class="keyword highlight">…</span>. (The span element simply marks a region of text, but has no effect unless it’s styled.)

These features are one (but unfortunately not the only) common source of confusion with CSS: if one may override general rules with specific ones but also provide multiple values for class, how do we keep track of which rules will apply to an element with multiple classes? A detailed discussion of the order of precedence for CSS rules is outside the scope of this tutorial. We recommend that those likely to work often with stylesheets read (and consider bookmarking) this W3Schools page.

One other thing CSS can do is match specific elements. We can label particular elements uniquely within a page using the id attribute, then refer to those elements using #name as a selector. For example, if we create a page that gives two spans unique IDs:

<html>
  <head>
    <link rel="stylesheet" href="selector-style.css" />
  </head>
  <body>
    <p>
      First <span id="major">keyword</span>.
    </p>
    <p>
      Full <span id="minor">explanation</span>.
    </p>
  </body>
</html>

then we can style those spans like this:

#major {
  text-decoration: underline red;
}
#minor {
  text-decoration: overline blue;
}

Internal Links

We can link to an element in a page using #name inside the link’s href: for example, <a href="page.html#place">text</a> refers to the #place element in page.html. This is particularly useful within pages: <a href="#place">jump</a> takes us straight to the #place element within this page. Internal links like this are often used for cross-referencing and to create a table of contents.

Bootstrap

CSS can become very complicated very quickly, so most people use a framework to take care of the details. One of the most popular is Bootstrap (which is what we’re using to style this website). Here’s the entire source of a page that uses Bootstrap to create a two-column layout with a banner at the top (Figure [f:htmlcss-bootstrap]):

<html>
  <head>
    <link rel="stylesheet"
          href="https://stackpath.bootstrapcdn.com/bootstrap/\
                4.1.3/css/bootstrap.min.css">
    <style>
      div {
        border: solid 1px;
      }
    </style>
  </head>
  <body>

    <div class="jumbotron text-center">
      <h1>Page Title</h1>
      <p>Resize this page to see the layout adjust dynamically.</p>
    </div>

    <div class="container">
      <div class="row">
        <div class="col-sm-4">
          <h2>First column is 4 wide</h2>
          <p>Text here goes</p>
          <p>in the column</p>
        </div>
        <div class="col-sm-8">
          <h2>Second column is 8 wide</h2>
          <p>Text over here goes</p>
          <p>in the other column</p>
        </div>
      </div>
    </div>

  </body>
</html>

The page opens by loading Bootstrap from the web; we can also download bootstrap.min.css and refer to it with a local path. (The .min in the file’s name signals that the file has been minimized so that it will load more quickly.)

The page then uses a style element to create an internal style sheet to put a solid one-pixel border around every div so that we can see the regions of the page more clearly. Defining styles in the page header is generally a bad idea, but it’s a good way to test things quickly. Oh, and a div just marks a region of a page without doing anything to it, just as a span marks a region of text without changing its appearance unless we apply a style.

The first div creates a header box (called a “jumbotron”) and centers its text. The second div is a container, which creates a bit of margin on the left and right sides of our content. Inside that container is a row with two columns, one 4/12 as wide as the row and the other 8/12 as wide. (Bootstrap uses a 12-column system because 12 has lots of divisors.)

Bootstrap is responsive: elements change size as the page grows narrower, and are then stacked when the screen becomes too small to display them side by side.

We’ve left out many other aspects of HTML and CSS as well, such as figure captions, multi-column table cells, and why it’s so hard to center text vertically within a div. One thing we will return to in Chapter 10 is how to include interactive elements like buttons and forms in a page. Handling those is part of why JavaScript was invented in the first place, but we need more experience before tackling them.

Exercises

Cutting Corners

What does your browser display if you forget to close a paragraph or list item tag like this:

<p>This paragraph starts but doesn't officially end.

<p>Another paragraph starts here but also doesn't end.

<ul>
  <li>First item in the list isn't closed.
  <li>Neither is the second.
</ul>

1. What happens if you don’t close a ul or ol list?

2. Is that behavior consistent with what happens when you omit </p> or </li>?

Mix and Match

1. Create a page that contains a 2x2 table, each cell of which has a three-item bullet-point list. How can you reduce the indentation of the list items within their cells using CSS?

2. Open your page in a different browser (e.g., Firefox or Edge). Do they display your indented lists consistently?

3. Why do programs behave inconsistently? Why do programmers do this to us? Why? Why why why why why?

Naming

What does the sm in Bootstrap’s col-sm-4 and col-sm-8 stand for? What other options could you use instead? Why do web developers still use FORTRAN-style names in the 21st Century?

Color

HTML and CSS define names for a small number of colors. All other colors must be specified using RGB values. Write a small JavaScript program that creates an HTML page that displays the word color in 100 different randomly-generated colors. Compare this to the color scheme used in your departmental website. Which one hurts your eyes less?

Units

What different units can you use to specify text size in CSS? What do they mean? What does anything mean, when you get right down to it?

Key Points

• HTML is the latest in a long line of markup languages.

• HTML documents contain elements (represented by tags in angle brackets) and text.

• Elements must be strictly nested.

• Elements can contain attributes.

• Use escape sequences beginning with ampersand to represent special characters.

• Every page should have one html element containing a head and a body.

• Use <!--...--> to include comments in HTML.

• Use ul and ol for unordered and ordered lists, and li for list elements.

• Use table for tables, tr for rows, th for headings, and td for regular data.

• Use <a href="...">...</a> to create links.

• Use <img src="..." title="..." alt="..." /> to include images.

• Use CSS to define appearance of elements.

• Use class and id to identify elements.

• Use selectors to specify the elements that CSS applies to.

Manipulating Pages

We have presented a lot of tools, but as yet no applications. As a reward for your patience, we will therefore work through several examples that show how to do useful things to web pages. These examples introduce some new concepts, the most important of which is the way in which HTML pages are represented in, and manipulated by, JavaScript.

One thing these examples don’t show is how to build interactive web pages. JavaScript was invented primarily to support buttons, forms, and the like, but we will need a bit more background before exploring them. Still, we can do a surprising number of useful things simply by playing with the content of pages.

Counting Paragraphs

Let’s begin by counting the number of paragraphs in a page:

<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <h1>Title</h1>
    <div id="fill"></div>
    <h2 id="one">First <em>emphasized</em></h2>
    <p>stuff</p>
    <h2 id="two">Second <code>with code</code></h2>
    <h3>stuff</h3>
    <h2 id="three">Third</h2>
    <p>stuff</p>

    <script>
      const counter = () => {
        const paragraphs = document.querySelectorAll('p')
        return paragraphs.length
      }
      console.log(`number of paragraphs: ${counter()}`)
    </script>
  </body>
</html>

This page has three main parts:

1. The head contains a meta tag that specifies the page’s character encoding, i.e., the scheme used to represent characters not found on a standard American keyboard in the 1970s. Character sets and character encodings are out of scope for this lesson; see this essay for an unfortunately timeless discussion.

2. The top half of the body has some headings and paragraphs for the JavaScript to play with. It also contains a div marked with class="fill" that our script will eventually fill in with a count.

3. The script itself is contained in a script tag at the bottom of the page; we will explore it in depth below.

When Scripts Run

We have put the script at the bottom of the page because we want to be sure that the page’s contents actually exist in memory before trying to process them. If we put the script tag and its contents at the top of the page, the browser might run our JavaScript after the page has been read but before its elements and text have been parsed and stored in memory. Race conditions like this bedevil web programming; we will see more robust ways to deal with them later.

Inside the script tag, we define a function called counter that takes no arguments, then use console.log to display the result of calling it. The only thing inside the function that we haven’t encountered before is the call document.querySelectorAll(’p’). As you might guess from its name, document is an object that gives us a handle on the page the script is in; it is created and provided automatically by the browser. Its querySelectorAll method finds all elements in the page that match the selector we provide. In this case, we’re looking for all paragraphs, so we simply search for ’p’.

To see the JavaScript in action, run a browser, open its developer tools so that you can see the JavaScript console, and then load the page. The page’s elements will be displayed as usual, and the console will show:

number of paragraphs: 2

Developer Tools

To open developer tools in Firefox, go to the main menu and select Tools > Web Developer > Toggle Tools. A tabbed display will open in the bottom of your page; choose Console to view the output of your JavaScript, or to write a little bit to run immediately. You can open a similar set of tools from View > Developer > JavaScript Console in Chrome, or by using Ctrl+Shift+I on Windows for Firefox, Chrome, or Microsoft Edge.

Showing results in the console is good enough for development work, but we would like to see the result in the page itself. To do this, we can replace the call to console.log with the two lines shown below:

const counter = () => {
  const paragraphs = document.querySelectorAll('p')
  return paragraphs.length
}
const fill = document.getElementById('fill')
fill.innerHTML = `number of paragraphs: ${counter()}`

Where document.querySelectorAll returns all nodes that match a selector, document.getElementById returns a single element that has the specified ID (which is set inside the element’s opening tag with id="some_name"). The variable fill is therefore assigned a reference to our div. We can then change the text inside that element by assigning to its innerHTML property. When we do this, JavaScript parses the string we provided as if it were HTML and creates whatever nodes it needs to represent the result. In this case, the content is just text, so JavaScript will create a single text node, store "number of paragraphs: 2" as its content, and add it to the in-memory structure that represents the page.

…at which point some magic happens behind the scenes. The browser stores the elements and text of the current page in a data structure called the Document Object Model, or more commonly the DOM. As shown in Figure 5.1, the DOM is organized as a tree: each element or piece of text is a node in the tree, and a node’s children are the elements contained within it. Any time the browser detects a change to the DOM, it automatically refreshes just as much of its display as it needs to. We can insert or remove text, change elements’ styles, or copy in entire sub-pages: each time, the browser will do only the work required to reflect that change as quickly as possible.

Creating a Table of Contents

Reporting the number of paragraphs is a good way to see how JavaScript works in the browser, but isn’t particularly useful (although counting the number of words is—we will tackle that in the exercises). Something we’re more likely to put in a real page is a table of contents, which takes only a little more code than what we’ve already seen:

(() => {
  const container = document.getElementById('fill')
  const headings = Array.from(document.querySelectorAll('h2'))
  const items = headings
        .map((h) => `<li><a href="#${h.id}">${h.innerHTML}</a></li>`)
        .join('')
  container.innerHTML = '<ul>' + items + '</ul>'
})()

Let’s start with the first and last lines, since they demonstrate a commonly-used idiom. We’ve seen how to define a function and then call it:

const f = (param) => {
  // body
}
f()

If we’re only going to call the function once, we might as well define it and call it immediately without giving it a name:

(param) => {
  // body
}(actual_value)

However, this doesn’t reliably work as written; in order to make JavaScript happy, we have to put parentheses around the function definition so that it’s clear exactly what’s being called:

((param) => {
  // body
})(actual_value)

() before the fat arrow means “this function doesn’t take any parameters”. The second () after the closing curly brace means “call the function”. If the function doesn’t take any arguments, this becomes:

(() => {
  // body
})()

which is a lot of parentheses in a row, but that’s what people write.

Let’s come back to the body of the function:

  const container = document.getElementById('fill')
  const headings = Array.from(document.querySelectorAll('h2'))
  const items = headings
        .map((h) => `<li><a href="#${h.id}">${h.innerHTML}</a></li>`)
        .join('')
  container.innerHTML = '<ul>' + items + '</ul>'

As before, the first line gets the div we’re going to fill in. The second line grabs all the h2 headings, which we have arbitrarily decided are the only things worthy of inclusion in the table of contents. We run the result of document.querySelectorAll through the function Array.from because the former’s result isn’t a JavaScript array: for reasons that probably made sense to someone, somewhere, it’s a thing called a NodeList that lacks most of Array’s useful methods.

Static Methods

Array.from is a static method: it belongs to the class as a whole, not to objects of that class. Static methods are primarily used to associate utility functions with classes, like dist in the example below. Unlike calls to instance methods like magnitude, static methods are called using class.method() rather than some_object.method().

We then have three lines that do most of the function’s work. The first tells us that items is going to be assigned something derived from headings; the second transforms the array of headings into an array of strings, and the third joins those strings to create a single string. Looking at the map call, each heading becomes a list item (li) containing a link (a) whose href attribute is the ID of the heading and whose displayed content (the text between <a...> and </a>) is the text of the heading. The href attribute’s value starts with #, which makes this a local link (i.e., it links to something inside the same page). If one of our h2 headings doesn’t have an id set, this map will fail; we’ll explore ways to handle this in the exercises.

Finally, the last line of the code shown above fills in the content of the container (i.e., the div) with an unordered list (ul) that contains all of the items we just constructed. Again, when we assign to an element’s innerHTML property, JavaScript parses the string we give it and constructs the HTML nodes we need. It would be marginally faster to build these nodes ourselves (which we will do in the exercises), but building and parsing strings is usually easier to read, and the performance differences are small enough in modern browsers that we should only worry about them if they actually prove themselves a problem.

Sortable Lists

Creating nodes allows us to add content to a page, but we can also rearrange the nodes that are there (Figure 6.1). Our next exercise is to sort the elements of a list, so that if the author writes:

<ul>
  <li>pee (P)</li>
  <li>cue (Q)</li>
  <li>are (R)</li>
</ul>

we will automatically rearrange the items to be:

<ul>
  <li>are (R)</li>
  <li>cue (Q)</li>
  <li>pee (P)</li>
</ul>

Our first attempt uses this as the HTML page:

<html>
  <head>
    <meta charset="utf-8">
    <script src="sort-lists.js"></script>
  </head>

  <body onload="sortLists()">

    <ul class="sorted">
      <li>first</li>
      <li>second</li>
      <li>third</li>
      <li>fourth</li>
      <li>fifth</li>
    </ul>

    <ol class="sorted">
      <li>one</li>
      <li>two</li>
      <li>three</li>
      <li>four</li>
      <li>five</li>
    </ol>

  </body>
</html>

and this as our initial script:

const sortLists = () => {
  const lists = Array.from(document.querySelectorAll('#sorted'))
  lists.forEach((list) => {
    const children = Array.from(list.childNodes)
          .filter(c => c.nodeName !== '#text')
    children.sort((left, right) =>
                  left.textContent.localeCompare(right.textContent))
    while (list.firstChild) {
      list.removeChild(list.firstChild)
    }
    children.forEach(c => list.appendChild(c))
  })
}

When we load the page, though, the items aren’t sorted. A bit of trial and error reveals that we have tripped over the race condition alluded to earlier: if we call our function in the onload attribute of the body tag, it is run when the page is loaded into memory but before the page’s content has been parsed and turned into a DOM tree. After searching online for “run JavaScript when page loaded”, we go back to this:

<html>
  <head>
    <meta charset="utf-8">
    <script src="sort-lists-event.js"></script>
  </head>

  <body>
    ...lists as before...
  </body>
</html>

and write our JavaScript like this:

const sortLists = () => {
  // ...function to sort lists...
}

document.addEventListener("DOMContentLoaded", (event) => {
  sortLists()
})

An event listener is a function that the browser calls when some kind of event occurs. In our example, the event we care about is “DOM content has been loaded”. When that occurs, the browser will call sortLists(). (The event parameter to our function will be given an object that stores details about what precisely happened. We don’t need that information now, but will use it later when we start handling button clicks and the like.)

Let’s return to the function:

const sortLists = () => {
  const lists = Array.from(document.querySelectorAll('.sorted'))
  lists.forEach((list) => {
    const children = Array.from(list.childNodes)
          .filter(c => c.nodeName !== '#text')
    children.sort((left, right) =>
                  left.textContent.localeCompare(right.textContent))
    while (list.firstChild) {
      list.removeChild(list.firstChild)
    }
    children.forEach(c => list.appendChild(c))
  })
}

As before, it starts by creating an array containing the nodes we want to operate on. (We use the selector .sorted (with a leading dot) to select everything with the class sorted, rather than #sorted, which would find nodes with the ID sorted.) This array will then have all the ul or ol lists that the function is to sort.

We process each list separately with lists.forEach. The callback function inside forEach uses Array.from to create an array containing the child nodes of the main list element, then filters that list to remove any that are text nodes. We need the Array.from call because (once again) the DOM doesn’t use a JavaScript array to store children, but a structure of its own devising that lacks the methods we want to call. We remove the text nodes because they represent the whitespace between list elements, such as newline characters after a closing </li> and before the next opening <li>.

Identifying Text Nodes

We could check c.nodeType instead of c.nodeName to spot text nodes, but we felt that nodeName made the code easier to understand. As always, we use !== for the comparison in order to prevent unpleasant surprises (Section 23.1).

Now that we have an array of the li elements to be sorted, we can use Array.sort to order them. Since we want to sort them by the text they contain, we have to provide our own sorting function that returns a negative number, 0, or a positive number to show whether its left argument is less than, equal to, or greater than its right argument. We use the textContent member of the node to get the text it contains, and the string object’s localeCompare to get a -1/0/1 result. All of this was discovered by searching online, primarily on the W3Schools site.

Unfortunately, searching for “remove all children from node” tells us that we have to do it ourselves, so we use a while loop to remove all the children (including the unwanted top-level text elements) from the ul or ol list, then add all of the children back in sorted order. Sure enough, the page now displays the nodes in the right order.

Bibliographic Citations

And so we come to the largest example in this lesson. HTML has a cite tag for formatting citations, but it doesn’t allow us to link directly to a bibliography entry. In order to minimize typing in scholarly papers, we’d like to find links like this:

<a href="#b">key1, key2</a>

and turn them into this:

[<a href="../bib/#key1">key1</a>, <a href="../bib/#key2">key2</a>]

The typed-in form is about as little typing as we can get away with; the displayed form then wraps the citations in [...] and turns each individual citation into a link to our bibliography. For now, we’ll assume that the bibliography can be found at ../bib/, i.e., in a file called index.html that’s in a directory called bib that’s a sibling of the directory containing whatever page the citation is in. This is very fragile, and we should be ashamed of ourselves, but we can tell ourselves that we’re going to fix it later and get on with learning JavaScript for now.

Our test page contains two bibliographic links and one non-bibliographic link:

<html>
  <head>
    <meta charset="utf-8">
    <script src="citations.js"></script>
  </head>
  <body>

    <p>As <a href="#b">Moreau1896</a> shows...</p>
    <p>
      We can look to <a href="#b">Brundle1982, Brundle1984</a>
      for answers.
    </p>
    <hr/>
    <p><em>Visit <a href="http://somewhere.org">the author's site</a>.</em></p>

  </body>
</html>

Here’s our function (Figure 6.2), which we’ll call from an event listener as before:

const citations = () => {
  Array.from(document.querySelectorAll('a'))
    .filter(link => link.getAttribute('href') === '#b')
    .map(link => (
      {node: link,
       text: link.textContent.split(',').map(s => s.trim())}
    ))
    .map(({node, text}) => (
      {node,
       text: text.map(cite => `<a href="../bib/#${cite}">${cite}</a>`)}
    ))
    .map(({node, text}) => (
      {node,
       text: `[${text.join(', ')}]`}
    ))
    .forEach(({node, text}) => {
      const span = document.createElement('span')
      span.innerHTML = text
      node.parentNode.replaceChild(span, node)
    })
}

There is a lot going on here, but it all uses patterns we have seen before. It starts by building an array of all the links in the document i.e., every a element:

  Array.from(document.querySelectorAll('a'))
  // output:
  // - <a href="#b">Moreau1896</a>
  // - <a href="#b">Brundle1982, Brundle1984</a>
  // - <a href="http://somewhere.org">the author's site</a>

(We show the nodes in comments to visualize what each step of the pipeline does.) We then filter this array to find the links pointing at #b, which is what we’re using to signal citations:

    .filter(link => link.getAttribute('href') === '#b')
  // output:
  // - <a href="#b">Moreau1896</a>
  // - <a href="#b">Brundle1982, Brundle1984</a>

We now have a problem. We could use a map call to get the text out of each link and process it, but then all we’d have is an array of strings. We’re going to want the nodes those strings came out of later on as well so that we can modify their href attributes, so somehow we have to pass the nodes and strings together through our pipeline. One option would be to create a two-element array for each:

    .map(link => [link, link.textContent.whatever])

but it’s more readable to create an object so that each component has a name:

    .map(link => (
      {node: link,
       text: link.textContent.split(',').map(s => s.trim())}
    ))
  // output:
  // - {node: <a href="#b">Moreau1896</a>,
  //    text: ['Moreau1896']}
  // - {node: <a href="#b">Brundle1982, Brundle1984</a>,
  //    text: ['Brundle1982', 'Brundle1984']}

Here, we are turning each link into an object whose "node" key has the link’s DOM node as its value, and whose "text" key has the node’s text, which we split on commas and trim to remove leading and trailing whitespace.

But we’re not done looking at this stage of our pipeline:

1. We don’t need to quote the names "node" and "text", though we could.

2. JavaScript’s String.split returns an array, so the value associated with "text" is an array. We then map over its elements to trim leading and trailing space from each.

3. If we wrote link => {node: link, text: whatever}, JavaScript would interpret the curly braces {...} as meaning, “Here is the body of a function,” and then complain because what’s in those curly braces clearly isn’t a function body. Putting parentheses around the curly braces, i.e., writing ({...}), tells JavaScript that the function is returning an object.

After all of this, the next stage of the pipeline is almost a relief:

    .map(({node, text}) => (
      {node,
       text: text.map(cite => `<a href="../bib/#${cite}">${cite}</a>`)}
    ))
  // output:
  // - {node: <a href="#b">Moreau1896</a>,
  //    text: [<a href="../bib/#Moreau1896"Moreau1896</a>]}
  // - {node: <a href="#b">Brundle1982, Brundle1984</a>,
  //    text: [<a href="../bib/#Brundle1982">Brundle1982</a>,
  //           <a href="../bib/#Brundle1984">Brundle1984</a>]}

All right, that’s not actually much of a relief, but it does make a strange kind of sense. First, if we have an object whose keys are called a and b, then the call f({a, b}) means, “Match the value of key a to a parameter called a and the value of key b to a parameter called b.” This is called destructuring, and can save a lot of wear and tear on our keyboard and eyes.

Second, if we have a variable called name, then define an object with {name}, JavaScript helpfully assumes that what we mean is {"name": name}, i.e., that we want a key called "name" with whatever value name currently has. This allows us to pass the value of node from call to call in our pipeline without typing anything more than its name.

And after all of this, the text.map call actually is a relief. The value associated with the key text is an array of strings, each of which is a bibliography key. All the map does is convert each to the text we want: a link that refers to ../bib/#citation_key and whose displayed text is also the citation key.

On to the next stage, which joins the string in text together to create a single string with commas between the entries and square brackets around the whole thing:

    .map(({node, text}) => (
      {node,
       text: `[${text.join(', ')}]`}
    ))

(We haven’t shown the output in comments because typesetting it would overflow the page and because our pseudo-HTML notation gets really confusing once we’re showing strings containing HTML that contains strings.)

The last stage in our pipeline uses forEach instead of map because we want to do something for each element of the array, but don’t need a value returned (because what we’re doing has the side effect of modifying the document):

    .forEach(({node, text}) => {
      const span = document.createElement('span')
      span.innerHTML = text
      node.parentNode.replaceChild(span, node)
    })

This is the point at which carrying the node itself forward through the pipeline pays off. We create a span element, fill it in by assigning to its innerHTML property, and then replace the original link node with the node we have just created. If we now add an event listener after our function to call it when the page is loaded, we see our citations formatted as we desired.

A Real-time Clock

We will wrap up this lesson with an example that is short, but hints at the possibilities to come:

<html>
  <head>
    <script>
      const startTime = () => {
        const today = new Date()
        const fields = [today.getHours(),
                        today.getMinutes(),
                        today.getSeconds()]
        const current = fields
              .map(t => `${t}`.padStart(2, '0'))
              .join(':')
        document.getElementById('current').innerHTML = current
        setTimeout(startTime, 1000)
      }

      document.addEventListener("DOMContentLoaded", (event) => {
        startTime()
      })
    </script>
  </head>

  <body>
    <p id="current"></p>
  </body>
</html>

Defining a function: check. Calling that function when the DOM is ready: check. What about inside the function? It’s pretty easy to guess that Date() creates an object that holds a date, and from the fact that we’re assigning that object to a variable called today, you might even guess that if we don’t specify which date we want, we get today’s values. We then pull the hours, minutes, and seconds out of the date and put them in an array so that we can turn each value into a two-character string, padded with a leading zero if necessary, and then join those strings to create a time like 17:48:02 to stuff into the element whose ID is current.

But what does setTimeout do? It tells the browser to run a function after some number of milliseconds have passed. In this case, we’re running the same function startTime a second from now. That call will change the displayed time, then set up another callback to run a second later, and so on forever. When we load the page, we see the current time updating itself second by second to remind us just how quickly life is passing by.

Exercises

What Encoding Is This?

1. Write a function that looks up the character encoding of the page the script is in and prints it to the console.

2. Extend the function to look up all the meta tags in the current page and print their names and values.

Word Count

1. Write a function called countWords that finds all the text nodes in a page, splits them on whitespace, and returns the total number of words in the page.

2. Write a second function called showWords that uses the first to find the number of words, then displays that number in a paragraph whose ID is wordcount.

A More Robust Table of Contents

1. What does the table of contents example generate if an h2 doesn’t have an id attribute?

2. Modify the example so that it only includes h2 elements that have an id attribute in the table of contents.

Explicitly Creating Nodes

Find documentation online for the two functions document.createElement and document.createTextNode, then rewrite the table of contents example to use these methods (and any others like them that you need) instead of assigning to a node’s innerHTML property.

Key Points

• Use a meta tag in a page’s header to specify the page’s character encoding.

• Pages are represented in memory using a Document Object Model (DOM).

• The document object represents the page a script is in.

• Use the querySelectorAll method to find DOM nodes that match a condition.

• Assign HTML text to a node’s innerHTML property to change the node’s content.

• Use ((params) => {...})(arguments) to create and call a function in a single step.

• An event listener is a function run by the browser when some specific event occurs.

• Create an event listener for ’DOMContentLoaded’ to trigger execution of scripts after the DOM has been constructed.

• Check the nodeType or nodeName property of a DOM node to find out what kind of node it is.

• Destructuring assignment allows us to assign to multiple variables by name in a single statement.

• Use setTimeout to trigger execution of a function after a delay.

• To make something run forever, have the function called by setTimeout set another timeout of the same function.

Dynamic Pages

In the beginning, people created HTML pages by typing them in (just as we have been doing). They quickly realized that a lot of pages share a lot of content: navigation menus, contact info, and so on. The nearly universal response was to create a template and embed commands to include other snippets of HTML (like headers) and loop over data structures to create lists and tables. This is called server-side page generation: the HTML is generated on the server, and it was popular because that’s where the data was, and that was the only place complex code could be run. (This tutorial uses a templating tool called Jekyll. It’s clumsy and limited, but it’s the default on GitHub.)

Server-side generation can be done statically or dynamically, i.e., pages can be compiled once, stored on disk, and served thereafter, or each page can be recompiled whenever it’s needed, which makes it easy to include dynamic elements like today’s top news story (Figure 7.1).

As browsers and JavaScript became more powerful, the balance shifted toward client-side page generation. In this model, the browser fetches data from one or more servers and feeds that data to a JavaScript library that generates HTML in the browser for display. This allows the client to decide how best to render data, which is increasingly important as phones and tablets take over from desktop and laptop computers. It also moves the computational burden off the server and onto the client device, which lowers the cost of providing data.

Many (many) JavaScript frameworks for client-side page generation have been created, and more are probably being developed right now. We have chosen React because it is freely available, widely used, well documented, simpler than many alternatives, and has a cool logo. Its central design principles are:

1. Page creators use functions to describe the HTML they want.

2. They then let React decide which functions to run when data changes.

We will show how to use it in pure JavaScript, then introduce a tool called JSX that simplifies things.

Hello, World

Let’s begin by saying hello:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello React</title>
    <script src="https://fb.me/react-15.0.1.js"></script>
    <script src="https://fb.me/react-dom-15.0.1.js"></script>
  </head>
  <body>
    <div id="app">
      <!-- this is filled in -->
    </div>
    <script>
      ReactDOM.render(
        React.DOM.h1(null, "Hello React"),
        document.getElementById("app")
      )
    </script>
  </body>
</html>

The head of the page loads two React libraries from the web; we will use locally-installed libraries later. The body contains a div with an ID to make it findable. When our script runs, React will put the HTML it generates into this div.

The script itself creates an h1 with the text “Hello, React” using React.DOM.h1, then finds the document element whose ID is "app" and uses ReactDOM.render to insert the former into the latter. This alters the representation of the page in memory, not the source of the page on disk; if we want to inspect the resulting HTML, we have to do so in the browser. Finally, we put the script at the bottom of the page so that the browser will have turned the HTML into a DOM tree in memory before the script runs. We will come back and fix this later.

Inspection Tools

If you are using the Firefox browser, you can open the developer tools pane by going to the main menu and selecting Tools… Web Developer… Toggle Tools. A tabbed display will open in the bottom of your page; choose Inspector to view the content of your page and page’s CSS. As you move your mouse around the page itself, corresponding structural elements will be highlighted. It’s actually pretty cool…

The first parameter to React.DOM.h1 is null in the example above, but it can more generally be an object that specifies the attributes we want the newly-created node to have. There are a few quirks in this—for example, we have to use fontStyle rather than font-style so that the attribute object’s keys look like legal JavaScript variables—but the mechanism is seductively easy to use:

  <body>
    <div id="app"></div>
    <script>
      const attributes = {
        'style': {
          'background': 'pink',
          'fontStyle': 'italic'
        }
      }
      ReactDOM.render(
        React.DOM.h1(attributes, "Hello Stylish"),
        document.getElementById("app")
      )
    </script>
  </body>

JSX

Writing nested functions is a clumsy way to write HTML, so most React programmers use a tool called JSX that translates HTML into JavaScript function calls. And yes, those JavaScript function calls then produce HTML—it’s a funny world. Here’s an example:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello JSX</title>
    <script src="https://fb.me/react-15.0.1.js"></script>
    <script src="https://fb.me/react-dom-15.0.1.js"></script>
    <script src="https://unpkg.com/babel-standalone@6/babel.js"></script>
  </head>
  <body>
    <div id="app"></div>
    <script type="text/babel">
      ReactDOM.render(
        <h1>Hello JSX</h1>,
        document.getElementById('app')
      )
    </script>
  </body>
</html>

Along with the two React libraries, this page includes a tool called Babel to translate a mix of HTML and JavaScript into pure JavaScript. To trigger translation, we add the attribute type="text/babel" to the script tag.

Why bother? Because as the example above shows, it allows us to write <h1>Hello JSX</h1> instead of calling a function. More generally, JSX lets us put JavaScript inside our HTML (inside our JavaScript (inside our HTML)), so we can (for example) use map to turn a list of strings into an HTML list:

  <body>
    <h1>JSX List</h1>
    <div id="app"></div>
    <script type="text/babel">
      const allNames = ['McNulty', 'Jennings', 'Snyder',
                        'Meltzer', 'Bilas', 'Lichterman']
      ReactDOM.render(
        <ul>{allNames.map((name) => <li>{name}</li> )}</ul>,
        document.getElementById('app')
      )
    </script>
  </body>

We have to use map rather than a loop because whatever code we run has to return something that can be inserted into the DOM, and for loops don’t return anything. (We could use a loop to build up a string through repeated concatenation, but map is cleaner.) And note: we must return exactly one node, because this is one function call. We will look in the exercises at why the curly braces immediately inside the <ul> element are necessary.

Note also that when we run this, the browser console will warn us that each list element ought to have a unique key property, because React wants each element of the page to be selectable. We will add this later.

Creating Components

One of the most powerful features of React is that it lets us create new components that look like custom HTML tags, but are associated with functions that we write. React requires the names of these components to start with a capital letter to differentiate them from regular tags. We can, for example, define a function ListOfNames to generate our list of names, then put that element directly in ReactDOM.Render just as we would put an h1 or p:

  <body>
    <h1>Create Component</h1>
    <div id="app"></div>
    <script type="text/babel">
      const allNames = ['McNulty', 'Jennings', 'Snyder',
                        'Meltzer', 'Bilas', 'Lichterman']

      const ListOfNames = () => {
        return (<ul>{allNames.map((name) => <li>{name}</li> )}</ul>)
      }

      ReactDOM.render(
        <ListOfNames />,
        document.getElementById('app')
      )
    </script>
  </body>

What we really want to do, though, is pass parameters to these components: after all, JSX is turning them into functions, and functions are far more useful when we can give them data. In React, all the attributes we put inside the component’s tag are passed to our function in a single props object:

  <body>
    <h1>Pass Parameters</h1>
    <div id="app"></div>
    <script type="text/babel">
      const allNames = ['McNulty', 'Jennings', 'Snyder',
                        'Meltzer', 'Bilas', 'Lichterman']

      const ListElement = (props) =>
            (<li id={props.name}><em>{props.name}</em></li>)

      ReactDOM.render(
        <ul>{allNames.map((name) => <ListElement name={name} /> )}</ul>,
        document.getElementById('app')
      )
    </script>
  </body>

If you look carefully, you’ll see that the name attribute passed to the use of ListElement becomes the value of prop.names inside the function that implements ListElement.

Before we map each component of our array within ReactDOM.render as in the examples above, the ListElement function gives us exactly one logical place to set attributes. Here we’ve used ListElement to italicize each element in our array and give them ids, but these functions can also be used for any transformation or calculation on each element to be rendered.

Developing with Parcel

Putting all of the source for an application in one HTML file is a bad practice, but we’ve already seen the race conditions that can arise when we load JavaScript in a page’s header. And what about require statements? The browser will try to load the required files when those statements run, but who is going to serve them?

The solution is to use a bundler (Figure 7.2) to combine everything into one big file, and to run a local server to preview our application during development. However, this solution brings with it another problem: which bundler to choose? As with front-end frameworks, there are many to choose from, and new ones are being added almost weekly. Webpack is probably the most widely used, but it is rather complex, so we will use Parcel, which is younger and therefore not yet bloated (but give it time).

Initiate a Project with Node Package Manager

If you’ve been coding along, so far we’ve created single HTML and JavaScript files, and we’ve used the Node command to run JavaScript files in the terminal, but we have not yet used the Node Package Manager npm. Before we install Parcel, we can turn the directory which we’ve been creating our HTML and JavaScript files into a project by typing:

$ npm init -y

To install Parcel, run:

$ npm install parcel-bundler

Once Parcel is installed, we can tell it to run one of our test pages like this:

$ node_modules/.bin/parcel serve -p 4000 src/dynamic/pass-params.html

Server running at http://localhost:4000
+ Built in 128ms.

Quitting Parcel

To leave Parcel use the keyboard interrupt Control-C.

This works because when NPM installs a library in a project’s node_modules directory, that library may put a runnable script in node_modules/.bin (note that it’s .bin with a leading ., not bin). When we ask Parcel to serve an application, it:

• looks in the named file to find JavaScript,

• looks recursively at what that file loads,

• copies some files into a directory called ./dist (which stands for “distribution”), and

• serves the application out of there.

Parcel also caches things in ./.cache so that it doesn’t need to do redundant work; both directories are normally added to .gitignore. To learn more about Parcel, see this short tutorial.

It’s very common to put tasks like “run my application” into NPM’s package.json file, just as older programmers would put frequently-used commands into a project’s Makefile. Look for the section in package.json whose key is "scripts" and add this:

  "scripts": {
    "dev": "parcel serve -p 4000",
    ...
  },

We can now use npm run dev -- src/dynamic/pass-params.html, since everything after -- is passed to the script being run. This doesn’t just save us typing; it also gives other developers a record of how to use the project. Unfortunately, there is no standard way to add comments to a JSON file…

Whoops

Note: if we accidentally specify the name of a directory like src/dynamic instead of the name of an HTML file, Parcel prints an error on the console saying “no entries found”. This happens because it is trying to read the directory itself as a file.

Multiple Files

Now that we can bundle things up, let’s move our JSX into app.js and load that in the head of the page:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello Separate</title>
    <script src="https://fb.me/react-15.0.1.js"></script>
    <script src="https://fb.me/react-dom-15.0.1.js"></script>
    <script src="https://unpkg.com/babel-standalone@6/babel.js"></script>
    <script src="app.js" type="text/babel"></script>
  </head>
  <body>
    <h1>Hello Separate</h1>
    <div id="app"></div>
  </body>
</html>

For now, the JavaScript in app.js is:

ReactDOM.render(
  <p>Rendered by React</p>,
  document.getElementById("app")
)

When we load this page we get the h1 title but not the paragraph. The browser console displays the message:

Error: _registerComponent(...): Target container is not a DOM element.

This is the same race condition that has bitten us before. After sighing in frustration and making another cup of tea, we decide that to keep things simple we will load the script in the body of the page:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello Bottom</title>
  </head>
  <body>
    <h1>Hello Bottom</h1>
    <div id="app"></div>
    <script src="./app.js" type="text/babel"></script>
  </body>
</html>

Installing React and ReactDOM

Now that we have made a project folder using npm init and have created a project.json file, we can install our first modules:

$ npm install react
$ npm install react-dom

In general, uses of the function require indicate that we need to install that library to access the module’s functions.

More importantly, we will rewrite app.js so that it loads the libraries it needs, because there’s no guarantee that libraries loaded in head will be available when app.js runs:

const React = require('react')
const ReactDOM = require('react-dom')

ReactDOM.render(
  <p>Rendered by React</p>,
  document.getElementById('app')
)

We don’t have to shut down the server and restart it every time we make changes like this, because Parcel watches for changes in files and relaunches itself as needed. Each time it does so, it looks at the libraries app.js loads and rebundles what it needs: right now, for example, dist/app.ef6b320b.js is 19930 lines long.

A more modern option than loading in the bottom is to add the async attribute to the script in the head of the page, which tells the browser not to do anything with the JavaScript until the page has finished building:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Hello Parcel</title>
    <script src="./app.js" type="text/babel" async></script>
  </head>
  <body>
    <div id="app"></div>
  </body>
</html>

Exercises

Those Damn Curly Braces

Our list-building example includes this line of code:

        <ul>{allNames.map((name) => <li>{name}</li> )}</ul>,

Why are the curly braces immediately inside the <ul> element necessary? What happens if you take them out?

Real Data

1. Create a file called programmers.js that defines a list of JSON objects called programmers with firstName and lastName fields for our programmers. (You can search the Internet to find their names.)

2. Load that file in your page like any other JavaScript file.

3. Delete the list allNames from the application and modify it to use data from the list programmers instead.

Loading constant data like this is a common practice during testing.

Ordering

What happens if you change the order in which the JavaScript files are loaded in your web page? For example, what happens if you load app.js before you load ListElement.js?

Multiple Targets

What happens if your HTML page contains two div elements with id="app"?

Creating a Component for Names

Create a new React component that renders a name, and modify the example to use it instead of always displaying names in <li> elements.

Striping

Suppose we want to render every second list element in italics. (This would be a horrible design, but once we start creating tables, we might want to highlight alternate rows in different background colors to make it easier to read.) Modify the application so that even-numbered list elements are <li>{name}</li> and odd-numbered list elements are <li><em>{name}</em></li>. (Hint: use the fact that a map callback can have two parameters instead of one.)

Key Points

• Older dynamic web sites generated pages on the server.

• Newer dynamic web sites generate pages in the client.

• React is a JavaScript library for client-side page generation that represents HTML elements as function calls.

• React replaces page elements with dynamically-generated content in memory (not on disk).

• React functions can be customized with elements.

• JSX translates HTML into React function calls so that HTML and JavaScript can be mixed freely.

• Use Babel to translate JSX into JavaScript in the browser.

• Define new React components with a pseudo-HTML element and a corresponding function.

• Attributes to pseudo-HTML are passed to the JavaScript function as a props object.

Visualizing Data

Tables and lists are great, but visualizations are often more effective—if they’re well designed and your audience is sighted, that is. There are even more ways to visualize data in the browser than there are front-end toolkits for JavaScript. We have chosen to use Vega-Lite, which is a declarative framework: as a user, you specify the data and settings, and let the library take care of everything else. It doesn’t do everything, but it does common things well and easily, and it interacts nicely with React.

Vega-Lite

Let’s start by creating a skeleton web page to hold our visualization. For now, we will load Vega, Vega-Lite, and Vega-Embed from the web; we’ll worry about local installation later. We will create a div to be filled in by the visualization—we don’t have to give it the ID vis, but it’s common to do so—and we will leave space for the script. Our skeleton looks like this (with lines broken for the benefit of the printed version):

<!DOCTYPE html>
<html>
<head>
  <title>Embedding Vega-Lite</title>
  <script src="https://cdn.jsdelivr.net/npm/vega@5"></script>
  <script src="https://cdn.jsdelivr.net/npm/vega-lite@3"></script>
  <script src="https://cdn.jsdelivr.net/npm/vega-embed@4"></script>
</head>
<body>

  <div id="vis"></div>

  <script type="text/javascript">
  </script>
</body>
</html>

We can now start filling in the script with the beginning of a visualization specification. This is a blob of JSON with certain required fields:

• $schema identifies the version of the spec being used (as a URL).

• description is a comment to remind us what we thought we were doing when we created this.

• data is the actual data.

...rest of page as before...
  <script type="text/javascript">
    let spec = {
      "$schema": "https://vega.github.io/schema/vega-lite/v2.0.json",
      "description": "Create data array but do not display anything.",
      "data": {
        "values": [
          {"a": "A", "b": 28},
          {"a": "B", "b": 55},
          {"a": "C", "b": 43},
          {"a": "D", "b": 91},
          {"a": "E", "b": 81},
          {"a": "F", "b": 53},
          {"a": "G", "b": 19},
          {"a": "H", "b": 87},
          {"a": "I", "b": 52}
        ]
      }
    }
  </script>
...rest of page as before...

In this case, we represent a two-dimensional data table as objects with explicit indices "a" and "b". We have to do this because JSON (like JavaScript) doesn’t have a native representation of two-dimensional arrays with row and column headers.

Once we have created our spec, we can call vegaEmbed with the ID of the element that will hold the visualization, the spec, and some options (which for now we will leave empty):

    let spec = {
      "$schema": "https://vega.github.io/schema/vega-lite/v2.0.json",
      "description": "Create data array but do not display anything.",
      "data": {
        "values": [
          // ...as above...
        ]
      }
    }
    vegaEmbed("#vis", spec, {})