What is optimized input?

Optimized input is content that contains the words you need to see, when you need to see them. What makes input optimized is that it fits what you need at that point in time.

I came up with the term while I was building Verba, the first adaptive immersion tool. The language learning community has spent years optimizing flashcards. No meaningful progress has been made on optimizing the input itself.

Optimized input is not the same as Stephen Krashen’s “optimal input”. Optimal input describes what good input is like: comprehensible, compelling, rich and abundant. Optimized input is input that fits one learner, at one moment in time.

What makes input optimized?

The words make input optimized. Optimized input contains the words you need to see at that point in time, because you’re due to review them. A word that doesn’t come back in time is a word you forget.

But of course, the right words aren’t all that makes input ideal. The ideal input at any point in time is also input that:

  • you’re interested in
  • has the right speech cadence for you
  • introduces you to new words that you’re ready to learn, or likely to understand

And the ideal input changes. Tomorrow you know a little more, so the ideal input for you is different.

How does Verba optimize input right now?

Right now, Verba does the fundamental part. The context algorithm checks each video for the words you’re due to review, and for the words you’ve seen before. Then it puts the videos that contain the most of them at the top.

Why optimize the input at all?

The rest of this page is why I think the input is the thing to optimize. It’s a short tour of the research I’ve spent the last few years reading.

Verba doesn’t hinge on any single study on this page. It’s built on a few principles from an area of linguistics called usage-based theory, which has growing support from linguists. Where something is still argued, I say so.

How do most people think language works?

Most people think language works like Lego. Words are the blocks, and grammar is the set of rules for putting the blocks together. You memorize the words, you learn the rules, and you build sentences.

Languages have been taught this way for centuries. In the 1950s, the linguist Noam Chomsky turned a version of the idea into a formal theory called generative grammar. The theory says we’re born with a system for grammar in the brain, and that this system is separate from the words we know.

It’s a neat picture, but real language doesn’t behave that way when you look at it closely.

Are grammar and vocabulary separate things?

Grammar and vocabulary are probably not separate things. A growing number of linguists see them as two ends of one continuum.

Take “keep an eye on”. You can’t work out what it means from the words and the rules. You just know the whole chunk. So is it vocabulary? It’s three words long.

Now take “not only X but also Y”. Part of it is fixed, and part of it is a slot you fill in: “not only smart but also hardworking”. So is it grammar? It has fixed words in it.

”What’s this coat doing on the floor?” is not a question about what the coat is doing. The pattern “What’s X doing Y?” carries its own meaning: X shouldn’t be there (Kay and Fillmore, 1999). And in “He sneezed the napkin off the table”, the verb “sneeze” doesn’t normally move things. The pattern around the verb is what tells you something got moved (Goldberg, 1995).

Linguists call these patterns constructions, and the field that studies them is construction grammar. The idea is that language is constructions at every size. There are single words, fixed chunks, chunks with slots, and fully open patterns. There’s no line where vocabulary stops and grammar starts.

Do we learn rules, or patterns?

Usage-based theory says we learn patterns, and we learn them from exposure. You don’t get given the rules. You hear a lot of language, and your brain picks up on what tends to go with what.

There’s good evidence that brains can do this. In a well-known study, 8-month-old babies listened to two minutes of made-up syllables with no pauses between the “words”. Afterwards, the babies could tell the words from the non-words. The only clue they had was which syllables tended to follow which (Saffran, Aslin and Newport, 1996).

Children seem to build up language the same way. They start with specific chunks like “I don’t know”. Later they work out which part is a slot: “I don’t like it”, “I don’t want it” (Tomasello, 2003).

So each time you meet a pattern in a new sentence, your sense of it gets sharper. You learn one more thing that can go in the slot, or one more situation the pattern fits.

What does the brain evidence say?

The brain evidence fits the idea that grammar and vocabulary are one system. Ev Fedorenko’s lab at MIT studies the parts of the brain that handle language.

In a 2020 study, her team went looking for any region in that network that responds more to grammar than to the meanings of words. They ran three experiments and didn’t find one. The same regions did both jobs (Fedorenko, Blank, Siegelman and Mineroff, 2020).

That doesn’t prove usage-based theory is right. But it’s what you’d expect to see if grammar and vocabulary are one continuum. It’s hard to square with a separate grammar system in the brain.

What do large language models have to do with it?

Large language models, like the one behind ChatGPT, come from the same line of research as usage-based theory. I find them a useful way to picture how language learning works.

The shared origin is a movement from the 1980s called connectionism (Rumelhart and McClelland, 1986). The idea was that a network of simple units can learn from experience by strengthening its connections, with no rules written in. Joan Bybee, one of the founders of usage-based theory, later wrote a paper with James McClelland, one of the founders of connectionism (Bybee and McClelland, 2005). Today’s language models are descendants of those early networks.

Nobody gives a language model grammar rules. It’s trained to do one thing: predict the next word. It reads a huge amount of text, and it comes out the other side writing grammatical sentences. So grammar doesn’t have to be built in. It can come out of exposure.

A 2021 study compared dozens of language models with recordings of human brain activity. The better a model was at predicting the next word, the closer its inner workings matched the brain’s language network (Schrimpf et al., 2021).

So I think of learning a language as a lot like training a model. Your brain is getting better at predicting what comes next. What it needs is a lot of varied examples, not a rulebook.

It’s an analogy, and it has limits. A language model reads thousands of times more words than a child ever hears. A child also learns in a world full of people and things, not from text. Some linguists argue that language models tell us very little about how humans learn (Kodner, Payne and Heinz, 2023). So language models show that learning from exposure is possible. They don’t show that it’s what your brain does.

Has anyone picked input for a learner before?

Yes. In 2004, researchers at Carnegie Mellon University built a reading tutor called REAP for people learning English. REAP kept a profile of each student: their reading level, their interests and the words they needed to learn. Then it searched a library of about five million web pages for texts that contained those words (Collins-Thompson and Callan, 2004).

It worked. In a classroom study, students who used REAP did better on vocabulary tests (Heilman, Zhao, Pino and Eskenazi, 2008).

REAP was a research system for English reading classes. Verba takes the same idea to video, and adds timing: it picks a video by the words you’re due to review.

What is still argued?

A lot is still argued, and you should know where.

  • Built-in grammar. Generative grammar still has serious defenders. Usage-based theory has growing support. It’s not the consensus.
  • How the learning happens. One proposal is that the brain learns from prediction errors: it guesses what’s coming, and updates when it’s wrong. It’s a proposal, not a settled fact.
  • Flashcards. Flashcards do work. One study taught people made-up words with flashcards, and the words went into memory like real words (Elgort, 2011). Each made-up word had a single meaning though. Real words have many uses.

What does this mean for how you learn a language?

For me, the research comes down to three principles. Verba is built on them.

1. You learn a word across many contexts, not one. In one study, people who met a new word in a different sentence each time learned its meaning better than people who saw the same sentence again and again (Bolger, Balass, Landen and Perfetti, 2008). So a review should be a new sentence.

2. A word’s meaning depends on the pattern it’s in. So a tool should tell you what a word means in this sentence, rather than hand you a list of meanings and leave it to you to figure out which one’s right.

3. Rare words don’t come back on their own. After the first few thousand words, a word you need might turn up once a week. So something has to bring it back.

That’s where optimized input starts. The word comes back when you’re due to see it, and it comes back in a new context. Adaptive immersion is the name for a tool that does that, and Verba is the first one.

Sources