1

Table of Contents
Books in APS - ESCI Annotation - ASTRIA review ............................................................................ 1
Books ESCI Tenets, unless you know better ones.... .............................................................. 1
1. Determining Query Intent....................................................................................................... 2
Books vs Non-Books Queries .................................................................................................. 2
Books query intents ................................................................................................................ 3
2. ESCI Annotation Guidelines .................................................................................................... 6
2.1 Title ................................................................................................................................... 6
2.2 Author ............................................................................................................................... 7
2.3 Broad ................................................................................................................................. 8
3. ESCI Annotation Resources ..................................................................................................... 9
3.1 Amazon-wide ESCI(+U) definitions - from ASTRIA ............................................................ 9
3.2 Determining Genre/Subject ............................................................................................ 10
3.3 Determining age-range ................................................................................................... 17
3.4 Complementarity in Books.............................................................................................. 18
3.5 Cross-PT queries.............................................................................................................. 19
3.6 Miscellaneous queries .................................................................................................... 22
5. Appendix .............................................................................................................................. 24
5.1 Books Query Sub-Intents ................................................................................................ 24

Books in APS - ESCI Annotation -

ASTRIA review
This document provides guidelines to judge search results for Books searches, according to the ESCI
framework. This is not (only) an SOP: these guidelines define the standards for optimal search relevance
for books.

Books ESCI Tenets, unless you know better ones....

1. We judge search results from the point of view of the customer: products in results must match
the customer query intent.
2. All ASINs on the search page are treated equally. We hold organic, sponsored and widgets
results to the same standards.
3. ESCI judgement is agnostic to customer context: we do not take into account subscription
status, past purchases or search behavior when judging keyword-ASIN pairs.
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 2

4. All query restrictions expressed with keywords (e.g. on format, binding, etc.) are taken into
account for ESCI judgement. At the same time, we do not assume restrictions unless they are
explicit, in particular: book queries are judged as format agnostic, unless a format is explicitly
mentioned by the customer.
5. Books ESCI annotation is a WW program. While examples in this doc mainly come from English-
speaking marketplaces, the general judgment criteria can be applied internationally. We will
draw marketplace-specific guidelines when there is a specific need (e.g. Japanese manga).

1. Determining Query Intent

The first step to judge search relevance for books is to put oneself into the customer shoes: would the
customer find the search results we present useful to satisfy her search? Answering this question
requires first of all an understanding of the customer search intent. Before we delve into books-specific
query intents, we have to first determine whether a query is related to books.

Books vs Non-Books Queries

From a books search perspective, a customer might be looking for a book, for a non-book or the product
type intent might be unclear (e.g. X-GL queries like “Harry Potter”).

Understanding if a customer is looking for books:

● the keywords contain the words “book”, Kindle, “audiobook”, “audible“, ”manga“, ”comic“ or
it’s a book ISBN
o Harry Potter books, Harry Potter books set
o Tokyo Ghoul book 1
o book gifts for children
o there's no place like here kindle
o walkin disaster audiobooks
o book about making jam and wine with fruit
o 9780435189112
● the keyword is an exact title/author/title+author match with at least one book
o atlas of the heart
o dave grohl
o hail mary andy weir
o winnie the pooh
● the keyword, while not an exact match with a book, when searched in APS produces book
results that are a suitable answer for a query. This is common for broad queries that customers
use to find a book on a topic/genre:
o things for retired dads to make
o pet bereavement story
For the purpose of ESCI annotation, we do not consider searches for the below as books queries:
● notebooks/journals
● agendas
● questions about Amazon programs/website functionalities, even if they return books, e.g.
“cancel kindle unlimited”, “amazon login”, “customer service”
We do consider books:
● calendars
● music scores, sheet music
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 3

● maps
● flash cards
● magazines
● the following keywords (and functional equivalents): “kindle unlimited”, “prime reading”,
“audible plus”, “kids plus”, “comixology unlimited”

Books query intents

When customers search for books, they typically type the title (fully or in part), the author or the
genre/subject of the book. We therefore define three book query intents.

Title

The customer searches by the full title, part of the title, series title or title+author name. These queries
should have one to a few (in the case of series) exact results.

Additional guidelines:
● when the keyword looks like a title, but there are no title exacts on Amazon, the auditor will
check on Google (a link is available in the annotation tool) whether a book title match exists
outside Amazon
● when the intent is ambiguous between title and broad, we judge as broad if there is more than a
few (> 4 as a rough reference) Exact matches, e.g. ayurvedic cookbook, metaverse book, ocean
book
Examples:
● this ear hustle unflinching stories of everyday prison life - full title search
● musical theory books for dummies - not a full title match, but customer intent is clearly looking
for this book
● ester the wonder pig - series name search (with typo)
● demon slayer 22 - volume in series spearfishing
● hail mary andy weir - title (partial) + author
● pre-k alphabet wipe-clean workbook: scholastic early learners (wipe-clean) - specific edition
spearfishing
● nerdy baby books - series query

SQ CATEGOR EXACT SUBSTITUTE COMPLEMENTARY IRRELEVANT

Y
Harry Potter Strip Harry Potter Harry Potter and the Half- Harry Potter and the Goblet The Haunting of
and the Goblet books and the Goblet Blood Prince of Fire Summary and Hill House
of Fire of Fire Harry Potter and the Guidebook (Irrelevant
Goblet of Fire Art Book Genre)

Pokémon Strip Pokémon One Piece/Naruto Death Note

complete series books complete series Desperado Pikachu (A (Irrelevant
part of the Pokémon Genre)
Complete Series)

Please Note : The way to understand this is that the complement is entirely about another book,
whereas the substitute is a different book in itself. It's a nuanced concept, but not a complex one once
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 4

you consider that only a limited types of books can be Complements and these are: summaries, study
guides, workbooks, exercise books and very few others.

Author

The customer searches by author name (fully or in part). These queries should have from one to several
exact results.
● jk rowling
● jk rowling books

Query Store Exact Substitute Complementary Irrelevant Exception

If the author spans
into 2 genres, then
a match with even
B07KRM53P one genre will be
R; The 5AM exact match. If the
club [same B0883FHYVK; SQ is a particular
B07RFSSYBH; genre of Summary/analy 0802145582, book then
Author All books by motivation sis of the History of India summary of the
(self- James digital the exact and self- author's work or [different author will be
help) clear text same author help] their works genre] irrelevant
1462742904,
B09NH3VDM
X,
0823444295
Books with
exact name,
book with
James as
author, book B002CSL0CK,
digital with James as secondary
Author James text illustrator contributor
Mid- Different genres
summer will ideally be
night’s irrelevant except
dream when their authors
Same [comedy Othello are the same
author by Mid-summer [tragedy by Analysis of mid- Different genre person. In that
different Shakesp night’s dream Shakespeare summer-night’s by another case, they will be a
genre eare] Books book ] dream author substitute.

Broad

This intent covers a variety of customer searches, anything that is not title or author: books by genre,
topic, subject, age group, etc. These queries should have many exact results.
● children's christmas carol book - topic/subject query
● ayurvedic cook books - topic+book type query
● psalms coloring book - topic+book type query
● risque regency romance - subgenre query
● adult alice in wonderland books - character query
● dinosaur writing book - topic+book type query
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 5

● the october revolution - topic query

● baby books - children age range query
● nkjv bible black - bible
● books under 10 dollars - price query

In the next section, we provide ESCI annotation guidelines by query intent and sub-intent. See Books
Query Sub-Intents for a full list of sub-intents with examples.

Category Broad Query Exact Substitute Irrelevant

Categories
1. Topic Cricket 0847868575

Books

B01N0J4CJ5

2. Genre Romantic B086BV6T6X The Lincoln Highway

3.Bible New 0195283805 1521113009

version
Bible

4.Price/fr Free (if customer is looking for “free books”, a book outside the specified price
ee Books price must be 0 in the local currency) range (or non-free) that would
B09VKWJJSW otherwise be an exact
match 0446521906

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 6

5.Best- New York Note that query constraints must be Any book ranked within
seller Best respected (e.g. “romance bestseller” will Books/Kindle Store Zeitgeist top
seller need to satisfy both bestseller and genre 2,000
guidelines to be judged Exact) any book with more than 1,000
-any Book with the Amazon Charts OR reviews
Editor’s Pick OR Bestseller badges B00CD36EFM
-any Book with “New York Times
bestseller” reported on the cover
0735222355
6.Cross PT Ronald B000SKJMQ0 B089PQSK47
Reagan

2. ESCI Annotation Guidelines

Before diving into ESCI criteria for books, readers unfamiliar with ESCI should first read 3.1 Amazon-wide
ESCI(+U) definitions - from ASTRIA, in order to understand where books guidelines fit within the overall
Amazon framework.

The following sections of chapter 3 are also critical for a complete understanding of the guidelines:
● Section 3.2 Determining Genre/Subject provides instructions to determine the genre/subject of
a book, often the most important element to judge substitutability between books. Examples
explain the varying level of detail required to judge genre substitutability: in most cases
determining the exact subgenre of the book is not necessary.
● The difference between a Substitute and a Complement, not always obvious in the media space,
is defined in more detail in section 3.4 Complementarity in Books.
● Section 3.5 Cross-PT queries explains in detail and with practical examples how to judge queries
returning results from multiple product types.
Important Note: format, age, price and other explicit constraints in the query must always be satisfied in
order to judge the query Exact.

2.1 Title
Exact
● Title spearfishing: the book(s) that exactly match the title in the search query
● Series spearfishing (volume number and/or title): the volume(s) in the series that exactly match
the title/volume in the search query
● Series: all books belonging to the series searched by the customer (e.g. as indicated by Series
grouping in search or detail page)
● Title+author: all books that exactly match the title+author combination in the search query
● Cross-PT: See 3.5 Cross-PT queries for a detailed explanation and examples
o For title spearfishing/series X-GL (e.g. Harry Potter, Jack Ryan, Avengers):

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 7

▪ all the media products that exactly match the keywords and are either the main
work(s) in the original medium (usually a book) or other media derived directly
from the original medium (e.g. film adaptation, screenplay, etc.) are Exacts.
Conversely, media products about the original work(s) are Substitutes.
o For brand/character/media tie-in (e.g. barbie):
▪ all media/non-media products that exactly match the keywords used by the
customer.
o For proper nouns keywords (people names, brands, characters), see Author Cross-
GL and 3.5 Cross-PT queries below
● ISBN/ASIN: the book exactly matching the ISBN/ASIN in the search query

Substitute
● Title spearfishing
o any book by the same author of the exact match
o any book within the same series or collection (use “book #x in series” displayed in
search to confirm this)
o any book of the same genre/subject of the exact match. See Determining
Genre/Subject.
● Series spearfishing:
o any book belonging to the same series of the exact match (use “book #x in series”
displayed in search to confirm this)
o any book by the author of the exact match
● Series:
o any book by the same author of the exact match series
o any book of the same genre/subject of the exact match
● Title+author, see Title spearfishing
● Cross-PT: See 3.5 Cross-PT queries for a detailed explanation and examples
o all non-media products that exactly match the keywords used by the customer.
o media products about the original work matching the customer query.

Complement
● Cross-PT: summaries, study guides, workbooks, etc. related to the main work the customer is
searching
● All other sub-intents: books that are meant to accompany/complement the exact match for the
title/series search (study guides, summaries, workbooks, etc.). See 3.4 Complementarity in
Books

Irrelevant

Anything not Exact, Substitute or Complement

2.2 Author
Exact
● Author: any book authored or contributed to by the author in the customer query. Any book
about the author in the customer query
● Publisher: any book published by the publisher in the customer query. Any book about the
publisher in the query.

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 8

Substitute
● Author: any book of the same genre(s)/subject(s) of the Exact matches for the query.
See Determining Genre/Subject.
● Publisher: not applicable

Complement

Workbooks, study guides, summaries of books by the author in the query

Irrelevant

Anything not Exact, Substitute or Complement

2.3 Broad
Exact
● topic/genre/sub-genre: any book matching the topic/genre specified in the
query. See Determining Genre/Subject.
● age range/reading level: any book suitable for the age-range specified in the customer query.
See Determining age-range
● Bible: any bible matching the customer query restrictions (e.g. version, leather cover, binding,
etc.)
● Price/Free: any book (in any format) matching the price constraints chosen by the customer (if
customer is looking for “free books”, price must be 0 in the local currency)
● Bestsellers: note that query constraints must be respected (e.g. “romance bestseller” will need
to satisfy both bestseller and genre guidelines to be judged Exact)
o any Book with the Amazon Charts OR Editor’s Pick OR Bestseller badges
o any Book with “New York Times bestseller” reported on the cover
o any book ranked within Books/Kindle Store Zeitgeist top 2,000
o any book with more than 1,000 reviews
● Cross-PT, person name (artist, celebrity, historical figure, e.g. “ronald reagan”)
o any media product (books, movies, music, etc.) authored by or about the person in the
customer query

Substitute
● topic/genre/sub-genre: a book in a different sub-genre/sub-topic from the one searched by the
customer, but part of the same genre (e.g. science fiction ASIN when the customer is looking for
“science fiction comedy”)
● age range/reading level: a children book for an age range adjacent to the one the customer was
looking for (e.g. a book suitable for 3-4 yrs old when customer is looking for “children books 5-6”
● Bible: any other Bible edition matching in part the customer query
● Price/free: a book outside the specified price range (or non-free) that would otherwise be an
exact match

Please note - For the purposes of this SOP, Books included in Kindle Unlimited are not Exacts for query
Free Books. You can consider them as Substitutes. Exacts will have price 0 in local currency. Books that
are not free are Substitutes (see next section) if they satisfy the rest of the query restrictions (if any)

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 9

● Bestseller:
o any book ranked between 2,000 and 10,000 in Books/Kindle Store
o any book having between 500 and 1000 reviews
● Cross-PT:
o any non-media product relevant to the query, See 3.5 Cross-PT queries for a detailed
explanation and examples

Complement

Not applicable

Irrelevant

Anything not Exact, Substitute or Complement.

Review Note: books are Exact matches… for relevance criteria for age-range.

3. ESCI Annotation Resources

The following sections provide additional details and guidance for Books ESCI judgement, diving deeper
into judgement criteria. We first report here Amazon-wide ASTRIA ESCI definitions that provide the
general semantics of ESCI categories from which Books-specific ESCI criteria are derived. We then dive
into genre/topic criteria for substitutability, age-range appropriateness and complementarity definition
in books.

3.1 Amazon-wide ESCI(+U) definitions - from ASTRIA

Exact – This product fully satisfies the query. A broad or ambiguous query could be satisfied by any of a
diverse set of products, so long as it is reasonable to expect that a customer who entered that query
might buy that product. A very specific query will only be satisfied if the product matches all of the
characteristics included in the query.

Substitute – This product differs from the one intended, but it is functionally equivalent. This will be the
same type of product, but may differ by brand, model, color, title, or other detail. If the requested
product is included in a bundle with other associated products, it is judged as Substitute, so long as it is
reasonable that the customer may want to buy both together.

Complement – This product is not the one intended, but it could be useful to the customer if they did
buy the intended product. This could be an accessory, a component, replacement parts, or even another
item designed to be used with the product.

Irrelevant – This product does not satisfy a reasonable understanding of the customer’s intent. It could
be entirely unrelated or it could differ by one crucial feature that would make it unsuitable for the
customer.

Unjudgeable – This judgment is only to be used if the product cannot be judged, either because the
query cannot be understood, or because the product is lacking enough information to determine if it
matches the query. An Unjudgeable query is one that is complete gibberish, is seeking help, is

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 10

attempting to navigate to a website, is in a non-market language, or is a string of numbers that does not
correspond to an ISBN or model number in Google searches.

3.2 Determining Genre/Subject

In order to determine whether a book is a Substitute for a customer query, we very often have to look at
the genre and/or the subject of the exact match book (or of the genre/topic mentioned in the keywords,
in case of broad queries).

Note: since determining the genre of a book can be time consuming, we can sometimes skip this step
and still come up with a decision on substitutability for a query-ASIN pair. Specifically, we consider an
ASIN Substitute if:
● it’s from the same author of the exact matching book of the customer query
● it’s part of the same series of the exact matching book of the customer query

Genre vs Subject

Genre refers to categories of works that are characterized by similar plots, themes, settings, situations,
and characters (reference). In other words, the genre describes what a book is.
Fiction and Non-fiction are macro genres and they each can be subdivided into sub-genres. Here are
some examples, check out this wikipedia article for a more complete list.

Fiction Genres (examples):

● Mysteries and Thrillers
● Historical
● Romance
● Science-Fiction
● Literary Fiction
● Children Fiction
Non-Fiction Genres (examples):
● Academic/textbook
● Biography
● Journalistic writing
● Self-help
● Travel guides
● Cookbooks
By contrast, the Subject describes what a book is about or its topic. Example of subjects are:
Architecture, Photography, Sports (in general or specific, e.g. football), etc.

In many cases, it will be necessary to determine both genre and subject in order to assign an ESCI label
to an ASIN. Let’s take a look at some examples to see why this is the case.

Query: fiction book with crypto - broad

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 11

Judgement: the customer is looking for a Fiction book (genre) on the topic of cryptocurrencies (subject).
Exact results have to match both restrictions.
The first result is an Exact match: as is easy to understand from the title and the description, it’s a fiction
story on the topic of Bitcoin, a cryptocurrency.
The second result, however, is an Irrelevant for the query, as it fails the genre restriction: it’s a non-
fiction book on cryptocurrencies.

Query: living with coronary heart disease - title

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 12

Judgement: the customer is looking for the book in the first result, which is a full title match. This Exact
match is a popular (i.e. non-professional) non-fiction (genre) book on cardiology (subject). Substitutes
have to match both these restrictions.
The second book is also a cardiology book (same genre), therefore, the result is Substitute

Query Exact Substitute Complementary Irrelevant

Call me by your name Call me by your My Policeman Summary of Call me The Intelligent Investor
1786495252 name (romance, LGBT, by your name (Business, Economy,
(Romance, LGBT, Historical, (Summary of the SQ) Self-help, Finance)
Contemporary, Coming Contemporary)
of age)

Gone with the wind Gone with the wind Girl with a Pearl Summary of Gone The August Sleepwalker
9382616594 Earring with the wind (Poetry, Academics)
(Classical, Historical (Historical, art, (Summary of the SQ)
Romance, War) romance)

Dune 0441172717 Dune Snow Crash Summary of Frank 101 Facts of Solar
(Science fiction, fantasy, (Science fiction, Herbert’s Dune System
adventure) fantasy) (summary of SQ) (Science, Academic,
Non-fiction)

The Lincoln Highway The Lincoln Highway The underground Summary of The Microsoft Excel 2016
B08WRHBXGN Railroad Lincoln Highway (Computer, Academics)
(Literary fiction, (Literary fiction, (Summary of the SQ)
Adventure, Historical) historical, cultural
race)

Matilda 0141365463 Matilda The Borrowers Summary of Matilda Letters to a Law

(Children, fantasy, (Children, classics, (Summary of the SQ) Student
humor, classics) adventure) (Non-fiction, self-help,
law)

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 13

The Maidens The Maidens Billy Summers Summary & Hard, Hard Religion
1250304458 (Mystery, thriller, Breakdown of The (Religion, history, non-
(Mystery, thriller, horror) Maidens fiction)
fantasy) (Summary of the SQ)

The Silent Patient The Silent Patient Murder on the Orient Summary of The Our White House:
1250301696 Express Silent Patient Looking In, Looking Out
(Psychological thriller, (Thriller, mystery) (Summary of the SQ) (Non-fiction, history,
Suspense, Mystery) children)

A Beautiful Mind A Beautiful Mind A Promised Land Lesson Plan for A Beloved
0571212921 (Biography, Politics, Beautiful Mind (Fiction, Literature,
(Biography, psychology, History) (Summary of the SQ) Classics)
memoir)

Think Like a Monk Think Like a Monk Ikigai Summary of Think A mercy
0008386595 (Self-help, spirituality, Like a Monk (Fiction, Literature,
(Self-help, Spirituality, philosophy) (Summary of the SQ) race)
philosophy, psychology)

Democracy in America Democracy in Fire and Fury Summary of Harry Potter and the
0226805360 America (Politics, history, Democracy in Deathly Hallows
(Politics, history, biography) America (Fiction, Adventure,
sociology) (Summary of the children, fantasy)
book)

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 14

Physics books Physics Bundle of Physics, Physics Workbook Breaking Dawn

1118230647 Chemistry, Biology (Workbook for the (Fiction, fantasy,
(Academic) (Set of academic SQ) romance)
books)

Ottolenghi Simple Ottolenghi Simple Plenty More Summary of City of Ashes

1607749165 (Cookbook) Ottolenghi Simple (Fiction, romance,
(Cookbook, health) (summary of SQ) Paranormal)

The Architecture The Architecture London Travel guide Insurgent

Lover’s Guide to Lover’s Guide to for Women (Fiction, science fiction,
London 1399001663 London (travel) adventure)
(travel)

The Soul Tells a Story The Soul Tells a The Case for Christ Lord of the Rings
0830832319 Story (Religion, spirituality, (Fiction, fantasy,
(Spirituality, religion) history) adventure)

A Note on Sub-genres

Some genres (especially in the realm of fiction) can be further broken down into sub-genres, e.g.
● Mysteries and Thrillers
o Thrillers
▪ Psychological thrillers
▪ ....
o Mysteries
▪ cozy mysteries
▪ detective stories
▪ ...
For the purpose of ESCI annotation, it is generally not necessary to determine which subgenre the exact
match should belong to: if an ASIN belongs to the same genre but a different subgenre of the Exact
match, it can still be considered as Substitute. Example:
Query: cyber warfare and the new world order - title

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 15

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 16

Judgement: the first result is the Exact match for the title query: a techno-thriller set in the future, so it
overlaps the genres of science fiction and thrillers. The following result is also a techno-thriller, although
not set in the future, it’s a Substitute.
The third result is an espionage thriller. While neither set in the future nor focused on technology, it can
still be considered a substitute as it belongs to one of the main genres of the first book (thriller). The
fourth result is Irrelevant, as it is (primarily) a historical fiction novel.

However, if the customer specifies a sub-genre in the query, ASINs not belonging to that subgenre are
Irrelevants, for example:
Query: cyber warfare fiction - broad (sub-genre)

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 17

Judgement: the first result is an Exact (covered in example above). The second result is irrelevant: it’s a
thriller unrelated to cyber-warfare.

Practical considerations

Judges can use the decision tree and the subject to macro-subject mapping provided in the Books IDQ
BISAC annotation SOP (BISAC labeling instructions.pdf). As a rule of thumb, books in the same
genre/macro-subject are substitutes, unless the customer query is for a specific sub-genre.
Browse nodes assignments (visible on detail page breadcrumbs and Bestseller/Zeitgeist rankings) can be
used to aid genre/subject determination, but are not the source of truth for genre, as misclassifications
are common: auditors should use judgement and question browse node assignments when they appear
incorrect.

3.3 Determining age-range

Detail page and face-out in Search contains age-range information that can be used for ESCI annotation
of age-related queries. When age-range information is not available or seems incorrect, the following
guidelines can be used to determine suitability for each age.

Note: books are Exact matches for an age range query if their age range overlaps with the one the
customer looks for. E.g. a book suitable for 3-5 yrs old is Exact for the query “books for children 5-6”.

Query CATEGOR EXACT SUBSTITUTE COMPLEMENTARY IRRELEVANT

Y
Book for Strip Book for 3-4 years/book for 2-3 Book for 5-6 Books stickers/book Book for 6-7 years (1+ year
3-4 years books years (when overlapping the age, years. covers/book mark. age gap will not be accepted/
can be considered EXACT. book for adults.

0-2
● Tactile books with flaps, mirrors, textures, and sounds.
● “See and say” books
● Simple board books with action language
● Maybe: longer picture books with a simple plot
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 18

3-5
● Board books with more text
● Books with wordplays, rhymes
● ABC books, learn numbers, etc
● Coloring books, puzzles/gadgets

6-8
● Easy non-fiction (books helping answer Why questions)
● Early chapter books, often illustrated: 48-64 pages (more if non-fiction)

8-12
● Most nonfiction for kids
● Illustrated with a lot of text (e.g. Diary of a Wimpy Kid)
● Light teen topics/issues (e.g. juvenile romance) and social topics (bullying,
racial/gender/religious) discrimination - especially 10-12
● 80-160 (max 200) pages
Above 12 years, we consider the book Young Adults (YA)

US Grades to age

Column 1 Column 2
Grade Age
Pre-kindergarten 3-5
Kindergarten 5-6
1st grade 6-7
2nd grade 7-8
3rd grade 8-9
4th grade 9 - 10
5th grade 10 - 11
6th grade 11 - 12
7th grade 12 - 13
8th grade 13 - 14
9th grade (High school) 14 - 15
10th grade (High school) 15 - 16
11th grade (High school) 16 - 17
12th grade (High school) 17 - 18

3.4 Complementarity in Books

Books that are meant to be read together with other books and/or their content is entirely based on
another book are considered complements. Common examples of complements include study guides,
summaries, reviews, companion books (e.g. exercise books for textbooks), anthologies, critical essays.

What is the difference between Substitutes and Complements? Consider this example:

Query: harry potter

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 19

Judgement: the first ASIN is a Substitute, as it is a book inspired by the Harry Potter Series. The second
ASIN is a Complement, as it is a study guide for one of the books (it isn’t about the same topic of the
Harry Potter Books, instead it’s about one of the Harry Potter books itself).

Rule of thumb! When in doubt between an ASIN being a Substitute or a Complement for a query, err
on the side of Substitute, because Complements are a well-defined category of books and exist in
much lower numbers than Substitutes.

3.5 Cross-PT queries

In these ESCI guidelines, we address judgement of cross-PT query results by assuming that customer are
looking for media products whenever the keyword has an Exact match with a media work (in music,
films/TV, books) and products from non-media GLs are related to that work. Therefore, in the case of
title/franchise searches:
all the media products that exactly match the keywords and are either the main work(s) in
the original medium or other media derived directly from the original medium (e.g. film
adaptation, screenplay, etc.) are Exacts. Conversely, media products about the original
work(s) are Substitutes.

Let’s look at a famous example to clarify:

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 20

Query: “harry potter”

Judgement: all Harry Potter movies and books are Exacts as they are either the primary work (the
books) or direct media derivations of the primary work (the film transpositions). All non-media results,
e.g. merchandising, are Substitutes.
Important! Not all media products are Exacts: a cookbook inspired by Harry Potter is not the primary
work identified by the keyword and therefore is a Substitute. Also, the Harry Potter movie Soundtrack
and music scores are Substitutes.

A different subset of cross-PT queries is represented by searches that use proper nouns as keywords.
These include people (artists, writers, historical figures), fictional characters and brand names.
● For people names (historical figures, celebrities, authors) and fictional characters:
o any media product authored by or about the person is an Exact
o merchandising and related products are Substitutes
● For brands and characters that are not primarily associated with a media work:
o any media or non-media product matching the customer query is an Exact
This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 21

o similar characters and brands are Substitutes (refer Amazon-wide guidelines by ASTRIA)
Query: “ronald reagan”

Judgement: Ronald Reagan was a US president, an actor and a books author. Therefore, all books
authored by him are Exacts. All books about the Reagan presidency, the Reagan family, etc. are Exacts.
All movies featuring Ronald Reagan as an actor are Exacts. All movies/TV shows about the Reagan
presidency, the Reagan family, etc. are Exacts.
T-shirts and other merchandising are Substitutes.

Query: “barbie”

Judgement: Barbie is a toy brand, however Exact results exist across books and video and are part of the
brand product line. All toys, movies/tv/books are Exacts.

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 22

3.6 Miscellaneous queries

Format Wrong:
Explanation: When query or store is specific to a book format and the ASIN is exact titled book of
different format, then it would be substitute due the format wrong.

Query Category Exact Substitute Complement Irrelevant

Kindle https://www.amazon.com/dp https://www.amazon
https://www.amazon.in/d /B09N7PYXNT .com/dp/054516207
p/0316311294 6
https://www.amazo
n.com/dp/B09DD17
The H3N
expanse
book
series
kindle
(entire book series in (different
kindle edition) genre/subject)
(hardcover format, not
kindle); (summary of a book from the
series)

https://www.amazon.com
/dp/B0971RBRV8

(one of the books from

series)

Strip https://www.amazon
books .com/dp/B09QQSNSJ
V
https://www.amazo https://www.amazon.com
https://www.amazon.com/dp
n.com/dp/B08GJZFB /dp/B075WTH7WM
/B08P4X37T5
YV s
A
Promised
Land
Hardcover

(summary of the book)

(Biography genre matches)
(different
genre/subject)

Language difference:
Query Exact Substitute Complementary Irrelevant

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 23

Harry Potter and Harry Potter and the Study Guide a Student Workbook Spanish Short
the prisoner of prisoner of Azkaban for Harry Potter and the Prisoner Stories for
Azkaban ASIN: B017V4NTFA of Azkaban Beginners
1973704110 ASIN: B097SN9G81

Theme based:
Query Exact Substitute Complementary Irrelevant

The Cancer Code: A Your Brain After Chemo This is my Cancer Sucks A Brain Tumor Changes
Cancer Revolutionary New ASIN: B06XCRYHVB Coloring Book: A Self Everything
Understanding of a Affirming Cancer Fighting ASIN: 1725287498
Medical Mystery Activity Book for Cancer
ASIN: B084RGT3H4 Warriors
ASIN: 1987596528

Ambiguous queries:
There are some ambiguous in nature. While service or membership seeking queries such as “cancel
kindle unlimited” continue to be disregarded, however, keywords (and functional equivalents): “kindle
unlimited”, “prime reading”, “audible plus”, “kids plus”, “comixology unlimited” all contextually hint
towards a customer searching for books and thus must be considered for judgement accordingly.
E.g. while the SQ: ‘kindle unlimited’ could entail all books to be marked as Exact, ‘audible plus’ hints at a
particular format of book and thus must be provided the relevant judgement, distinguishing format.

1. When customer is looking for a service offered by Amazon, its to be considered exact for
instance -Kindle Unlimited [Query "Kindle unlimited" should return books available within the
program, as identified by the badge in search]

2. When the SQ is just a letter for instance A, it can be considered Other unless the letter has some
specific meaning. R is programming language, in this scenario books related to programming
language R will be marked EXACT and related to other programming languages can be marked
SUBSTITUTE.

Media Vs non-media/merchandise Products:

All the media products that exactly match the keywords and are either the main work(s) in the original
medium or other media derived directly from the original medium are Exacts. Conversely, non-media
products about the original work(s) are Substitutes.
E.g. all Harry Potter movies and books are Exacts as they are either the primary work (the books) or
direct media derivations of the primary work (the film transpositions).
However, all non-media results, e.g. merchandising (https://www.amazon.com/s?k=harry+potter),
are Substitutes.
Similarly for people names (historical figures, celebrities, authors) and fictional characters – any
media product authored by or about the person is an Exact. Whereas, merchandising and related
products are Substitutes

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.
 24

5. Appendix
5.1 Books Query Sub-Intents

Column 1 Column 2 Column 3

Intent Sub-intent Example
Title spearfishing
title+author https://www.amazon.com/s?k=terry%20james%20lawless
series https://www.amazon.com/s?k=laura+chambers
collection https://www.amazon.com/s?k=my+america+books
series https://www.amazon.com/s?k=darth+vader+volume+3
spearfishing
multiple https://www.amazon.com/s?k=accidentally+in+love
authors
cross-GL https://www.amazon.com/s?k=american+gods
ISBN https://www.amazon.com/s?k=978-1880684924
Author author https://www.amazon.com/s?k=sandra+bonyton+books
publisher https://www.amazon.com/s?k=openstax+math
Broad topic https://www.amazon.com/s?k=augmented+reality+book
category https://www.amazon.com/s?k=maps+of+national+parks
no matches https://www.amazon.com/s?k=creating+your+own+career
children https://www.amazon.com/s?k=elephant+story+books+for+kids
genre https://www.amazon.com/s?k=mystery+book+authors
bible https://www.amazon.com/s?k=giant+print+kjv+bible
price/free https://www.amazon.com/s?k=prime+free+books
age https://www.amazon.com/s?k=coloring+book+for+12+year+old+girls
bestsellers https://www.amazon.com/s?k=new+york+times+best+sellers+2018
reading https://www.amazon.com/s?k=geometry+workbook+grade+5
level
religion https://www.amazon.com/s?k=studying+the+old+testament+cotton

This material cannot be shared with third parties without TELUS International written consent.
DO NOT COPY. DO NOT DISTRIBUTE.