Lesson 1 – Getting started with

Terminology
What is Artificial Intelligence?

Artificial intelligence, often abbreviated to AI, is the largest, broadest

category of tools in this space. It is a sub-discipline of computer science,
and as an academic discipline, it’s been around since 1955-- perhaps not as
new as you might think!

Artificial intelligence (AI) is a discipline of computing science that studies

and builds systems to complete tasks we typically associate with cognitive
functions – such as reasoning, strategizing and problem-solving, without
requiring an explicit solution for every variation.

These algorithms, processes and methodologies allow a computer system to

perform tasks that would normally require advanced intellect. AI can appear
as a component in a larger system, a computer application, a digital
agent or an autonomous machine.

The field of AI has both data-driven approaches- such as machine learning-

and non-data-driven approaches such as search, logic systems, constraint
satisfaction, and optimization.
What is Machine Learning?

Machine learning is only a part of AI, and right now it’s the biggest part
that’s being adopted into industry.

Machine learning is a set of computational techniques that use data to create

models that make predictions about future data. These models independently
learn and continuously adapt to changing environments without being
explicitly programmed. Machine learning is a crucial component in many
artificial intelligence systems.

Industry is particularly interested in adopting applied machine learning,

and investing in advanced research in the field, because of the focus on
using historical data to inform future opportunities for systems
improvement, discoveries, and augmenting human-cognitive capacity.
Artificial Intelligence (AI)
Artificial intelligence is a discipline of computing science that allows a system to
complete tasks we typically associate with cognitive functions – such as
reasoning, strategizing and problem solving, without requiring an explicit
solution for every variation.
Machine Learning (ML)
 Machine learning is a set of computational techniques that use data to create
models that make predictions about future data. These models independently
learn and can be made to continuously adapt to changing environments without
being explicitly programmed. Machine learning is a crucial component in many
artificial intelligence systems.
More Terminology
There's lots of other related terms and areas when we talk about AI and ML.
Let's look at some common ones below.

DATA SCIENCE
Data science is a field of study that combines domain expertise, programming skills, and
knowledge of mathematics and statistics to extract meaningful insights from data. Data
science practitioners may use machine learning algorithms to generate insights which
analysts and business users can translate into business value. The focus of data science
is extracting, explaining and interpreting insights from data.

DEEP LEARNING
Deep Learning is a subfield of machine learning focused on algorithms inspired by the
structure and function of the human brain, called artificial neural networks.
The advancements of deep learning are rapid today because we now have fast enough
computers and enough data to train large neural networks.

The key advantage of deep learning is scalability. As we construct larger neural

networks and train them with more and more data, their performance continues to
increase. This is generally different from other machine learning techniques that reach a
plateau in performance.

Organizations looking to adopt these techniques will generally leverage a large, existing
dataset in their space- such as ImageNet.
AUTOMATION
Automation is a term for technology applications where human input is minimized. This
includes business process automation (BPA), robotic process automation (RPA),
personal applications such as home automation, and more.

AI can be used to support automation, but not all automation involves AI.

Most automation that uses machine learning today does so in an assistive manner,
providing recommendations to humans who decide whether to trust them or not and
then take action.

ROBOTICS
Robotics is the interdisciplinary study of science and engineering dedicated to the
design, construction and use of mechanical machines, called robots which attempt to
replicate human actions.

Similar to automation, you can have robotics with or without AI.

Lesson 2 - The AI Revolution
1760-1840 - The Industrial Revolution
AI and ML are the continuations of a larger pattern of human history; the
industrial revolution changed society in many ways, and one of the major things
was the dramatic reduction in the cost of manufacturing. This reduced cost
allowed for a new level of scalability- spawning new industries and jobs.

In the picture below, the Jacquard Loom automated fabric production with
unlimited patterns and spawned a fashion industry that couldn’t have existed
before! Interestingly, the loom used replaceable punched cards to control its
sequence of operations, which is also considered an essential step in the history
of computing hardware.

1947 - The Digital Revolution

The digital or computing revolution in the mid to late twentieth century
significantly reduced the cost of computation and created new jobs and
industries over time. For example, the software development industry, the
computer gaming industry, and the Internet would not have been possible
without this revolution.

We are still seeing advancements from this revolution today.

1990s - The AI Revolution
91% of tech executives and 84% of the general public believe AI constitutes the
next technology revolution (source).

The AI revolution is in its very early stages, but it’s reasonable to believe that it
will follow the same historical pattern- reducing the cost of prediction.

Let's examine the three major factors contributing to making AI

applications possible for organizations today. (source)

1. Increased computing power

– Improvements in computing power and capacity. Today’s processors can be 40 to 80
times faster than the quickest versions available in 2013; cloud solutions offer much
cheaper computing and storage services on demand.
2. Exponential growth of data
– The world creates an unprecedented amount of data every day, feeding algorithms the
raw material needed to produce new insights. The enormous diversity in the data being
generated means that organizing and analyzing these data are extremely challenging.
Still, there is an unprecedented opportunity to extract value from data that were not
available in the past.
3. Progress in algorithms
– The techniques and algorithms underlying AI have continued to be developed. Recent
advances are delivering increases in the accuracy of classification and prediction.
Lesson 3 - AI Hierarchy of Needs
The foundation of all machine learning projects is data.

1. IDENTIFY & COLLECT

The foundation of most AI projects (particularly machine learning projects) is data, and
the first step is identifying and gathering the relevant data from various sources. Data
that is relevant to the problem needs to be collected and accurately reflect the ground
truth. Furthermore, it needs to encompass all the diversity and variety that will be
present in the operational environment.

Any data you collect and use should be done with the proper consent, privacy, and
regulatory considerations. Data can be collected from:
• internal sources such as logs and customer data
• sensors or other instrumentation
• data mining and scraping the web
• external sources such as StatsCan, and many more.

2. MOVE & STORE

Data usually needs to be moved and made accessible to the AI system. This
includes combining data from various sources, of different types and
timelines, into a single source.

The infrastructure must be in place to move the data from where it is

generated to where it will be cleaned and used. Data storage infrastructure
must also be in place and processes to ensure that the data is accessible and
in a usable format.

It would be best to consider your problem's storage needs for future use and
reference. For example, you'll likely have different requirements if the data
is needed in real-time or if it can be stored and used in batches.

3. EXPLORE & TRANSFORM

Data needs to be fully understood and cleaned. High-quality AI systems can
only be made from high-quality data. Data exploration, transformation and
understanding require both technical infrastructure and domain expertise
to accomplish. Clear business strategy and objectives must also be in place
to provide the context and purpose for the work.

Data cleaning involves removing or updating incomplete, incorrect or

otherwise biased data samples. It also involves standardizing the data to
ensure appropriate representation. In addition, data from various sources
will need to be aligned.
 4. SELECT & LABEL
Decide which data points are essential for your problem and how you will measure that
importance.

One of the most important parts of your data is the answer to the prediction question
you are asking your AI system. The label is the answer to your question- for example if
your problem is to identify spam email- the label for each email is whether it is spam or
not- this needs to be included in the data you will use for training your model. Some
data is labeled during collection or as part of the system, and other data needs to be
manually labeled (preferably by experts).

Selecting the data components that are relevant to your problem can be an iterative
process- you want to have the minimal number of components for faster processing but
enough to get accurate and reliable results. In addition, explore, Transform, Select and
Label steps will need to be performed on new operational data when you deploy your
system, so be sure to capture these processes.

Aggregate and Label - This step also requires both technical infrastructure and domain
expertise. Labels for classification or regression need to be both relevant and valid, and
proxies and metrics likewise need to be relevant and valid. Data labeling processes that
are efficient and have measures and checks for accuracy need to be in place.

5. LEARN
The first step in AI is to train a baseline model with a standard algorithm. This will
give you an idea of how well your data represents the problem, as well as the level
of technical feasibility.

6. OPTIMIZE
Then experiment and iterate in order to try and improve your model. Try various
algorithms and parameters and make alterations to your data transformations and
selection.

7. AI
The output of the optimized level is a model that meets both your business and
technical requirements. An AI system is not set-it-and-forget-it and will need
ongoing observation and maintenance.
To read more about the AI Hierarchy of Needs, check out the original article
here: https://hackernoon.com/the-ai-hierarchy-of-needs-18f111fcc007

ML Project Timeline

In the timeline of an AI project, you may find it surprising that the machine
learning modelling component is a relatively small amount of your time.
Instead, you will spend most of your time on data tasks- collecting,
understanding, cleaning and merging your data in preparation for
modelling.

If stakeholders' do not understand the importance of setting the data

foundation of their machine learning projects, they may become frustrated
with the amount of time spent on these steps.
Lesson 4 - The 5Ps: How AI Generates
Value
We identify these five common ways AI can be applied to get measurable value for
your organization. Let's take a closer look!

1. Project
As companies grow in technology and customer base, they usually face one

challenge: big data. Traditional forecasting methods work well with small

data. However, we need new methods to address the volume of data today.

Machine learning allows you to combine data, expertise, and effort to

generate more accurate projections, reducing noise in your forecasting

models.

Machine learning can help you more accurately forecast supply and

demand, allowing for better planning, reduced costs and increased profits.

Let's look at some use cases of Project

HONESTDOOR
A company using projection here in Amii's home city of Edmonton.
HonestDoor offers a daily property value estimate for residential properties using a
proprietary machine learning model.
HonestDoor uses land, property, mortgage and forecast data to predict property prices
that update daily, giving customers transparency in their real-estate purchasing
decisions.

CHINA AGRICULTURE
"BEIJING, April 20 (Xinhua) -- Winter wheat yields in north China can be better
predicted by a new hybrid forecast model powered by machine learning, according to a
study by the Institute of Atmospheric Physics under the Chinese Academy of Sciences.

China is the world's largest wheat producer and consumer. As such, accurate yield
forecasting is a top concern for researchers. The new hybrid model proposes an
approach that combines machine learning and dynamical atmospheric prediction.
Developed by Chinese and American scientists, the new model was applied to north
China over the sub-seasonal-to-seasonal period.

As an emerging statistical model, machine learning can better describe the non-linear
relationship between input and forecast and has obvious advantages in yield forecast
compared with a linear model.

The study results indicate that the hybrid model generally outperforms conventional
models, with one metric that tells how far the prediction values are from the real values,
decreasing by 30 percent to 55 percent compared with conventional models.

The results also showed that the new model achieved the best prediction three or four
months before the harvest season.

The study demonstrates that the coupling of machine learning and dynamical
atmospheric prediction is a useful tool for yield forecast, which could provide support to
agricultural practitioners, policy-makers, and agricultural insurers.
The study was published in the journals Remote Sensing, Weather and Forecasting, and
Atmospheric and Oceanic Science Letters. "
Published
at: https://english.news.cn/20220420/bb466c241b454a8eb62f510d6a8e3449/c.html

2. Produce
Artificial intelligence can also add value to your business by streamlining

and improving the production process.

By maximizing the time systems run, reducing costs, and increasing the

output potential, machine learning can help you improve your bottom

line. There are lots of applications of machine learning in production.

Let's look at some use cases of Produce!

DOT
DOT is a farm automation company out of Regina, Saskatchewan. They created an
autonomous diesel-powered tractor platform that is adaptable for many types of farm
equipment.

Using artificial intelligence, they were able to reduce the time spent by farmers on
equipment, increasing their productivity in other areas.

DOT was bought out by Raven Autonomy in South Dakota in 2017.

Dot Technology Corp (Dot) CEO Rob Saik explains how the Dot-ready planter prototype was
designed using SeedMaster Manufacturing technology and the row crop e...
VIEW ON YOUTUBE

BIO-STREAM DIAGNOSTICS
For the past two years, an Edmonton-based biotech company Bio-Stream Diagnostics
has been working on a new approach to COVID testing. By combining advanced
biosensors with artificial intelligence, their work has the potential to be faster, cheaper
and more accurate than current rapid tests. Watch more:
VIEW ON YOUTUBE
3. Prevent
We can use machine learning to make predictions about risks and anomalies

in order to make smart decisions at critical moments. This prevents costs

and poor outcomes.

Let's look at some use cases of Prevent!

Explore the tabs below for real examples of organizations using machine
learning to anticipate and proactively prevent problems.

ZEST AI + DISCOVER
Discover, a leading U.S. consumer lender, turned to ZestAI in 2018 to help build one of
the largest AI-based credit scoring solutions in the financial services industry. Zest data
scientists and analysts worked closely with Discover to tap its trove of consumer
spending data to build a new model for its $7.5 billion personal loans business. Its new
ZAML model assesses hundreds of applicant data points,
up to 10 times more than Discover’s credit model had used before to get a more
accurate view of borrower risk.

In the new machine learning model, for example, a history of discount-store shopping
will boost an applicant’s chances of getting a personal loan, while an applicant writing
the full legal name of an employer on a loan application will lower it. In addition,
applicants who call Discover from a landline or cellphone, rather than Skype or other
internet-phone services, will be considered safer bets because they’re easier to trace
back to an individual.

Source: Wall Street Journal

OKAKI
Amii has helped Calgary-founded public health informatics company OKAKI plan and
executes an AI project for opioid risk prediction.
Read more: https://www.amii.ca/latest-from-amii/okaki-opioid-case-study/
How did they make this significant leap? Educating its workforce. Working with Amii, OKAKI
educated its entire management and core technical team on the language and concepts behind
AI. This allowed the company to create a shared lexicon within and between teams. Learning
how to ask the right questions.

4. Promote

This is probably something you are familiar with in your own lives- targeted

advertising is one of the main value functions here! And we see it

happening more often and more accurately all the time- I’m sure you’ve

started shopping for a particular item and then notice as the ads around you

start adapting to that new interest. ML can also help organizations with

setting and adjusting prices and tailoring offers.

Let's look at some use cases of Promote!

LOKNOW
LoKnow is an Edmonton marketing-tech and data company. Listen to Nolan talk about
how they are using unsupervised machine learning for applications in advertising.

HUDSON'S BAY COMPANY

The infamous Canadian department store, Hudson's Bay Company (HBC), wanted to
reach more people with their display advertisements. So they leveraged a fully
automated solution that uses machine learning to optimize creatives, bids and audience
targeting called Smart Display.

The system mixes and matches assets added by HBC to create a variety of unique ads,
including headlines, descriptions and images. Their implementation saw an increase of
40% in revenue over their manual methods and a 43% increase in conversions.

Source: https://www.thinkwithgoogle.com/intl/en-ca/marketing-resources/data-
measurement/marketing-machine-learning-how-brands-canada-can-get-
started/(opens in a new tab)
5. Personalize

The final way we see business value-added through AI is

personalization. Today’s consumers want speedy response time, and in this

age of information overload, they want relevant customized experiences.

Let's look at some use cases of Personalize!

NETFLIX
Recommendation systems are a common application of machine learning for
personalization. Netflix uses them to personalize an individual experience, from which
shows to recommend, to which images to display for a particular show and more!

75% of what consumers watch on Netflix comes from product recommendations based
on machine learning algorithms.

Check out other cool applications of machine learning happening at

Netflix: https://research.netflix.com/research-area/machine-learning

We invest heavily in machine learning to continually improve our member experience and
optimize the Netflix service end-to-end. As researchers, we innovate u...
VIEW ON YOUTUBE
INSURANCE: HUK24
A German insurance firm is using an AI recommendation engine to personalize
insurance sales and services. Unlike the potentially inconsistent intuition of a human
agent, the insurance engine can use advanced analytics to parse the vast amounts of
existing data to deduce what individual consumers need.
With permission, the engine can also consider the customer’s unique characteristics,
such as web history and device usage. This information can then feed into product
recommendations and AI-based chatbots ready to answer questions. For example, when
a price-conscious customer is shopping for insurance, the engine can prioritize offerings
such as usage-based insurance or telematics offerings that reward safe driving.

Source: https://www.mckinsey.com/industries/financial-services/our-
insights/revolutionizing-insurance-the-personalized-insurance-engine
Lesson 5 – Refining your AI Problem
Working Backwards

A common problem-solving strategy is to work backwards- consider the

ideal future state you are looking to achieve and then find the path to get
there. We apply this same process to identifying and scoping AI problems.

At this point we are focusing on the business of machine learning rather than
the science. Often organizations mis-step by trying to take a machine
learning solution and fit it onto their data or business, instead we want to
align machine learning solutions with organizational problems and
objectives.

What makes a good question?

A good question for machine learning has three properties:

1. It's about a single data point: a property in the previous example, but it could
also be a product, a person or a component.
2. It's specific and well-defined: our question in the previous example was very
narrow. What is the fire risk of a single property for this year?
3. Asks for a prediction, not an explanation: machine learning is used to predict
an unknown value based on the patterns it finds in the data it has seen, so be
sure you are asking for a prediction.

Take the following quiz to test your knowledge on identifying good machine
learning questions.
Is this a good machine learning question:

What impact are delinquent accounts having on our business?

Yes
No
Lesson 6 - AI Red Flags

How can you tell if you’ve gone wrong along your problem scoping journey
or in the execution of your AI solution? Just as it’s important for us to
understand how AI can be used, it’s also imperative to understand its
limitations.

AI projects are iterative and experimental- there is an expected cyclical

nature of adjusting, testing and evaluating. So how do you know if you
picked the right project? How do you know if you are making progress or if
you need to adjust the course?

Let's take a look at some common AI 'red flags'- or potential pitfalls- that
organizations can run into when working on AI projects.

Watch out for Red Flags!

AI systems are very powerful, but they are also expensive and require a lot of expertise.
If your question can be answered by standard analytics, such as statistics and reporting,
you should use those methods instead.

1. Problematic data: All data is biased, as all systems and their institutions are
biased. When the data you are using is irrevocably biased toward a particular
outcome and there’s no way to compensate, in the bias toward that outcome, you
should consider reformulating your problem or using solutions other than AI.
 2. Unsound functional mappings: When the mapping from input to output is not
a valid association. Just because you can get an AI system to map an input to an
output, doesn’t mean that the connection is actually valid.

3. Subjective metrics: When a chosen metric(s) are more subjective than

objective. Is the metric about opinion? For an AI system to come up with a
good answer, they need to know what a good answer looks like. If the good
answer (and how you get to the good answer) is very subjective and prone to
biases, the system is going to replicate (and likely amplify) the biases and
subjectivity that it was trained on.

4. Unclear Metrics: When key metrics are unclear, multiple, and/or conflicting. Is
context needed to clarify a metric intent? Metrics can also often have unclear or
multiple definitions that need context to clarify the meaning and purpose of the
metric, something that can be difficult to capture adequately with an AI system.

5. Optimizing for a single metric: When the system maximizes one aspect of the
problem, sacrificing other important factors. This is not a problem just in AI.
There are countless examples from other contexts in which optimizing for a
single metric caused serious problems. It pretty much always results in things
being sacrificed that shouldn’t be, people gaming the system, runaway feedback
loops, and so on.

Learn from the mistakes of others. You can't live long enough to

make them all yourself.

Eleanor Roosevelt

I found a red flag!

We can't anticipate every pitfall, but we can learn from others' mistakes.
Let's examine some examples where these AI red flags occur.

When standard analytics are enough

To understand when standard analytics is enough- read the following short
article by Cassie Kozyrkov, who succinctly and entertainingly explains when
using machine learning is not a good idea with a toy example.

Imagine that you've just managed to get your hands on a dataset from a clinical trial. Exciting! To
help you get in character, I made up some data for you to look at: Pretend that these datapoints
map out the relationship between the treatment day (input "feature ") and the correct dosage of
some miracle cure in milligrams (output "prediction ") that a patient should receive for over the
course of 60 days.
READ MORE MEDIUM

1. Problematic data

AI Essay Scoring Engines

In a 2019 survey conducted by Motherboard, it was found that Natural

language processing (NLP) artificial intelligence systems—often called

automated essay scoring engines—were either the primary or secondary

graders on standardized tests in at least 21 US states.

Of those 21 states, three said every essay is also graded by a human, and 18

said only a small percentage of essays are randomly selected for a human

grader to double-check.

"Essay-scoring engines don’t actually analyze the quality of writing.

They’re trained on sets of hundreds of example essays to recognize patterns

that correlate with higher or lower human-assigned grades. They then

predict what score a human would assign an essay, based on those

patterns."

In various studies, one vendor (Educational Testing Service (ETS)) found

that its engine gave higher scores to some students, particularly those from

mainland China, than expert human graders. It also tended to underscore

African Americans, Arabic, Spanish, and Hindi speakers—even after

attempts to reconfigure the system to fix the problem.

“If we make an adjustment that could help one group in one country, it’s

probably going to hurt another group in another country,” said Brent

Bridgeman, a senior ETS researcher."

This is an example where the data is problematic and making adjustments

only changes the problem, rather than resolves it.

Source: https://www.vice.com/en/article/pa7dj9/flawed-algorithms-are-grading-

millions-of-students-essays(opens in a new tab)

2. Unsound functional mappings

Pictures → Criminals?

A research team from Harrisburg University wrote a paper claiming that

their facial recognition system could predict criminality from a facial photo.

This team successfully mapped the input of facial photos to predictions of

criminality. Furthermore, they claimed it to be 80% accurate and unbiased

against race.

The underlying theory of this project - that there is a valid correlation

between facial features and criminality - is really, really unsound.

There were initial statements that this paper was accepted for a Springer

anthology, but after an online open letter signed by dozens of academics

(and others) protesting this project, Springer denied that it had ever been
accepted, and the paper was withdrawn from all venues. This was in early

2020.

Source: https://www.secureidnews.com/news-item/face-recognition-fact-from-

fiction/#(opens in a new tab)

3. Subjective metrics

Emergency Room Triage

Different physicians have different approaches to how prioritizing patients.

It seems objective, but subjective subtleties are involved, including

physicians' training and experiences.

If we choose physician notes as our method for constructing a metric for

triaging, it’s likely the metric will be subjective, as physicians are.

Source: https://www.huffingtonpost.ca/alan-katz/emergency-room-waits-

canada_b_16132058.html(opens in a new tab)

4. Unclear metrics

Route Planning

When we open up our favourite route planning application, we are usually

looking for the best route to get us from point A to point B.

But what exactly does the "best" route mean? It could be the shortest time,

the shortest distance, the fewest turns or traffic lights, or the route that

takes us past our favourite coffee shop.

AI systems need a clear definition of the metrics to use, and sometimes

those definitions are not the same for everyone.

5. Optimizing for a single metric

Bing Search Engine

Microsoft found that the worse the search results from the Bing search

engine returned, the higher the advertising revenue.

“The worse the search results returned by the Bing search engine, the

researchers concluded, the higher the ultimate ad revenue. This makes

sense when you consider that poor results will drive people to keep

searching, providing the search engine with more chances to show

advertisements.

“But obviously, that’s a terrible long-term strategy and it would lead to

worse revenue over the life of the product,” said Clark.

Source: https://www.datanami.com/2018/02/20/getting-metrics-right-important-

machine-learning/(opens in a new tab)

Further reading
If you are curious if you are off the right start with your AI project and want
to evaluate it further- take a run through the checklist here!
Lesson 7 - The Machine Learning Process
Lifecycle Framework
The Machine Learning Process Lifecycle (MLPL)

What is it?
Machine Learning (ML) has been experiencing explosive growth in
popularity due to its ability to learn from data automatically with minimal
human intervention. As ML is implemented and applied more in business
settings, ML practitioners need to develop methods to describe the timing
of their project work to their employers or clients.

Our tool for achieving this is the ML Process Lifecycle, a framework adapted
by the Amii team leveraging already-existing knowledge from
organizations like Microsoft, Uber, Google, Databricks and Facebook.

The MLPL is a technology-independent framework that is abstract enough

to be flexible across problem types and concrete enough for
implementation. To fit our clients’ needs, we’ve also decoupled the
deployment and exploration phases, provided process modules and defined
key artifacts for each stage. The MLPL ensures we are able to capture
learnings throughout the process.

The ML Process Lifecycle (MLPL) is a framework that captures the iterative

process of developing an ML solution for a specific problem.

ML project development and implementation is an exploratory and

experimental process where different learning algorithms and methods are
tried before arriving at a satisfactory solution. The journey to reach an ML
solution that meets business expectations is rarely linear – as an ML
practitioner advances through different stages of the process and more
information is generated or uncovered, they may need to go back to make
changes or start over completely.

The MLPL tries to capture this dynamic workflow between different stages
and the sequence in which these stages are carried out.
Why do we need it?
There are a few important reasons why an organization should use the
MLPL:
• Risk Mitigation: The MLPL standardizes the stages of an ML project
and defines standard modules for each of those stages, thereby
minimizing the risk of missing out on important ML practices.
• Standardization: Standardizing the workflow across teams through an
end-to-end framework enables the users to easily build and operate
ML systems while being consistent and allowing inter-team tasks to
be carried out smoothly.
• Tracking: The MLPL allows you to track the different stages and the
modules inside each of the stages. As an exploration task, there are a
lot of attempts that will never be used in the final ML solution but
have required significant investment. The MLPL allows you to track
the resources that have been spent on these experiments and to
evaluate for future iterations.
• Reproducibility: Having a standardized process enables an
organization to build pipelines for creating and managing
experiments, which can be compared and reproduced for future
projects.
• Scalability: A standard workflow also allows an organization to
manage multiple experiments simultaneously.
• Governance: Well-defined stages and modules for each stage will help
in better audits to assess if the ML systems are designed appropriately
and operating effectively.
• Communication: A standard guideline helps in setting the expectations
and effectively facilitates communication between teams about the
workflow of the projects.
Four Stages of the MLPL
 Now that we understand what the ML Process Lifecycle (MLPL) is and why
it’s important, we will take a look at the framework itself and go through
the key aspects of each stage.

Step 1
Stage 1: Business Understanding & Problem Discovery

This stage is for formulating and scoping your business problem.

Tasks in this stage include:

• Identifying the business requirements.

• Scoping & defining a problem that can be solved using ML.
• Identifying how your ML solution will resolve your problem.
• Determining if the idea is feasible for your organization.
• Identifying the value of the solution for your organization.

Step 2
Stage 2: Data Acquisition & Understanding

This stage is for exploring the data you have available and identifying the
possibilities and restrictions for its use in ML.

Tasks in this stage include:

• Determining if you can answer your ML question using this data.

• Identifying if you need to acquire different or additional data.
• Iterating on your problem and data until you have alignment- i.e. a
ML question that can be answered with the data you have or will
obtain.
Step 3
Stage 3: ML Modelling & Evaluation

This stage is for obtaining results & assessing the viability of our solution.

This is where the ML algorithms come in. Organizations often think they
need to start at this stage, but it's critical to get the first two stages right
before you start modelling.

Tasks in this stage include:

• Identifying your metrics of evaluation.

• Creating a baseline model.
• Evaluating your model.
• Experimenting with adjustments to create additional models.
• Compare model performance and evaluate again the problem scope.
 • Communicate results, refine and iterate

Step 4
Stage 4: Verification & Validation

This stage is where we validate if the ML problem is addressing the initial

business problem. An ideal project should arrive at this stage only once, but
given how quickly a project evolves for various reasons, there is a possibility
that this stage may have to be revisited. Good communication among all the
stakeholders and clarity in the problem definition will minimize the amount
of times this stage will need to be visited.

Tasks in this stage include:

• Verifying agreement on the level of performance.

• Validating the solution meets our organizational objectives.
• Documenting and communicating the results to key stakeholders
Summary

At the completion of the MLPL, you should have a well-performing ML

model. From here, an organization may decide to develop this model into a
product or service as a tangible component, deploying it into production
and maintaining it. This phase is not captured in the MLPL.

Read more on our blog: https://www.amii.ca/latest-from-amii/ml-process-lifecycle-

part-2

Iterate & Experiment

Unlike most other projects, machine learning projects are highly iterative
and may require a lot more experiments and revisions than expected. It's
important to know this before you get started so everyone is on the same
page with expectations.

In an idealistic world, your project would be very linear:

But the reality will often involve a few (or a lot) more lifecycle switches.

You want to make better product recommendations to your customers on your e-

commerce site.

Match the task with the correct MLPL stage.

 Question

• You create a data warehouse with the sales transactions from the last 2 years.
• You measure the success of recommendations based on if the customer buys it
within 2 days.
• You beta test your model on customers and it achieves 53% accuracy.
• You present the new recommendation model for the shoe department to the
staff.
Lesson 8 - How it Works
It may seem intimidating to learn how machine learning works. Let's walk
through an example scenario to distill the main concepts without getting into the
technical details.

It's not possible to call every alumni- there are too many, and you have limited staff and
time. What factors would help you decide who to call?
1. Income, graduation year, program of study, previous donations
2. I would pick randomly

After going through the scenario, you should have a better understanding of
the main principles behind machine learning, such as:

• machine learning algorithms analyze examples of your data

to build mathematical models.
• Those models exploit connections between each factor and
the desired output (such as income and donation or
donation amount).
• Machines can consider and weigh many, many more factors
than humans can.
• The model can be used to predict the output for new
data (probability of donation or donation amount).
The Machine Learning Process

We can formalize the machine learning process in the following graphic.

On the left side, we have the technical components, or the science of,
building our machine learning model. On the right, we have the integration
of the organizational processes and the business of machine learning to
achieve an ongoing competitive advantage from our model.
Lesson 9 - Types of Machine Learning
There are three general categories under the umbrella of machine learning-

supervised learning, unsupervised learning and reinforcement learning.

Before we can take a closer look at each of them, we need to understand the

concept of labeled data.

What is labeled data?
To understand what a label is, we first need to understand the term feature.
A feature is a specific characteristic of a data point. First, let's look at some
examples of features.

Recall the example of calling university alumni for donations:

An image of an example dataset with nine icons of a generic human bust, each one with
their first name, graduation year, whether or not they previously donated and their
annual income range.

We can also look at this data in another format:

A table version of the same dataset of people with their first names, graduation year,
whether or not they previously donated and their annual income range.

If we look at each row in the table, we see a single person and their
characteristics. So the row is sometimes referred to as a record or data
point. Each column contains a specific characteristic- also referred to as
the features of the data set.

A labeled dataset simply has this column completed for the feature that you
are interested in predicting. In this case, if we are trying to predict whether a
person will donate or not, we can say this dataset is labeled because we have the
"Previously Donated?" column complete for our dataset.

On the other hand, if we wanted to predict the amount that a person will
donate, this dataset is not labeled because we don't have the amount in our
features.
Let's look at another example
A classic example of machine learning is labeling images of cats. We have
the following data set of images:
A collection of images of small animals, including many cats, a fox, a rabbit, a dog and a
squirrel.

Is this dataset labeled to predict whether an image contains a cat or not?

Yes
No

A labeled dataset for the problem of classifying images as cats or not would
need to have extra data. Take a look at the new version of this dataset,
where a human has examined each imaged and tagged it as CAT or NOT.
The same image with animals, but now each image has the word CAT or NOT written on
it on a coloured background.

Now you know the difference between labeled and unlabeled data! Back to
the original conversation- the three types of machine learning.

Supervised learning.

First up, supervised learning. Supervised learning uses patterns in historical

data to predict labels. So now you see why we needed to learn about labeled

data.

The more accurate and robust the labels in your dataset, the better

predictions it can make.

Supervised learning can be used for two major

tasks: classification and regression.

Classification tasks predict labels that are a category or class- like spam or

not spam.

Regression tasks predict labels that are numeric- like this house is valued at

$456,000.

Some examples of supervised learning in the wild.

MOSQUITO IDENTIFICATION
Rapid and accurate identification of mosquitoes that transmit human pathogens such as
malaria is an essential part of mosquito-borne disease surveillance. The identification
of mosquitoes that transmit malaria can be difficult—some species are nearly
indistinguishable even to trained taxonomists.

Researchers have shown the effectiveness of using supervised machine learning

to classify mosquito sex, genus, species and strain.
Couret, et al. at the University of Rhode Island, applied a Convoluted Neural Network
(CNN) to a library of 1,709 two-dimensional images of adult mosquitos. The mosquitoes
were collected from 16 colonies in five geographic regions and included one species not
readily identifiable to trained medical entomologists.

Using the library of labeled species, the researchers trained the CNN to distinguish the
mosquitoes on various traits. They achieved over 98% accuracy on their various
classification tasks.

Source: https://medicalxpress.com/news/2020-12-artificial-intelligence-id-
mosquitos.html

AIRBNB
Airbnb's Smart Pricing tool helps owners predict the probability that a listing will be
booked at a given price on a given calendar date. This helps hosts price their listing
based on what they want to optimize (e.g., occupancy rate vs. price per stay).

Smart Pricing takes into account over 70 different factors in their price calculation.
These factors, plus host set controls, determine the best price for each available night.
Some factors Smart Price considers are:

• Lead-time: as a check-in date approaches, the price will update

• Market popularity: if more people are searching for homes in an area, the price will
update
• Seasonality: as we move into or out of high season, the price will update
• Listing popularity: if the listing gets a lot of views and bookings, the price will update
• Listing details: if amenities are added, such as WiFi, the price will update
• Bookings history: future prices are partly based on the prices of successful bookings. If
the host sets a price higher than Smart Pricing suggests and gets a successful booking at
that price, the algorithm will update to reflect that.
• Review history: prices update as you get more positive reviews from successful stays.

Source: https://blog.atairbnb.com/smart-pricing/

Match the example with the appropriate type of supervised learning.

Question

• An app to identify the plant type from an image.

• An e-commerce site predicting prices for their products.
• An e-commerce site recommending products based on other customers'
purchase similarities.

Match once more the example with the appropriate type of supervised learning.

• A car dealership is scoring their customers on long-term value into 1, 2, 3 or 4.

• A medical research company is predicting drug dosages for patients.
 • A design software that removes noise from visual data to improve picture
quality.

Unsupervised learning.

In unsupervised learning, the system examines data without labels. Instead,

it looks for patterns, relationships and anomalies.

They are usually used to find associations and groups or anomalies.

Clustering is a common task in unsupervised learning.

Recommender systems are a common application of Unsupervised

Learning. Another application of unsupervised learning is detecting unusual

or “out of the pattern” transactions- such as fraud.

Some examples of unsupervised learning in the wild.

COMPUTER SECURITY WITH CISCO
Check out the following video to see some examples of how unsupervised learning
techniques are used for anomaly detection.

Curious if machine learning in security is hopeful or just hype? Check out our video for some
guidance on what machine learning is, how it is helpful in se...
VIEW ON YOUTUBE

GAINING INSIGHT
A team at Lawrence Berkeley National Laboratory collected 3.3 million abstracts of
published materials science papers and fed them into an algorithm called Word2vec. By
analyzing relationships between words the algorithm was able to predict discoveries
of new thermoelectric materials years in advance and suggest possibilities for future
research.

Vahe Tshitoyan, lead author, said the project was motivated by the difficulty making
sense of the overwhelming amount of published studies. "In every research field there's
100 years of past research literature, and every week dozens more studies come out,"
he said. "A researcher can access only fraction of that. We thought, can machine learning
do something to make use of all this collective knowledge in an unsupervised manner—
without needing guidance from human researchers?"

They’ve released the predictions the system has made with recent abstracts, and made
their design available so anyone can train their own version for other disciplines.
Source: https://techxplore.com/news/2019-07-machine-learning-algorithms-uncover-
hidden-scientific.html

Match the example with the correct type of machine learning.

• You ask a child to sort toys, and they can do it based on any characteristics they
like.
• You ask a child to sort toys into stuffed toys, trucks and building materials.

Reinforcement learning.

Reinforcement learning (RL) systems learn from continuous

experience rather than labels or historical patterns. Learning happens

through interaction.

For every action, the system takes, a reward is given, and the system's goal

is to maximize the accumulated rewards.

Action control is a type of reinforcement learning where the system chooses

an action in response to feedback from the environment. This is used for

HVAC and electrical system control and can result in a significant increase

in efficiency. It’s also used in autonomous cars and game-playing systems

such as Google’s AlphaGo.

Some examples of reinforcement learning in the wild.

IMPROVING PROSTHESIS CONTROL
In the Bionic Limbs for Improved Natural Control (BLINC) lab, Amii Fellow, Dr. Patrick
Pilarski and his team are using reinforcement learning to smooth the interaction
between a person with an amputation and a prosthesis.

Some state-of-the-art devices currently used by people with amputations have over a
dozen possible grip patterns that can be manually selected by the user. A robotic arm
with so many available motions presents a problem since there exist more degrees of
freedom than there are available control signals from the human user. One solution to
this problem is for the user to switch between all available joints or grip patterns in a
predesigned, optimized order.

The BLINC team extended this idea of an optimized, fixed order by creating an RL
system, which learned to predict which joint a prosthesis user intended to actuate next,
which allowed the device to reorder the available joints or grip patterns adaptively for a
situation-specific switching list.

In the video below you watch and hear Rory Dawson demonstrate how this works from
time 4:59-7:02.

Edwards, Ann & Dawson, Michael & Hebert, Jacqueline & Sutton, Richard & Chan, K & Pilarski, Patrick. (2014).
Adaptive Switching in Practice: Improving Myoelectric Prosthesis Performance through Reinforcement Learning.
10.13140/2.1.4117.8880.

Rory Dawson speaks at The Tea Time Talks with his presentation "Adaptive Switching for
Improved Control of Robotic Prostheses"---The Tea Time Talks are a ser...
VIEW ON YOUTUBE

CAR PARK
Watch this simulation of a car learning to park. Unfortunately, in many reinforcement
learning applications, the system has no background on the task it's trying to achieve-
so it will perform very poorly while it learns what actions are working and which ones
are not.

"In this project, the AI is rewarded with small positive signals for getting closer to the
parking spot, which is outlined in red, and gets a larger reward when it actually reaches
the parking spot and stops there. The final reward for reaching the parking spot is
dependent on how parallel the car stops in relation to the actual parking position.
Suppose the car stops at a 90° angle to the actual parking direction, for instance. In that
case, the AI will only be rewarded a very small amount relative to the amount it would
get for stopping completely parallel to the actual direction. The AI is penalized with a
negative reward signal when it either drives further away from the parking spot or if it
crashes into any obstacles."

An AI learns to park a car in a parking lot in a 3D physics simulation. The simulation was
implemented using Unity's ML-Agents framework (https://unity3d.com...
VIEW ON YOUTUBE

Given data with labels, train a machine learning model to predict the label of a
new data point.
Given data without labels, train a machine learning model to identify a pattern
between data points.
Given information about the environment, learn to choose the best actions by
maximizing rewards.
Lesson 10 - Python Notebook: Supervised
Learning
Use this link to access the supervised learning notebook. Your access is read-
only, so you have to copy the file to your own personal drive before you can
use the file.