Google Cloud Computing Foundation Course

Evan Jones
Technical Curriculum Developer
Google Cloud

Lecture-79
Building Bespoke ML models

In this topic, you will take a high-level look at the complexities behind developing Bespoke
machine learning models. But how also AI platform Google's managed machine learning service
makes it easier for machine learning developers, data scientists, and data engineers to take their
machine learning projects from ideation to production and deployment.

(Refer Slide Time: 00:22)

Earlier, you are introduced to the idea that leveraging machine learning can split into three areas.
In this topic, you look at the more complex but also the most adaptable option of those three.

299
(Refer Slide Time: 00:36)

As a starting point, let us talk a little bit about TensorFlow. TensorFlow is an open-source, high-
performance library for numerical computation, not just about machine learning, any numerical
computation. In fact, people have used TensorFlow for all kinds of GPU computing. For
example, you can use TensorFlow to solve partial differential equations. These are very useful in
domains of like fluid dynamics. As a numeric programming library, TensorFlow is appealing
because you can write your own computation code in a high-level language like Python and have
it be executed in a very fast way that matters at scale.

300
(Refer Slide Time: 01:11)

TensorFlow works because you create a Directed Acyclic Graph or a DAG to represent your
computation. Directed means it has a direction of flow. Acyclic means that it can't feed into
itself, meaning that it isn't a circle and Graph because it has those nodes and edges. In this
schematic, those nodes represent mathematical operations things like adding, subtracting and
multiplying, and also very more complex functions.

Here, for example, softmax, matrix multiplication, and so on are mathematical operations that
are part of this directed graph. Connecting the nodes are those edges the lines the input and the
output of those mathematical operations. The edges represent arrays of data. Essentially, the
result of computing the cross-entropy is one of those three inputs to this bias add operation, and
the output of the bias add operation is sent along to the matrix multiplication operation or
MatMul. The other input to the MatMul, you need two inputs to multiply matrix together. The
other input is the variable, or that is the weight.

301
(Refer Slide Time: 02:26)

So, where does the name TensorFlow come from? In math, a simple number like 3 or 5 is called
a scalar. A vector is a one-dimensional array of numbers. In physics, a vector I know everyone's
trying to remember it right now is something that has what magnitude and direction, but in
computer science, we as a vector to mean one dimensional or 1D arrays. A 2-dimensional array
is called a matrix. A 3-dimensional array well, we just call it a 3D Tensor.

So, a scalar, vector, matrix, 3D Tensor, 4D Tensor, and so on. A Tensor is, therefore, an n-
dimensional array of data. So, your data in TensorFlow are Tensors. They flow through the
graph. Hence TensorFlow.

302
(Refer Slide Time: 03:23)

You can build a DAG in Python, stored in a saved model, and restore it in a C++ program for
low latency predictions. You can use the same Python code and execute it both on CPUs and
GPUs. This provides language and hardware portability.

(Refer Slide Time: 03:45)

Like most software libraries, TensorFlow contains multiple abstraction layers. The lowest level
of abstraction is a layer that is implemented to target the different hardware platforms like
running ML on your mobile device. In the latest version of TensorFlow, you also get the
Accelerate Linear Algebra or XLA framework, a faster compiler for all the math that underpins

303
your ML models. Again very low-level stuff here, they should probably know exists but
probably wouldn't interface with directly.

On top of that, hardware and moving up to more abstraction is the execution engine for
TensorFlow written in C++ for highly efficient operations. You could write me an entire
TensorFlow function if you wanted to in C++ and register it as a TensorFlow operation.
Generally, data scientists will use the APIs, which are a bit more abstract and next on our list.
Next are the familiar front-end SDKs or software development kits where we can use C++,
Python, Go, Java, etc. to access your TensorFlow operations. I will be honest with you, though a
lot of my work in TensorFlow uses pre-built ML ingredients provided through the Keras and
dataset APIs. Keras is a high-level neural networks API written in Python, and it can run on top
of TensorFlow. Here is how friendly and abstract the Keras library is.

(Refer Slide Time: 05:13)

Remember the deep neural network before that found the dog that is hiding in the laundry basket
if you are going to use Kares creating and adding those layers into that DNN would simply be
model dog ad and how large you want to layer to be in units. There are some other concepts for
image models like Softmax and ReLU layers that you will pick up when working with these

304
different algorithms, but the code itself for building these layers is just like stacking block after
block.

(Refer Slide Time: 05:40)

Now, let's talk about data size. If you have a small dataset like one that just fits in memory pretty
much any ML framework will work or Python and so on. They have these statistical packages
that often in three or four lines of code, and it will just work. But these are in-memory data sets.
Once your data sets get larger, these packages would not work. You will need to split your data
into batches and then train. However, you are also needed to distribute your training over many,
many machines.

Now, sometimes people think that they can take a shortcode to keep the training simple by
getting a bigger and bigger and bigger single machine with lots of GPUs, but that's not the case.
Scaling out is the answer, not scaling up. Another common shortcut that people take is just to
sample their data. So, it is small enough to let the ML use on the hardware that they happen to
have. They are limiting the effectiveness of ML by not using all of the data for the machine on a
model form. Using all of that data and then devising a plan to collect ten times that they currently
have is often the difference between an ML that does not work an ML that appears magical.

305
(Refer Slide Time: 06:53)

Some of the other major improvements to ML happen with human insights come into the
problem. In ML, you bring human insights that's what your experts know about the use case of
the data set in the form of refinements to existing features or the addition of new features in a
process that is often called feature engineering. You will also need to pre-process your raw data,
scale it, and code it. And these two things and a large data set also need to be distributed and then
done in the Cloud for scale.

Now, that is just on the training side. Once you have completed and successfully trained your
model, you want to deploy it for production. At that point, the performance characteristic
changes instead of just thinking how long it takes to train on your training data set. You must
also think about how it is going to support the number of prediction queries per second or QPS
that you are going to need.

That requires your solution being able to scale the prediction code as necessary to support the
users who need to make those timely predictions. Now the type of questions that you will ask
yourself. Here, in the serving of your model are what if the underlying model changes you have
retrained it? What if the parameters used in the model need to change? What if the number of
inputs or the data changes? Do you really want to expose all this to your users? What if the client
isn't in the language that you used to train the model?

306
You can invoke the TensorFlow API from pretty much any programming language. You can use
Cloud server resources to automatically scale out to as many of QPS or queries per second that
you need for those predictions. But these are things that need to be managed, and that can be a
challenge if you need to rapidly scale out that solution.

Earlier, we mentioned feature engineering and how to build those pipelines to pre-process your
training data before training. This same pre-processing must also happen at prediction time. But
beyond is cleaning up the data, there are various ways that your trained model could be a bit
different from your prediction one. Using a standard, like an AI platform, helps minimize these
issues. Once you have rarely talked about your prediction, inputs will often be systematically
different from those you use to train your model. In subtle in hard to detect ways, maybe the
average of some column has shifted, or the variance has grown over time. This is not a
phenomenon that we call training, serving, skew, and detecting it requires continual data
collection and re-examination.

307
(Refer Slide Time: 09:36)

AI platforms simplify this for you, ensuring that the trained model is what you actually run. In
other words, it helps you handle that training serving skew. A platform will keep track of all
those pre-processing and feature engineering steps for you as well as allow you to version your
model over time. In training, the AI platform will help you distribute that pre-processing, train
your model multiple times iteratively, and deploy your train model to the Cloud for predictions.

Now when we talk about predictions, the ML model is accessible through something like a
simple REST API, and it includes all the pre-processing and feature transformation that you did.
So, the client code can simply supply the raw input variables and get back a prediction. An AI
platform can also distribute the model as needed to supply a high number of queries per second.
Those are the people that want to make predictions with your trained model. With ML, you need
both high-quality executions at training and prediction time.

If you have a great model, but it is super slow for those timely predictions, no one will use it.
While the computation of a TensorFlow model once is relatively cheap. The point of an ML
model is to make those predictions for lots of incoming requests.

308
(Refer Slide Time: 10:56)

Let's look at a diagram that provides a high-level overview of the stages in the ML workflow.
The blue fill boxes will indicate where the AI platform provides managed services and APIs for
your use. You must have access to a large data set of training data that includes the attribute that
lets the label the correct answer for machine learning, and that is what you want to infer or
predict for the future based on all the other data that you have. Your other inputs are called
features.

For example, assume that you want your model to predict the sale price of a house. Begin with a
large dataset, describing the characteristics of houses in a given area, including things like the
sale price of each house and when it was sold. After you have sourced your data, you have to
analyze it and understand the data and pre-processing to get it ready for machine learning. In this
pre-processing step, you transform valid clean data into the format that best suits your model's
needs.

TensorFlow has already had a lot of pre-processing libraries that you can use automatically with
the AI platform. In addition, consider other GCP services that we talked about like BigQuery,
Cloud Dataproc, Cloud Dataflow, Cloud Data Fusion, and Cloud Dataprep to help you with
those transformations. A lot of the work and machine learning is just getting that clean data
ready for machine learning. Then you can develop your model using established ML techniques

309
or by defining new operations and approaches. You can start learning how to create your model
by working through the documentation provided by TensorFlow, SciKit-Learn, and XGBoosts.

AI platform provides the services you need to train and then evaluate your model in the Cloud.
When training your model, you feed it that data or those input features for which you already
know the value of that target data attribute the historical right answer again that is called the
label. You then run the model to predict those target values for your training data so that your
model can adjust its settings to fit itself to the data better and predict the target value more
accurately. This is the whole learning part of machine learning.

Similarly, when evaluating your trained model, you feed it data that includes the target values.
You compare the results of your model predictions to the actual values for the evaluated data,
and then you use statistical techniques appropriate to your model to gauge its success. That is
how well it learned. You can then tune the model by changing the operations or the settings that
you use to control for training purposes. These are called hyper parameters, such as the number
of training steps to run in training. This technique of adjusting those model knobs is called hyper
parameter tuning.

AI platform provides tools to upload your trained ML model to the Cloud. So, then you can send
prediction requests to the model. To deploy your trained model on the AI platform, you must
first save your model using the tools provided by your machine learning framework. This
involves serializing the information that represents your trained model into a file then you can
then simply deploy for prediction in the Cloud.

On GCP, you would upload the saved model to a Google Cloud Storage Bucket and then create a
model resource on the AI platform, and you just specified that Cloud storage path to where your
saved model is located. AI platform provides the services you need to request predictions from
your model in the Cloud. There are two ways to get predictions from trained models- Online
Prediction, sometimes called HTTP prediction, and Batch Prediction. In both cases, you pass
input data to your Cloud-hosted machine learning model, and you get those inferences for each
data instance.

310
And you can monitor the predictions on an ongoing basis. AI platform provides API is to
examine all your running jobs. In addition, various GCP tools support your deployed model's
operation, such as the entire suite of tools within Stackdriver. AI platform provides various
interfaces from managing your model and model versions, including a REST API, the G-Cloud
AI platform command-line tool, and through the GCP console.

311
 Google Cloud Computing Foundation Course
Evan Jones
Technical Curriculum Developer
Google Cloud

Lecture-80
Cloud AutoML

(Refer Slide Time: 00:11)

In this topic, you learn how Cloud AutoML allows you to train high-quality custom machine
learning models with minimal effort or machine learning expertise. Cloud AutoML is a suite of
machine learning products that enables users with a limited machine learning experience to train
high-quality models specific to their business needs. Cloud AutoML leverages more than ten
years of proprietary Google research technology to help the users ML models achieve faster
performance and more accurate predictions.

312
(Refer Slide Time: 00:33)

To put Cloud AutoML into context, let's look at what it takes to solve an ML problem. To solve
an ML problem without the benefit of a managed service, it is up to you to wrangle your data,
code the model, and put together all the infrastructure to serve the predictions. This can be
prohibitively complex and very time-consuming.

(Refer Slide Time: 00:52)

Earlier, you saw how the AI platform lets developers and data scientists build and run superior
learning models in production. As shown here, there is a considerable reduction in the required
training and serving infrastructure, as well as the overall amount of model code. However, there

313
is still a requirement to provide extensive training data, and the process is still a time consuming
one.

(Refer Slide Time: 01:17)

What is immediately notable with Cloud AutoML is that there is no requirement on the user's
side to develop a model or provide training and serving infrastructure. In addition, far less
training data is required, and the results are achieved a lot faster.

(Refer Slide Time: 01:34)

314
The ability of Cloud AutoML to efficiently solve an ML problem is largely due to how it
simplifies these complex steps associated with custom ML model building. Two Cloud AutoML
products apply to what you can see- AutoML Vision and AutoML Video Intelligence.

(Refer Slide Time: 01:52)

With AutoML Vision, you can upload images and train custom image models through an easy-
to-use graphical interface. You can optimize your model for accuracy, latency, and size. AutoML
Vision Edge allows you to export your custom trained model to an application in the Cloud or an
array of devices at the edge. You can train models to classify images through labels that you
choose. Alternatively, Google's data labeling service allows you to use their team to help
annotate your images, or videos, or text.

Later, you will complete a lab where you use Cloud AutoML Vision to train a custom model to
recognize the different types of Clouds.

315
(Refer Slide Time: 02:37)

AutoML Video Intelligence makes it easy to train custom models to classify and track objects
within videos. It is ideal for projects that require custom entity labels to categorize content,
which is not covered by the pre-trained Video Intelligence API. Two Cloud AutoML products
apply to what you can hear, AutoML Natural Language and AutoML Translation.

(Refer Slide Time: 03:10)

With AutoML Natural Language, you can train custom ML models to classify, extract, and
detect sentiment. This allows you to identify entities within documents and then label them based

316
on your domain-specific keywords or phrases. The same applies to understand the overall
opinion, feeling, or attitude expressed in a block of text that's tuned to your domain-specific
sentiment scores.

(Refer Slide Time: 03:32)

AutoML Translation allows you to upload translated language pairs and then train a custom
model where translation queries return specific results for your domain and scale and adapt to
meet your needs.

(Refer Slide Time: 03:46)

317
AutoML Tables reduces the time it takes to go from raw data to top-quality production-ready
machine learning models from months to just a few days. There are many different use cases for
AutoML Tables. For example, if you are in retail, you can better predict customer demand. You
can preemptively fill gaps and maximize your revenue by optimizing product distribution,
promotions, and pricing. If you are in the insurance business, you could first see and optimize a
policyholders' portfolios risk and the return by zeroing in on the potential for large claims or the
likelihood of fraud.

In marketing, you can better understand your customer. For example, what're the average
customers' lifetime value? You can make the most of marketing spend using AutoML Tables to
estimate predicted purchasing value, volume, frequency, lead conversion probability, and churn
likelihood.

318
 Google Cloud Computing Foundation Course
Evan Jones
Technical Curriculum Developer
Google Cloud

Lecture-81
Googles Pre-trained ML APIs

In the previous topic, you learned how you could build custom ML models with minimal effort
or ML expertise by leveraging the suite of ML products offered through Cloud AutoML. When
using Cloud AutoML, you define a domain-specific label the training dataset used to create the
custom ML model you require if you do not need a domain-specific dataset. However, Google
Suite of pre-trained ML APIs might meet your needs. In this topic, you will explore some of
those APIs and apply them through a series of labs.

(Refer Slide Time: 00:35)

API is like the Vision API, Natural Language API, or Translation API, or are already trained for
common ML use cases like image classification. They save you time and the effort of building,
curating, and training a new dataset. So, you can jump ahead right to your predictions. For pre-
trained models, Google has already figured out a lot of those hard problems for you.

319
(Refer Slide Time: 01:03)

Let's explore some of these pre-trained Machine Learning APIs. Let's start with the Cloud
Vision API. There are three major components that all roll up into this RESTful API, and behind
the scenes, each of these are powered by many ML models and years of research. The first is
detecting what an image is and then classifying it. The Vision API picks out the dominant entity.
For example, a car or a cat within an image from a broad set of object categories. This allows
you to detect broad sets of objects within your images easily.

Facial detection can detect when a face appears in photos along with the associated facial
features such as the eyes, the nose, and the mouth placement and the likelihood of over eight
attributes like joy and sorrow. However, facial recognition isn't supported, and Google doesn't
store facial detection information on any Google server. You can use the API to easily build
metadata on your image catalog enabling new scenarios like image-based searches or
recommendations.

320
(Refer Slide Time: 02:09)

Next, our images with text like scanned documents or a sign out there on the row, the Vision API
uses optical character recognition or OCR to extract the text of a wide range of languages into a
selectable and searchable format.

(Refer Slide Time: 02:26)

321
Lastly is a bit of intuition from the web, and it uses the power of Google Image Search. Does the
image contain entities that we know like the Eiffel Tower or a famous person? And then
landmark detection allows you to identify a popular natural and man-made structures along with
the associated latitude and longitude of the landmark. And the logo detection allows you to
identify product logos within an image.

You can build metadata on your image catalog extract text, moderate offensive content, or enable
new marketing scenarios through image sentiment analysis. You can also analyze images
uploaded in the request or integrate with image storage on Google Cloud Storage.

(Refer Slide Time: 03:11)

There are two APIs that apply to speech. The Cloud Text-to-Speech API converts text into
human-like speech in more than 180 voices across more than 30 languages and variants. It
applies research in speech synthesis and Google's powerful neural networks to deliver high
fidelity audio with this API. You can create lifelike interactions with users that transform
customer service device interaction and other applications.

The Cloud Speech-to-Text API enables you to convert real-time streaming or pre-recorded audio
into text. The API recognizes 120 language invariants to support a global user base. You can
enable voice command and control, transcribe audio from call centers, and so on.

322
(Refer Slide Time: 04:04)

The Cloud Translation API provides a simple programmatic interface for translating an arbitrary
string into any supported language. The API is highly responsive. So, web sites and applications
can integrate with it for fast dynamic translation of source text from the source language to a
target language, for example, from French to English. Language detection is also available in
cases where the source language is unknown.

Let us look at a short video that shows how Bloomberg, a global leader in business and financial
data news and insight, applies the Cloud Translation API to reach all of their customers
regardless of language.

323
(Refer Slide Time: 04:44)

The Cloud Natural Language API offers a variety of natural language understanding
technologies. It can do syntax analysis, breaking down sentences into tokens, identify the nouns,
verbs, and adjectives, and other parts of speech. And then also figure out the relationships among
the words. It can also do entity recognition. In other words, it can parse text and flag mentions of
people, organizations, locations, events, products, and media. Sentiment analysis allows you to
understand customer opinions to find actionable product and UX insights.

324
(Refer Slide Time: 05:22)

The Cloud Video Intelligence API supports the annotation of common video formats and allows
you to use Google Video Analysis technology as part of your applications. This REST API
enables you to annotate videos, stored in Google Cloud Storage with video and one frame per
second contextual information. It helps you identify key entities that are the nouns that are within
your video and when they occur. You can also use it to make your content more accessible,
searchable, and discoverable.

325
 Google Cloud Computing Foundation Course
Evan Jones
Technical Curriculum Developer
Google Cloud

Lecture-82
Quiz

(Refer Slide Time: 00:04)

Now let’s test what you have learned so far with a short quiz. Which machine learning tool will
be the best option for somebody who has limited application development or data science skills?
If you said Cloud AutoML, yes, that is a great option for when you want to leverage machine
learning, and you are not an application developer or a data scientist.

326