Course Resources

Google Cloud Certification

Google Cloud Certification
Foundation
Level up into
Professional
Data Roles
Security /
Networking
Developer /
DevOps
User
Certification
Certified Fellow
Google Cloud Certification
Foundation
Level up into
Professional
Data Roles /
Machine Learning
Security /
Networking
Developer /
DevOps
Google
Workspaces
Certified Fellow
Why Certify?

Milestone for learning

Keeping your skills up to date

Help advance your career

Achieve a higher market value

Self motivation
Scenario - Bowtie Inc.
 Bowtie Inc.

Bowtie manufacturing company

Global company, headquartered in Montreal, Canada
~300 employees globally with 100 in Sales
In-store staff, IT, marketing, manufacturing, finance, …
Sales people, managers, sewers, etc…
Offices/Stores in Montreal, London, Los Angeles
 Bowtie Inc.

2 racks in each on-premises location

Global inventory updated upon sales and new stock
POS systems in each store/office location
All office/store infrastructure connected and backed up
to tape in Montreal HQ
Management is extremely thrifty but have finally
come to terms with spending money
 Global Infrastructure

Offices Sales
Showroom
Stores
Los Angeles • POS system
London • Website Sales
• Inventory Updates
• Email & File Access
• Voip phone & chat
• Sales Predictions

Connection Head Office

 Current Issues

Legacy on-premises hardware is out of warranty

Performance issues from store to store - POS systems
Lack of HA and Scalability
Tape backups are slow and offsite storage is costly
Outdated IT skills - a lot of manual intervention
Rush to open stores to supply demand globally
Favourable Results

Reliable and stable connection between stores

Fast and cost-effective cloud backup
Deploy into new regions quickly when required
Base infrastructure is scalable and low cost
No manual intervention - Automation
Connect with Me
The Instructor

https://www.linkedin.com/in/antonitzavelas/

@antoniscloud

[email protected]
 Course Community
Slack Community https://techstudyslack.com
1000’s of students
#general #google #google-ace
Social learning
Share all the things
What is Cloud Computing?
What is Cloud Computing?
What is Cloud Computing?
What is Cloud Computing?

The delivery of a shared pool of on-demand

computing services over the public internet, that can
be rapidly provisioned and released with minimal
management effort or service provider interaction.
The 5 Characteristics of Cloud
The 5 Characteristics of Cloud

On-demand Self Service

Provision resources automatically
without requiring human interaction
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access

Available over the network
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access

Resource Pooling
Pooled resources to support a multi-tenant
model allowing multiple customers to share
the same applications or the same physical
infrastructure
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access
Resource Pooling

Rapid Elasticity
Rapidly provision and de-provision
any of the cloud computing resources
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access
Resource Pooling
Rapid Elasticity

Measured Service
Resource usage can be monitored,
controlled and reported using
metering capabilities
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access
Resource Pooling
Rapid Elasticity
Measured Service
The 5 Characteristics of Cloud

On-demand Self Service

Broad Network Access
Resource Pooling
Rapid Elasticity
Measured Service
 Benefits of Cloud

Agility • Flexibility for provisioning resources

• Innovate faster
Cost • Pay as you go
• Trade capital expenditure for
variable expense
 Benefits of Cloud

Speed • Resources on demand

• Scriptable infrastructure
Global • Global data centres
• Disaster recovery becomes easier
• High availability
 Benefits of Cloud

Security • Always up-to-date

• Physical security
• Encryption at rest and in transit
• Compliance
Cloud Deployment Models
Cloud Deployment Models
Multi-
Public Public Public Cloud

Hybrid

Anthos

AWS Outposts

Private Azure Stack

 Cloud Deployment Models
Public Cloud - 1 public cloud
Multi-Cloud - 2 or more public clouds
Private Cloud - on-premise cloud
Hybrid Cloud - private cloud + public cloud
Hybrid Environment - on-premise data center
+ public cloud
Cloud Service Models

XaaS
Cloud Service Model Concepts
Traditional
On-premises
APPLICATION

DATA

RUNTIME

CONTAINER

OPERATING SYSTEM

VIRTUALIZATION

PHYSICAL SERVERS

NETWORK & STORAGE

DATA CENTER
Cloud Service Model Concepts
Traditional
On-premises
APPLICATION

Infrastructure Stack
DATA

RUNTIME Parts managed by you

CONTAINER

OPERATING SYSTEM
Parts managed by the vendor
VIRTUALIZATION
Priced by unit of consumption
PHYSICAL SERVERS

NETWORK & STORAGE

DATA CENTER
Cloud Service Model Concepts
Traditional
On-premises DC Hosted
APPLICATION APPLICATION

DATA DATA

RUNTIME RUNTIME

CONTAINER CONTAINER

OPERATING SYSTEM OPERATING SYSTEM

VIRTUALIZATION VIRTUALIZATION

PHYSICAL SERVERS PHYSICAL SERVERS

NETWORK & STORAGE NETWORK & STORAGE

DATA CENTER DATA CENTER

You Manage Vendor Manages

 Infrastructure as a Service - IaaS
Traditional
On-premises DC Hosted IaaS
APPLICATION APPLICATION APPLICATION

DATA DATA DATA

RUNTIME RUNTIME RUNTIME

CONTAINER CONTAINER CONTAINER

OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM

VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION

PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS

NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE

DATA CENTER DATA CENTER DATA CENTER

You Manage Vendor Manages Unit of Consumption

 Platform as a Service - PaaS
Traditional
On-premises DC Hosted IaaS PaaS
APPLICATION APPLICATION APPLICATION APPLICATION

DATA DATA DATA DATA

RUNTIME RUNTIME RUNTIME RUNTIME

CONTAINER CONTAINER CONTAINER CONTAINER

OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM

VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION

PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS

NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE

DATA CENTER DATA CENTER DATA CENTER DATA CENTER

You Manage Vendor Manages Unit of Consumption

 Software as a Service - SaaS
Traditional
On-premises DC Hosted IaaS PaaS SaaS
APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION

DATA DATA DATA DATA DATA

RUNTIME RUNTIME RUNTIME RUNTIME RUNTIME

CONTAINER CONTAINER CONTAINER CONTAINER CONTAINER

OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM

VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION

PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS

NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE

DATA CENTER DATA CENTER DATA CENTER DATA CENTER DATA CENTER

You Manage Vendor Manages Unit of Consumption

 Shared Responsibility Model
Traditional
On-premises DC Hosted IaaS PaaS SaaS
APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION

DATA DATA DATA DATA DATA

RUNTIME RUNTIME RUNTIME RUNTIME RUNTIME

CONTAINER CONTAINER CONTAINER CONTAINER CONTAINER

OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM OPERATING SYSTEM

VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION VIRTUALIZATION

PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS PHYSICAL SERVERS

NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE NETWORK & STORAGE

DATA CENTER DATA CENTER DATA CENTER DATA CENTER DATA CENTER

You Manage Vendor Manages

Shared Responsibility Model
Google Cloud Global Infrastructure
Google Cloud Global Infrastructure

Submarine Cable Investments

Current Network

Points of Presence
Google Cloud Global Infrastructure

Submarine Cable Investments

Current Network

Points of Presence

13 Subsea Cables
Hundreds of thousands of miles of fiber cables
Google Cloud Global Infrastructure

Subsea cable runs support the connections

between continents
Google Cloud Global Infrastructure

24 regions 73 zones
144 network edge locations 200+ countries and territories
Google Cloud Global Infrastructure
Google Cloud Global Infrastructure

Response Response

Request Request
POP Google Data
Point of Presence
(edge network) Center
Geography and Regions
Zone Region Multi-Region

us-east4-a

us-east4-b

us-east4-c

nam3
us-east4 N Virginia/ USA
S Carolina
Geography and Regions

us-east4-a
Zone
A zone is a deployment area for Google
Cloud resources within a region. The
us-east4-b
smallest entity in Google’s global network

us-east4-c
A single failure domain within a region
Deploy closer to users for optimal latency
Geography and Regions
Zone Region Multi-Region

us-east4-a

us-east4-b

us-east4-c

nam3
us-east4 N Virginia/ USA
S Carolina
Geography and Regions

Region
Regions are independent geographic
areas that are sub-divided into zones

For fault tolerance and high availability

Intercommunication <5ms between
us-east4 zones within a region
Geography and Regions
Zone Region Multi-Region

us-east4-a

us-east4-b

us-east4-c

nam3
us-east4 N Virginia/ USA
S Carolina
Geography and Regions

Multi-Region
Multi-Regions are large geographic
areas, that contain two or more
regions
Allows Google services to maximize
redundancy and distribution within and
nam3 across regions
N Virginia/ High availability (geo-redundant)
S Carolina
Geography and Regions
Zone Region Multi-Region

us-east4-a

us-east4-b

us-east4-c

nam3
us-east4 N Virginia/ USA
S Carolina
 Geography and Regions
Zone - A zone is a deployment area for Google Cloud
resources within a region. The smallest entity of Google’s
global infrastructure.
Region - Regions are independent geographic areas
that are sub-divided into zones
Multi-Region - Multi-Regions are large geographic areas,
that contains two or more regions
 Jane Lark
Tony

Lisa Izzy
Compute Service Options
Compute Service Options

Options
Complete control and Flexibility
Flexible container technology
Managed application platform
Serverless Environments
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

Compute Engine
 Compute Engine
Virtual Machines (VMs) called instances
Choose region and zone to deploy

You decide the operating system and the software you decid

Use public or private images to create instances

Pre-configured images and software packages available in G

 Compute Engine
Manage multiple instances using instance groups
Add/remove capacity using autoscaling with
instance groups
Attach/detach disks as needed
Can be used with Google Cloud Storage
Use SSH to connect directly
Considered to be IaaS
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

Google Kubernetes Engine (GKE)
 Google Kubernetes Engine (GKE)
Container-orchestration system for automating deploying,
scaling, and managing containers
Built on open-source Kubernetes
Flexibility to integrate with on-premise Kubernetes
Uses Compute Engine instances as nodes in a cluster.
A cluster is a group of nodes or Compute Engine instances
Considered CaaS
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

App Engine
 App Engine

Fully managed, serverless platform for developing and hosti

Provisions servers and scales your app instances based

on demand
Build your app in Go, Java, .NET, Node.js, PHP, Python,
or Ruby
Connect with other Google services seamlessly
Integrates with Web Security Scanner to identify threats
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

Cloud Functions
 Cloud Functions

Serverless execution environment for building and connectin

Simple, single-purpose functions that are attached to

events
Triggered when an event being watched is fired
Your code executes in a fully managed environment
No need to provision any infrastructure
 Cloud Functions
Cloud Functions can be written using JavaScript,
Python 3, Go, or Java runtimes
Use Cases:
• Data processing or ETL operations (video transcoding)
• Webhooks to respond to HTTP triggers
• APIs that compose loosely coupled logic
• Mobile backend functions
• FaaS
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

Cloud Run
 Cloud Run
Fully managed compute platform for deploying and
scaling containerized applications quickly and securely
Built upon an open standard Knative
Abstracts away all infrastructure management
Known as serverless for containers
Any language, any library, any binary
Considered Faas
Compute Service Options

IaaS CaaS PaaS FaaS

More Flexibility Less Flexibility

Storage & Databases
Storage Options
Storage Options

Cloud Storage
 Cloud Storage
Consistent, scalable, large-capacity, highly durable
object storage
11 9’s Durability (99.999999999%)
Unlimited storage with no minimum object size

Use Cloud Storage for content delivery, data lakes, and bac

Available in different storage classes and availability

Cloud Storage
Storage Classes

Standard Nearline Coldline Archive

Maximum Low-cost archival Even lower-cost Lowest-cost
availability storage archival storage archival storage
No limitations Accessed <1/mo Accessed <1/qtr Accessed <1/yr
Cloud Storage
Availability

Region Dual-region Multi-region

Single Region Pair of regions Large geographic area

= region
Storage Options
Storage Options

Filestore
 Filestore
Fully managed NFS file server
NFSv3 compliant
Store data from running applications
Use with VM instances and Kubernetes clusters
 Filestore
Fully managed NFS file server
Use with VM instances and Kubernetes clusters
Storage Options
Storage Options

Persistent Disks
Persistent Disks
Durable block storage for instances

Standard Solid State (SSD)

Lower latency/higher iops

Available in zonal and regional options

Persistent Disks
Durable block storage for instances

Standard Solid State (SSD)

Lower latency/higher iops

Available in zonal and regional options

 Database Options

SQL / Relational NoSQL

Database Options

SQL / Relational
SQL / Relational Options

Cloud SQL Cloud Spanner

Fully managed Scalable relational
database service database service
PostgreSQL, MySQL, Support transactions,
and SQL Server strong consistency and
synchronous replication
High availability across
zones High availability across
regions and globally
Database Options

NoSQL
 NoSQL Options

Bigtable Datastore Firestore Memorystore

Fully managed, scalable Fast, fully managed, serverless, NoSQL, realtime Highly available in-
NoSQL database NoSQL document database database memory service for Redis
and Memcached
High throughput with low For mobile, web and IoT Optimized for offline use
latency apps Fully Managed
Cluster resizing without
Cluster resizing without Multi-region replication downtime
downtime
ACID transactions
 Storage

SQL / Relational NoSQL

Networking Services
Networks, Firewalls and Routes
Networks, Firewalls and Routes

Virtual Private Cloud (VPC)

 Virtual Private Cloud (VPC)

Virtualized network within Google Cloud

Core networking service
Global resource
Each VPC contains a default network

Additional networks can be created in your project, but netw

Networks, Firewalls and Routes
Firewalls and Routes

Firewall Rules Routes

Govern traffic coming Advanced networking
into instances on a functions for your instances
network
Specifies how packets
Default network has a leaving an instance should
default set of firewall rules be directed
Custom rules can be
created
Load Balancing

Distributing Workloads
across multiple instances
Load Balancing

HTTP(S) Load Network Load

Balancing Balancing
Distribute traffic across regions to Distribute traffic among server
ensure that requests are routed to the instances in the same region based
closest region or, in the event of a on incoming IP protocol data, such
failure or over-capacity, to a healthy as address, port, and protocol
instance in the next closest region.

Distribute traffic based on content type

Cloud DNS

Google Cloud
DNS
Publish and maintain DNS records by
using the same infrastructure that
Google uses.

Work with managed zones and DNS

records through the CLI, API, or SDK
Advanced Connectivity

Cloud VPN Direct Interconnect

Connect your existing Connect an existing network
network to your VPC through to your VPC using a highly
an IPsec connection. available, low-latency,
enterprise-grade connection.
Advanced Connectivity

Direct Peering Carrier Peering

Exchange internet traffic Connect your infrastructure to
between your business Google's network edge
network and Google at one through highly available,
of Google's broad-reaching lower-latency connections by
edge network locations using service providers
Resource Hierarchy
 What is a resource?
Service-level resources
• Compute Instance VM’s
• Cloud Storage buckets
• Cloud SQL databases
Account-level resources
• Organization
• Folders
• Projects
 Resource Hierarchy
Configure and grant access to the various resources

Service Level
 Resource Hierarchy
Configure and grant access to the various resources

Account Level
 Resource Hierarchy
Configure and grant access to the various resources

Service Level + Account Level

Resource Hierarchy
 Resource Hierarchy Structure
Resources are organized Bowtie Inc

hierarchically using a parent/child Dept A Dept B Dept C

relationship
Team 1 Team 2

Designed to map organizational

structure to Google Cloud
Product X Product Y

Better management of permissions

Project X Project Y

and access control VMs Storage

 Resource Hierarchy Structure

Policies controlled by IAM Bowtie Inc

Access control policies and Dept A Dept B Dept C

configuration settings on a parent Team 1 Team 2

resource are inherited by the child Product X Product Y

Each child object has exactly one Project X Project Y

parent. VMs Storage

 Resource Hierarchy
bowtieinc.co
Domain (cloud level)

Bowtie Inc Organization (root node)

Payments Billing
Dept A Dept B Dept C
Profile Account
Folders
• Grouping mechanism and
Account-level Team 1 Team 2
Owns isolation boundary
Pays For
Cloud Product X Product Y
Resources

Project
Labels
Projects
Project X Project Y
• Core organizational component
Labels Resource
Labels Resources
VMs Storage
• Categorize resources Service-level
• Any service-level resource
 Resource Hierarchy
bowtieinc.co
Domain (cloud level)

Bowtie Inc Organization (root node)

Dept A Dept B Dept C

Folders
• Grouping mechanism and
Account-level Team 1 Team 2
isolation boundary
Owns
Cloud Product X Product Y
Resources

Project
Labels
Projects
Project X Project Y
• Core organizational component
Labels Resource Resource
Labels
VMs Storage s
• Categorize resources Service-level
• Any service-level resource
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
Resource Hierarchy

Bowtie Inc

Dept A Dept B Dept C

Team 1 Team 2

Product X Product Y

Project X Project Y

VMs Storage
 Jane Lark
Tony

Lisa Laura
Create a Free Tier Account
Create a Free Tier Account

Google Cloud

Free Tier
Create a Free Tier Account

Google Cloud Google Cloud

Free Tier Always Free

3 month free trial with $300 Always free, with limited
credit access

Ends when credit is used, or Usage above free usage limits,

after 3 months, whichever billed at standard rates
happens first
You have an upgraded billing account
Cannot be a business account (no free tier account)
Create a Free Tier Account

What’s needed
Fresh new gmail address
Google Cloud Credit card (for verification)
Private Browsing Session
Free Tier • Chrome - Incognito Session
• Firefox - Private Browsing
• Edge - InPrivate

https://console.cloud.google.com/freetrial
DEMO
Securing your GCP Account
Securing your Google Account

Username + Password

GCP Account

[email protected]
******
Securing your Google Account

Username + Password
EASY TO BYPASS

GCP Account

[email protected]

******
2 Step Verification

Know Have
2 Step Verification

1234 5678
1234 5678
1234 5678
1234 5678
Securing your Google Account

Username + Password + 2 Step Verification

GCP Account

[email protected]
******
2SV

2FA
******
Securing your Google Account

Username + Password + 2 Step Verification

VERY SECURE !

GCP Account

[email protected]

******
2SV
 DEMO

https://console.cloud.google.com
GCP Console Overview
DEMO
Cloud Billing
 Resource Hierarchy
bowtieinc.co
Domain (cloud level)

Bowtie Inc Organization (root node)

Payments Billing
Dept A Dept B Dept C
Profile Account
Folders
• Grouping mechanism and
Account-level Team 1 Team 2
Owns isolation boundary
Pays For
Cloud Product X Product Y
Resources

Project
Labels
Projects
Project X Project Y
• Core organizational component
Labels Resource
Labels Resources
VMs Storage
• Categorize resources Service-level
• Any resource created in GCP
Billing Account and Payment Profile
Google-level Cloud-level

Payments
Billing Account
Profile

Project X Project Y

Linked to
Pays For
Cloud
Resources
 Billing Account
Defines who pays for a given set Google-level
Payments
Cloud-level
Billing

of Google Cloud resources

Profile Account

Tracks all costs incurred by

Google Cloud usage Project X Project Y

Linked to

Linked to a Payments profile Pays For

Cloud
Resources

Can be linked to one or more projects

Billing specific roles and permissions to control access
 Billing Account
Self-service (online) or Invoiced Google-level
Payments
Cloud-level
Billing

(offline) payments available

Profile Account

Sub-accounts can be used for

resellers Project X Project Y

Linked to

Billing account can pay for projects Pays For

Cloud

in a different organization Resources

Projects that are not linked to a Cloud Billing

account cannot use paid Google Cloud services
Billing Account and Payment Profile
Google-level Cloud-level

Payments
Billing Account
Profile

Project X Project Y

Linked to
Pays For
Cloud
Resources
 Payments Profile
Processes payments for all Google-level
Payments
Cloud-level
Billing

Google services
Profile Account

Stores all payment methods

Project X Project Y

Single pane of glass for viewing Linked to

invoices and payment history

Pays For
Cloud
Resources

Controls who can view and receive invoices

Individual or Business profile types - cannot be changed
DEMO
Cost Management and Budget Alerts
DEMO
Export billing data
 Billing Export
Billing export enables granular billing data (such as
usage, cost details, and pricing data) to be exported
automatically to BigQuery for detailed analysis
Not retroactive

• Daily cost detail data

• Pricing data
DEMO
Setting up an admin user
Super Admin account

Irrevocable admin permissions

Grant Organization Admin role
(or any other role)
Recover accounts at the domain level
Admin user account architecture
Google-level Cloud-level

Payments
Billing Account
Profile

Billing Account Billing Account

Administrator User
Linked to
Pays For
Cloud [email protected] [email protected]
Resources

My First Project Project Tony

Setting up an Admin account

Roles Needed
Billing Account User

Admin user
Setting up an Admin account

New Google account

Existing Google account
DEMO
Cloud SDK and CLI
 Cloud SDK

Set of command line tools that allow you to manage resource

• gcloud
• gsutil
• bq
• kubectl

interactive automated scripts

Cloud SDK

Do everything
….and more
Infrastructure as Code
Autocompletion
Powershell
 Cloud SDK
user account service account
single machine google account
best practice multiple machines

gcloud init gcloud components

initialize, authorize, and setup install, update, delete sdk components

gcloud auth login gcloud config

authorize access for gcloud configure accounts and projects

gcloud command format

gcloud + compute + instances + create + example-instance-1 --zone=us-central1-a
component entity operation positional arguments flags
Quickstart Guide
DEMO
Managing the Cloud SDK
DEMO
Cloud Shell and Editor
Cloud Shell
DEMO
Limits and Quotas
 Limits and Quotas

Hard limit on how much of a particular Google Cloud resource

Resets after Must be explicitly

specified time released

Rate Quota Allocation Quota

Enforcement of Quotas

Resource
Protection Countable
Management
Monitoring and alerting on quotas

Quotas

429

ResourceExhausted
Viewing your quota
DEMO
Cloud IAM
 Principle of Least Privilege
A user, program, or process should have only the bare
minimum privileges necessary to perform its function

storage.objects.create
storage.objects.get
storage.objects.delete
storage.objects.list
Identity and Access Management (IAM)

Who Has what access For which

(identity) (Role) resource

Cloud IAM let’s you manage access control by defining who,

has what access, for which resource
Policy Architecture
Policy Member Condition Metadata
etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
Binding compute.instances.stop
compute.instances.setMachineType
configure
audit logging
compute.instances.delete
Members

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
compute.instances.stop configure
compute.instances.setMachineType audit logging
compute.instances.delete
The Who
Google Account
Any email address that's associated with a Google Account, including gmail.com or
other domains.
Members Service Account
An account for an application instead of an individual end user.
Member
Google Groups
A named collection of Google Accounts and service accounts

G Suite Domain
Google Accounts that have been created in an organization's G Suite account

Cloud Identity Domain

Google Accounts in an organization that are not tied to any G Suite applications or
features

AllAuthenticatedUsers
A special identifier that represents all service accounts and all users on the internet who have
authenticated with a Google Account

AllUsers
A special identifier that represents anyone who is on the internet, including authenticated and
unauthenticated users
Roles

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
compute.instances.stop configure
compute.instances.setMachineType audit logging
compute.instances.delete
 Permissions
Determines what operations are allowed on a resource
Correspond one-to-one with REST API methods
Not granted to users directly
You grant roles which contain one or more permissions

compute.instances.list
service resource verb
 Roles
Collection of permissions
You cannot grant a permission to the user directly
You grant a role to a user and all the permissions that the role
contains.
Role
Permissions
compute.instances.list
Example compute.instances.get
compute.instances.start
compute.instances.stop
compute.instances.setMachineType
compute.instances.delete
Roles
Primitive Predefined Custom
Roles historically Finer-grained access Tailor permissions to
available in the Google control than the primitive the needs of your
Cloud roles organization
Owner
Editor
Viewer
Avoid using these roles if
possible

Launch Stages
alpha beta ga
in testing tested and awaiting generally
approval available
Conditions

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
compute.instances.stop configure
compute.instances.setMachineType audit logging
compute.instances.delete
Conditions

Member Condition Metadata Used to define and enforce

etags conditional, attribute-based
Further Concurrency
constrains control access control for Google Cloud
binding resources.
version
Specifies Conditions allow you to choose
schema version
granting resource access to
Role
Permissions
identities only if configured
compute.instances.list Audit Config
conditions are met
compute.instances.get
compute.instances.start Used to When a condition exists, the access
compute.instances.stop configure request is only granted if the
compute.instances.setMachineType audit logging condition expression = true
Metadata

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
compute.instances.stop configure
compute.instances.setMachineType audit logging
compute.instances.delete
Metadata

Member Condition Metadata To help prevent a race condition

etags when updating the policy, IAM
Further Concurrency supports concurrency control through
constrains control
binding the use of an etag field in the policy
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get To avoid breaking your existing
compute.instances.start Used to integrations on new feature releases
compute.instances.stop configure that rely on consistency in the policy
compute.instances.setMachineType audit logging structure, new policy schema versions
compute.instances.delete are introduced
Audit Config

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
compute.instances.stop configure
compute.instances.setMachineType audit logging
compute.instances.delete
Audit Config

Member Condition Metadata

etags
Further Concurrency
constrains control
binding
version
Specifies Determines which
schema version
permission types are
Role
Permissions logged, and what
compute.instances.list Audit Config identities, if any, are
compute.instances.get exempted from
compute.instances.start Used to
compute.instances.stop configure logging
compute.instances.setMachineType audit logging
compute.instances.delete
Policy Architecture
Policy Member Condition Metadata
etags
Further Concurrency
constrains control
binding
version
Specifies
schema version
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
Binding compute.instances.stop
compute.instances.setMachineType
configure
audit logging
compute.instances.delete
 Resource Hierarchy
bowtieinc.co
Domain (cloud level)

Bowtie Inc Organization (root node)

Policy Inheritance
Dept A Dept B Dept C

Folders
• Grouping mechanism and
Team 1 Team 2
isolation boundary
Owns
Cloud Product X Product Y
Resources

Projects
Project X Project Y
• Core organizational component
Resource
VMs Storage s
• Any service-level resource
 Jane Lark
Tony Federlagen
Bowtie

Lisa Laura
Deelightful
Policies and Constraints
Policy Architecture

Policy Member Condition Metadata

etags
Collection of statements that define Further Concurrency
who has what type of access constrains control
binding
version
A policy is attached to a resource and
Specifies
is used to enforce access control schema version
whenever that resource is accessed
Role
Permissions
compute.instances.list Audit Config
compute.instances.get
compute.instances.start Used to
Binding compute.instances.stop
compute.instances.setMachineType
configure
audit logging
Binds one or compute.instances.delete

more members with a

single role and any context-
specific conditions
 {
"bindings": [
Policy Statement
{
"role": "roles/storage.admin",
"members": [
"user:[email protected]"
]
},
{
"role": "roles/storage.objectViewer",
"members": [
"user:[email protected]"
],
"condition": {
"title": “Expires_January_1_2021",
"description": "Do not grant access after Jan 2021",
"expression":
"request.time < timestamp('2021-01-01T00:00:00.000Z')"
}
}
],
"etag": "BeEEja0YfWJ=",
"version": 3
Policy Statement
bindings:
- members:
- user: [email protected]
role: roles/storage.admin
- members:
- user: [email protected]
role: roles/storage.objectViewer
condition:
title: expirable access
description: Do not grant access after Jan 2021
expression: request.time < timestamp('2021-01-01T00:00:00.000Z')
etag: BeEEja0YfWJ=
version: 3
Policy Statement

tonybowtieace@cloudshell:~ (project-tony-286016)$ gcloud projects get-iam-policy

project-tony-286016

bindings:
- members:
- serviceAccount:[email protected]
role: roles/compute.serviceAgent
- members:
- serviceAccount:[email protected]
- serviceAccount:[email protected]
role: roles/editor
- members:
- user:[email protected]
role: roles/owner
etag: BwWtFEMmKL0=
version: 1
Policy Statement
tonybowtieace@cloudshell:~ (project-tony-286016)$ gcloud projects get-iam-policy project-tony-286016

bindings:
- members:
- serviceAccount:[email protected]
role: roles/compute.serviceAgent
- members:
- serviceAccount:[email protected]
- serviceAccount:[email protected]
role: roles/editor
- members:
- user:[email protected]
role: roles/owner
etag: BwWtFEMmKL0=
version: 1

gcloud projects get-iam-policy <project-id>

gcloud resource-manager folders get-iam-policy <folder-id>
gcloud organizations get-iam-policy <organization-id>
 Policy Versions
bindings:
- members:
- serviceAccount:[email protected]
role: roles/compute.serviceAgent
- members:
- serviceAccount:[email protected]
- serviceAccount:[email protected]
role: roles/editor
- members:
- user:[email protected]
No Condition Statement
role: roles/owner
etag: BwWtFEMmKL0=
version: 1
bindings:
Use version 1
- members:

- user: [email protected]

role: roles/storage.admin

- members:

- user: [email protected]

Condition Statement role: roles/storage.objectViewer

condition:

title: expirable access

Needs version 3 description: Do not grant access after Jan 2021

expression: request.time < timestamp('2021-01-01T00:00:00.000Z')

etag: BeEEja0YfWJ=

version: 3
 Policy Limitations
1 policy per resource (including organizations, folders, projects)
1500 members or 250 Google groups per policy
Up to 7 minutes for policy changes to fully propagate across GCP
Limit of 100 conditional role bindings per policy
 Conditions
Condition attributes are either based on resource or based on
details about the request (timestamp, originating/destination IP
address)
bindings:
- members:
- user: [email protected]
role: roles/storage.admin
- members:
- user: [email protected]
role: roles/storage.objectViewer
condition:
title: expirable access
description: Do not grant access after Jan 2021
expression: request.time < timestamp('2021-01-01T00:00:00.000Z')
etag: BeEEja0YfWJ=
version: 3
Time based conditions
bindings:
- members:
- user: [email protected]
role: roles/storage.admin
- members:
- user: [email protected]
role: roles/storage.objectViewer
condition:
title: Business_hours_access
description: Business hours access Monday-Friday
expression: request.time.getHours(“America/Toronto") >= 9 &&
request.time.getHours("America/Toronto") <= 17 &&
// Days of the week range from 0 to 6, where 0 == Sunday and 6 == Saturday.
request.time.getDayOfWeek("America/Toronto") >= 1 &&
request.time.getDayOfWeek(“America/Toronto”) <= 5
etag: BeEEja0YfWJ=
version: 3
 Resource based conditions
bindings:
- members:
- user:[email protected]
role: roles/owner
- members:
- group:[email protected]
role: roles/compute.instanceAdmin
condition:
title: Dev_only_access
description: Only access to development* VMs
expression: (resource.type == 'compute.googleapis.com/Disk' &&
resource.name.startsWith('projects/project-cat-bowties/regions/us-central1/disks/development')) ||
(resource.type == 'compute.googleapis.com/Instance' &&
resource.name.startsWith('projects/project-cat-bowties/zones/us-central1-a/instances/development')) ||
(resource.type != 'compute.googleapis.com/Instance' &&
resource.type != 'compute.googleapis.com/Disk')
etag: BwWEmjveluK=
version: 3
 Condition Limitations
Limited to specific services
Primitive roles are unsupported
Members cannot be allUsers or allAuthenticatedUsers
Limit of 100 conditional role bindings per policy
20 role bindings for same role and same member
AuditConfig Logs
auditConfigs:
- auditLogConfigs:
- logType: DATA_READ
- logType: ADMIN_READ
- logType: DATA_WRITE
service: allServices
- auditLogConfigs:
- exemptedMembers:
- [email protected]
logType: ADMIN_READ
service: storage.googleapis.com
Service Accounts
Service Accounts
Google Cloud

VPC
Project A

Project B Project C
Web App

service account
Service Account types

User-managed Default Google-managed

User created Using some GCP services create Managed by Google, and they
You choose the name user-managed service accounts are used by Google services

Automatically granted the Editor Some are visible, some hidden

role for the project
Name ends with "Service Agent" or
"Service Account”

User-managed [email protected]

Default [email protected]
[email protected]
Service Account Keys

Google managed User managed

Key Management Key Management
None Key storage
All handled by Google Key distribution
Key revocation
Key rotation
Protecting the keys from unauthorized users
Key recovery
Service Account Permissions

Google Cloud

VPC

Project-redbt
Editor

Viewer

Project level or Service Account level

Service account impersonation

Access Scopes

Legacy method

Changing to custom service account allows use of IAM roles

 Use of Service Accounts

Binding

Impersonation
Attach to
resource
 Best Practices
Audit service accounts and keys using either
the serviceAccount.keys.list() method or the Logs Viewer page in the console.

Delete service account external keys if you don’t need them

Grant the service account only the minimum set of permissions required to
achieve their goal

Create service accounts for each service with only the permissions required for
that service

Take advantage of the IAM service account API to implement key rotation
Cloud Identity
Cloud Identity

Identity as a Service (IDaaS)

User and Group Management
Identity Federation with Active Directory
Cloud Identity
Device Management

Security

Single Sign-on
(SSO)

Reporting

Directory Management
Device Management
Security

1234 5678 2 Step

1234 5678
1234 5678
Verification
1234 5678
Single Sign-on (SSO)

App A

App B

App C
Reporting

Audit Logs
Directory Management

Azure AD

GCDS

Active Directory
Google Cloud Directory Sync (GCDS)
On-premises environment Google Cloud

bowtieinc.co bowtieinc.co
AD Forest

bowtieinc.co
bowtieinc.co One way sync
Google Cloud
AD Domain Organization
GCDS
Cloud Identity
bowtieinc.co Single sign-on Other Google
AD FS services

Third-party

Corporate
SaaS apps
 Jane Lark
Tony

Lisa Izzy
IAM Best Practices
 Least Privilege
Apply only the minimal access level required for what’s needed

Predefined roles over primitive roles

Grant roles at the smallest scope

Child resources cannot restrict access granted on it’s parent

Restrict who can create and manage service accounts

Be cautious with owner roles

 Resource Hierarchy
Mirror your Google Cloud resource hierarchy structure to your organization
structure
Use projects to group resources that share the same trust boundary

Set policies at the organization level and at the project level rather than at the resour

Use the security principle of least privilege to grant IAM roles

Grant roles for users or groups at the folder level instead of setting it at the project
level, if spanning across multiple projects
 Service Accounts
When using service accounts, treat each app as a separate trust boundary
Do not delete service accounts that are in use by running services

Rotate user managed service account keys

Name service account keys to reflect use and permissions

Restrict service account access

Don’t check in service account keys into source code

 Auditing
Use Cloud Audit Logs to regularly audit IAM policy changes
Audit who can edit IAM policies on projects

Export audit logs to Cloud Storage for long-term retention

Regularly audit service account key access
Restrict log access with logging roles

Role logging.viewer
Policy Management
Organization-level policy
Role
Permissions
compute.instances.get
compute.instances.start
 Policy Management
To grant access to all projects in your Organization, use an organization-level
policy
Grant roles to a Google group instead of individual users where possible

When granting multiple roles to a particular task, create a Google group

instead
Role
Permissions
compute.instances.get
compute.instances.start
Networking Refresher
Internet Protocol - IP
Open Systems Interconnection Model (OSI)
Communications between a computing system

Application
Presentation

Session

Transport
Network

Data Link
Physical
OSI Model
Protocol
HTTP - HTTPS - DHCP - DNS
Application SSH - Telnet

Presentation

Session

Transport TCP - UDP IP Packets

Network IPv4 - IPv6 IP address

Subnets

Data Link
Physical
OSI Model
Protocol
HTTP - HTTPS - DHCP - DNS
Application SSH - Telnet

Presentation

Session

Transport TCP - UDP IP Packets

Network IPv4 - IPv6 IP address

Subnets

Data Link
Physical
Internet Protocol - IP

Internet Protocol v4 (IPv4)

Internet Protocol v6 (IPv6)

IPv4 Classful Addressing

Dotted decimal
192.168.255.255
octet

4,294,967,296
IPv4 Classful Addressing
0.0.0.0 - 127.255.255.255
A 2,147,483,648 addresses
4,294,967,296
128 networks
128.0.0.0 - 191.255.255.255
B 1,073,741,824 addresses
16,384 networks
192.0.0.0 - 223.255.255.255
C 536,870,912 addresses
2,097,152 networks

D
E
 Private IP addresses
Defined by standard RFC1918

Single Class A
10.0.0.0 – 10.255.255.255
16,777,216 addresses
16 Class B
172.16.0.0 – 172.31.255.255
1,048,576 addresses
256 Class C
192.168.0.0 – 192.168.255.255
65,536 addresses
Classless Inter-Domain Routing (CIDR)

192.168.0.0/16
Network address Prefix
Classless Inter-Domain Routing (CIDR)
192.168.0.0/16

65,536
addresses

192.168.0.0 - 192.168.255.255
Classless Inter-Domain Routing (CIDR)

192.168.0.0/17 192.168.0.0 - 192.168.128.0 -

192.168.128.0/17
32,768 addresses 192.168.127.255 192.168.255.255 32,768 addresses
 Classless Inter-Domain Routing (CIDR)

192.168.0.0/18 192.168.128.0/18
16,384 addresses 192.168.0.0 - 192.168.128.0 - 16,384 addresses
192.168.63.255 192.168.191.255

192.168.64.0 - 192.168.192.0 -
192.168.127.255 192.168.255.255
192.168.64.0/18 192.168.192.0/18
16,384 addresses 16,384 addresses
Networking Refresher Part 2
Classless Inter-Domain Routing (CIDR)

192.168.0.0/16
Dotted decimal notation Prefix
Classless Inter-Domain Routing (CIDR)

192.168.0.0/16
Dotted decimal notation Prefix

Helpful Reference
192.168.0.0/8 16+ million IP addresses

fixed 192.168.0.0/16 65,536 IP addresses

192.168.0.0/24 256 IP addresses

192.168.1.2/32 1 IP address 0.0.0.0/0 All IP addresses

OSI Model
Protocol
HTTP - HTTPS - DHCP - DNS
Application SSH - Telnet

Presentation

Session

Transport TCP - UDP IP Packets

Network IPv4 - IPv6 IP address

Subnets

Data Link
Physical
IP Version 6
Octet

192.168.0.250
Dotted decimal notation
Hextet

1452:0db8:0000:0000:0000:fe02:0042:8452
Hexadecimal notation

1452:0db8:0:0:0:fe02:0042:8452
1452:0db8::fe02:0042:8452
Shortened
IP Version 6
2001:de3::/64
Network address Prefix

2001:de3:0000:0000:0000:0000:0000:0000
Start address

2001:de3:0000:0000:ffff:ffff:ffff:ffff
End address

0.0.0.0/0 ::/0 All addresses

IPv4 IPv6
OSI Model
Protocol
HTTP - HTTPS - DHCP - DNS
Application SSH - Telnet

Presentation

Session

Transport TCP - UDP IP Packets

Network IPv4 - IPv6 - ICMP IP address

Subnets

Data Link
Physical
IP - TCP/UDP

IP Packet
Source IP Address Destination IP Address

Protocol Port Number Data

(Source/Destination)

TCP - Transmission Control Protocol

UDP - User Datagram Protocol
OSI Model
Protocol
HTTP - HTTPS - DHCP - DNS
Application SSH - Telnet

Presentation

Session

Transport TCP - UDP IP Packets

Network IPv4 - IPv6 - ICMP IP address

Subnets

Data Link
Physical
Application Layer

HTTP(S) UDP-Port 53 Port 22

Virtual Private Cloud (VPC)
 Virtual Private Cloud (VPC)

Virtualized network within Google Cloud

A VPC is a Global resource
Encapsulated within a Project
VPC’s do not have any IP address ranges associated
with them
Firewall rules control traffic flowing in and out of the
VPC
 Virtual Private Cloud (VPC)
Resources within a VPC can communicate with one
another by using internal (private) IPv4 addresses

Support only for IPv4 addresses

Each VPC contains a default network
2 Network types: Auto Mode or Custom Mode
Virtual Private Cloud (VPC)

Project Internet

Network A Network B Network C

us-east1
us-central1
europe-west1
asia-east1
australia-southeast1
southamerica-east1
northamerica-northeast1
Default VPC
10.128.0.0/9
Project /20 subnet in each region
Route to Default Internet Gateway
Default

10.142.0.0/20 us-east1
10.128.0.0/20 us-central1
10.132.0.0/20 europe-west1
10.140.0.0/20 asia-east1
10.152.0.0/20 australia-southeast1
10.158.0.0/20 southamerica-east1
10.162.0.0/20 northamerica-northeast1
DEMO
Virtual Private Cloud (VPC)

Project

Network

Region: us-west1 Region: us-central1 Region: europe-west1

subnet-1 subnet-2 subnet-3

VPC Network Subnets
 Subnets
A subnetwork of a VPC
Each VPC network consists of one or more subnets
and each subnet is associated with a region
The name or region of a subnet cannot be changed
after you have created it
Primary and secondary ranges for subnets cannot
overlap with any allocated range
 Increasing subnet IP space
Must not overlap with other subnets in the same VPC
network
Inside the RFC 1918 address-space
Network range must be larger than the original
Once subnet has been expanded you cannot undo it

/20 /16
 Reserved IP Addresses

Network - First address

Default Gateway - Second address

Second-to-last address - Google Cloud future use

Broadcast - Last address

Routing and Private Google Access
 Routing
Routes define the network traffic path from one
destination to the other
In a VPC routes consists of a single destination (CIDR)
and a single next hop
All routes are stored in the routing table for the VPC
Each packet leaving a VM is delivered to the next hop of
an applicable route based on a routing order
Routing Types

System-generated
Default
Subnet Route

Custom Routes
Static Route
Dynamic Route
Routing Types

System-generated
Default
Subnet Route
 Default Route
Path to the Internet
Path for Private Google Access
Can be deleted only by replacing with custom route
Lowest priority
 Subnet Route
Routes that define paths to each subnet in the VPC
Each subnet has at least one subnet route whose
destination matches the primary IP range of the subnet
When a subnet is created, a corresponding subnet route is
created for both primary and secondary IP range
Cannot delete a subnet route unless you modify or delete
the subnet
Routing Types

Custom Routes
Static Route
Dynamic Route
 Static Route
Can use the next hop feature
Can be created manually
Static routes for the remote traffic selectors are created
automatically when creating Cloud VPN tunnels
Static Route parameters

Every route in the project must have a unique name.

The VPC network

IPv4 CIDR block containing systems receiving the

incoming packets

Lower numbers indicate higher priorities

0 being the highest priority

Network tags so that the route only applies to

tagged instances

Next hop for static routes

 Dynamic Route
Managed by one or more Cloud Routers
Dynamically exchange routes between a VPC and
on-premises networks
Destination IP ranges outside the VPC network
Used with dynamically routed VPNs and Interconnect
Routing Order
Special return paths

Special Routes

Project

Network

Region: us-west1 Region: us-central1

subnet-1
subnet-2
 Private Google Access
Internet

Traffic to Google
APIs and Services Google APIs
Other Private Access
and Services
Traffic to the Internet
Public IP addresses
Options
Internal subnet route
Project
Private Google Access
Network Internet gateway
for on-premises hosts
VPC routing
Private Services Access
Region: us-west1 Region: us-central1

subnet-1 Serverless VPC Access

subnet-2
Private Google Access On

VM1 VM2
10.30.0.5 192.168.2.7 + Public IP
IP Addressing
 IP Addressing
IP Address
Decision starts here

Internal (Private) External (Public)

OPTIONAL

Alias IP Auto Custom Ephemeral Static

PROMOTE TO STATIC

Ephemeral Static Ephemeral Static

PROMOTE TO STATIC PROMOTE TO STATIC

 Internal IP Addressing
Not publicly advertised and used only within a network

Resources with internal IP addresses communicate with each other privately

Configure multiple internal IP addresses, representing

containers or applications hosted in a VM, without
using a separate network interface Internal (Private)
IP ranges can be assigned from the subnet's primary Auto mode VPC
or secondary ranges. automatically selected if available

Custom VPC
Alias IP Auto Custom must be selected manually
OPTIONAL PROMOTE TO STATIC

The IP address comes

from the region's subnet Ephemeral Static Ephemeral Static
IP addresses are
PROMOTE TO STATIC
released only when the You must specify which subnet the IP
instance or forwarding address comes from
rule is deleted
Assigns the IP address to your project
until you explicitly release it
IP Addressing
IP Address

Internal (Private) External (Public)

OPTIONAL

Alias IP Auto Custom Ephemeral Static

PROMOTE TO STATIC

Ephemeral Static Ephemeral Static

PROMOTE TO STATIC PROMOTE TO STATIC

 External IP Addressing
External IP addresses are needed to communicate with the internet,
with resources in another network, or a public Google Cloud service
External (Public)
Sources from outside a Google Cloud VPC network can address a
specific resource by the external IP address

Only resources with an external IP address can send and receive Ephemeral Static
traffic directly to and from outside the network.

PROMOTE TO STATIC
Automatically assigned

IP addresses are
released when stopping
and restarting or deleting Assigns the IP address to your project
until you explicitly release it

Available as a regional or global

resource
 Internal IP address reservation
1. Reserve a specific address and then associate it with a specific resource
2. Specify an ephemeral internal IP address for a resource and then promote
the address
10.12.4.3
1A. Create reserved
internal IP Address
Unreserved Reserved
Not in use Not in use

2A. Create VM instance 1B. Create VM instance

with automatically with reserved IP Address
allocated IP Address

Unreserved Reserved
In use 2B. Create reserved In use
internal IP Address

10.12.4.3
 External IP address reservation
1. Reserve a new static external IP address and then assign it to a resource
2. Specify an ephemeral external IP address for a resource and then promote
the address

Regional IP address
gcloud compute addresses create ADDRESS_NAME \ --region REGION

Global IP address
gcloud compute addresses create ADDRESS_NAME \ --global \ --ip-version [IPV4 | IPV6]
IP Addressing
IP Address

Internal (Private) External (Public)

OPTIONAL

Alias IP Auto Custom Ephemeral Static

PROMOTE TO STATIC

Ephemeral Static Ephemeral Static

PROMOTE TO STATIC PROMOTE TO STATIC

VPC Firewall rules
VPC Firewall rules

incoming OR outgoing
Project NOT BOTH
Network

Region: us-west1 Region: us-central1

subnet-1
subnet-2 protocol
ports
sources
destinations
target
 Implied and pre-populated rules
TCP, UDP, ICMP, GRE
Metadata Server:
169.254.169.254 TCP: PORT 25
DHCP
DNS
Instance Metadata
Implied Rules
NTP
allow egress deny ingress
Firewall rule characteristics
OR

BOTH

IPv4
OR

BOTH
Firewall rule components
The VPC network

Numerical priority

Direction of the connection

Action on match (permit or block)

Dropdown
Defines the instance

Source IP
Source Tags
Source Service account

Protocols and ports

[DEMO] Custom VPC
 Custom VPC w/ Private Google
Access Internet

Traffic to Google
APIs and Services
Cloud Storage
Traffic to the Internet

Internal subnet route

Project

custom Internet gateway

VPC routing

Region: us-east1 Region: us-east4

private 10.0.5.0/24 public 10.0.0.0/24

Private Google Access On

private-instance public-instance
10.0.5.x 10.0.0.x + Public IP
 Custom VPC w/ Private Google
Access Internet

Cloud Storage
Traffic to the Internet

Project

custom Internet gateway

VPC routing

Region: us-east1 Region: us-east4

private 10.0.5.0/24 public 10.0.0.0/24

 Custom VPC w/ Private Google
Access Internet

Cloud Storage
Traffic to the Internet

Internal subnet route

Project

custom Internet gateway

VPC routing

Region: us-east1 Region: us-east4

private 10.0.5.0/24 public 10.0.0.0/24

private-instance public-instance
10.0.5.x 10.0.0.x + Public IP
DEMO
VPC Peering
 VPC Peering
Private connectivity across two VPC networks (RFC 1918)
Peer across the same or different projects and
organizations
Reduces network latency
Increases network security
Reduces network costs
 VPC Peering

network-b

PEERING Region: us-east1 PEERING

subnet-b 10.2.0.0/20
CIDR range CANNOT overlap

network-a network-c

Region: us-east1 network-a > network-b > network-c Region: us-east1

subnet-b 10.0.0.0/20
Transitive peering is NOT supported subnet-c 10.3.0.0/20

Internal
DNS
To allow ingress traffic from VM instances in a peer network, you must create ingress allow firewall
rules. By default, ingress traffic to VMs is blocked by the implied deny ingress rule.
VPC Peering Demo

bowtieinc project-tony-286016

bowtieinc-a bowtieinc-b

Region: us-east1 PEERING Region: us-east4

subnet-a 10.0.0.0/20 subnet-b 10.4.0.0/20

Shared VPC
 Shared VPC
Communication through internal IP
Shared VPC Admin
Service Project Admin
Service Project A Service Project B
• Project-level permissions
• Subnet-level permissions

VM1 VM2

Standalone Project
Project
Host project OR
Network
Service project
cannot be both Host Project
Region: us-west1

Shared VPC Network

subnet-1 10.18.5.0/24

Region: us-west1 Region: us-central1

subnet-1 10.0.2.0/24 subnet-2 10.10.4.0/24

VM3
10.18.5.2

Internal IP for VM1 Internal IP for VM2

10.0.2.15 10.10.4.6
Multiple Hosts Project

Dev-A Service Project Dev-B Service Project Prod-A Service Project Prod-B Service Project

devtest-a devtest-b prod-a prod-b

Development - Host Project Production - Host Project

Development Network Production Network

Region: us-west1 Region: us-central1 Region: us-west1 Region: us-central1

subnet-1 10.0.2.0/24 subnet-2 10.10.4.0/24 subnet-1 10.0.2.0/24 subnet-2 10.10.4.0/24

Internal IP for Internal IP for Internal IP for Internal IP for

devtest-a devtest-b prod-a prod-b
10.0.2.15 10.10.4.6 10.0.2.15 10.10.4.6
Hybrid Environment

Service Project A Service Project B

On-premises
Network
prod-a prod-b

Customer VPN
Gateway

Host Project

Shared VPC Network

Region: us-west1 Region: us-central1

subnet-1 10.0.2.0/24 subnet-2 10.10.4.0/24

Cloud VPN

Internal IP for Internal IP for

prod-a prod-b
10.0.2.15 10.0.2.15

Internet
Two-tier web service

Tier 1 Service Project Tier 2 Service Project

Instance Tier 1 Instances Instance Tier 2 Instances
Group Group

External IP HTTP(S) Multiple Instances Internal Multiple Instances

Address Load Balancing Load Balancing
External Client

Host Project

Shared VPC Network

Region: us-west1

subnet-1 10.0.2.0/24 subnet-2 10.0.3.0/24

Internal IPs for Internal IPs for

Internal IP Address
Tier 1 Instances Tier 2 Instances

10.0.2.3, 10.0.2.4, 10.0.2.5 10.0.3.9 10.0.3.3, 10.0.3.4, 10.0.3.5

VPC Flow Logs
VPC Flow Logs
1 of every 10 packets are captured

Cloud Logging Cloud Storage

(Stackdriver)
bowtieinc

bowtieinc-a

30 day Region: us-east1 Region: us-east4 long-term

log storage storage
subnet-a subnet-b
10.0.0.0/20 10.4.0.0/20
 Use cases
Network Monitoring
• Real-time visibility into network throughput and
performance
Analyze network usage and optimize network traffic expenses
Network forensics when incidents occur
Real-time security analysis
• Stream to Pub/Sub and integrate with SIEM (Splunk,
Rapid7, LogRhythm)
 Record Format
InstanceDetails
Additional Fields project_id
Core Fields Metadata vm_name
src_instance region
Base IpDetails VpcDetails
dest_instance zone
connection src_ip project_id
src_vpc
start_time src_port vpc_name
dest_vpc
end_time dest_ip GeographicDetails subnetwork_name
src_location
bytes_sent dest_port continent
dest_location
packets_sent protocol country ClusterDetails
rtt_msec src_gke_details cluster_name
region
reporter dest_gke_details cluster_location
city
asn
PodDetails
pod_name
GkeDetails
Cluster pod_namespace

Pod
ServiceDetails
Service
service_name
service_namespace
Sample Log
 Sample Log

Expand all Collapse all

compute.googleapis.com%2Fvpc_flows }
DNS Fundamentals
What is DNS?

Domain Name System

www.google.com 172.217.164.196
What is DNS?

Domain Name System

www.google.com 172.217.164.196

query reply

google.com

www 172.217.164.196

ZONE FILE
DNS Record Nameserver
Why DNS

Stored on each computer

HOSTS Updates were difficult to manage
Not scalable
Why DNS

Organization
Domain Name Structure
A dynamic system

Stored on each computer

HOSTS Updates were difficult to manage
Not scalable
DNS Structure

antonit.com.
ROOT
 DNS Structure
IANA

Root domain
13 root servers
Root
zone
. a.root-servers.net (a-m)

Top-level domains gTLD - .com

com org net io ccTLD - .ca, .uk, .it
TLD name servers

Second level domains

google antonit wikipedia speedtest cantrill
Authoritative name servers

Sub-domain of cloud training learn

parent
 How DNS works
DNS root
name servers
DNS recursive
resolver
ISP

query www.google.com www.google.com

www.
google.
com TLD name
reply 172.217.164.196 go to name server for domain google.com
server for .com
CACHE
www.google.com
172.217.164.196 CACHE
www.google.com

DNS client 172.217.164.196

Root Root
hints hints

Authoritative name
server for
www.google.com
DNS Record Types
SOA

NS CNAME
NS
Name Server (NS)

Root domain
. Records for .co

root zone

co
TLD name servers bowtieinc.co NS ns-cloud-d1.googledomains.com
bowtieinc.co NS ns-cloud-d2.googledomains.com

.co zone

Authoritative name servers bowtieinc

Records for bowtieinc.co

bowtieinc.co zone
 A
AAAA
A and AAAA records
IPv4

bowtie
www A 52.54.92.195
www AAAA 2001:4860:4802:32::a

bowtieinc.co zone

IPv6
CNAME
CNAME records

bowtieinc.co A 52.54.92.195
bowtie

shop CNAME bowtieinc.co

ftp CNAME bowtieinc.co

bowtieinc.co zone
TXT
TXT records

Please verify your

domain
Add TXT record
bowtie-approved
bowtieinc TXT tony-loves-cats

bowtieinc.co zone
TXT
TXT records

You’re good to
go!

bowtieinc TXT tony-loves-cats

TXT bowtie-approved

bowtieinc.co zone
MX
MX records

bowtieinc bowtieinc.co A 52.54.92.195 SMTP

domain name
MX query
MX 5 mail
MX 10 ASPMX.L.GOOGLE.COM. [email protected]
priority
bowtieinc.co zone
SOA
PTR records

bowtieinc
www A 52.54.92.195
Pointer record 195.92.54.52.in-addr.arpa PTR bowtieinc.co

bowtieinc.co zone
SOA
SOA records

bowtieinc.co
origin = ns-cloud-b1.googledomains.com
bowtieinc mail addr = cloud-dns-hostmaster.google.com
serial = 3
refresh = 21600
retry = 3600
expire = 259200
minimum = 300

bowtieinc.co zone
Network Address Translation (NAT)
 Network Address Translation - NAT
Translates local private IP(s) to public IP(s) before transferring packets

Originally designed to deal with the scarcity of free IPv4 addresses

IPv6 networks do not require NAT as their are no shortage of addresses

Provides security and privacy

Types of NAT

Static NAT - 1 private IP to 1 public IP

Dynamic NAT - 1 private IP to 1 public IP in pool of public addresses

Port Address Translation (PAT) - Multiple private IPs to 1 public IP

 Static NAT
Source = SRC Destination = DEST

SRC IP: 54.5.4.9 SRC IP: 54.5.4.9

DEST IP: 192.168.0.15 DEST IP: 73.6.2.33
ROUTER
The destination responds back to The destination responds back to
source by using it’s given public (NAT DEVICE) source by using it’s given public
IP address IP address
bowterest

Packets are generated with the The source address in the packet 54.5.4.9
private source IP and the public is translated from a private IP to a
192.168.0.15 destination IP in the header TRANSLATION public IP and the source in the
packet is delivered as a public IP
192.168.0.15 73.6.2.33
SRC IP: 192.168.0.15 SRC IP: 73.6.2.33
DEST IP: 54.5.4.9 DEST IP: 54.5.4.9

Internal/ Private Public

The NAT device maps a private IP with a public IP in a NAT table

Mapping private IP: public IP or 1:1
Telephone Analogy for Static NAT

Public
x1337 Phone Number
514-555-8437

Laura George

Internal/
Private
 Dynamic NAT
Source = SRC Destination = DEST

SRC IP: 192.168.0.13

DEST IP: 54.5.4.9
192.168.0.13 ROUTER
(NAT DEVICE)
fashiontube
Public IP’s are allocated
dynamically from a pool and 73.6.2.33 After finishing communication,
assigned to a private IP that is the public IP is returned back to
communicating over the public the pool for use by another
internet 73.6.2.34
device

TRANSLATION bowterest
Private IPs Public IP Pool
192.168.0.37 73.6.2.33
192.168.0.13 73.6.2.34 54.5.4.9
192.168.0.37
Internal/ Private Public

The NAT device maps a private IP with a public IP in a NAT table

Public IPs are allocated randomly and dynamically from a pool
Telephone Analogy for Dynamic NAT
Bowtie Inc Public
Phone Number
514-555-8437
514-555-8438
x1335 514-555-8439

x1336
George

Internal/
x1337 Private
 Port Address Translation - PAT
Source = SRC Destination = DEST

Return traffic uses port 443 and 62.88.44.88 as

the source on port 443 along with the public IP
SRC IP: 192.168.0.13 SRC PORT: 35535 of the NAT device and the destination port
DEST IP: 62.88.44.88 DEST PORT: 443
192.168.0.13 ROUTER
(NAT DEVICE)
SRC IP: 73.6.2.33 SRC PORT: 8844
DEST IP: 62.88.44.88 DEST PORT: 443
SRC IP: 192.168.0.14 SRC PORT: 35536
DEST IP: 62.88.44.88 DEST PORT: 443
SRC IP: 73.6.2.33 SRC PORT: 8845
DEST IP: 62.88.44.88 DEST PORT: 443
192.168.0.14
SRC IP: 73.6.2.33 SRC PORT: 8846 fashiontube
SRC IP: 192.168.0.15 SRC PORT: 35537
DEST IP: 62.88.44.88 DEST PORT: 443
DEST IP: 62.88.44.88 DEST PORT: 443 62.88.44.88
Private IP Private Port Public IP Public Port TCP/443
192.168.0.13 35535 73.6.2.33 8844
192.168.0.14 35536 73.6.2.33 8845
192.168.0.15 192.168.0.13 35537 73.6.2.33 8846

Internal/ Private Public

The NAT device records the source IP and source port in a NAT table
ource IP is replaced with a public IP and public source port are allocated from a pool that allows overloading -
Telephone Analogy for PAT

x8844 514-555-8437

Laura Lark George

Internal/ Private
 Jane Lark
Tony

Lisa Izzy
Cloud DNS
 Cloud DNS
Host authoritative name servers and allow authoritative
DNS lookups (DNS as a Service)
100% SLA - Globally Resilient
Host zones through managed name servers
• Public Zone - visible to the internet

• Private Zone - visible only within your network

Cloud DNS

Records
Record Sets
DEMO
Virtualization Fundamentals
What is Virtualization?
Lark
Kernel

User Mode
Application
Or SYSTEM CALL
Non Privileged Mode
Operating System
Kernel Mode
Or
Privileged Mode
Hardware

used unused
What is Virtualization?
Kernel Kernel

User Mode Application

Or SYSTEM CALL
Non Privileged Mode
Operating System
Kernel Mode
Or
Privileged Mode
Hardware

used unused
What is Virtualization?

Kernel Kernel

User Mode Application

Or
Non Privileged Mode
Operating System
Kernel Mode
Or
Privileged Mode
Hardware

Unstable
What is Virtualization?
Web App Database

Application
unused
Operating System

Hypervisor (VMM)
used
Hardware

Enables multiple operating systems to run alongside each other, sharing

the same physical computing resources.
 Emulation
Web App Database

Application
Virtual Machines
(guest) Operating System

Binary translation Hypervisor (VMM)

Host Hardware
 Para-virtualization
Web App Database

Application
Virtual Machines
Modified
(guest)
Operating System
HYPERCALL
Hypervisor (VMM)

Host Hardware
 Hardware-assisted Virtualization
Web App Database

Application
Virtual Machines
(guest) Operating System
Can access the
underlying hardware Hypervisor (VMM)

Host Hardware

Virtualization-aware
 Kernel Level Virtualization
VM VM VM
Operating System
Hypervisor
Application
(Nested Virtualization)
Operating System

Kernel Linux Kernel

Host Hardware
Compute Engine
 Compute Engine
Virtual machine = Instance (IaaS)
Multiple instance sizes and types
Per second billing
Launched in a VPC network
Host is available in a Zone
Multi-tenant host or Sole-tenant node
Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Operating
Public Image Custom Image Marketplace System

Standard (HDD) Balanced (SSD) SSD Storage

Default Custom Networking

Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Many machine types - General, compute, memory

Intel or AMD
vCPU = single hardware hyper-thread on CPU
Network throughput = 2Gbps per vCPU
Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Operating
Public Image Custom Image Marketplace System

Standard (HDD) Balanced (SSD) SSD Storage

Default Custom Networking

 Machine Configuration

Operating
Public Image Custom Image Marketplace System

Public Image - Linux or Windows

Custom Image - Private Images (Snapshots/existing disk)
Marketplace - OS + software
Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Operating
Public Image Custom Image Marketplace System

Standard (HDD) Balanced (SSD) SSD Storage

Default Custom Networking

 Machine Configuration

Standard (HDD) Balanced (SSD) SSD Storage

Performance vs Cost
Standard - Spinning Hard Drive
Balanced - Solid State Drive (alternative to SSD)
SSD - Solid State Drive
Local SSD - Physically attached (swap disk)
Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Operating
Public Image Custom Image Marketplace System

Standard (HDD) Balanced (SSD) SSD Storage

Default Custom Networking

 Machine Configuration

Default Custom Networking

Auto, default, custom networks

Many available regions and zones
Ingress/egress firewall rules (IP ranges, tags, instances)
Network load balancing
Regional/global load balancing
Machine Configuration

Cores (vCPU)
Predefined Custom
Memory

Operating
Public Image Custom Image Marketplace System

Standard (HDD) Balanced (SSD) SSD Storage

Default Custom Networking

Compute Engine Machine Types
 Predefined machine type families
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 Machine Types
Generation
Type

Series
e2-standard-32 vCPU’s
E2 1 64
standard
2 80
N2
highmem 4 96
N2D 8 128
GPU’s only available highcpu
N1 + GPU 16 160
for N1 series 30 208
M1 ultramem 32 224
M2 40 416
megamem
C2 48

F1 medium 60

G1 small

micro
 Predefined machine type families
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 Standard machine type
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU

NVIDIA Tesla K80 Standard

NVIDIA Tesla P4
High-memory Balance of CPU and memory
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 High-memory machine type
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
NVIDIA Tesla P4
High-memory High memory to CPU ratio
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 High-CPU machine type
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
NVIDIA Tesla P4
High-memory High CPU to memory ratio
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
General-purpose machine family
Available for custom

e2-micro
e2-small General-purpose Day-to-day computing at a lower cost
e2-medium
Web serving
E2
Shared core N1 App serving
f1-micro N2 Back office applications
g1-small N2D
Small-medium databases

Microservices
GPU
Standard Virtual desktops
NVIDIA Tesla K80
High-memory
NVIDIA Tesla P4 Development environments
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 General-purpose

min
vCPU’s Memory

2 0.5
e2-standard-32 2 4 8 16 32
max 32 128 E2 standard

highmem 2 4 8 16

highcpu 2 4 8 16 32

Support up to 32 vCPUs and 128 GB of memory

Lowest on-demand pricing
General-purpose machine family
Available for custom

e2-micro Balanced price/performance across

e2-small General-purpose
e2-medium
a wide range of VM shapes
E2 Web serving
Shared core N1
App serving
f1-micro N2
g1-small N2D Back office applications

Medium-large databases
GPU Cache
Standard
NVIDIA Tesla K80 Media/streaming
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 General-purpose

min
vCPU’s Memory

2 0.95
n1-standard-32 2 4 8 16 32 48
max 96 624 N1 + GPU standard
64 80 96
TPU
highmem 2 4 8 16 32 48
64 80 96

highcpu 2 4 8 16 32 48
64 80 96

Up to 96 vCPUs and 624 GB of memory

Only machine type for GPU and TPU support
Larger sustained use discount than N2
 General purpose

min
vCPU’s Memory

2 0.5
n2-standard-32 2 4 8 16 32 48
max 80 640 N2 standard
64 80

highmem 2 4 8 16 32 48
64 80

highcpu 2 4 8 16 32 48
64 80

Support up to 80 vCPUs and 640 GB of memory

Workloads that can take advantage of the higher clock frequency
Higher per-thread performance
 General purpose

min
vCPU’s Memory

2 0.5
n2d-standard-32 2 4 8 16 32 48
max 224 896 N2D standard
64 80 96 128 224

highmem 2 4 8 16 32 48
64 80

highcpu 2 4 8 16 32 48
64 80 96 128 224

Up to 224 vCPUs and 896 GB of memory

Largest general-purpose machine type
Higher memory-to-core ratios
 Compute-optimised machine family
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Ultra high performance for compute-intensive workloads
NVIDIA Tesla K80
Standard
HPC
High-memory
NVIDIA Tesla P4 Electronic Design Automation (EDA)
High-CPU
NVIDIA Tesla T4 Gaming
NVIDIA Tesla V100
Single-threaded applications
NVIDIA Tesla P100
 Compute-optimised

min
vCPU’s Memory

4 16
c2-standard-32
max 60 240 C2 standard 4 8 16 30 60

Compute intensive workloads

Highest performance per core
Cannot use regional persistent disks
 Memory-optimised machine family
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80 Ultra high-memory workloads
High-memory
NVIDIA Tesla P4
High-CPU Large in-memory databases like SAP HANA In-
NVIDIA Tesla T4
memory analytics
NVIDIA Tesla V100
NVIDIA Tesla P100
 Memory-optimised

M1
min
vCPU’s Memory
40 32
m1-ultramem-40
max 160 3844 M1 ultramem 40 80 160

megamem 96
M2 vCPU’s Memory
min 40 32 M2 ultramem 208 416
max 160 11,776 416
megamem

Intensive memory use

In-memory databases/ in-memory analytics
Cannot use regional persistent disks
 Shared core machine type
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Burstable workloads, cost-effective,
Standard
non-resource
NVIDIA Tesla intensive
K80 applications
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 Shared Core

e2-small 2
E2 micro
vCPU’s Memory
small 2
min 2 1
medium 2
max 2 4
N1 f1-micro 1

g1-small 1

Physical core available for short periods of time

CPU bursting capabilities
 Custom machine types
Available for custom
e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1
Customer defined CPU andM2memory
f1-micro N2
custom workloads
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
Custom

custom 2 48 224
E2 standard
4 64
N2 highmem 8 80
N2D 16 96
+ GPU highcpu 32 128
N1
 Predefined machine categories
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Graphics-intensive workloads
Ultra-memory
GPU
Standard
NVIDIA Tesla K80
High-memory
NVIDIA Tesla P4
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
 Predefined machine categories
Available for custom

e2-micro
e2-small General-purpose Compute-optimised Memory-optimised
e2-medium
E2 C2 M1
Shared core N1 M2
f1-micro N2
Standard
g1-small N2D
Mega memory
Ultra-memory
GPU
Standard
NVIDIA Tesla K80 1 2 4 8 16 30 32 40 48
High-memory
NVIDIA Tesla P4 60 64 80 96 128 160 208 224 416
High-CPU
NVIDIA Tesla T4
NVIDIA Tesla V100
NVIDIA Tesla P100
Managing Instances
Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone
Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone

Provisioning $$$: none

Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone

Staging $$$: none

Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone

Running $$$: instance, static IPs, disks

Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone

Suspend Stopping $$: static IPs, disks Terminated $$: static IPs, disks
Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone
Staging State
Staging

internal IP
external IP

system
image

Boot
Shielded VM’s
Shielded VM’s

Boot Process

Virtual Trusted
Integrity
Secure Boot Platform Module
Monitoring
(vTPM)

Measured
Boot
Running State
Running

Startup script set/get

metadata

SSH | RDP
Export system
image

Modify/Repair
Snapshot
persistent disk
Live Migrate

Migrate instance
to different zone
 Metadata

Project Bowtie

Linux and
Windows
scripts
Guest environment

curl http://metadata.google.internal/computeMetadata/v1/project/
curl http://metadata.google.internal/computeMetadata/v1/instance/
Running State
Running

Startup script set/get

metadata

SSH | RDP
Export system
image

Modify/Repair
Snapshot
persistent disk
Live Migrate

Migrate instance
to different zone
VM access

SSH RDP
Requires firewall rule Requires firewall rule
allow - tcp:22 allow - tcp:3389
Google Cloud console Connect using RDP
Cloudshell Powershell terminal
using CloudSDK Requires setting Windows
OS Login (use 2SV) password
Manually creating SSH key pair RDP Chrome extension
3rd party RDP client
Running State
Running

Startup script set/get

metadata

SSH | RDP
Export system
image

Modify/Repair
Snapshot
persistent disk
Live Migrate

Migrate instance
to different zone
 Live Migration

Project Bowtie

Network A

Region: us-west1
us-west1-a us-west1-b

subnet-1 10.0.2.0/24

bowtievm bowtievm

gcloud compute instances move bowtievm --zone us-west1-a --destination-zone us-west1-b

Instance Lifecycle
Resume Suspend

Reset

Provisioning Staging Running Stopping

vCPU Startup script set/get Availability

internal IP Shutdown
+ metadata Policy
external IP Script
Memory

SSH | RDP
Root disk Export system
Persistent disk
system
image Terminated Delete
image
Modify/Repair
Additional Disks
Boot Snapshot
persistent disk
Live Migrate

Migrate instance to
different zone
Compute Engine Billing
 Compute Engine Pricing
Each individual vCPU and each GB of memory is
billed separately - resource based

All vCPUs, GPUs, and GB of memory are charged by the sec

Instance uptime - number of seconds between when

you start an instance and when you stop an instance
(terminated)
 Reservations
Ensuring resources are available for when you need it
• Future increases in demand
• Planned or unplanned spikes
• Backup and disaster recovery
• Buffer

Include sustained use and committed use discounts

Apply only to Compute Engine, Dataproc and GKE VM’s
 Discount types
Sustained use discounts
Committed use discounts
Preemptible VM’s
 Sustained use discounts
Automatic discounts applied to vCPU, GPU and memory
up to 20% up to 30%
Usage level (% of % at which incremental is Usage level (% of % at which incremental is
month) charged month) charged

0% - 25% 100% of base rate 0% - 25% 100% of base rate

25% - 50% 80% of base rate 25% - 50% 86.78% of base rate

50% - 75% 60% of base rate 50% - 75% 73.3% of base rate

75% - 100% 40% of base rate 75% - 100% 60% of base rate

N2 | N2D N1 (predefined and custom)

(predefined and custom) Memory optimized
Compute optimized Shared-core
GPU’s
Sole tenant nodes
Applying sustained use discounts
Legend

= vCPU

n1-standard-4+12

n1-standard-4

Week 1 Week 2 Week 3 Week 4

Upgrade instance
Discount for 4vCPU’s
10% 20% 30%

Discount for additional 12vCPU’s

10%
 Committed use discounts
Purchased 1 year or 3 year contracts in return for
deeply discounted prices
vCPUs
Predictable/steady-state resources memory
GPUs
Local SSD
57% discount for most resources
70% for memory-optimized machine types
Apply at the project level, as well as share discounts
across multiple projects
Preemptible VMs

80% cheaper
Fixed pricing
Within 24 hours
No charge if <10min

Live migration/ Auto restart

Fault-tolerant applications
Storage Fundamentals
Types of Storage

Block File Object

2
DATA ID
METADATA
1 6

7 12 3 8

11 4 9 10 5
 Block Storage

Spinning Evenly sized blocks

hard drives Uniquely identifiable
Mountable
2

Solid State 1 6 Bootable

drives
7 12 3 8

11 4 9 10 5
File Storage

NFSv3
Network File System
Directory tree structure
Mountable
Not bootable
Object Storage

Unstructured data
Infinitely scalable
DATA ID
METADATA Not mountable
Not bootable

Cloud Storage
 Storage Performance Terms
I/O I/O queue depth IOPS Throughput Latency

4kb
256kb
4mb
I/O operations/s MB/s ms

Sequential Access Random Access

Performance
Persistent Disk Snapshots
 Persistent Disk Snapshots
Backup and restore of persistent disks
Global resources
Support for zonal and regional PDs
Incremental and automatically compressed
Snapshots are stored in Cloud Storage
Stored in regional or multi-regional location
Creating Snapshots

Bowtie-disk

Snapshot 1 (full snapshot)

INCREMENTAL

Snapshot 2
REFERENCE
from snapshot 1
Blocks that are different than snapshot 1
INCREMENTAL

Snapshot 3 REFERENCE
from snapshot 2
Blocks that are different than snapshot 2
Deleting Snapshots

Bowtie-disk

Snapshot 1 (full snapshot)

INCREMENTAL

Snapshot DELETED
2
REFERENCE
REFERENCE
fromfrom
snapshot 1
snapshot 1
Blocks that are different than snapshot 1
INCREMENTAL

Snapshot 3 REFERENCE
from snapshot 2
Blocks that are different than snapshot 2
 Scheduled Snapshots
Snapshot schedules
• Best practice for backups

• Must be in same region as pd

Attach Create with disk

Retention policy
options Source disk deletion rule
 Managing Snapshots
1 snapshot = 10min
Create regular schedules
Eliminate excessive snapshots images
Set schedule to off-peak hours
Windows - create VSS snapshots
Deployment Manager
Configuration

Defines the structure of your deployment

Must contain resources section
• list of resources to create
Configuration

3 Components
Name
Type
Properties
Configuration

name

Name
User-defined string name: larks-instance-dont-touch
Configuration
base type

Type
Base type type: [API].[VERSION].[RESOURCE]

Composite type type: gcp-types/[PROVIDER]:[RESOURCE]

type: gcp-types/compute-v1:addresses
Configuration

properties

Properties
Parameters for the resource type
zone, machineType, boot, sourceImage
Templates

Jinja Python
Templates

Template property
Deployment

deploy . update . delete

 Deployment

deploy . update . delete

gcloud deployment-manager deployments create bowtiedeploy --config bowtie-deploy.yaml
 Deployment

manifest

deploy . update . delete

gcloud deployment-manager deployments create bowtiedeploy --config bowtie-deploy.yaml
 Deployment

manifest

deploy . update . delete

gcloud deployment-manager deployments create bowtiedeploy --config bowtie-deploy.yaml
 Best Practices
Break up your configurations
Use references - enforces order resources are created
Preview your deployments using --preview flag
Automate the creation of resources
Use version control
• Previous known good config
• Audit trail
• Use config for CI/CD
Load Balancing
 Load Balancing
Distributes user traffic across multiple instances
Single point of entry with multiple backends
Fully distributed and software defined
Global and Regional
Serve content as close as possible to users
Autoscaling with health checks
Load Balancing Types
External HTTP(S) SSL Proxy Load TCP Proxy Load
Global
Load Balancing Balancing Balancing

Internal HTTP(S) Internal TCP/UDP TCP/UDP Network

Regional
Load Balancing Load Balancing Load Balancing

External HTTP(S) SSL Proxy Load TCP Proxy Load TCP/UDP Network
External
Load Balancing Balancing Balancing Load Balancing

Internal HTTP(S) Internal TCP/UDP

Internal
Load Balancing Load Balancing

HTTP(S) TCP UDP

Traffic External HTTP(S) Load Balancing TCP Proxy Load Balancing Network Load Balancing
Type Internal HTTP(S) Load Balancing Network Load Balancing Internal TCP/UDP Load Balancing
Internal TCP/UDP Load Balancing
 Load Balancer Types

HTTP(S) SSL Proxy TCP Proxy Network Internal

Backend Services
Health Checks

Session Affinity

Service Timeout

Traffic Distribution

Backends

Defines how Cloud Load Balancing distributes traffic

 HTTP(S) traffic management
Cross-region load balancing Content-based load balancing

Oregon Boston Switzerland Oregon Switzerland

IPv6 IPv4

bowtieinc.co URL map configuration:

/static/*=>backend service: static
/video=>backend service: video
/images=>backend service: images
GFE GFE GFE /static /video /images

us-west2-b us-east1-b europe-west6-a backend service: static backend service: video backend service: images

Instance Instance Instance

Group Group Group
Compute Compute Compute
Engine Engine Engine us-west2-b us-east1-b europe-west6-a

Compute Compute Compute Instance Instance Instance

Engine Engine Engine Group Group Group
Compute Compute Compute
Compute Compute Compute Engine Engine Engine
Engine Engine Engine

Compute Compute Compute

Engine Engine Engine

Compute Compute Compute

Engine Engine Engine
 HTTP(S) Load Balancer
Layer 7 Load Balancer
Laura in California Lark in New York Lisa in Zurich
Single Unicast IP address
Implemented on Google Front Ends (GFE)
bowtieinc.co Global, external, internal
Google Cloud Global Load Balancing HTTPS and SSL for encryption in transit
GFE GFE GFE
IPv4/IPv6 traffic
IPv6 traffic terminates at LB and is served
us-west2-b us-east1-b europe-west6-a
as IPv4 to backend
Distribute traffic by location or by content
Instance Group Instance Group Instance Group

Compute Compute Compute

Forwarding rules in place to distribute defined targets
Engine Engine Engine to target pools
Compute
Engine
Compute
Engine
Compute
Engine
URL maps direct requests based on rules
SSL certificates must be used for HTTPS (Google
Compute Compute Compute
Engine Engine Engine managed or self-managed)
Ports 80, 8080; 443 (HTTPS)

Global, proxy-based Layer 7 load balancer behind a single external IP address

 SSL Proxy
Laura in California Lark in New York
Client SSL sessions terminated at the load balancer

SSL traffic Global and external

connection 1
Distribute traffic by location only
Google Cloud Global Load Balancing (SSL Proxy)
SSL termination Single Unicast IP address
1.3.3.7:443
Layer 4 Load Balancer
connection 2 Support for TCP with SSL offload
us-west2-b us-east1-b IPv4/IPv6 traffic
IPv6 traffic terminates at LB and is served
Instance Group Instance Group as IPv4 to backend
Compute Compute
Engine Engine Forwarding rules in place to distribute defined targets
to target pools
Compute Compute
Engine Engine
Used for other protocols that use SSL; Websockets
Compute Compute
and IMAP over SSL
Engine Engine

Reverse proxy load balancer that distributes SSL traffic coming from the internet to VMs
 TCP Proxy
Laura in California Lark in New York

Client TCP sessions terminated at the load balancer

TCP traffic
connection 1 Forward traffic as SSL or TCP
Google Cloud Global Load Balancing (TCP Proxy) Intelligent routing: Route to locations that have capacity
TCP termination
1.3.3.7:110 Single Unicast IP address
Layer 4 Load Balancer
connection 2
Global and external
us-west2-b us-east1-b
Distribute traffic by location only
Instance Group Instance Group
Intended for non HTTP traffic
Compute Compute
Engine Engine IPv4/IPv6 traffic
Compute Compute
Engine Engine
IPv6 traffic terminates at LB and is served
as IPv4 to backend
Compute Compute
Engine Engine
Supports many well-known TCP ports

Reverse proxy load balancer that distributes TCP traffic coming from the internet to VMs
 Network Load Balancer
Laura in California Lark in New York Not a proxy
Responses from backend go directly to client
Regional and external
Network Load
Balancer
Network Load
Balancer
Network TCP/UDP Supports either TCP or UDP; not both
LB provides regional
Direct load balancing
server Support traffic on ports that are not supported by TCP
return proxy and SSL proxy
SSL decrypted by backends not by load balancer
Traffic distributed by protocol, scheme and scope
bowtieapproved.com bowtieinc.co
No TLS offloading or proxying
us-west2 us-east1
Multiple forwarding rules reference one target pool
Compute Compute
Engine Engine Other protocols use target instances
Self managed SSL certificates

Pass-through load balancer that distributes TCP and UDP traffic to VMs
 Internal Load Balancer
Laura in California Lark in New York

HTTP(S) Load Layer 4 Load Balancer

Balancer

Regional and internal

bowtie-project

bowtie-network
Supports either TCP or UDP; not both
VPC Routing
Balances internal traffic between instances
us-west2 us-east1 us-west2 Cannot be used to balance internet traffic

Web Frontend Web Frontend Web Frontend Traffic sent to backend directly; does not terminate
client connections
Instance Instance Instance
Group Group Group
When using forwarding rules
subnet-1 subnet-1 subnet-1 You must specify at least one and up to 5 ports by
Internal Load Internal Load Internal Load number
Balancer Balancer Balancer

You must specify ALL to forward traffic to all ports

Middleware Middleware Middleware

Instance Instance Instance

Group Group Group

Pass-through load balancer that distributes TCP and UDP traffic to VMs
 Load Balancer Types

HTTP(S) SSL Proxy TCP Proxy Network Internal

Instance Groups and Instance Templates
Instance Groups
A collection of VM instances that you can manage as a single entity.

Instance Group

Managed Instance Groups Unmanaged Instance Groups

Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

A collection of VM instances that you can manage as a single entity.

Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

Stateless serving workloads: website frontend, web servers, web apps

 Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

Stateless batch: high-performance or high throughput compute workloads

Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

Stateful workloads: use stateful managed instance groups

Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating
Managed Instance Groups (MIGs)

Autohealing
STATELESS
High AvailabilityKeeps VMs in RUNNING state
Regional (multi-zone)
Recreate VMs not in RUNNING state
STATELESS
Load Balancing
Application-based autohealing BATCH
Scalability
Recreate VMs when app is frozen or has crashed
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating
Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)

Zonal or Regional STATELESS

Load Balancing BATCH
Scalability
Regional provides higher availability
Autoscaling
Zonal MIGs are in one zone only STATEFUL
Google recommends regional MIGs WORKLOADS
Updates Auto-updating
 Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Load balancing can use instance groups to serve traffic
Autoscaling STATEFUL
Work together to know how much traffic can be handled WORKLOADS
Updates
LB health checks do not Auto-updating
send traffic to unhealthy instances
 Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Dynamically add or remove instances
Updates Auto-updating from the MIG
Scale up to meet load demands
Shrink as the load decreases to reduce costs
Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

Deploy new versions of software to instances

Update deployment happens automatically
Perform rolling updates
Partial rollouts for canary testing
Managed Instance Groups (MIGs)

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

Preemptible Instances
Containers
Network and Subnet
Unmanaged Instance Groups

Autohealing
STATELESS
High Availability
Regional (multi-zone)
STATELESS
Load Balancing BATCH
Scalability
Autoscaling STATEFUL
WORKLOADS
Updates Auto-updating

A collection of VM instances that you can manage as a single entity.

 Instance Templates
Resource used to create VM instances and MIGs
Existing instance template cannot be
updated or changed
Machine type
Boot disk image Save the configuration of an existing
VM and create a new one
Container image gcloud instance-templates create
Labels
To make changes, you can create
another one with similar properties
using the console.
Use to create
MIG or VM Use custom or public images

If you want to create a group of identical instances, you must use

an instance template to create a MIG
Instance Groups

Instance Templates

Managed Instance Groups Unmanaged Instance Groups

Introduction to Containers
Containers

consistent . efficient . standardized

Virtual Machines vs Containers

VM VM VM VM
App A App A App A App A
Runtime Runtime Runtime Runtime

Separate OS Guest OS Guest OS Guest OS Guest OS

per VM
Hypervisor
Infrastructure

Large overhead in CPU, memory and disk

Virtual Machines vs Containers

App I App J App K App L

Runtime Runtime Runtime Runtime
VM VM VM VM
App E App F App G App H
App A App B App C App D
Runtime Runtime Runtime Runtime
Runtime Runtime Runtime Runtime
App A App B App C App D
Guest OS Guest OS Guest OS Guest OS Runtime Runtime Runtime Runtime

Container Engine
Hypervisor
Host Operating System
Infrastructure Infrastructure

Abstracted OS
Container
 Docker Image breakdown

FROM ubuntu:12.04

Command # install the necessities

RUN apt-get update
Expose apt-get install -y apache2
echo 'Hello, bowtie lovers’ > /var/www/index.html
Working directories Layers ENV APACHE_RUN_USER www_data
ENV APACHE_RUN_GROUP www_data
Installed Software ENV APACHE_LOG_DIR /var/log/apache2

# define the port number the container should expose

Base Image EXPOSE 8080

# run the command

Docker Image CMD [“-D", “FOREGROUND"]

Starts with base image Dockerfile - used to create image

Each line in dockerfile creates a layer
All layers are read-only
docker image docker container
fashionista R/W layer Read/Write
Command Command
Expose Expose
Working directories Working directories Read Only
docker run fashionista
Installed Software Installed Software
Base Image Base Image

Docker Image Docker Container

Docker container is created from a Docker image

Containers can use the same image, yet will always have a
different read/write layer
 Container Registry

Dockerfile

Container
Image Container Registry
Docker Hosts
GKE and Kubernetes Concepts
What is Kubernetes?
Orchestration platform for containers

automate . schedule . run

What is GKE?

Cluster
What is GKE?

Cloud Load Balancing

Cluster
Node Pools
Automatic scaling
Automatic upgrades
Node auto-repair
Logging and Monitoring
Cluster Architecture

One or more Control Planes

Cluster
One or more Nodes
Control Plane responsible for
scheduling and management
Nodes run containerized apps
Nodes responsible for Docker runtime

Control Plane Nodes

Cluster Architecture

Cluster

Control Plane Endpoint of the cluster

API server Point of interaction with the cluster (API calls or kubectl)
kube scheduler Discovers and assigns newly created pods
kube controller
manager Runs all controller processes
kubectl
cloud controller
manager Runs controllers specific to the cloud provider
etcd Key-value store that stores the state of the cluster
Cluster Architecture

Cluster Node Node

kubelet kubelet
Agent for communication
with Control Plane
kube-proxy kube-proxy
Control Plane
runtime runtime
API server

kube scheduler

kube controller
manager
kubectl
cloud controller kubelet kubelet
manager kube-proxy kube-proxy
Network connectivity
runtime runtime
etcd Runs containers

Node Node
GKE Abstraction

Cluster Node Node

kubelet kubelet
kube-proxy kube-proxy
Control Plane runtime runtime

API
cluster
server
IP

kube scheduler
kube controller
manager
cloud controller
kubectl manager

etcd
kubelet kubelet
kube-proxy kube-proxy

runtime runtime

Node Node
 Node Pools

Group of nodes within a cluster with the same

cluster-1
configuration
custom-node-pool default-node-pool
One or multiple nodes

Custom node pools - Useful for pods that

require more resources

Manually or automatically upgraded

 Cluster Types
Zonal Clusters
Single-zone Multi-zonal Regional

us-east1-b us-east1-b us-east1-c us-east1-d us-east1-b us-east1-c us-east1-d

cotton-cluster cotton-cluster cotton-cluster

Control Control Control Control Control

Plane Plane Plane Plane Plane

Node Node Node Node Node Node

Node Node Node Node Node Node

Node Node Node

Node Node Node Node Node Node

 Cluster Types
Private Cluster

customer managed project google-managed project

bowtie-vpc

cotton-cluster

vpc
Control
peering Plane
Node Node Node

internet
 Cluster Version

Release Channel
Rapid Regular (default) Stable Specific Version
Several weeks 2-3 months after 2-3 months after Use a specific supported
after upstream releasing in Rapid releasing in Regular version of Kubernetes for
open source GA a given workload
Kubernetes Components

Cluster Node Node

kubelet kubelet
kube-proxy kube-proxy
Control Plane runtime runtime

API server

kube scheduler
kube controller
manager
cloud controller
kubectl manager

etcd
kubelet kubelet
kube-proxy kube-proxy

runtime runtime

Node Node
GKE Cluster and Node Management
 Node Pools

Group of nodes within a cluster with the same

cluster-1
configuration
custom-node-pool default-node-pool
One or multiple nodes

Custom node pools - Useful for pods that

require more resources

Manually or automatically upgraded

 Cluster Types
Zonal Clusters
Single-zone Multi-zonal Regional

us-east1-b us-east1-b us-east1-c us-east1-d us-east1-b us-east1-c us-east1-d

cotton-cluster cotton-cluster cotton-cluster

Control Control Control Control Control

Plane Plane Plane Plane Plane

Node Node Node Node Node Node

Node Node Node Node Node Node

Node Node Node

Node Node Node Node Node Node

 Cluster Types
Private Cluster

customer managed project google-managed project

bowtie-vpc

cotton-cluster cotton-cluster

vpc
Control
peering
Node Node Node Plane

internet
 Cluster Version

Release Channel
Rapid Regular (default) Stable Specific Version
Several weeks 2-3 months after 2-3 months after Use a specific supported
after upstream releasing in Rapid releasing in Regular version of Kubernetes for
open source GA a given workload
 Cluster upgrades
Control plane and nodes do not always run the same version
at all times
A control plane is always upgraded before its nodes
• Zonal - Cannot launch or edit workloads during upgrade
• Regional - Each control plane is upgraded one by one
Auto-upgrade enabled by default - best practice
Manual upgrade - cannot upgrade control plane more than
one minor version at a time
• Maintenance window and exclusions available
 Node and Node pool upgrades
Auto-upgrade enabled by default - best practice
Manual upgrade available
• Maintenance window and exclusions available
Pods scheduled to run on another node during upgrade
Upgrade is complete only when
• All nodes have been recreated
• Cluster is in the desired state
 Surge Upgrades
Control the number of nodes GKE can upgrade at a time
Use surge upgrade parameters
max-surge-upgrade max-unavailable-upgrade
Num of additional nodes added to the Num of nodes that can be simultaneously
node pool during an upgrade unavailable during an upgrade

Higher number = More parallel upgrades Higher number = More disruptive

During upgrades, GKE brings down at most the sum of (max-surge-upgrade + max-unavailable-upgrade)
Pods and Object Management
Kubernetes Objects

Kubernetes Object
persistent entity
Object spec Object status

desired state current state

described by you described by
Kubernetes

Represents the state of the cluster

Kubernetes Objects
 Manifest file

Version of the Kubernetes API

The kind of object you want to create
Identifies the object (name, UID, namespace)

The desired state for this object

 Pod concepts

Pod
container container

shared storage volumes

shared storage
for containers

Auto-assigned unique IP shared networking

Pod concepts

Pod Pod Pod

container container container

shared storage shared storage shared storage

shared networking shared networking shared networking

Replicas
Pod concepts

Node
Remains on the node until:
The pod’s process is complete

The pod is deleted

Pod
The pod is evicted from the node due to
container lack of resources

shared storage The node fails

shared networking
 Namespaces
Version of the Kubernetes API
The kind of object you want to create

The namespace for this object

default
kube-system
kube-public
kube-node-lease
 Labels

Key/value pair used for your object

key value
Pod lifecycle
initial pod phase for at least one container all containers in the pod
container(s) to start in the pod is running have terminated successfully

create scheduled finish

pending running succeeded

unknown failed

state of the pod could

one or more containers in
not be obtained due to
the pod have terminated
communication error
unsuccessfully
 Creating pods

The kind of object you want to create

Specifies how many instances of a pod will run

Specification for a pod

(pod template)
 Workloads
Deployments - runs multiple replicas of your app and automatically repla

StatefulSets - used for apps that requires persistent storage

DaemonSets - ensures that every node in the cluster runs a copy of
a pod
Jobs - used to run a finite task until completion
CronJobs - similar to jobs but runs until completion on a schedule
ConfigMaps - configuration info for any workload to reference
Kubernetes Services
Kubernetes approach to networking

10.211.33.7 NEW 10.63.52.98

10.34.89.26 10.133.72.51

Ephemeral
State =
What is a service?

Service
 What is a service?

Persistent single IP

Internal and external cluster

Service
Load balancing

Scaling

Service Service
Service components
services.yaml deployment.yaml

DNS name of the

service
Forward requests to
pods with this label

The type of service it is

Port number exposed

internally in cluster

Port that containers are

listening on
Service components
services.yaml deployment.yaml

DNS name of the

service
Forward requests to
pods with this label

The type of service it is

Port number exposed

internally in cluster

Port that containers are

listening on
Selector and Labels

bowtie-cluster selector:
app: inventory
Service
10.176.133.7 80

192.168.2.1 192.168.7.1

192.168.2.15 80 80 192.168.2.41

labels: Node
labels: 2
Nginx app: inventory Nginx app: shop

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
Service Types

ClusterIP Multi-port Services

NodePort ExternalName
LoadBalancer Headless
ClusterIP

SDK or Cloud Shell (GKE)

bowtie-cluster
Service
10.176.133.7 80 port

192.168.2.1 192.168.7.1

192.168.2.15 80 target-port 80 192.168.2.41

Node 2
Nginx Nginx

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
 ClusterIP

Name of the service

Label used for the selector

Service Type

Port number exposed internally in cluster

Port that containers are listening on
NodePort
[NODE_IP]:32002

pre-configured range: 30000-32767

bowtie-cluster

10.216.72.4 32002 NodePort 32002 10.216.72.5

10.176.133.7 80
Service

10.35.1.17 80 target-port 80 10.35.3.1

Node 2
Nginx Nginx

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
 NodePort

Name of the service

Label used for the selector

Service Type

Port number exposed internally in cluster

Port that containers are listening on
Nodeport within the specified port range
LoadBalancer
Load Balancer

bowtie-cluster
Service
10.176.133.7 80 port

192.168.2.1 192.168.7.1

192.168.2.15 80 target-port 80 192.168.2.41

Node 2
Nginx Nginx

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
 LoadBalancer

Name of the service

Label used for the selector

Service Type

Port number exposed internally in cluster

Port that containers are listening on
Multi-port Services
Load Balancer

bowtie-cluster
Service
10.176.133.7 9752 80 ports

192.168.2.1 192.168.7.1

192.168.2.15 9752 80 target-port 80 192.168.2.41

Node 2
App-B Nginx Nginx

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
 Multi-port Services

Name of the service

Label used for the selector

Service Type

Port number exposed internally in cluster

Port that containers are listening on

Port number exposed internally in cluster

Port that containers are listening on
ExternalName

bowtie-sql2.bowtieinc.private

bowtie-cluster
Service
10.176.133.7 80

192.168.2.1 192.168.7.1

192.168.2.15 80 80 192.168.2.41
Node 2
Nginx Nginx

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
ExternalName

Internal DNS
name
External DNS
name redirect
Headless

bowtie-cluster
Service
DNS: private-bowtie 80 port

192.168.2.1 192.168.7.1

192.168.2.15 80 target-port 80 192.168.2.41

Node 2
nginx1 nginx2
nginx1.private-bowtie nginx2.private-bowtie
bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
Headless

Service Type
Service Types

ClusterIP Multi-port Services

NodePort ExternalName
LoadBalancer Headless
Ingress for GKE
Ingress
HTTPS Load Balancer bowtieinc.co

Ingress
/products /discontinued
bowtie-cluster
Service Service
80 port 80

192.168.2.1 192.168.7.1

192.168.2.15 80 target-port 21337 192.168.2.41

Node 2
products discontinued

bowtieinc-pod-1 bowtieinc-pod-2
80
Node 1 Node 2
Ingress
products-service.yaml
ingress.yaml

discontinued-service.yaml
Network Endpoint Group (NEG)
Global Load
IP Address Balancer
traffic
Forwarding Rule

Target HTTP Proxy

kind: Ingress
URL Map

Backend Service Backend Service

/products /discontinued
GKE Cluster

Network Endpoint Network Endpoint

Group (NEG) Group (NEG)

Pod
PodIP
PodIP
IP
Pod
PodIP
PodIP
IP kind: Service
 Health Checks
Default and inferred parameters are used if there are no specified health check par

Should be explicitly defined by using a Backend Config custom resource

definition (CRD)

• Anthos Ingress controller

• >1 container
• Specific port for LB health check
Backend service’s health check
• healthCheck parameter of a BackendConfig CRD referenced by service
 SSL Certificates
Load Balancer

Self-managed
Google-managed Self-managed
as Secrets
• Completely managed by • Managed and shared with • Provision your own
Google Google Cloud certificates

• Do not support wildcard • Provision your own • Create a secret to hold

domains certificates the certificate

• List the certificate in • Refer to the secret for use

annotation for use

Multiple certificates: specify in Ingress manifest

GKE Storage Options
 GKE Storage Options

DATABASE NAS OBJECT STORAGE BLOCK STORAGE

Cloud SQL Cloud Spanner Datastore Filestore Cloud Storage Persistent Disk
Docker Storage

Pod

container

Bind Mount tmpfs

cannot be shared amongst stored in the host’s
other containers memory

Volumes
Filesystem can be shared amongst
other containers Memory

docker area
 Kubernetes Storage Abstractions
Ingress
Volumes
Basic storage unit that
decouples the storage from the
container and tie it to the pod
Service

ReplicaSet
Volume Persistent
Ephemeral Volume
Storage Pod Pod Pod
Durable
Storage

Persistent Volume
Claim

ConfigMap Secret
Persistent Volume
Volumes
created when the Pod is created
terminated when pod is terminated or deleted

Pod

Container

Pod spec
how directory is created
storage medium used
directory's initial contents

Volume
 Types of volumes
emptyDir
• empty directory that containers in the Pod can read and write from
ConfigMap
• provides a way to inject configuration data into Pods
Secret
• used to make sensitive data available to applications
Downward API
• used to make Downward API data available to applications
PersistentVolumeClaim
• provision durable storage to be used by applications
 Persistent Volume
Ingress

Service

ReplicaSet

Pod Pod Pod

Requests:
Persistent Volume Specific size
Claim Access Mode
Storage Class
Persistent Disk
persist independently Persistent Volume
dynamically provisioned
Storage Classes

StorageClass resources

Default storage class

 Persistent Volume - Provisioning
Ingress

Service

ReplicaSet

Pod Pod Pod StatefulSet

Persistent Volume
Set reclaim policy to Claim
RETAIN
to prevent persistent
volume deletion
Persistent Volume
 Persistent Volume Access
Access Modes

ReadWriteOnce ReadOnlyMany ReadWriteMany

mounted as read-write mounted as read-only mounted as read-write
by a single node by many nodes by many nodes

Regional persistent disks Zonal persistent disks

Replicate between 2 zones If no zone specified, one is chosen at random
Can failover workloads (HA) Pods referencing disk are scheduled in same zone

***Size of persistent disks determine IOPS

Cloud VPN
 Cloud VPN
Connects your peer network to your VPC network through
an IPsec VPN connection.

On-premesis Cloud provider Another Cloud VPN

IPsec tunnel over the public internet

Encrypted by one VPN gateway, and then decrypted by the other
VPN gateway.
 Cloud VPN
Regional Service
Site to site VPN only (no site to client)
Allows Private Google Access for on-premises hosts
Supports up to 3Gbps per tunnel
Dynamic and static routing
Supports IKEv1 and IKEv2 using Shared Secret
 Types of Cloud VPN

Classic VPN HA VPN

99.9% SLA 99.99% SLA
Static and dynamic routing Dynamic routing only
1 external IP address for a 2 external IPs to be configured
single interface for 2 interfaces
Deprecating functionality in 2021 New default VPN
Classic VPN External IP

Interface 0
Regional External IP

on-premises network
bowtie-project

bowtie-network us-east1

On-premises VPN
Classic Cloud VPN
Gateway Gateway
Internet 36.91.33.7

VPN tunnel 0 (encrypted)

HA VPN External IP
and BGP IP
Interface 0
Regional External IP

bowtie-project on-premises network

bowtie-network

us-east1 On-premises VPN

VPN tunnel 0 (encrypted) Gateway 1
HA Cloud VPN
Gateway
bowtie-gw-a Internet
VPN tunnel 1 (encrypted)
On-premises VPN
Gateway 2

Cloud Router
router-a
(ASN 65001)
on-premises subnets
and resources

(ASN 65002)
BGP BGP

External IP
and BGP IP

Interface 1
Regional External IP
 When to use Cloud VPN
Public internet access is needed
Peering location is not available
Budget constraints
High speeds/ low latency not needed
Outgoing traffic (egress) from GCP
Cloud Interconnect
 Cloud Interconnect
Low latency, highly available connection between your on-
premises and Google Cloud VPC networks
Directly accessible internal IP addresses - Private Google Access
Does not traverse the public internet
Dedicated connection
Not encrypted
Expensive
Dedicated Interconnect

On-premises network

bowtie-network (bowtieinc) Colocation facility Subnet: 192.168.0.0/24

bti-interconnect (bowtieinc)

us-east1: 10.68.0.0/20

Zone 1
On-premises
Cloud Router Router
Compute Google Peering
Link-local address: Edge Link-local address:
10.68.0.7 162.76.13.37 162.76.13.38 192.168.0.16

8 x 10 Gbps connections (80 Gbps total)

2 x 100 Gbps connections (200 Gbps total)
Partner Interconnect
Service provider network On-premises network

bowtie-network

us-east1

Compute

10.68.0.7 192.168.0.16

Colocation facility

Cloud Router On-premises

Google Peering Service Provider Service Provider Router
ASN: 65002 Edge Peering Edge Peering Edge 162.76.13.3
Link-local address: 8
162.76.13.37 internet

50 Mbps to 50 Gbps VLAN attachments (50Gbps total)

 Direct Peering
Direct peering connection between your on-premises network and
Google's edge network
100 locations in 33 countries
Direct egress pricing available
Direct Peering connection with Google is FREE
 CDN Interconnect
Enables select third-party CDN providers to establish direct peering
links with Google's edge network

Direct traffic from VPC networks to the provider's network

Reduced pricing on egress costs

 When to use Cloud Interconnect
Prevent traffic from traversing the public internet
Dedicated physical connection
Extension of your VPC network
High speed/low latency is needed - 200 Gbps
Heavy outgoing traffic (egress) from GCP
Private Google Access
App Engine Overview
 App Engine Overview
Fully managed, serverless platform to develop and host web apps
PaaS service
Code or containers - Python, Java, Node.js, Go, Ruby, PHP, or .NET
Autoscaling based on load
Versions - Allow for rollbacks, migrating or traffic splitting
Support for connecting to external storage
Standard and Flexible environments
 Standard and Flexible environments

App Engine
Standard Flexible
Apps run in sandbox environment Apps run in docker containers
Specific versions of runtimes used Any version of runtimes used
Run for free or at very low cost No free quota available
Designed for sudden and Designed for consistent traffic
extreme spikes of traffic
Pricing based on VM resources
Pricing based on instance hours
Managed VMs
 Deploying an application
gcloud app deploy Top level
One or more services

Application

Loosely-coupled
Service 1 Service 2
Versions of the service logical components

Version Version Version Version

Instance Instance Instance Instance

VMs provided to run

versions of the service
 Managing Instances
Automatically create and shut down instances
Specify a number of instances to run
Specify a scaling type
app.yaml
Automatic scaling
• based on metrics like request rate and response latencies
Basic scaling
• creates instances when your application receives requests

Manual scaling

• specifies the number of instances that continuously run

Traffic Migration

Service

Version 1 Version 2

Instance Instance Instance Instance

Traffic Splitting

Service

90% traffic 10% traffic

Version 1 Version 2

Instance Instance Instance Instance

Cloud Functions
 Cloud Functions
Serverless
FaaS - Function as a Service
Runtime - Python, Java, Node.js, Go, .NET core
Event-driven
beta
Triggers - HTTP, Pub/Sub, Cloud Storage (Firestore, Firebase)
Billing - time + resources provisioned (memory)
Free Tier
How Cloud Functions work

trigger
How Cloud Functions work

event data

stateless

trigger
Cloud Functions
Cloud Storage
 Cloud Storage
Consistent, scalable, large-capacity, highly durable
object storage - not file or block
Worldwide accessibility and worldwide storage locations
Use for data files, text files, pictures, videos
Excels for content delivery, big data sets and backups
Buckets and Objects
Cloud Storage buckets

basic container Upon creation

that holds your globally unique name
data geographic location
NAME
Organize your data storage class
Access control access control
labels (optional)
Bucket

Region Dual-region Multi-region

Standard Nearline Coldline Archive
Uniform Fine-grained

Standalone IAM IAM + ACLs

Cloud Storage objects

Object

Bucket
object data metadata name:value

properties
name, storage class, etc

flat namespace
/bowties/spring2021/plaidbowtie.jpg
Storage Classes
Hot data Cold data

Standard Nearline Coldline Archive

Maximum Low-cost for infrequently Very low-cost for infrequently Lowest-cost
availability accessed data accessed data archival storage
No storage 30 day min. 90 day min. 365 day min.
duration storage duration storage duration storage duration
Analytical Data backup and Data backup and Cold data storage
workloads and data archiving data archiving disaster recovery
transcoding

$0.02 /GB/month $0.01 /GB/month $0.004 /GB/month $0.0012 /GB/month

$ Data access $$ Data access $$$ Data access
Storage Classes
Hot data Cold data

Standard Nearline Coldline Archive

>99.99% multi-regions 99.95% multi-regions 99.95% multi-regions 99.95% multi-regions
dual-regions dual-regions dual-regions dual-regions
99.99% regions 99.9% regions 99.9% regions 99.9% regions

99.999999999% annual durability

 Access Control
Can be used together

IAM Access Control Signed URLs Signed Policy

List (ACL) Documents
standard IAM time-limited read/write
permissions defines who has access access URL
specify what can be
to your buckets and
access the object for the uploaded to a bucket
permissions inherited objects, as well as what
level of access they duration of time you
hierarchically
have specify
 IAM and ACLs

IAM Access Control

Recommended over
List (ACL)
ACLs Granular permissions
Two levels of granularity Entry = permission + scope
project or bucket level
CAUTION: ACLs overlap IAM roles
Roles available: Primitive, Standard, Legacy
Legacy roles are equivalent to ACLs
Access Control

Signed URLs
Access an object using the URL for the duration of time you specify

Allows users without credentials to perform specific actions on a resource

Actions are taken as a user or service account

Do not need an account - just the URL

gsutil signurl -d 10m private-key.json

gs://bowties/spring2021/plaidbowtie.jpg
Cloud Storage
Object Versioning and Lifecycle Management
Versioning

Object
generation
metageneration current version
v4
immutable Bucket
v3
objects are never edited in place
v2
always replaced with a new version
v1
replacement is marked as end of object
lifecycle and beginning of a new one
 Object Lifecycle Management

Setting a Time to Live (TTL) for objects

Delete or archive non-current versions
Downgrade storage class to save $$$

Use cases
Downgrade the storage class of objects older than 365 days to Coldline Storage
Delete objects created before January 1, 2020
Keep only the 3 most recent versions of each object in a bucket with versioning enabled
Object Lifecycle Management

Age
CreatedBefore

Rules Conditions CustomTimeBefore

DaysSinceCustomTime
DaysSinceNoncurrentTime
Action
IsLive
Any set of conditions 1 or multiple MatchesStorageClass
NoncurrentTimeBefore
Delete
for any action NumberofNewerVersions
SetStorageClass

Example > delete object > condition met > delete object
> change storage > condition met
class

> change sc to Nearline > condition met > move to Coldline

> change sc to Coldline > condition met
 Cloud Storage considerations
Changes are in accordance to object creation date
Once an object is deleted, it cannot be undeleted
Lifecycle rules can take up to 24 hours to take effect
Test lifecycle rules in development first
Cloud SQL
 Cloud SQL
Fully managed, relational database service (RDBMS)
DBaaS (Database as a Service)
Low latency, transactional, relational db workloads
MySQL, PostgreSQL and SQL Server - NEW
Replication - Read Replicas
High Availability
 Cloud SQL
On-demand and automatic backups
Point in time recovery
30TB storage capacity
Automatic storage increase
Encryption at rest and in transit
Billed for instance, persistent disk and egress traffic
 Cloud SQL

us-east1

us-east1-b

Shared-core HDD or SSD

db-f1-micro, db-g1-small Up to 30TB
Standard
db-n1-standard-1 > 96 Persistent Disk
Cloud SQL
High memory
db-n1-highmem-2 > 96 Connecting
to Cloud SQL
• Public or Private IP
• Cloud SQL Proxy
• Authorize a network
• External applications
Cloud SQL Proxy

TCP standard port

TCP secure tunnel

Client Applications Proxy Client Proxy Server Instance

Local connection
 Replication
Primary Instance

us-east1 europe-west3 On-premises

network

us-east1-b us-east1-c europe-west3-a

External
Cloud SQL Read Replica

Cross-region
Read Replica

Read Replica

read-only

To create read replicas Automated backups must be enabled

Binary logging must be enabled point-in-time recovery to be enabled
One backup must have been created after binary logging was enabled
Promoting replicas
Primary Instance

us-east1 europe-west3

us-east1-b us-east1-c europe-west3-a

Cloud SQL Cross-region

Read Replica

Manual promotion Read Replica

Reasons for promoting replicas Regional migration (planned)

Disaster recovery (unplanned)
Promoting replicas
Primary Instance

us-east1 europe-west3

us-east1-b us-east1-c europe-west3-a

Cloud SQL Cross-region

Read Replica

Manual promotion Read Replica

Promote to primary
High Availability
Client application

us-east1

us-east1-b us-east1-d

IP Address X

Primary Instance Standby Instance

Regional
Persistent Disk

Persistent Disk Persistent Disk

Synchronous Replication
High Availability - Failover
Client application

us-east1

us-east1-b us-east1-d

IP Address X

Primary Instance Standby Instance

Regional
Persistent Disk

Persistent Disk Persistent Disk

Synchronous Replication
High Availability - Failback
Client application

us-east1

us-east1-b us-east1-d

IP Address X

Primary Instance Failback Standby Instance

Regional
Persistent Disk

Persistent Disk Persistent Disk

Synchronous Replication
Backups

Types of backups
us-east1 us-west-3
On-demand
Create at any time
us-east1-b Persist until you delete them

Cloud SQL Cloud SQL

Automated
Backup Backup 4 hour backup window
Cloud SQL Occur everyday
7 most recent backups are retained

Persistent Disk Point in time recovery (PITR)

Recover an instance to a specific point in time
Always creates a new instance
Cloud Spanner
 Cloud Spanner
Fully managed relational database service that is both
strongly consistent and horizontally scalable
DBaaS (Database as a Service)
Supports schemas, ACID transactions, and SQL queries
Globally distributed
Handles replicas and sharding
Synchronous data replication
 Cloud Spanner
Automatic scaling and node redundancy
Up to 99.999% availability
Data layer encryption, audit logging, IAM integration
Designed for financial services, ad tech, retail and
global supply chain, gaming

Pricing: $0.90 /node/hr + $0.30/GB/mo.

Instances
Instance
Configuration Node count

us-east1

Nodes

us-east1-b us-east1-c us-east1-d

Instance

Cloud Spanner Cloud Spanner Cloud Spanner Full copy of

Leader Replica Replica
database

3 node minimum recommended for Production

Instances

us-east1
Nodes
us-east1-b us-east1-c us-east1-d

Instance

Cloud Spanner Cloud Spanner Cloud Spanner

Leader Replica Replica

Cloud Spanner Cloud Spanner Cloud Spanner

Leader Replica Replica

Cloud Spanner Cloud Spanner Cloud Spanner

Leader Replica Replica
 Performance

10,000 queries QPS of reads or 2,000 QPS of writes

2TB of storage per node
Add nodes to increase data throughput and QPS
Scale nodes automatically using Cloud Monitoring metrics
triggered by Cloud Functions
NoSQL Databases
NoSQL Databases
 Cloud Bigtable
Use cases Built for real-time app serving &
Time-series data
large-scale analytical workloads
Regional Service
Marketing data
Automated replication
Financial data
Store large amounts of
IoT data
single-keyed data
Graph data Add nodes when you need them
Cluster resizing
Storage Engine uses
Ideal data source for MapReduce
Batch MapReduce operations operations
Stream processing/analytics
High-priced
Machine-learning applications

Fully managed, wide-column NoSQL database designed for terabyte to petabyte-scale

workloads that offers low latency and high throughput.
 Cloud Datastore
Use cases
Product Catalogs High-availability of reads and writes

User profiles Atomic transactions

Transactions based Automatic scaling

on ACID properties SQL-like query language (GQL)
Strong and eventual consistency
Encryption at Rest
Datastore emulator
Being retired in favour of Cloud
Provides local emulation of the production Firestore in 2021
Datastore environment
Component of the Google Cloud SDK's
gcloud tool

Fully managed, highly scalable NoSQL document database built for automatic scaling,
high performance, and ease of application development
Firestore for Firebase
Serverless
Fully Managed Document Multi-region replication
Data
Flexibility
Collection Expressive querying
Realtime updates
Offline support

Free tier Secure

Realtime Database
Simpler version of Firestore

Flexible, scalable NoSQL cloud database to store and sync data for client and
server-side development
 Firebase

A mobile app development platform that provides tools and cloud services to help
enable developers to develop apps faster and more easily
 Memorystore
Fully Managed

Fully managed
High Availability
Scale as needed
Secure
Use cases
Always up to date
Caching

Gaming (leaderboards, user profiles)

Stream processing

Fully managed service for either Redis or Memcached in-memory data store to build
application caches
NoSQL Databases
Big Data Overview
 What is Big Data?
Massive amounts of data that would typically be too
expensive to store, manage, and analyze using
traditional database systems.

Traditional databases are not cost effective

• No flexibility for storing unstructured data
• Inability to accommodate “real time” data
• Lacks support for petabyte-scale data volumes
• Apache Hadoop & NoSQL to the rescue
• Extremely complex to deploy and manage
 Why is Big Data so important?
When this data is captured, formatted, manipulated,
stored and then analyzed, can help a company make
better decisions (business value).
• Gain useful insight
• Increase revenue
• Get or retain customers
• Improve operations
• Better with Machine Learning
Big Data Services
Big Data Services

Big Query
 Big Query
Fully managed, petabyte scale, low cost analytics data
warehouse
Serverless
Real-time analytics insertion
Use Standard SQL for querying
Process external data
• Dataproc, Dataflow, Cloud Storage, Big Table, Cloud SQL,
Google Drive
• Parquet, ORC, Google sheets
 Big Query

Data Transfer Service (DTS)

• 145 Services - Teradata, Amazon S3, Azure Blob, etc.
Run open source data science workloads
• Spark, Tensorflow, Dataflow, Apache Beam, MapReduce
Automatic backups
Automatic high availability
Data Governance and security
• Geographic data control
• Data encryption at rest and in-transit
Big Data Services
Big Data Services

Composer Dataflow Dataproc

Managed workflow Fully managed Fully managed Spark
orchestration service, processing service and Hadoop service
built on Apache for executing Apache
Can be used to
Airflow Beam pipelines for
replace on-prem
batch and realtime
Hadoop infrastructure
data streaming
Big Data Services
Big Data Services

DataLab Pub/Sub Dataprep

An easy-to-use Fully-managed, real-time Serverless, intelligent
interactive tool for data messaging service that data service for visually
exploration, analysis, allows you to send and exploring, cleaning, and
visualization, and receive messages preparing structured and
machine learning. between independent unstructured data for
applications. analysis, reporting, and
machine learning
Big Data Services
Machine Learning
 What is Machine Learning?

Functionality that enables software to perform tasks

without any explicit programming or rules.
• Trained to recognize patterns in collected data using
algorithmic models
• Collected data includes video, images, speech or text
• Cloud is an efficient place for ML due to the use of
massive computation at scale
• Better with Big Data
 What can Machine Learning do?
Categorize images such as photos, faces, or satellite
imagery
Look for keywords in text documents or emails
Flag potentially fraudulent transactions
Enable software to respond accurately to voice commands
Translate languages in text or audio
Machine Learning Platform
Machine Learning
Sight

Vision Video Intelligence

Pre-trained machine Pre-trained machine
learning models that allow learning models that
you to assign labels to automatically recognize a
images and quickly classify vast number of objects,
them into millions of places, and actions in
predefined categories stored and streaming video
Machine Learning
Language

Natural Language Translation

Derive insights from Translation enables you to
unstructured text using dynamically translate between
Google machine languages using Google’s
learning pre-trained or custom
machine learning models
Machine Learning
 Conversation

Dialog Flow Speech-to-Text Text-to-Speech

Natural language Accurately convert Enables developers to
understanding platform that speech into text using synthesize natural-
makes it easy to design and Google’s AI sounding speech with
integrate a conversational technologies 100+ voices, available
user interface into your in multiple languages
application or device and variants
Machine Learning
AutoML

AutoML
Fully Managed suite of
machine learning
products
Operations Suite ( Formerly Stackdriver)
Operations Suite
Available for GCP and AWS
VM monitoring with agents
Available for on-premises environments
Google Cloud native integration

Monitoring Logging Error Reporting Debugger Trace Profiler

Application Performance Management (APM)

A suite of tools for logging, monitoring, and application diagnostics

Operations Suite

Monitoring Logging Error Reporting Debugger Trace Profiler

Application Performance Management (APM)

Operations Suite

Monitoring Logging Error Reporting Debugger Trace Profiler

Application Performance Management (APM)

 Cloud Monitoring
EC2

Collects metrics to provide insights

Dashboards and charts
Workspaces are needed to use cloud monitoring
Agents are recommended to monitor VMs
Works together with cloud logging
Support to monitor GKE
Monitoring Logging Error Reporting
Alerting Debugger Trace Profiler

Application Performance Management (APM)

Collects measurements, or metrics, to help you understand how your applications
and system services are performing
 Cloud Logging
Concepts Types of Logs
Audit Logs who did what, where, and when
Logs Viewer only shows logs from one project
Access Transparency Logs actions taken
Log Entry records a status or an event
by Google staff
Logs are a named collection of log
Agent Logs
entries within a GCP resource
Retention period how long
your logs are kept Real-time log management and analysis
Tight integration with monitoring
Platform, system and application logs
Export logs to other sources

Monitoring Logging Error Reporting Debugger Trace Profiler

Application Performance Management (APM)

Central repository for log data from multiple sources

Error Reporting

ERROR

Counts, analyzes, and aggregates all the errors

in your GCP environment
Alerts you when a new application error occurs
Integrated into Cloud Functions and GAE Standard
Issue tracking integration
In beta for GCE, GKE, GAE Flexible, AWS EC2
Monitoring Logging Error Reporting Debugger Trace Profiler
Go, Java, Node.js, .Net, PHP, Python, Ruby
Application Performance Management (APM)

Real time error monitoring and alerting

 Debugger

Debug a running application with no latency

“Snapshot” the call stack in your application
Logpoints allow you to inject logging into running services
Can be hooked into remote Git repo - Github, GitLab, Bitbucket
Can be installed on non-GCP environments
Monitoring Logging Error Reporting Debugger Trace Profiler
Java, Go, Node.js, Python, .Net, PHP, Ruby
Application Performance Management (APM)
Inspect the state of a running application in real time, without stopping or slowing
it down
 Trace

Helps to understand how long it takes your

application to handle incoming requests (latency)
Collects latency data from cloud resources and apps
Integrated with GAE Standard
Can be installed on GCE, GKE, and GAE
Can be installed on non-GCP environments
Monitoring Logging Error Reporting Debugger Trace Profiler
C#, Go, Java, Node.js, PHP, Python, Ruby
Application Performance Management (APM)

Collects latency data from App Engine, HTTPS load balancers and applications
 Profiler

Helps discover patterns of resource consumption

Low-profile
Needs profiling agent to be installed
Can be installed on GCE, GKE, GAE
Can be installed on non-GCP environments
Monitoring Logging Go,
Error Java, Node.js,
Reporting Python
Debugger Trace Profiler

Application Performance Management (APM)

Continuously gathers CPU usage and memory allocation information from your
applications
Operations Suite

Monitoring Logging Error Reporting Debugger Trace Profiler

Application Performance Management (APM)