0% found this document useful (0 votes)

110 views

Google Analytics BigQuery Setup V2

Uploaded by

sathishkumar R

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

110 views

Google Analytics BigQuery Setup V2

Uploaded by

sathishkumar R

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 64

Google Analytics BigQuery Setup

Written by Himanshu Sharma, Founder of Optimize Smart

© Copyright 2023 Optimize Smart. All rights reserved.

No part of this publication can be reproduced or transmitted in any form or by any means, mechanical or electronic
including photocopying and recording or by any information storage and retrieval system, without permission in
writing from the author (except by a reviewer, who may quote brief passages and/or show brief video clips in a
review).

Disclaimer: Great efforts have been made to ensure the accuracy, timeliness and completeness of the contents
(written text, graphics) provided in this book. However neither the author nor Optimize Smart make any guarantee
/warranty of accuracy, timeliness and completeness of the information provided in this book or any website this
book links out to. No warranty/guarantee may be created or extended by a sales representative or written sales
materials. Similarly neither the author nor Optimize Smart make any guarantee/warranty that the websites linked
to this book are free from viruses, adware or other malicious programs. Websites listed in this book may have
changed or disappeared between when this book was written and when it is read. Neither the author nor Optimize
Smart will be liable for any losses or damages (including but not limited to: commercial, special, incidental,
consequential damages) arising from the use of the contents in this book. This book is for information purposes
only. The advice and strategies contained herein may not be suitable for every situation. If professional assistance is
required, the services of a competent consultant/agency should be sought. The fact that an organization or website
is referred to in this book, as a citation and/or potential source of further information does not mean that the
author or Optimize Smart endorses the information the organization or website may provide and/or
recommendations it may make.

Copyright Optimize Smart 2023 Page 2

About the author
● Founder, OptimizeSmart.com

● Over 15 years of experience in digital

analytics and marketing

● Author of four best-selling books on digital

analytics and conversion optimization

● Nominated for Digital Analytics Association Awards for Excellence

● Runs one of the most popular blogs in the world on digital analytics

● Consultant to countless small and big businesses over the decade

Website: www.optimizesmart.com

Linkedin: https://www.linkedin.com/in/analyticsnerd

Facebook: https://www.facebook.com/optimizesmart

Copyright Optimize Smart 2023 Page 3

Following are our most downloaded ebooks for
career advancement:

#1 Sales and ROI Accelerator (150+ pages)

WHAT'S INSIDE: My step-by-step blueprint for generating record sales and ROI
by leveraging analytics data.

#2 Set Up Your Google Analytics 4 (GA4) Account Correctly And Fast (70 pages)

WHAT'S INSIDE: Learn to set up your GA4 account correctly and fast using this
62 points checklist.

FAQ: Do you show “How” to do each item on the checklist? If so, with
screenshots?
Yes. There are links to the articles with detailed step by step instructions.

FAQ: Does this ebook cover GTM too?

Yes.

#3 Google Tag Manager Data Layers (100+ pages)

WHAT'S INSIDE: My step-by-step blueprint for getting started with data layers.
Get the only ebook on GTM data layers ever published. Learn the JavaScript
behind it.

Copyright Optimize Smart 2023 Page 4

#4 Learn to Read E-Commerce Reports in Google Analytics (100+ pages)

WHAT'S INSIDE: My step-by-step guide to reading both standard and enhanced

e-commerce reports in Google Analytics. E-commerce reports are the most
valuable reports in Google Analytics.

#5 Do you want better skills in digital analytics and marketing? If yes, then
register for the free training:

Here’s what we’re going to cover…

1. Why digital analytics is the key to online business success.

2. The number 1 reason why most marketers are not able to scale their
advertising and maximize sales.
3. Why Google and Facebook ads don’t work for most businesses & how to
make them work.
4. Why you won’t get any competitive advantage in the marketplace just by
knowing Google Analytics.
5. The number 1 reason why conversion optimization is not working for your
business.
6. How to advertise on any marketing platform for FREE with an unlimited
budget.
7. How to learn and master digital analytics and conversion optimization in
record time.

Copyright Optimize Smart 2023 Page 5

Get helpful tips on a daily basis
If you are the type of person who finds it helpful to receive short tips on building
your website traffic, improving conversions, fixing attribution issues and learning
about analytics in general, then follow me on LinkedIn. I post a few short tips each
day.

Click here and follow me on LinkedIn

Introduction to BigQuery
Google BigQuery is one of the products of the Google Cloud Platform.

The big query is an enterprise-level data warehouse from Google which is used to
provide business intelligence in the form of reports and dashboards.

It is a data storage and management system which is used to bring data from
several data sources (like Google Analytics, Google Ads, Facebook etc) for the
purpose of reporting and analysis.

When you use BigQuery, you can manipulate Google Analytics data in a way that is
many times simply not possible by using the Google Analytics user interface.

For example, certain dimensions and metrics combinations can not be queried
together whether you use the Google Analytics user interface or Google Analytics
API. But BigQuery has no such limitations.

This is one of the biggest advantages of using BigQuery. It makes advanced data
segmentation and analysis possible.

Copyright Optimize Smart 2023 Page 6

It removes most of the limitations which come when you use the GA user
interface or API for querying analytics data.

Prerequisites for using BigQuery

Following are the prerequisites for using Google BigQuery:

#1 You need a good working knowledge of SQL so that you can query data in
BigQuery. This is the primary requirement.

#2 You need a Google Cloud Platform account with billing enabled. In order to
enable the billing, you would need a valid credit card.

How to send Google Analytics data to

BigQuery without using Google
Analytics 360
Normally you would require access to GA360 in order to send Google Analytics
(aka Universal Analytics) data to BigQuery.

But with the help of a third-party solution/connector, it is possible to send data

from Google Analytics to BigQuery without 360.

Copyright Optimize Smart 2023 Page 7

Note: Here I am assuming that you are using the Google Cloud Platform for the
first time.

You should now see a screen like the one below:

Step-3: Type ‘bigquery’ in the search box:

Step-4: Click on the first search result:

Copyright Optimize Smart 2023 Page 8

You should now see a screen like the one below:

Step-5: Click on the ‘CREATE PROJECT’ button:

Copyright Optimize Smart 2023 Page 9

Step-6: Name your project (say ‘Google Analytics’):

Step-7: Click on the ‘EDIT’ button to change the project ID:

Copyright Optimize Smart 2023 Page 10

You should now see a screen like the one below:

Step-8: Change the project ID to something which is easier to recognize:

Copyright Optimize Smart 2023 Page 11

Note: Finding a meaningful project ID is not going to be easy as many IDs have
already been taken. Once you change the ID, you cannot change it later.

Step-9: Click on the ‘Create’ button.

You should now see a screen like the one below:

Copyright Optimize Smart 2023 Page 12

You now have access to the BigQuery sandbox.

Step-10: Click on the ‘Upgrade’ button at the top right-hand side:

Step-11: Click on the ‘CREATE BILLING ACCOUNT’ button:

Copyright Optimize Smart 2023 Page 13

Step-12: Select your billing country from the ‘Country’ drop-down menu and then
read and accept the terms of service by clicking on the checkbox:

Step-13: Click on the ‘Continue’ button.

Step-14: Set your account type, enter tax information, name, address and credit
card details and then click on the ‘START MY FREE TRIAL’ button:

Copyright Optimize Smart 2023 Page 14

Copyright Optimize Smart 2023 Page 15
Step-15: Navigate to https://console.cloud.google.com/bigquery and make sure
that ‘Google Analytics’ project is selected.

Step-16: Click on your project ID (this ID could be different in your case):

Copyright Optimize Smart 2023 Page 16

Step-17: On the right-hand side of your screen, find and click on the button
‘Create Dataset’:

Step-18: Name your dataset (say GA_data_set) and then click on the ‘Create
dataset’ button at the bottom:

Copyright Optimize Smart 2023 Page 17

We are going to use this new data set for storing data from Google Analytics.

You would now need to use a third-party solution for sending Google Analytics
data to BigQuery. This is going to cost you based on your account usage.

Regardless of how much it costs you, it would most likely still be cheaper than
paying $150k per year to Google for using the GA360 (which comes with the free
connection to BigQuery).

You can either search for such third-party solutions online or use the one that I
use which is ‘Supermetrics for BigQuery‘. They provide a free 14 days trial of their
solution. No credit card is required.

Step-19: Navigate to ‘Supermetrics for BigQuery‘ page.

Copyright Optimize Smart 2023 Page 18

Step-20: Click on the ‘Start a free trial’ button:

Step-21: Fill out the form and then click on the ‘Send’ button:

Copyright Optimize Smart 2023 Page 19

You should now be redirected to the page like the one below in your Google Cloud
platform account:

Step-22: Click on the ‘Select a project’ drop-down menu:

Step-23: Click on ‘Google Analytics’ project:

Copyright Optimize Smart 2023 Page 20

Step-24: Type ‘supermetrics google analytics’ in the search box:

Step-25: Press the enter key and then click on ‘Google Analytics by Supermetrics’:

Copyright Optimize Smart 2023 Page 21

Step-26: Click on the ‘Enroll’ button:

You should now see the ‘Data Source enrolled‘ status:

Copyright Optimize Smart 2023 Page 22

Note: If you are using the BigQuery Sandbox then you will not able to enrol as this
Supermetrics connector requires active billing. So when you click on the ‘Enroll’
button, you should see a message box like the one below:

Step-27: Click on the ‘CONFIGURE TRANSFER’ button:

Copyright Optimize Smart 2023 Page 23

You should now see a screen like the one below which is about creating a data
transfer:

Copyright Optimize Smart 2023 Page 24

Step-28: Keep the data source (‘Google Analytics by Supermetrics’) intact and
move on to the next step:

Step-29: Type a meaningful name for the ‘Transfer config name’ field:

Step-30: Keep the scheduling options (which specify when the transfer will run) to
‘Start now’ (unless you want to change it) and move on to the next step:

Copyright Optimize Smart 2023 Page 25

Step-31: Keep the ‘Repeats’ setting to ‘Daily’ to have new data added once a day
and move on to the next step:

Copyright Optimize Smart 2023 Page 26

Note: The ‘Start date and run time’ setting is locked for editing when the ‘Start
now’ setting is selected.

Step-32: Select the data set you created earlier from the drop-down menu:

Copyright Optimize Smart 2023 Page 27

We are going to use this data set for storing Google Analytics data in BigQuery.

Step-33: Click on the ‘CONNECT SOURCE’ button:

Step-34: Click on the ‘Accept Agreement’ button:

Copyright Optimize Smart 2023 Page 28

Step-35: Click on ‘Sign in with Google’ button:

Step-36: Click on the name of the Google account which is associated with both
your Google Analytics account and BigQuery project:

Copyright Optimize Smart 2023 Page 29

Step-37: Click on the button ‘Authorize with Google Analytics’:

Step-38: Click on the name of the Google account which is associated with your
Google Analytics account:

Copyright Optimize Smart 2023 Page 30

Step-39: Click on the ‘Allow’ button:

Copyright Optimize Smart 2023 Page 31

Step-40: Click on the ‘Continue’ button:

You should now see a screen like the one below:

Copyright Optimize Smart 2023 Page 32

Step-41: Select the Google Analytics view (from which you send data to BigQuery)
from the ‘Accounts’ drop-down menu:

Step-42: Click on the ‘Submit’ button.

You should now see the ‘Source Connected‘ message below ‘Third party
connection‘:

Copyright Optimize Smart 2023 Page 33

Step-43: Click on the ‘SAVE’ button to save the transfer and also start the initial
data transfer:

Copyright Optimize Smart 2023 Page 34

You should now see a screen like the one below:

Through the ‘Run History’ section, you can monitor the current progress of all of
your data transfers.

Copyright Optimize Smart 2023 Page 35

When the data transfer is in progress you see the message ‘The transfer run is in
progress:

When the data transfer is complete you see the message ‘The transfer run has
completed successfully:

Copyright Optimize Smart 2023 Page 36

Step-44: Click on the ‘SQL WORKSPACE’ link from the left-hand side navigation:

Step-45: Navigate to the data set named ‘GA_data_set’ (which we created

earlier).

This data set lists all the data tables automatically created by supermetrics:

Copyright Optimize Smart 2023 Page 37

Step-46: Click on the data tables one by one to see what data it contains.

For example, GA_CONTENT_ data table contains the GA content data:

Copyright Optimize Smart 2023 Page 38

The GA_EVENTS_ data table contains the GA event data:

At this point, you can query a particular set of data by clicking on the ‘Query Table’
button:

Copyright Optimize Smart 2023 Page 39

That’s how you can send Google Analytics data to BigQuery without using Google
Analytics 360.

Copyright Optimize Smart 2023 Page 40

How to connect Google Analytics 4
with BigQuery?
Now let’s dive into this step-by-step guide to connecting your Google Analytics 4
property to BigQuery.

Follow the steps below to connect Google Analytics 4 with BigQuery and then
send the GA data to it:

Step-1: Navigate to your BigQuery account:

https://console.cloud.google.com/bigquery

Step-2: At the top of your screen click on the drop-down menu:

Step-3: Click on the ‘New Project’ button:

Copyright Optimize Smart 2023 Page 41

We are going to use this new project for collecting and querying Google Analytics
4 data into BigQuery.

Step-4: Name your new project and then click on the ‘Create’ button:

Step-5: At the top of your screen click on the drop-down menu:

Copyright Optimize Smart 2023 Page 42

Step-6: Click on the name of the project you want to switch to (in our case ‘DBRT
GA4’):

Congratulations! You are now in the ‘DBRT GA4’ project.

You will know that you are in the right project because the project name will
appear at the top of your screen:

Copyright Optimize Smart 2023 Page 43

Step-7: Log in to your Google Analytics 4 property and click on ‘Admin’ under the
left-hand side reporting menu.

Step-8: Click on ‘BigQuery Linking’ under ‘Product Linking’:

Copyright Optimize Smart 2023 Page 44

Step-9: Click on the ‘Link’ button:

Step-10: Click on ‘Choose a BigQuery project’:

Copyright Optimize Smart 2023 Page 45

Step-11: Select the BigQuery project where you want to send your GA4 data and
then click on the ‘Confirm’ button. In our case that project is ‘DBRT GA4’:

Step-12: Select your data location and then click on the ‘Next’ button.

Data location is the cloud region where your data is stored.

Copyright Optimize Smart 2023 Page 46

Step-13: ‘Edit’ your data stream(s) if required. Otherwise, move on to the next
step. By default, all of the data streams are selected.

You have the option to select a specific data stream for which you will be
exporting the data to BigQuery. Just click on ‘Edit’ if you want to edit your data
streams:

Copyright Optimize Smart 2023 Page 47

Step-14: Click on the checkbox “Include advertising identifiers for mobile app
streams” if you have a mobile app and you want to export mobile advertiser
identifiers. Otherwise, move on to the next step.

Step-15: Select the frequency of your data import to BigQuery by selecting both
‘Daily’ and ‘Streaming’ settings and then click on the ‘Next’ button:

Copyright Optimize Smart 2023 Page 48

You should now see a screen like the one below:

Step-16: Click on the ‘Submit’ button. You should now see a screen like the one
below:

Congratulations!

Copyright Optimize Smart 2023 Page 49

Your GA4 account is now successfully linked to your Big Query project.

Step-17: Navigate to https://console.cloud.google.com/apis/dashboard

Step-18: Make sure your BigQuery project for GA4 is selected:

Step-19: Click on the ‘+ENABLE APIS AND SERVICES’ button:

You should now see a screen like the one below:

Step-20: Search for “Big Query” and then click on the ‘Big Query API’ button:

Step-21: Click on the ‘Manage’ button:

Step-22: Click on the ‘Credentials’ link from the left-hand side navigation menu:

Step-23: You would now need to add a firebase service account.

This account would be used to export Google Analytics 4 data to BigQuery. To do

that click on the ‘+create credentials’ button:

Step-24: Click on the ‘Service account’ link to create a new service account:

You should now see a screen like the one below:

Step-25: Enter [email protected] as service
account name:

Step-26: Click on the ‘Create’ button:

You should now see a screen like the one below:

Step-27: Click on the ‘DONE’ button.

You should now see your new service account listed like the one below:

Step-28: Wait for a full 24 hours.

It usually takes around 24 hrs for your GA4 data to be available in your BigQuery
project.

Step-29: Navigate back to your BigQuery account and make sure that your project
used for collecting GA4 data is selected:

Step-30: Scroll down and then click on the project ID of the project which collects
GA4 data:

You should now be able to see a data set named “analytics_<property_id>“:

For each Google Analytics 4 property that is linked to BigQuery, a single dataset
named “analytics_<property_id>” is added to your BigQuery project.

Property ID refers to your Analytics Property ID, which you can find in the
property settings for your GA4 property.

In my case, the property ID is 207472454

That’s why the name of my data set is ‘analytics_207472454‘

This data set contains the following two data tables which contain your GA4
data:

1. events_(<number of days>)

2. events_intraday_<current date>

You are doing Google Analytics all

wrong. Here is why...

I have dealt with hundreds of Google Analytics accounts in my career.

I have seen a lot of issues from incorrect tracking code, selecting the wrong KPIs to
analyzing data without using custom reports or advanced segments.

But do you know the biggest issue of all in Google analytics?....

It is the “misinterpretation of analytics data”.

Many marketers make the mistake of crediting conversions to the wrong

marketing channel.

And they seem to be making this mistake over and over again.

They give the credit for conversions to the last touchpoint (campaign, ad, search
term...).

They can’t help themselves because they believe that the Google Analytics reports
are ‘what you see is what you get’.

But they are actually ‘what you interpret is what you get’.

This has resulted in marketers making wrong business decisions and losing money.

All the data you see in Google Analytics reports today lies to you unless you
know exactly how to interpret it correctly.

For example, let's talk about direct traffic.

All untagged or incorrectly tagged marketing campaigns from display ads to emails
could be reported as direct traffic by Google.

Whenever a referrer is not passed, the traffic is reported as direct traffic by

Google.

Mobile applications don’t send a referrer. Word/PDF documents don’t send a

referrer.

‘302 redirects’ sometimes cause the referrer to be dropped. Sometimes browsers

don’t pass the referrer.

During an HTTP to HTTPS redirect (or vice versa) the referrer is not passed
because of security reasons.

All such traffic is reported as direct traffic by Google.

So on the surface, it may look like that most people are visiting your website
directly but this is not usually the case.

But this analysis does not end here, because you are still not looking at the
complete picture.

People do not always access your website directly and then make a purchase
straight away.

They are generally exposed to multiple marketing channels ( display ads, social
media, paid search, organic search, referral websites, email etc) before they
access your website directly.

Before they make a purchase.

So if you are unaware of the role played by prior marketing channels, you will
credit conversions to the wrong marketing channels.

Like in the present case to direct traffic.

To get this type of understanding you need to understand and implement web
analytics.

But you learn data analysis and data interpretation from web analytics and not
from Google Analytics.

The direction in which your analysis will move will determine the direction in
which your marketing campaigns will move.

You get that direction from ‘web analytics’ and not from ‘Google Analytics’.

Web/Digital analytics is not about Google Analytics (GA) or Google Tag Manager
(GTM). It is about analyzing and interpreting data, setting up goals, strategies
and KPIs.