bq-project

Google BigQuery (BigQuery) Project

This blueprint contains all the necessary Terraform modules to build and deploy a BigQuery project on Google Cloud.

Introduction Google BigQuery (BigQuery)

Google BigQuery is a fully-managed, serverless data system in which querying data is made possible. Database does not need to be constantly monitored, and users can levarage data and analyze the data.

1. The Rotation Period rotation_period is set to 90 days indicated by 7776000s seconds,
2. The Destory Schedulded Duration is destroy_scheduled_duration is set to 30 days indicated by 2592000 seconds.
3. The IAM Permissions and Roles roles/cloudkms.cryptoKeyEncrypterDecrypter is assigned

Pre-requisite for Google BigQuery (BigQuery)

1. The Principal (user or group) must enablw BigQuery API in their Google Cloud Project
2. Have access to the GCP Project ID
3. You will need an existing project with billing enabled and a user with the “Project owner” IAM role on that project.
4. Note: to grant a user a role, take a look at the Granting and Revoking Access documentation.

Disclaimer

• The present GCP Terraform Module in this project is set up and intended to be implemented in either a FedRAMP-High or IL5 (Impact Level 5) environment using the Assured Workloads within the Google Cloud Platform (GCP) organization.
• Assured Workloads in both environments ensures that sensitive data and workloads in GCP adhere to the rigorous security standards mandated by the DoD, making it suitable for government agencies.

Variables

name	description	type	required	default
dataset_description	Provides a discription of the deployed BigQuery Dataset.	string	✓
dataset_id	This is the dataset id.	string	✓
kms_key_names	Key names and base attributes. Set attributes to null if not needed.	map(object({…}	✓
kms_keyring_name	Keyring attributes.	object({…})	✓
main_project_id	Project ID.	string	✓
region	GCP Region to deploy into.	string	✓
tables	BigQuery tables.	map(map(string))		{}

Outputs

name	description	sensitive
dataset_name	Dataset name.
keyring	Keyring name.
materialized_view_ids	Materialized view IDs.
materialized_views	Materialized views.
table_ids	Table IDs.
tables	Tables.
view_ids	View IDs.
views	Views.

How to deploy the Terraform Code. The Deployment Steps

You should see this README and some terraform files.

1. Update the Variables in the variables.tf and also the properties within the keys variables. For reference update the following variables and associated properties

• project_id with your GCP Project ID
  
• email with your email address
  
• location with the GCP Location
  
• keyring with the location of the keyring and the name of the keyring, for example
  

  default = {
    location = "us-east4"
    name     = "may-bq-keyring"
  }

• keys with the right properties, update the updated-the-runner-key-name , labels = { "team" = , iam = { roles/cloudkms.cryptoKeyEncrypterDecrypter = ["user:YOUR-EMAIL-ADDRESS]

2. There is a sample terraform.tfvars.sample available as well.
3. Although each use case is somehow built around the previous one they are self-contained so you can deploy any of them at your will. The usual terraform commands will do the work. To provision this example, run the following from within this directory:

terraform init to get the plugins
terraform plan to see the infrastructure plan
terraform apply to apply the infrastructure build
terraform destroy to destroy the built infrastructure

It will take a few minutes. When complete, you should see an output stating the command completed successfully, a list of the created resources. The Output will look like following

Outputs:

id = "projects/my-project/datasets/dataset_name"
keyring = {
  "id" = "projects/my-project/locations/us-east4/keyRings/may-bq-keyring-"
  "location" = "us-east4"
  "name" = "may-bq-keyring-8"
  "project" = "my-project"
  "timeouts" = null /* object */
}
materialized_view_ids = {}
materialized_views = {}
self_link = "https://bigquery.googleapis.com/bigquery/v2/projects/my-project/datasets/dataset_name"
table_ids = {}
tables = {}
view_ids = {}
views = {}