BigQuery: Updates samples for BigQuery Beta 2 (do not merge until release) (GoogleCloudPlatform#1178)

* updates client.extract_table() sample

* updates client.load_table_from_file() sample

* updates client.load_table_from_uri() sample

* updates parameterized query samples

* updates query samples

* updates dataset quickstart

* fixes typo in sample

* updates simple app sample

* updates snippets

* updates stream data sample

* updates user credentials sample

* removes job_id generation

* Update BQ lib to 0.28.0.

Use max_results in list_rows.

* updates public dataset queries to use 'bigquery-public-data' instead of outdated 'publicdata'

* updates readme and help sections

Author: alixhami

bigquery/cloud-client/README.rst

@@ -70,6 +70,18 @@ Install Dependencies
 Samples
 -------------------------------------------------------------------------------
 
+Simple Application
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+
+
+To run this sample:
+
+.. code-block:: bash
+
+    $ python simple_app.py
+
+
 Quickstart
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
@@ -82,7 +94,7 @@ To run this sample:
     $ python quickstart.py
 
 
-Sync query
+Query
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 
@@ -91,26 +103,35 @@ To run this sample:
 
 .. code-block:: bash
 
-    $ python sync_query.py
+    $ python query.py
 
-    usage: sync_query.py [-h] query
+    usage: query.py [-h] [--use_standard_sql]
+                [--destination_table DESTINATION_TABLE]
+                query
     
-    Command-line application to perform synchronous queries in BigQuery.
+    Command-line application to perform queries in BigQuery.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python sync_query.py \
-              'SELECT corpus FROM `publicdata.samples.shakespeare` GROUP BY corpus'
+        $ python query.py '#standardSQL
+              SELECT corpus
+              FROM `bigquery-public-data.samples.shakespeare`
+              GROUP BY corpus
+              ORDER BY corpus'
     
     positional arguments:
-      query       BigQuery SQL Query.
+      query                 BigQuery SQL Query.
     
     optional arguments:
-      -h, --help  show this help message and exit
+      -h, --help            show this help message and exit
+      --use_standard_sql    Use standard SQL syntax.
+      --destination_table DESTINATION_TABLE
+                            Destination table to use for results. Example:
+                            my_dataset.my_table
 
 
-Async query
+Parameterized Query
 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 
@@ -119,23 +140,29 @@ To run this sample:
 
 .. code-block:: bash
 
-    $ python async_query.py
+    $ python query_params.py
 
-    usage: async_query.py [-h] query
+    usage: query_params.py [-h] {named,positional,array,timestamp,struct} ...
     
-    Command-line application to perform asynchronous queries in BigQuery.
+    Command-line app to perform queries with parameters in BigQuery.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python async_query.py \
-              'SELECT corpus FROM `publicdata.samples.shakespeare` GROUP BY corpus'
+        $ python query_params.py named 'romeoandjuliet' 100
+        $ python query_params.py positional 'romeoandjuliet' 100
     
     positional arguments:
-      query       BigQuery SQL Query.
+      {named,positional,array,timestamp,struct}
+                            samples
+        named               Run a query with named parameters.
+        positional          Run a query with positional parameters.
+        array               Run a query with an array parameter.
+        timestamp           Run a query with a timestamp parameter.
+        struct              Run a query with a struct parameter.
     
     optional arguments:
-      -h, --help  show this help message and exit
+      -h, --help            show this help message and exit
 
 
 Snippets
@@ -202,20 +229,21 @@ To run this sample:
 
     $ python load_data_from_file.py
 
-    usage: load_data_from_file.py [-h] dataset_name table_name source_file_name
+    usage: load_data_from_file.py [-h] dataset_id table_id source_file_name
     
     Loads data into BigQuery from a local file.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python load_data_from_file.py example_dataset example_table         example-data.csv
+        $ python load_data_from_file.py example_dataset example_table \
+            example-data.csv
     
     The dataset and table should already exist.
     
     positional arguments:
-      dataset_name
-      table_name
+      dataset_id
+      table_id
       source_file_name  Path to a .csv file to upload.
     
     optional arguments:
@@ -233,25 +261,26 @@ To run this sample:
 
     $ python load_data_from_gcs.py
 
-    usage: load_data_from_gcs.py [-h] dataset_name table_name source
+    usage: load_data_from_gcs.py [-h] dataset_id table_id source
     
     Loads data into BigQuery from an object in Google Cloud Storage.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python load_data_from_gcs.py example_dataset example_table         gs://example-bucket/example-data.csv
+        $ python load_data_from_gcs.py example_dataset example_table \
+            gs://example-bucket/example-data.csv
     
     The dataset and table should already exist.
     
     positional arguments:
-      dataset_name
-      table_name
-      source        The Google Cloud Storage object to load. Must be in the format
-                    gs://bucket_name/object_name
+      dataset_id
+      table_id
+      source      The Google Cloud Storage object to load. Must be in the format
+                  gs://bucket_name/object_name
     
     optional arguments:
-      -h, --help    show this help message and exit
+      -h, --help  show this help message and exit
 
 
 Load streaming data
@@ -265,24 +294,25 @@ To run this sample:
 
     $ python stream_data.py
 
-    usage: stream_data.py [-h] dataset_name table_name json_data
+    usage: stream_data.py [-h] dataset_id table_id json_data
     
     Loads a single row of data directly into BigQuery.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python stream_data.py example_dataset example_table         '["Gandalf", 2000]'
+        $ python stream_data.py example_dataset example_table \
+            '["Gandalf", 2000]'
     
     The dataset and table should already exist.
     
     positional arguments:
-      dataset_name
-      table_name
-      json_data     The row to load into BigQuery as an array in JSON format.
+      dataset_id
+      table_id
+      json_data   The row to load into BigQuery as an array in JSON format.
     
     optional arguments:
-      -h, --help    show this help message and exit
+      -h, --help  show this help message and exit
 
 
 Export data to Cloud Storage
@@ -296,25 +326,26 @@ To run this sample:
 
     $ python export_data_to_gcs.py
 
-    usage: export_data_to_gcs.py [-h] dataset_name table_name destination
+    usage: export_data_to_gcs.py [-h] dataset_id table_id destination
     
     Exports data from BigQuery to an object in Google Cloud Storage.
     
     For more information, see the README.rst.
     
     Example invocation:
-        $ python export_data_to_gcs.py example_dataset example_table         gs://example-bucket/example-data.csv
+        $ python export_data_to_gcs.py example_dataset example_table \
+            gs://example-bucket/example-data.csv
     
     The dataset and table should already exist.
     
     positional arguments:
-      dataset_name
-      table_name
-      destination   The desintation Google Cloud Storage object.Must be in the
-                    format gs://bucket_name/object_name
+      dataset_id
+      table_id
+      destination  The destination Google Cloud Storage object. Must be in the
+                   format gs://bucket_name/object_name
     
     optional arguments:
-      -h, --help    show this help message and exit
+      -h, --help   show this help message and exit
 
 
 

bigquery/cloud-client/README.rst.in

@@ -16,13 +16,15 @@ setup:
 - install_deps
 
 samples:
+- name: Simple Application
+  file: simple_app.py
 - name: Quickstart
   file: quickstart.py
-- name: Sync query
-  file: sync_query.py
+- name: Query
+  file: query.py
   show_help: true
-- name: Async query
-  file: async_query.py
+- name: Parameterized Query
+  file: query_params.py
   show_help: true
 - name: Snippets
   file: snippets.py

bigquery/cloud-client/export_data_to_gcs.py

@@ -19,47 +19,43 @@
 For more information, see the README.rst.
 
 Example invocation:
-    $ python export_data_to_gcs.py example_dataset example_table \
+    $ python export_data_to_gcs.py example_dataset example_table \\
         gs://example-bucket/example-data.csv
 
 The dataset and table should already exist.
 """
 
 import argparse
-import uuid
 
 from google.cloud import bigquery
 
 
-def export_data_to_gcs(dataset_name, table_name, destination):
+def export_data_to_gcs(dataset_id, table_id, destination):
     bigquery_client = bigquery.Client()
-    dataset = bigquery_client.dataset(dataset_name)
-    table = dataset.table(table_name)
-    job_name = str(uuid.uuid4())
+    dataset_ref = bigquery_client.dataset(dataset_id)
+    table_ref = dataset_ref.table(table_id)
 
-    job = bigquery_client.extract_table_to_storage(
-        job_name, table, destination)
+    job = bigquery_client.extract_table(table_ref, destination)
 
-    job.begin()
-    job.result()  # Wait for job to complete
+    job.result()  # Waits for job to complete
 
     print('Exported {}:{} to {}'.format(
-        dataset_name, table_name, destination))
+        dataset_id, table_id, destination))
 
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(
         description=__doc__,
         formatter_class=argparse.RawDescriptionHelpFormatter)
-    parser.add_argument('dataset_name')
-    parser.add_argument('table_name')
+    parser.add_argument('dataset_id')
+    parser.add_argument('table_id')
     parser.add_argument(
-        'destination', help='The desintation Google Cloud Storage object.'
+        'destination', help='The destination Google Cloud Storage object. '
         'Must be in the format gs://bucket_name/object_name')
 
     args = parser.parse_args()
 
     export_data_to_gcs(
-        args.dataset_name,
-        args.table_name,
+        args.dataset_id,
+        args.table_id,
         args.destination)

bigquery/cloud-client/load_data_from_file.py

@@ -19,7 +19,7 @@
 For more information, see the README.rst.
 
 Example invocation:
-    $ python load_data_from_file.py example_dataset example_table \
+    $ python load_data_from_file.py example_dataset example_table \\
         example-data.csv
 
 The dataset and table should already exist.
@@ -30,38 +30,37 @@
 from google.cloud import bigquery
 
 
-def load_data_from_file(dataset_name, table_name, source_file_name):
+def load_data_from_file(dataset_id, table_id, source_file_name):
     bigquery_client = bigquery.Client()
-    dataset = bigquery_client.dataset(dataset_name)
-    table = dataset.table(table_name)
-
-    # Reload the table to get the schema.
-    table.reload()
+    dataset_ref = bigquery_client.dataset(dataset_id)
+    table_ref = dataset_ref.table(table_id)
 
     with open(source_file_name, 'rb') as source_file:
         # This example uses CSV, but you can use other formats.
         # See https://cloud.google.com/bigquery/loading-data
-        job = table.upload_from_file(
-            source_file, source_format='text/csv')
+        job_config = bigquery.LoadJobConfig()
+        job_config.source_format = 'text/csv'
+        job = bigquery_client.load_table_from_file(
+            source_file, table_ref, job_config=job_config)
 
-    job.result()  # Wait for job to complete
+    job.result()  # Waits for job to complete
 
     print('Loaded {} rows into {}:{}.'.format(
-        job.output_rows, dataset_name, table_name))
+        job.output_rows, dataset_id, table_id))
 
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(
         description=__doc__,
         formatter_class=argparse.RawDescriptionHelpFormatter)
-    parser.add_argument('dataset_name')
-    parser.add_argument('table_name')
+    parser.add_argument('dataset_id')
+    parser.add_argument('table_id')
     parser.add_argument(
         'source_file_name', help='Path to a .csv file to upload.')
 
     args = parser.parse_args()
 
     load_data_from_file(
-        args.dataset_name,
-        args.table_name,
+        args.dataset_id,
+        args.table_id,
         args.source_file_name)