Skip to content

Connect BigQuery to Infograph

What it brings in: Every table and view the user can read.

Connect BigQuery 4 steps to set up

What you need

  • A Google Cloud project with BigQuery, and permission to create service accounts and grant IAM roles in it.
  • Permission to create a service account key. Organizations created on or after 3 May 2024 block key creation by default, with the iam.disableServiceAccountKeyCreation policy.
  • BigQuery bills each query by the bytes it scans, and every sync scans the tables you pick.

Steps

  1. Create a service account

    In the Google Cloud console, go to IAM & Admin, then Service accounts, and select your project. Click Create service account, give it a name such as infograph, then click Create and continue.

  2. Give it read access

    Grant two roles on the project. BigQuery User lets it run queries and list your datasets. BigQuery Data Viewer lets it read the tables. To limit what Infograph can read, grant Data Viewer on single datasets instead of the project. Click Continue, then Done. The same in gcloud, with your project ID:

    gcloud iam service-accounts create infograph --display-name="Infograph"
gcloud projects add-iam-policy-binding [project-id] \
  --member="serviceAccount:infograph@[project-id].iam.gserviceaccount.com" \
  --role="roles/bigquery.user"
gcloud projects add-iam-policy-binding [project-id] \
  --member="serviceAccount:infograph@[project-id].iam.gserviceaccount.com" \
  --role="roles/bigquery.dataViewer"

  3. Create a JSON key

    Click the service account's email, open the Keys tab, then Add key and Create new key. Choose JSON and click Create. The file downloads once and cannot be downloaded again, so keep it somewhere safe.

    gcloud iam service-accounts keys create infograph-key.json \
  --iam-account=infograph@[project-id].iam.gserviceaccount.com

  4. Paste it in Infograph and choose your tables

    Open BigQuery in Infograph and paste the whole file into Service Account Key (JSON). Leave Billing Project blank to use the key's own project: Infograph runs its queries there and lists the datasets in it. Fill in Dataset to see only one dataset. Press Connect, pick the tables you want, and Infograph starts loading them.

    Connect

Troubleshooting

Each entry is the message Infograph shows when a connection fails. Open it for the fix.

Those credentials were not accepted.

Paste the whole key file, from the first { to the last }. A partial paste, or one missing client_email, private_key, project_id or token_uri, is not a valid key. If it is complete, the key may have been deleted or disabled, or the service account itself disabled or deleted: check its Keys tab and create a new key. Also check the service account has the BigQuery User role on the project in Billing Project, or on the key's own project if that is blank.

Common problems

These happen in BigQuery itself, where Infograph cannot see them.

A dataset or table is missing

Infograph lists the datasets the service account can see in the project it runs in, and the tables and views in them. Grant BigQuery Data Viewer on the missing dataset, or check Billing Project names the project the dataset lives in. External tables are not listed. Then open the connection and press Re-discover streams.

Syncs cost more than expected

Every sync scans the tables you picked, and BigQuery bills by the bytes scanned. Pick only the tables you need, use a less frequent schedule for large ones, and watch the project's billing reports. Listing your tables does not run a billed query.

Queries fail with access denied

Without BigQuery Data Viewer, the service account can list tables but not read their rows. Grant it on the project or on each dataset you sync. A view also needs read access to the tables it queries, unless it is an authorized view.