Add BigQuery database to apache superset GKE

[victorpergher]

I'm looking for a way to add BigQuery to Apache Superset.
I installed Superset using the GCP marketplace (marketplace.gcr.io/google/superset).
It's working perfectly, but now I need to add the connection to BigQuery.
In this documentation https://superset.apache.org/docs/configuration/databases/#installing-drivers-in-docker-images i can find the installation instructions, but it's for those who use Docker Compose and I'm using Kubernetes.
Is there a step-by-step guide on how to do this?

[dosubot[bot]]

Hey @victorpergher! I'm here to help you with any bugs, questions, or contributions you might have. Let's tackle this BigQuery issue together!

To add a BigQuery connection to Apache Superset deployed on GKE, you'll need to install the sqlalchemy-bigquery driver and configure the connection using GCP Service Account credentials. Here's a step-by-step guide for Kubernetes:

1. Modify the Superset Deployment:

   • Update your values.yaml file to include the sqlalchemy-bigquery package. You can do this by adding a bootstrapScript to install the package when the Superset pods are initialized:

     bootstrapScript: |
       #!/bin/bash
       pip install sqlalchemy-bigquery==1.6.1 &&\
       if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi

   • Apply the changes using Helm to update your deployment ^[1].

2. Create a Custom Docker Image (Optional):

   • For production environments, it's recommended to build a custom Docker image that includes the necessary Python packages. Modify the Dockerfile to add the sqlalchemy-bigquery package:

     FROM apache/superset:latest
     
     RUN pip install sqlalchemy-bigquery==1.6.1

   • Build and push this image to your container registry, then update your Helm chart to use this custom image by setting the image.repository and image.tag in your values.yaml file ^[2].

3. Configure the BigQuery Connection:

   • In the Google Cloud Platform, create a Service Account with the necessary permissions for BigQuery and download the JSON key file.

   • In Superset, add a new database connection for BigQuery. Use the connection string format:

     bigquery://{project_id}
     

   • In the Advanced tab, add the JSON key file content to the Secure Extra field in the following format:

     {
       "credentials_info": {
         "type": "service_account",
         "project_id": "...",
         "private_key_id": "...",
         "private_key": "...",
         "client_email": "...",
         "client_id": "...",
         "auth_uri": "...",
         "token_uri": "...",
         "auth_provider_x509_cert_url": "...",
         "client_x509_cert_url": "..."
       }
     }

By following these steps, you should be able to successfully connect Superset to BigQuery in your GKE environment ^[3][4].

To continue talking to Dosu, mention @dosu.