Connect to BigQuery

This guide walks you through connecting FeatureMeshd to Google BigQuery.

Prerequisites

• FeatureMeshd installed
• TLS certificates configured (see Configure TLS )
• Google Cloud account with BigQuery access
• gcloud CLI tool (optional)

Steps

1. Create Service Account & Key

The service account key (key.json) will be used by your BigQuery clients (command line tools, GUIs, etc.) to authenticate with BigQuery through the proxy.

Using Google Cloud Console

1. Go to the Google Cloud Console
2. Navigate to "IAM & Admin" > "Service Accounts"
3. Click "Create Service Account"
4. Enter "featuremeshd-sa" as the service account name
5. Click "Create and Continue"
6. Select "BigQuery Data Viewer" role
7. Click "Done"
8. Find your new service account in the list
9. Click on the email address
10. Go to the "Keys" tab
11. Click "Add Key" > "Create new key"
12. Choose "JSON" format
13. Click "Create" to download the key file as key.json

Using gcloud CLI

# Create service account
gcloud iam service-accounts create featuremeshd-sa \
    --display-name="FeatureMeshd Service Account"

# Grant permissions
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
    --member="serviceAccount:featuremeshd-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/bigquery.dataViewer"

# Download key
gcloud iam service-accounts keys create key.json \
    --iam-account=featuremeshd-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

2. Configure FeatureMeshd

Create or update your config.yaml:

tls:
  certificate_chain: "/certs/rootCA.pem"
  private_key: "/certs/rootCA-key.pem"

listeners:
  - name: "main_https"
    protocol: https
    address: "0.0.0.0:10443"

routes:
  - domain: "bigquery.googleapis.com"
    target:
      address: "bigquery.googleapis.com:443"
      tls: true
    middlewares:
      - name: bigquery
        wasm: >-
          /etc/envoy/proxy-wasm-plugins/bigquery.wasm
  - domain: "www.googleapis.com"
    target:
      address: "www.googleapis.com:443"
      tls: true

3. Update Docker Compose

Update your docker-compose.yml:

services:
  featuremeshd:
    image: featuremeshd:latest
    ports:
      - "10443:10443"
    volumes:
      - ./config.yaml:/config.yaml
    environment:
      - FEATUREMESHD_REGISTRY_URL=http://your-registry-url

4. Using the Service Account Key

After setting up the proxy, you can use the downloaded key.json with your BigQuery clients:

Command Line Example

export GOOGLE_APPLICATION_CREDENTIALS=./key.json
bq --ca_certificates_file=/path-to-ca.pem --proxy_address=localhost --proxy_port=10443 query 'SELECT * FROM mydataset.mytable LIMIT 5;'

Python Client Example

from google.cloud import bigquery
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "./key.json"
os.environ["HTTPS_PROXY"] = "http://localhost:10443"

client = bigquery.Client()
query = "SELECT * FROM mydataset.mytable LIMIT 5"
results = client.query(query)

Connecting to BigQuery with GUI Tools

To connect DBeaver or DataGrip to BigQuery:

1. Start a new BigQuery connection in your tool
2. Set proxy to localhost:10443
3. Enable SSL and use Java's cacerts
4. Add your BigQuery service account key for auth

DBeaver Example

1. Right-click → "New Connection" → "Google BigQuery"
2. Enter your project ID
3. Add these driver properties:
   • ProxyHost: localhost
   • ProxyPort: 10443
   • sslTrustStore: Path to JVM cacerts (usually /Library/Java/JavaVirtualMachines/jdk*/Contents/Home/lib/security/cacerts)
   • sslTrustStorePwd: changeit

Test the connection and you're ready to go!

5. Test Connection

# Using the service account key
curl --proxy http://localhost:10443 \
  "https://bigquery.googleapis.com/bigquery/v2/projects/YOUR_PROJECT_ID/datasets"

Troubleshooting

Common issues:

1. Permission Issues

   • Verify service account roles
   • Check key.json permissions
   • Validate project access

2. Connection Problems

   • Check proxy settings
   • Verify network access
   • Review BigQuery API enablement