Documentation
Book a demo

CI/CD

Integrating GCP CloudBuild

Cloudsmith can be used as a target for all the assets created using Google CloudBuild. This guide shows you how to use the Cloudsmith CLI to upload/push a Python package to your Cloudsmith repo.

All formats are supported.

Getting started

Integrating Cloudsmith as part of your Google CloudBuild project is as simple as installing the Cloudsmith CLI during your build and then using the cloudsmith push command to upload the artifacts from your build process to your Cloudsmith repository.

Adding your API key to Google Cloud

In order to use the Cloudsmith CLI with Google CloudBuild, we recommend you to create a Cloudsmith service account and use its API key instead of a personal API key. This is more secure, easier to audit, and better suited for CI/CD environments like Google CloudBuild.

Once you’ve created a service account and generated its API key, we recommend that you add the key as an environment variable using Google Secret Manager.

Note

📘 Authentication best practices

We do not recommend adding the API-Key itself directly into the cloudbuild.yaml file for your Google CloudBuild project, as it will then be revealed in any resulting logs from the build.

Step 1: store the API key in Google Secret Manager

  1. Go to the Secret Manager page in the Google Cloud Console.
  2. Click + CREATE SECRET.
  3. In the Name field, enter CLOUDSMITH_API_KEY.
  4. In the Secret value field, paste your Cloudsmith service account API key.
  5. Leave other settings as default or configure as needed.
  6. Click CREATE SECRET.

Step 2: grant Cloud Build access to the secret

Your Cloud Build service account (usually [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com) needs permission to access the secret.

  1. Go to the Secret Manager page.
  2. Click on the CLOUDSMITH_API_KEY secret.
  3. Go to the PERMISSIONS tab.
  4. Click + GRANT ACCESS.
  5. In New principals, enter your Cloud Build service account.
  6. Under Select a role, choose Secret Manager Secret Accessor.
  7. Click SAVE.

Step 3: reference the secret in cloudbuild.yaml

To reference a secret stored in Google Secret Manager in your Google CloudBuild cloudbuild.yaml file you would use the following syntax:

yaml
steps:
  - name: 'python'
    secretEnv: ['CLOUDSMITH_API_KEY']

For further details on using Google Secret Manager with Google CloudBuild see Use secrets from Secret Manager.

Authenticating with OpenID Connect (OIDC)

OpenID Connect (OIDC) removes the need to store a Cloudsmith API key in Secret Manager. In each build, the Cloudsmith CLI gets an ID token for the Cloud Build service account and exchanges it for a short-lived Cloudsmith access token. There is no long-lived credential to rotate or protect.

To authenticate with OIDC:

  1. Create a Cloudsmith service account and give it access to the target repository. The CLI authenticates as this service account. Note its slug.

  2. Find the Google service account that runs your builds. Open the Settings page of Cloud Build in the Google Cloud console, or check the serviceAccount field of your build configuration or trigger. Depending on when the project was created, the default is one of these accounts:

    • PROJECT_NUMBER@cloudbuild.gserviceaccount.com (legacy Cloud Build service account)
    • PROJECT_NUMBER-compute@developer.gserviceaccount.com (Compute Engine default service account)

  3. Get the numeric unique ID of that service account:

    shell
    gcloud iam service-accounts describe SERVICE_ACCOUNT_EMAIL --format='value(uniqueId)'

  4. Create an OIDC provider in your Cloudsmith workspace with these settings:

    • Provider URL: https://accounts.google.com
    • Required OpenID token claims: a sub claim set to the unique ID from step 3. To allow all user-managed service accounts in the project instead, use an email claim with a wildcard.
    • Service accounts: the Cloudsmith service account from step 1.

    For more information, see Google Cloud in the OpenID Connect documentation.

  5. Enable the IAM Service Account Credentials API in your Google Cloud project:

    shell
    gcloud services enable iamcredentials.googleapis.com --project=PROJECT_ID

  6. Grant the service account the Service Account OpenID Connect Identity Token Creator role on itself:

    shell
    gcloud iam service-accounts add-iam-policy-binding SERVICE_ACCOUNT_EMAIL \
  --member='serviceAccount:SERVICE_ACCOUNT_EMAIL' \
  --role='roles/iam.serviceAccountOpenIdTokenCreator'

  7. In cloudbuild.yaml, set the CLOUDSMITH_WORKSPACE and CLOUDSMITH_SERVICE_SLUG environment variables. Then install the CLI with the gcp extra and run it. Do not add CLOUDSMITH_API_KEY to secretEnv, because the CLI uses an API key before it tries OIDC.

    yaml
    steps:
  - name: 'python'
    entrypoint: bash
    args:
      - '-c'
      - |
        set -eu
        python3 -m pip install --upgrade 'cloudsmith-cli[gcp]'
        cloudsmith whoami --verbose
        cloudsmith push FORMAT WORKSPACE/REPOSITORY FILENAME

options:
  env:
    - 'CLOUDSMITH_WORKSPACE=WORKSPACE'
    - 'CLOUDSMITH_SERVICE_SLUG=SERVICE_ACCOUNT_SLUG'

  8. Check the output of cloudsmith whoami --verbose in the build log. When OIDC authentication succeeds, the output includes lines similar to these:

    text
    Authentication Method: OIDC Auto-Discovery
  Source: OIDC via Google Cloud (org: WORKSPACE, service: SERVICE_ACCOUNT_SLUG)

Why the IAM role is required

Some Cloud Build workers do not provide an ID token through the metadata server. On these workers, the CLI asks the IAM Service Account Credentials API for the ID token. For more information, see Google Cloud in the CLI documentation.

Shell variables in cloudbuild.yaml

Cloud Build replaces $VARIABLE with a substitution before the step runs. To read a shell environment variable in a script, write it as $$VARIABLE. For example, use $$CLOUDSMITH_WORKSPACE.

If your build uses a user-specified service account, Cloud Build requires a logging option. Add logging: CLOUD_LOGGING_ONLY under options, or set a logsBucket.

Adding the Cloudsmith CLI to your CloudBuild project

The Cloudsmith CLI ships as a standalone binary, so no Python or other runtime is required for the CLI itself. To add the Cloudsmith CLI to your Google CloudBuild Project, add a build step to the cloudbuild.yaml file that runs the install script. The install script prints the installation result as key=value lines; the step below parses bin_dir from that output and adds it to PATH:

yaml
steps:
  - name: 'python'
    entrypoint: bash
    args:
      - '-c'
      - |
        set -eu
        apt-get update && apt-get install -y ca-certificates curl
        install_output="$(curl -fsSL https://install.cloudsmith.com/raw/versions/latest/cli.sh | sh -s -- --version 1.20.1)"
        bin_dir="$(printf '%s\n' "$install_output" | sed -n 's/^bin_dir=//p')"
        export PATH="$bin_dir:$PATH"

Uploading a built artifact to Cloudsmith

To upload an artifact from a build to a Cloudsmith repository, add the cloudsmith push command to the same build step:

yaml
        cloudsmith push FORMAT WORKSPACE/REPOSITORY FILENAME

Please see the Cloudsmith CLI documentation for more details of the syntax of the cloudsmith push command and the Supported Formats page for examples of the cloudsmith push command for each supported format.

Example cloudbuild.yaml

Here’s a complete working example that builds a Python package and uploads it to a Cloudsmith repository:

yaml
steps:

  - name: 'python'
    entrypoint: bash
    secretEnv: ['CLOUDSMITH_API_KEY']
    args:
      - '-c'
      - |
        set -eu
        echo "Installing Cloudsmith CLI..."
        apt-get update && apt-get install -y ca-certificates curl
        install_output="$(curl -fsSL https://install.cloudsmith.com/raw/versions/latest/cli.sh | sh -s -- --version 1.20.1)"
        bin_dir="$(printf '%s\n' "$install_output" | sed -n 's/^bin_dir=//p')"
        export PATH="$bin_dir:$PATH"

        echo "Setting up virtualenv..."
        python3 -m venv venv
        source venv/bin/activate
        pip install --upgrade pip setuptools wheel

        echo "Building package..."
        python setup.py sdist bdist_wheel

        echo "Authenticating with Cloudsmith..."
        cloudsmith whoami 

        echo "Pushing package to Cloudsmith..."
        cloudsmith push python WORKSPACE/REPOSITORY dist/*.whl

availableSecrets:
  secretManager:
    - versionName: projects/your-gcp-project-id/secrets/CLOUDSMITH_API_KEY/versions/latest
      env: CLOUDSMITH_API_KEY

Note

📘 Replace WORKSPACE/REPOSITORY with your actual Cloudsmith Workspace and repository slugs.

Also update your-gcp-project-id with your actual Google Cloud project ID.

Support

As always, if you have any questions about integration or would like some general advice, please contact support.