# Cloudsmith Documentation > The universal artifact registry for secure software distribution and dependency management. Cloudsmith docs cover repositories, package formats, integrations, authentication, CI/CD workflows, and more. Machine-readable API specs: - [Cloudsmith API v1 (Swagger 2.0)](https://api.cloudsmith.io/swagger/?format=openapi) - [Cloudsmith API v2 (OpenAPI 3)](https://api.cloudsmith.io/v2/openapi/?format=json) --- # Documentation # Common Use Cases This section outlines the primary and supporting use cases for Cloudsmith, detailing the benefits and key features for each. **[Artifact Management for the SDLC](/artifact-management)**: Centralized storage, management, and control of all software artifacts in one location. **[Software Distribution](/software-distribution)**: A platform to securely and reliably distribute software to customers at scale. **[Software Supply Chain Security](/supply-chain-security)**: Identify and mitigate security vulnerabilities and license compliance risks. **[Software Supply Chain Observability](/logs-and-observability)**: Get insights about your artifact management and distribution workflows. ## Artifact Management for the Software Development Lifecycle (SDLC) At its core, Cloudsmith provides a centralized, single source of truth for all your software artifacts. This is fundamental to modern DevOps practices, enabling organizations to store, manage, and control the vast number of packages, containers, and other assets generated throughout the software development lifecycle. By having one place to manage all software assets, teams can streamline their workflows, improve collaboration, and gain crucial visibility over their development processes. ### Benefits - **Universal Package Support**: Manage all your different package formats in one place, reducing the complexity and toolchain sprawl associated with polyglot development environments. - **Centralized Control and Visibility**: Gain a comprehensive overview of all your software assets, making it easier to manage dependencies, track versions, and understand what's in your software. - **Improved Developer Productivity**: Developers can easily push and pull artifacts using familiar, native tooling without needing to learn a new set of commands. This reduces friction and speeds up development cycles. - **Reliable and Repeatable Builds**: By managing all dependencies and artifacts in a central repository, you ensure that builds are consistent and repeatable, eliminating "works on my machine" issues. - **Scalability**: Cloudsmith's cloud-native architecture is designed to scale with your organization's needs, from small teams to large enterprises, without the overhead of managing your own infrastructure. ### Learn more about related features **[Multi-format Repositories](/formats)**: Support for over 30 package formats, including Docker, Maven, npm, Python, and RubyGems. **[Native Tooling Integration](/formats/python-repository)**: Seamless integration with the package managers and tools your developers already use. Start with our Python examples. **[Role-Based Access Controls (RBAC)](/accounts-and-teams/about-privileges)**: Fine-grained permissions to control who can read, write, and manage artifacts. **[Upstream Proxying and Caching](/repositories/upstreams)**: Proxy and cache public repositories to protect against upstream outages and improve performance.
## Software Distribution For organizations that need to distribute their software to end-users and customers, Cloudsmith offers a robust and secure platform. It simplifies the complexities of software distribution at scale, ensuring that your customers can access the software they are entitled to quickly and reliably, anywhere in the world. ### Benefits - **Global, High-Speed Delivery**: Leverage Cloudsmith's global Content Delivery Network (CDN) to ensure fast and reliable software distribution to your customers, regardless of their location. - **Secure and Controlled Access**: Protect your intellectual property by managing who can download your software through entitlement tokens and fine-grained access controls. - **Simplified Customer Experience**: Provide your customers with a seamless and straightforward way to download and install your software. - **Reduced Infrastructure Overhead**: Offload the complexities of building and maintaining a global distribution infrastructure to Cloudsmith. - **Usage Insights**: Understand how your software is being consumed with detailed analytics on downloads and bandwidth usage. ### Learn more about related features **[Entitlement Tokens](/software-distribution/entitlement-tokens)**: Securely grant access to your private repositories for specific customers or users. **[Public, Open Source, and Private Repositories](/repositories)**: Choose the right level of access for your different software packages. **[Broadcasts](/software-distribution/broadcasts)**: Enterprise grade features to share your repository artifacts to the world. **[Detailed Usage Logs and Analytics](/logs-and-observability/)**: Track download statistics, bandwidth consumption, and more to understand user engagement.
## Software Supply Chain Security Ensuring the integrity and security of the software you build and use is paramount. Cloudsmith provides a suite of features designed to help you secure your software supply chain by identifying and mitigating risks from vulnerabilities and non-compliant licenses in your third-party and open-source dependencies. ### Benefits - **Reduced Risk of Vulnerabilities**: Proactively identify and block packages with known vulnerabilities from entering your software supply chain. - **License Compliance**: Automatically scan for and identify the licenses of your dependencies, ensuring compliance with your organization's policies. - **Proactive Threat Mitigation**: Quarantine suspicious or non-compliant packages for manual review before they can be used by developers. - **Improved Security Posture**: Gain a comprehensive understanding of the security and license profile of all the artifacts in your repositories. - **Automated Policy Enforcement**: Define and enforce security and license policies across your entire organization, ensuring consistency and reducing manual overhead. ### Learn more about related features **[Enterprise Policy Manager](/supply-chain-security/epm)**: Define granular, automated policies for vulnerability severity, license compliance, and more. **[Vulnerability Scans](/supply-chain-security/vulnerability-scanning)**: Automatically scan your artifacts for known vulnerabilities and receive detailed reports. **[Package Quarantine](/artifact-management/package-quarantine)**: Automatically move packages that violate policies to a quarantine repository for further inspection. **[Digital Signatures](/supply-chain-security/signing-keys)**: Verify the integrity and authenticity of your software artifacts.
## Software Supply Chain Observability Regardless of your primary goal, having deep visibility into your software supply chain is crucial. Cloudsmith's observability features provide the insights you need to understand what's happening across your artifact management and distribution workflows. This allows you to monitor usage, troubleshoot issues, and make data-driven decisions. ### Benefits - **Actionable Insights**: Gain a clear understanding of how your artifacts are being used, who is accessing them, and where potential bottlenecks exist. - **Proactive Troubleshooting**: Identify and diagnose issues quickly with detailed logs and event feeds. - **Informed Decision-Making**: Use rich metadata and analytics to make better decisions about your software development and distribution strategies. - **Comprehensive Audit Trail**: Maintain a complete record of all activities within your repositories for security and compliance purposes. - **Integration with Existing Tools**: Feed observability data into your existing monitoring and analytics platforms for a unified view of your operations. ### Learn more about related features **[Detailed Audit Logs](/logs-and-observability/audit-logs)**: A comprehensive, immutable log of all events that occur within your organization and repositories. **[Client and logs and stats](/logs-and-observability)**: Visualize and analyze data on downloads, bandwidth, and storage. **[Datadog Integration](/integrations/integrating-with-datadog)**: The Datadog Cloudsmith Integration is a simple and effective visualization tool for your workspace. **[Webhook Integrations](/developer-tools/webhooks)**: Push real-time event data to your other tools and services.
# About Cloudsmith ## What is Cloudsmith? Cloudsmith is a fully managed, cloud-native artifact management platform that provides a single source of truth for all software artifacts. It serves as both the control plane and data plane for your software supply chain, helping teams control, secure, and distribute artifacts at global scale. Cloudsmith supports all major [package formats](/formats) - such as Docker, Python, Maven, npm, Helm, ML models, and more - and integrates with CI/CD systems, developer workflows, and security tools. Artifacts are delivered through a global Package Delivery Network (PDN) for fast, reliable access anywhere in the world. ### Differentiators Cloudsmith is designed for organizations that want to: - **Move to the cloud**: Replace on-premises registries with a fully managed, globally available service. - **Secure their software supply chain**: Enforce policies, scan for vulnerabilities, and verify artifact provenance. - **Adopt AI**: Support AI-enabled development workflows, where models, datasets, and other artifacts become part of the software supply chain. Cloudsmith provides a single platform to manage these alongside traditional software artifacts, with consistent security and governance. - **Improve developer experience**: Provide developers with fast access to artifacts, CI/CD integrations, and reliable global delivery. - **Distribute software globally**: Deliver artifacts to customers, partners, or internal teams with low latency worldwide. Cloudsmith organizes artifacts into a simple hierarchy: - [**Workspace**](/workspaces): The top-level scope for managing users, billing, and policies. - [**Repository**](/repositories): A logical grouping of packages. Repositories are multi-format and can host any supported package type. - [**Package**](/artifact-management): An immutable software artifact, such as a container image, library, or dataset. For more details about these concepts, see [Key Concepts](/about-cloudsmith/key-concepts), or explore some practical examples in the [Common Use Cases](/about-cloudsmith/common-use-cases) section. Not sure where to start? Start your trial. --- **[Artifact Management for the SDLC](/artifact-management)**: Centralized storage, management, and control of all software artifacts in one location. **[Software Distribution](/software-distribution)**: A platform to securely and reliably distributing software to customers at scale. **[Software Supply Chain Security](/supply-chain-security)**: Identify and mitigate security vulnerabilities and license compliance risks. **[Software Supply Chain Observability](/logs-and-observability)**: Get insights about your artifact management and distribution workflows. ## Developer tools Cloudsmith is built by developers, for developers. We understand the workflows and processes that developers use and need, and we try to ensure that what we build brings value to our customers, and generally makes their lives easier. We maintain a set of tools to programmatically manage your workspaces, repositories, and artifacts. Along with other elements within Cloudsmith that helps you manage your organization: like teams, user accounts, service accounts, or tokens to conveniently automate configuration through scripts or any of the existing tooling you already use. **[Cloudsmith CLI](/developer-tools/cli)**: Integrate smoothly with Cloudsmith without requiring explicit plugins or tools **[REST API](/api)**: Retrieve information, upload packages, or take actions programmatically. **[Visual Studio Code Extension](/developer-tools/vscode)**: View and manage your Cloudsmith packages directly within VS Code. **[Webhooks](./developer-tools/webhooks)**: Receive notifications when particular actions take place within Cloudsmith. **[Terraform Provider](./developer-tools/terraform-provider)**: Automate administrative actions with our Terraform provider.
We believe in what we have built. We love to talk to other developers about what we have created, how it can fit into their development processes, and how we can help. If you have questions or would like to chat, please just [contact us](https://cloudsmith.com/company/contact-us). # Key concepts ## Concepts and components of Cloudsmith Cloudsmith is a cloud-native artifact management platform that serves as both the control plane and data plane for your software supply chain. It provides a single, secure source of truth for all software artifacts - packages, container images, ML models, datasets, and more - delivered reliably through a global package delivery network (PDN). Cloudsmith supports all major package formats and integrates with CI/CD pipelines, security tools, and developer workflows. It enables organizations to store, secure, scan, and distribute artifacts with full visibility, provenance, and policy enforcement. Here are some of the key concepts and components of Cloudsmith: ### Workspace A workspace is the top level scope in Cloudsmith. It is where you manage users, billing, policies, repositories and registries for your organization. Each workspace can contain one or more repositories, each with its own access controls. ### Repository A repository is a logical grouping of artifacts within a workspace. Repositories can be format-specific (e.g., Docker, Python, Maven) or universal. Each repository has its own permissions, policies, upstream mirrors, and retention rules. ### Artifact An artifact is an immutable software asset stored in a repository. Examples include container images, libraries, Helm charts, SBOMs, and ML models. ### Package Artifact that is “published” for use by others. It is usually the output of a build process. Examples include Docker image (package); a ZIP file containing source code is a package, not a source, because it is built from some other source, such as a git commit. ### Dependency Artifact that is an input to a build process but that is not a source. ### Version and Tag Artifacts can be referenced using immutable versions (such as semantic versions or digests) or mutable tags (such as latest). Tags resolve to specific artifact versions within a repository. ### Access Control Cloudsmith uses role-based access control (RBAC), API tokens, and service accounts to manage access. Permissions can be scoped to workspaces or repositories, and control individual actions such as pushing, pulling, or managing artifacts. ### Policies Repositories can enforce governance rules using Enterprise Policy Manager (EPM). Policies can block artifacts, restrict usage based on licenses, or require provenance metadata before publication or distribution. ### Provenance and Metadata Artifacts stored in Cloudsmith include metadata such as checksums, SBOMs, and provenance records. This enables traceability, compliance, and reproducibility across the software supply chain. ### Upstreams and Caching Cloudsmith can proxy and cache upstream registries such as PyPI, npmjs, Docker Hub, and Hugging Face. Cached upstreams ensure availability, improve build performance, and provide full visibility into external dependencies. ### Universal Package Support Cloudsmith "speaks" the native protocol for a large number of packaging technologies (e.g. Python \+ Ruby \+ Maven/Java, etc.), as well provide APIs for easy/agnostic manipulation; so you have immediate compatibility with all of your tools. ### Continuous Security Cloudsmith continuously scans artifacts for vulnerabilities using multiple threat-intelligence sources. ### Retention Rules Repositories can be configured with retention policies that automatically remove artifacts based on version count, age, or tags. ### Package Delivery Network (PDN) Cloudsmith's globally distributed PDN ensures reliable, low-latency artifact delivery worldwide. It supports high-velocity teams and production environments at scale. ### Audit and Client Logging Cloudsmith records all artifact actions \- push, pull, delete, and promote \- and provides near real-time logs and metrics for compliance, debugging, and usage insights. ### Enterprise Features Enterprise plans include advanced policy enforcement, continuous security, custom MSAs, negotiated SLAs, cross-region replication, and support for annual usage commitments. ### Broadcasts Share branded public repositories via a public facing URL using your own domain, and monitoring usage from your users. # Log In Learn how to access your Cloudsmith [Workspaces](/workspaces). ## Username and password Go to [https://app.cloudsmith.com/login](https://app.cloudsmith.com/login) and enter your e-mail address and password. You'll be taken to your Workspace overview page. ## SAML Go to [https://app.cloudsmith.com/login/saml](https://app.cloudsmith.com/login/saml) or click **Login using SAML** on the main login page. Enter your **workspace name** in the form and click **Continue**. **SAML and Primary Workspace** To use SAML, your workspace must be set as your primary workspace. See [primary workspace](/workspaces#primary-workspace) for learn more. Complete any required MFA steps required by your administrator. ## Device verification When signing in from a new device, you may be asked to verify it. Click the verification link sent to your email. # Sign Up Anyone can [start a free Cloudsmith trial](https://app.cloudsmith.com/signup) to explore the platform on your own. For teams with complex use cases - whether scale, performance or security related - a guided proof of value (POV) is often the better path. A POV gives you a tailored experience, with hands-on support from our team. [Request a guided POV](https://cloudsmith.com/book-a-demo) [Image: Sign Up] Start a free trial: 1. Go to [https://app.cloudsmith.com/signup](https://app.cloudsmith.com/signup). 2. Enter your **work email**, then click **Continue**. 3. Add your **first name** and **last name**, and create a **password**. 4. Review and accept the **Terms** and **Privacy Policy**, then click **Continue**. 5. Open the verification email from Cloudsmith and click the **verification link**. 6. Click **Start your free trial**. Now you can [sign-in](/about-cloudsmith/sign-in) into your new Cloudsmith account. # Teams and Accounts Privileges Cloudsmith provides a powerful and flexible permissions system to ensure you can implement a least-privilege approach for your software supply chain. Access control is managed at three distinct levels: the Workspace, Teams, and individual Repositories. It is helpful to think of Cloudsmith's permissions as a top-down hierarchy, where settings at a higher level provide defaults that can be refined or overridden by settings at a more specific level. - [**Workspace Level**](/workspaces/privileges): The broadest level of control. It defines user roles for the entire workspace and sets default permissions that apply to all repositories within it. - [**Team-Based Permissions**](/accounts-and-teams/teams): Teams are groups of workspace users. You don't assign privileges to a team directly; instead, you grant a team specific access to a repository. All members of that team then inherit those repository privileges. This is the primary mechanism for managing access for groups of users. - [**Repository Level**](/repositories/repository-privileges): The most granular level of control. Here, you can override workspace defaults for a specific repository, grant unique permissions to specific users and teams, and fine-tune what actions are permissible within that repository. A user's final, **effective privilege** is the greatest privilege granted to them from any of these levels. To learn more about privileges, visit each of the sections linked above, or complete our [Permissions guide](/guides/managing-permissions). --- title: "API Key" --- # API Key In order to use the [Cloudsmith API](/api), or any other integrations or tools that make use of the Cloudsmith API, you will first need to get your API Key. An API Key provides Read and Write access. If you want Read-only access, please use an [Entitlement Token](/software-distribution/entitlement-tokens). API Keys and Entitlement Tokens should be treated as secrets to prevent unwarranted access. ## Getting your API Key There are two ways to retrieve your API Key: - Via the Cloudsmith web app - Via the Cloudsmith CLI ### Via the Cloudsmith web app On the top right corner, click on your user icon, then click on [Personal API Keys](https://app.cloudsmith.com/settings/api-keys) and click **Refresh** to view the API Key. Refreshing the API key will permanently disable the current API key. Make sure you store it in a proper secret management tool to access it later. [Image: Edit provider settings] ### Via the Cloudsmith CLI You can retrieve your API key using the cloudsmith login command: ``` cloudsmith login Login: you@example.com Password: PASSWORD Repeat for confirmation: PASSWORD ``` > Please ensure you use your email for the 'Login' prompt. ## Adding IP-Based restrictions to your API-Key By default, you can use your API-Key from anywhere. If you wish to restrict the use of your API-Key to a specific IP address or range, you can add the CIDR address/mask to the Allow List in the [API Key Restrictions](https://app.cloudsmith.com/settings/api-keys/restrictions) tab: [Image: Setting API restrictions for a user API key] One you have added your CIDR address/mask, just click the "+ Add" button to apply your restriction. ## Refreshing a Personal/User API Key If you lose the API key associated with your account, or suspect it has been compromised, you'll need to refresh the key, generating a new one. To do this, under your profile menu, select Personal API Keys. [Image: Personal key menu item] Click the "Refresh API Key" button to generate a new API key. [Image: Refreshing the key] # Teams and Accounts In Cloudsmith, you can define the identities (Accounts and Teams) and their privileges within your Workspaces with a high level of detail. There are two different types of **accounts**: - **Members**: use them to give access to humans in your organization. - **Services**: for any automation or agents that might require access to your Workspaces. Effective account **privileges** are divided into three main levels: - Workspace - Team - Repository Learn more about them in the [Teams and Account's privileges](/accounts-and-teams/about-privileges) documentation or in the Workspace or Repository settings. # Service Accounts Services let you create a Cloudsmith API Key not tied to a specific user. Services are ideal for any use case where you have automated processes (such as a CI/CD pipeline or similar) that require access to repositories. If your use case only requires short-term or restricted read access to a repository, e.g. in software distribution, use an [Entitlement Token](/software-distribution/entitlement-tokens). To create or manage Services for your organization, you need to have Owner or Manager permissions. ## Creating a Service To create a Service, go to Services on your Cloudsmith organization's accounts page, and click "Create Service": [Image: Accounts section] Fill out the service form, and give your service a name. Optionally, add a description and select any teams in your organization to associate with the service. [Image: Create service men] Once you create your service account, Cloudsmith will assign a `slug`, which will be used as an API-level identifier for the account. Note that service account slugs are automatically truncated at 30 characters, and may have random characters appended to ensure uniqueness. ## Getting a Service Username The Service username is displayed in the NAME column: [Image: Service username] ## Getting a Service API Key Copy Your API Key Immediately: - Immediately copy the API key to your clipboard after it is created and store it securely. - You will not be able to retrieve the API key again from Cloudsmith. However, you can rotate the key if needed. **Securely Store Your API Key** Use a trusted password manager or a dedicated secrets vault to store your API key securely. [Image: Storing your key] ## Refreshing a Service API Key If you lose the API key associated with a Service account, or suspect it has been compromised, you'll need to refresh the key, generating a new one. To do this, go to the Accounts->Services section of our app. Under the API KEY column select the affected key, and click the "Refresh API Key" button to generate a new API key. [Image: Service accounts menu, refresh API key] You must then confirm that you wish to refresh the Service API Key [Image: Confirm you want to refresh the service] ## Deleting a Service Click the red "Delete Service" button to permanently delete the Service and its associated API Key: [Image: Delete the service] You must then confirm that you wish to delete the Service: [Image: Confirming you want to delete the service] # Teams ## Create a Team Creating a team requires that you have first created a workspace (See [Workspaces](/workspaces/workspace-create) for full details). To create a team: 1. Navigate to Teams in the main menu 2. Click "New Team" [Image: New Team button] You are then presented with the Add New Team form: [Image: Add New Team form] A Team name is required while the description is optional. "Visibility" controls if non-members can see the team or not: | | | | :------ | :------------------------------- | | Visible | non-members can view the team | | Hidden | non-members cannot view the team | {/* TODO: we might need to add here a reference about team privileges at the repository settings */} You can add users (also known as "Accounts") to Cloudsmith directly via the web app, or you can use [SCIM](/authentication/scim). # Invite Users To invite people to your organization: 1. Click on the "Accounts" Tab 2. Click the blue "Invite Users" button [Image: Invite Users button] You will then be presented with the "Invite user(s)" form: [Image: Invite New Users form] To invite new users, enter one (or more) complete email addresses for the user(s) you wish to invite. To save time, you can bulk invite new users however they must share the same Organization role ## Roles When inviting users, you specify a role that all invited users will assume when they join the organization. These roles are: | Organization Role | Description | | :---------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------- | | Owner | Can manage organization administration and settings and have all permissions implicitly. They can manage other owners and delete the account | | Manager | Can manage organization settings, teams and non-manager user roles (excluding owners), and inherit privileges from the organization and team memberships. | | Member | Can see other members and visible teams, and inherit privileges from teams they belong to. | | Collaborator | Can only see other team members and inherit privileges from teams they belong to. | Owners are the "superusers" of the organization. Managers have access to organization settings, can manage the organization itself and can join teams, but managers have to join teams explicitly to get access to repositories due to the principle of least privilege. # Artifact Management With Cloudsmith for **Artifact Management**, you control how software components move through your SDLC, both what you build and what you consume. Artifact Management includes both: - the artifacts your teams **generate** (like binaries and container images) - and the dependencies they **consume** from internal and third-party sources These software components and their corresponding metadata are collectively known as **artifacts**. Effective artifact management ensures these critical assets are versioned, secured, and reliably accessible, forming the backbone of a modern software supply chain. As complexity grows, particularly with popular formats such as Maven and npm, it becomes important to manage packages through a package management system such as Cloudsmith. ## What's a package? A package bundles software files with metadata (name, version, dependencies). Packages are typically versioned to provide a better and more manageable understanding of what software is being deployed. While related, there's a key difference between artifacts and packages. An **artifact** is a raw output, like a `.jar` file or a Docker image. A **package** is an artifact bundled with metadata (name, version, dependencies) that tools can understand. **all packages are artifacts, but not all artifacts are packages.** For a more detailed comparison, please see [Artifacts vs. Packages: What Is the Difference?](https://cloudsmith.com/blog/artifacts-vs-packages-what-is-the-difference) Cloudsmith is a universal artifact management platform supporting 28+ formats, with native tooling integrations and registry upstreams to proxy and cache popular sources. Publish and deliver via native interfaces (e.g., Maven Publish) or the Cloudsmith CLI, API, or UI. ## What package types are supported? See the full list in [Supported Formats](/formats). Each format gets first-class support with consistent controls for access, policy and visibility. ## Package Search Syntax Quickly find packages with [Package Search Syntax](/artifact-management/search-filter-sort-packages), combining fields like name, version or dependency to narrow results. ## Package Actions You can manage your packages using different tools: - Cloudsmith CLI - Cloudsmith web app - Native tooling (docker, pip, npm, etc.) - Cloudsmith API Here's a list of supported actions with references to learn more about each of them. | Action | Description | | :------------------------------------ | :------------------------------------------------------------------------- | | [Identification](/artifact-management/identifying-a-package) | Get package ID | | [Upload](/artifact-management/package-upload) | Publish from your development environment or CI pipeline | | [Download](/artifact-management/download-a-package) | Download packages and dependencies to any environment | | [Tag](/artifact-management/package-tags) | Tag a package | | [Copy](/artifact-management/copy-a-package) | Copy a package from one repository to another | | [Move](/artifact-management/package-move) | Move a package from one repository to another | | [Delete](/artifact-management/delete-a-package) | Delete a package from a repository | | [Quarantine](/artifact-management/package-quarantine)| Package Quarantine | | [Resynchronize](/artifact-management/resync-a-package) | Republish (delete/add) a package (usually to retry a package sync failure) | | [Share Private](/artifact-management/) | Share a private package | | [About Package SBOMs](/artifact-management/sbom/) | Learn how to get your package SBOM and sign it with cosign | 📘 **Promote Packages** Cloudsmith allows you to "promote" packages between repositories through either a [move](/artifact-management/package-move) or [copy](/artifact-management/copy-a-package) function, preventing unnecessary uploads/downloads, for an accelerated pipeline. ## Package groups [Package Groups](artifact-management/package-groups) provide a streamlined, high-level overview of your repository by consolidating all component versions into a single entry for each package. ## Retention rules [Retention rules](artifact-management/retention-rules) automate repository storage management by systematically deleting packages based on configurable criteria for count, size, age, or a filtered search query. ## Artifact Management Policies ### Package Deny policies Package [deny policy rules](/policy-management/deny-policy) let you control which packages can be downloaded from your workspace's repositories. By defining these rules, organizations can enforce stricter security measures and maintain tighter control over their software artifacts. ### Block Until Scan [Block Until Scan](/policy-management/block-until-scan) is a security feature designed to enhance the integrity and security of software packages served by Cloudsmith, guaranteeing that all relevant security and compliance policy checks (licenses, vulnerabilities, package deny policies) are fully completed before a package is made available for download. To learn more about it, browse to [Supply Chain Security > Policies > Block Until Scan](/policy-management/block-until-scan). # Package Groups Package Groups provides a per package view of a repository, as opposed to the default view for a repository which shows each individual version of every package. This can be useful to give you an overview of the packages contained within your repository, especially if you have a large number of versions for each package. The Package Groups view groups packages by package format and package name: [Image: Package Groups] Where: | Information | Description | | :---------- | :--------------------------------------------------------------------- | | Format | The format of the package | | Name and version | The name and version of the | | Count | The total number of artifacts within the package group | | Downloads | The total number of downloads for all versions of the package | | Size | The size of the Package | | Uploaded | The date / time of the most recent upload of the package to Cloudsmith | Clicking on any package name or in the dropdown arrow in the right will take you to the default list view of artifacts within a group. # Recently Deleted Packages The Recently Deleted Packages page acts as a recycle bin for your deleted packages, giving you the opportunity to restore or permanently delete them before they are automatically removed. This is useful in cases of accidental deletion. Recently Deleted Packages is available at both the workspace and repository level. The workspace-level view aggregates deleted packages across all repositories, while the repository-level view is scoped to a single repository. [Image: Recently deleted packages page] ## Package retention Deleted packages are retained for 7 days from the time of deletion, after which they are permanently and automatically removed. ## Restore packages To restore an individual package, click the arrow icon on the right-hand side of the table row. To restore multiple packages at once, select the packages you want to restore, then choose **Restore** from the actions menu at the bottom of the page. Packages will be restored to their original repository. [Image: Restore packages] ## Permanently delete packages To permanently delete an individual package, click the trash icon on the right-hand side of the table row. To permanently delete multiple packages at once, select the packages you want to remove, then choose **Permanently delete** from the actions menu at the bottom of the page. Once permanently deleted, packages can no longer be restored. [Image: Delete packages] ## Deleted repositories When a repository is deleted, all of its packages are transferred to Recently Deleted Packages. If you restore a package from a deleted repository, the repository will be automatically restored as well. # Retention Rules Cloudsmith Retention Rules automate artifact data management by deleting packages that exceed a repository's configured limits. The configured limits can be based on: - The number of packages (count) - The total size of packages (bytes) - The age of the package (days) For more granular retention rules, artifacts in a repository can be divided up into distinct groups based on: - Package Name - Package Format - Package Type - A [search query](/artifact-management/search-filter-sort-packages) Each repository has one configurable retention rule. This rule runs automatically when a new package is uploaded to a repository or when the latest uploaded package is resynced. Packages that fall outside of the repository’s configured limits will be deleted. Retention Rules can be configured through the Cloudsmith Web App, the Cloudsmith API, or the Cloudsmith Terraform provider. ## How Retention Rules work Cloudsmith uses a "funnel" approach to identify which packages should be evaluated and deleted. ### The “Funnel”: Identifying the evaluation groups Cloudsmith enables granular control by evaluating packages as distinct subgroups rather than only treating the repository as a single unit. These groups are defined by your specific grouping configurations and package filters. - If no groupings or filters are configured, the entire repository is processed as a single group. - If multiple groupings or filters are enabled, they are applied hierarchically (nested). This creates specific, distinct groups for evaluation. #### Grouping examples | **Groupings enabled** | **Resulting evaluation group(s)** | | -------------------------- | ---------------------------------------------------------------------------- | | Format | One group for every unique format in the repository. | | Format + Name | One group for every unique name within each unique format in the repository. | | Format + Type | One group for every unique type within each unique format in the repository. | | Format + Name + Type | One group for every unique type within each unique name within each unique format in the repository. | | Grouping + Package Filters | Only packages matching your search query within each enabled grouping. | Packages that have not synced successfully are ignored and excluded from the evaluation groups. ### The deletion logic Once the evaluation groups are identified, Cloudsmith checks them against your configured limits. If any limit is exceeded, deletions are triggered based on specific sorting methods: | **Limit Type** | **Sorting Method** | **Action** | | -------------- | ------------------ | ---------- | | Limit by Count | Upload Date | Deletes the oldest packages until the count limit is met. “Oldest” is determined by the date the package was uploaded into Cloudsmith. Manually resyncing a cached package does not change the uploaded timestamp. | | Limit by Days | Upload Date | Deletes packages older than X days. “Oldest” is determined by the date the package was uploaded into Cloudsmith. Manually resyncing a cached package does not change the uploaded timestamp. | | Limit by Size | Semantic Version | Deletes the oldest packages until the group size is within limits. "Oldest" is determined using the semantic version of a package. | At least one limit (Days, Count, or Size) must be greater than 0 for a retention rule to execute. If all are set to 0, no deletions will occur. ## When are Retention Rules evaluated? Retention rules are not a continuous background process; they are event-driven. Evaluations only occur in response to specific triggers. ### Trigger events A Retention Rule is evaluated when: - A new package is uploaded and successfully synchronized. - The most recently uploaded package is manually resynced. Resyncing an older package will not trigger an evaluation. ### The scope of evaluation If you have configured groupings or package filters, the evaluation will be isolated to the distinct group associated with the package that invokes the trigger event. - **Group by Format**: Uploading a Docker image triggers an evaluation only for Docker packages. All other package formats are excluded from the evaluation. - **Nested Groups (Format + Name)**: Uploading a Docker image named `test-docker` only triggers an evaluation for that specific name within the Docker format. ### Why some packages may persist Because rules are event-driven and group-specific, packages may persist beyond the configured limits of a rule if there is no activity in their configured group. **For example**: You have a 90-day retention rule and **Group by Format** enabled. You upload Python packages every day, but haven't uploaded a Docker image in 120 days. Your existing Docker images will not be deleted until a new Docker image is added or the latest Docker image is resynced, which triggers a Docker format evaluation. For manual resyncs, the package that triggers the evaluation is never deleted. ### How the cutoff date is calculated When **Limit by Days** has been enabled, Cloudsmith will delete packages based on their upload date. Cloudsmith determines age-based deletions by using a "cutoff date" relative to the triggering package, not the current calendar date. **For example**: If your most recent package is 100 days old and your limit is 90 days, the cutoff is 190 days. Only packages 190 days or older are identified for deletion. The package used to trigger the evaluation (the latest one) is never deleted, even if it exceeds your configured parameter. To properly delete this 100-day-old package, you must upload a newer package to shift the cutoff date forward. ### How many packages will be deleted The processing limit is 1000 deletions per evaluation cycle. Any packages exceeding this limit stay in the repository until the next evaluation is triggered. ## Enabling Retention Rules Retention Rules for a repository are disabled by default. Go to the Repository Settings and, in the left menu, click on **Retention Rules**. Then, click the **Enable** button in the colored banner. To disable a Retention Rule, follow the same steps and click the **Disable** button in the colored banner. Alternatively, you can use the [Cloudsmith Repo Retention Partial Update](/api/repo/retention/partial-update) to update a repository’s retention rule or configure a repository’s retention rule using the Cloudsmith Terraform Provider. ## Configuration parameters | Name | API | Description | | --- | --- | --- | | Enabled | `retention_enabled` | Activates Retention Rules for the repository. A value of `true` enables the rule, and `false` disables it. | | Group packages by Name | `retention_group_by_name` | If enabled, evaluation applies to groups of packages by name rather than all packages. | | Group packages by Format | `retention_group_by_format` | If enabled, evaluation applies to packages by package formats rather than across all package formats. | | Group packages by Type | `retention_group_by_package_type` | If enabled, evaluation applies to packages by package type (for example, binary vs source) rather than across all package types. | | Package Filters (Query String) | `retention_package_query_string` | A package search expression that filters packages in each evaluation group. Refer to the Cloudsmith search syntax documentation. | | Limit by Days | `retention_days_limit` | Number of days to retain a package. Packages older than this are selected for deletion. Set to `0` to disable. “Oldest” is based on upload date; resyncing doesn’t change it. | | Limit by Count | `retention_count_limit` | Maximum number of packages to retain. Set to `0` to disable. If 5 packages meet criteria and the limit is 4, one is deleted; if only 3 meet criteria, none are deleted. “Oldest” is based on upload date; resyncing doesn’t change it. | | Limit by Size | `retention_size_limit` | Maximum total size (bytes) of packages to retain. Set to `0` to disable. "Oldest" is determined using semantic version when reducing size. | From the Cloudsmith Web App, use the sliders to configure limits, and then click the green **Update** button to apply the changes. To learn how to configure retention rule parameters via our Cloudsmith API, visit the [API reference](https://docs.cloudsmith.com/api) documentation. For Terraform, visit the [Cloudsmith Terraform Provider](https://registry.terraform.io/providers/cloudsmith-io/cloudsmith/latest/docs/resources/repository_retention) documentation. ## Docker and multi-architecture images We recommend exercising caution when applying retention rules to Docker repositories containing multi-architecture images. ### Key considerations Cloudsmith evaluates and deletes Docker artifacts as individual entities. Because the retention rule engine does not currently evaluate the relationship between a manifest list and its associated images, automated deletion can lead to repository inconsistency, such as: - **Broken manifests**: A manifest list may remain while one or more of its required images are deleted. - **Orphaned images**: Platform-specific images may remain after their parent manifest list has been removed. Download statistics are not recorded against Docker Manifests. If you use package filters based on download activity, specifically the `last_downloaded` or `downloads` package filter options, Cloudsmith may inadvertently delete manifests because they appear "unused." This often results in orphaned platform images remaining in the repository. If your workflow relies heavily on manifest lists, consider using package filters (query strings) to exclude critical tags from retention, or use the Cloudsmith API to manage cleanup to ensure all associated artifacts are completely removed. # Package Search Syntax Cloudsmith provides a powerful and flexible query language that enables you to instantly find any package, in any repository, based on a comprehensive set of attributes. This syntax can be used in multiple areas of the product. Our search syntax allows you to move beyond simple name-based searches and build highly specific queries using a wide range of criteria, including package format, version, size, upload date, dependencies, and even policy violation status. By combining these terms with boolean operators (AND, OR, NOT) and advanced matching logic, you can construct precise filters to isolate the exact artifacts you need, directly from the Web UI, API, or CLI. **Boolean evaluation** The search syntax is fully "boolean", in that you can combine queries together with `AND` (i.e. find this AND that), or `OR` (i.e. find this OR that), or `NOT` (i.e. find NOT this). This is optional, if you don't put any boolean operators (AND/OR) in, we'll just assume that you want `AND` by default. ## Search Terms |Search By|Search Terms Example| |:---|:---| |Name (`name`)|`name:my-package` (package name contains "my-package"; use anchors for exact match)| |Filename (`filename`)|`filename:my-package.ext` (filename contains "my-package.ext"; use anchors for exact match)| |Tag (`tag`)|`tag:latest` (a package tag contains "latest"; use anchors for exact match)| |Version - String-based (`version`)|`version:^1.1.0$` (package version is exactly 1.1.0)`version:1.1.0*post1` (package version contains 1.10 followed by post1).| |Version - Semantic (`version`)|`version:1.1.0` (package version is "1.1.0")`version:>1.1.0` (package version is greater than "1.1.0")`version:<1.1.0` (package version is less than "1.1.0")`version:~=1.1.0` (package version is greater or equal to 1.1.0, but less than 1.2.0)| |Prerelease (`prerelease`)|`true` (packages are prerelease) `false` (packages are not prerelease)| |Architecture (`architecture`)|`architecture:x86_64` (architecture is "x86_64")| |Distribution (`distribution`)|`distribution:el/7` (distribution is "el", release is "7")| |Format (`format`)|`format:deb` (format is "deb" [debian])| |Status (`status`)|`status:in_progress` (status is "in_progress")| |File Checksum (`checksum`)|`checksum:5afba` (checksum contains "5afba")| |Downloads (`downloads`)|`downloads:>8` (more than 8 downloads)`downloads:<1000` (at least 1, but less than 1000 downloads)| |Package Type (`type`)|`type:binary` (Binary Packages)`type:source` (Source Packages)`type:combined` (Binary and Source Packages)| |Size in Bytes (`size`)|`size:>50000` (size is greater than 50000)`size:<10000` (size is less than 10000)| |Uploaded Date (`uploaded`)|`uploaded:’1 day ago’` (uploaded more than one day ago) `uploaded:’August 14, 2019 EST’` (uploaded on Aug 14th)| |Last Downloaded Date (`last_downloaded`)|`last_downloaded:’1 day ago’` (downloaded more than one day ago) `last_downloaded:’August 14, 2019 EST’` (downloaded on Aug 14th)| |Entitlement Token Identifier (`token`)|`token:3lKPVJPosCsY` (packages visible for specified token)| |Dependencies (`dependency`)|`dependency:log4j` (search for packages that have a dependency with "log4j" in the name).`dependency:log4j=1.2.17` (search for packages dependencies name including "log4j" & version matching 1.2.17).`dependency:log4j<2.0.0` (search for packages dependencies with name including "log4j" & version less than 2.0.0).| |Repository (`repository`)|`repository:repo-name` (Search for packages within the repository named "repo-name")| |**Debian**: Component|`deb_component:component-name` (Search for packages by Debian Component)| |**Docker**: Image Digest|`docker_image_digest:sha256:abcdef` (Search for packages by Docker Image Digest)| |**Docker**: Layer Digest|`docker_layer_digest:sha256:abcdef` (Search for packages by Docker Layer Digest)| |**Maven**: GroupID|`maven_group_id:io.cloudsmith` (Search for packages by Maven GroupID)| |Packages that have violated a policy (`policy_violated`)|`true` (packages that have violated a policy)`false` (packages have not violated a policy)| |Packages that have violated a vulnerability policy (`vulnerability_policy_violated`)|`true` (packages that have violated a vulnerability policy)`false` (packages have not violated a vulnerability policy)| |Packages that have violated a license policy (`license_policy_violated`)|`true` (packages that have violated a license policy) `false` (packages have not violated a license policy)| |Packages that have violated a deny list policy (`deny_policy_violated`)|`true` (packages that have violated a deny list policy)`false` (packages have not violated a deny list policy)| **For all queries you can use**: `~` for negation. (Example: `~foo`) **For string queries you can use**: `^` - to anchor to start of term. (Example: `^foo`) `$` - to anchor to end of term. (Example: `foo$`) `*` - for fuzzy matching. (Example: `foo*bar`) **For number or date queries you can use**: `>` - for values greater than. (Example: `>foo`) `>=` - for values greater / equal. (Example: `>=foo`) `<` - for values less than. (Example: `'1 month ago'`, that becomes `uploaded > dd/mm/yy` (i.e, within the last month). **For version queries you can use**: `>` - for versions greater than. (Example: `>1.2.3.`) `>=` - for versions greater / equal. (Example: `>=1.2.3`) `<` - for versions less than. (Example: `<1.2.3`) `<=` - for versions less / equal. (Example: `<=1.2.3`) `~=` - for versions greater than or equal to, up to the next incompatible version. (Example: `~=1.2.0`) Remember: You can use any combination of `AND`, `OR` and `NOT`. You can use parentheses to group terms if required, as `AND` has a higher precedence than `OR`, and it may change the meaning of queries without it. ## A Short Working Example Suppose that you had the following packages: - AlphaLib, Version: 1.0 - AlphaLib, Version: 1.1 - AlphaLib, Version: 2.0 - BetaLib: Version: 0.9 To find "AlphaLib" packages that are in the version 1.x series only, your query would be: `name:^AlphaLib$ AND version:~=1.0.` Which would provide the following results: - AlphaLib, Version: 1.0 - AlphaLib, Version: 1.1 To bring the query down, we added the following parts: - `name:^AlphaLib$` - Find a package name that is **exactly** "AlphaLib". Without the `^` (starts-with) and `$` (ends-with), the query would be fuzzy, and potentially find packages called "AlphaLib2" or "FreeAlphaLib" too. - `AND` - We want to search for both the name and version together. Both must match for the results. - `version:~=1.0` - Find package versions that are at-least 1.0, up to the next incompatible version (2.0). ## Searching Packages via the Cloudsmith web app At the top of every page is the search box: Additional help and examples are available by hovering over the "? Search syntax" at the right side of the search box. ## Searching Packages via the Cloudsmith API Please see the Cloudsmith full API reference to learn how to: - List all of the packages in a [Workspace](/api/packages/list). - List all of the packages in a [Repository](/api/packages/list). Then, you can use the `query` field in the URL request to search a package by any of the next fields: - `name` - `filename` - `version` - `distribution` - `architecture` - `format` - `status` For example, to retrieve all packages named `ms`: ```bash curl -sL --request GET \ --url "https://api.cloudsmith.io/packages///?query=name:ms" \ --header 'Authorization: Bearer ' ``` Alternatively, you can use `jq` to search for a package in a repository by using **any of the fields** available in the `json` returned by the request. For example: ```bash curl -sL "https://api.cloudsmith.io/packages///" \ --header 'Authorization: Bearer ' \ | jq '.[] | select(.name == "requests")' ``` ## Searching Packages via the Cloudsmith CLI To search packages using the Cloudsmith CLI, you use the `cloudsmith list packages` command in combination with the `-q` option: ```shell cloudsmith list packages OWNER/REPOSITORY -q "SEARCH_TERMS" ``` **Example** To search for all debian packages: ```shell cloudsmith list packages cloudsmith/examples -q "format:deb" ``` **Example** To search for the latest rpm and composer packages: ```shell cloudsmith list packages cloudsmith/examples -q "tag:latest AND (format:rpm OR format:composer)" ``` **Example** To search for all python packages larger than 2KB with more than 50 downloads: ```shell cloudsmith list packages cloudsmith/examples -q "format:python AND downloads:>50 AND size:>2048" ``` # Sorting Packages --- ## Sorting Packages via the Cloudsmith web app At the top of each repository packages page is the sorting dropdown: The available sorting options are: - Name (A-Z) - Name (Z-A) - Uploaded At (Oldest FIrst) - Uploaded At (Newest First) - Size (Smallest First) - Size (Biggest First) - Total Downloads (Ascending) - Total Downloads (Descending) **Multiple Sorts** Note that the UI only supports sorting by one field at a time. ## Sorting Packages via the Cloudsmith API Please see the Cloudsmith full API reference to learn how to: - List all of the packages in a [Workspace](/api/packages/org/list) (Organization). - List all of the packages in a [Repository](/api/packages/list). Then, use the `sort` option to rank the result by any or many or the sort fields listed in the next section. For example, to retrieve all of the packages of the workspace `` organized by ascending number of downloads: ```bash curl -sL --request GET \ --url "https://api.cloudsmith.io/packages//?sort=downloads" \ --header 'Authorization: Bearer ' ``` Sorting search results is simple with Cloudsmith. A collection of one or more fields can be supplied in CSV (comma-separated value) format, and the results will be sorted based on the order of the fields provided. ## Sort Fields | Sort By | Sort Field Example | | :--------------------------------- | :-------------------------------------------------------------------- | | Name (`name`) | `-name` (sort by name alphabetically) | | Format (`format`) | `-format` (sort by format alphabetically e.g. alphine, maven, python) | | Version - String-based (`version`) | `-version` (sort by version, greatest first e.g. 1.2.1, 1.2.0, 1.1.0) | | Status (`status`) | | | Size in Bytes (`size`) | `-size` (sort by size in bytes, largest package first) | | Downloads (`downloads`) | `downloads` (sort by downloads ascending e.g. 0, 10, 50, 1,000) | | Date (`date`) | `-date` (sort by uploaded time, most recent first) | **For all fields you can use**: `-` to sort by descending order. (Example: `-date`, `-version`) ## A Short Worked Example Suppose that you had the following packages: | Name | Format | Size | Version | Downloads | | :--------------------- | :----- | :--- | :------ | :-------- | | cloudsmith-example-cli | Python | 722 | 1.2.0 | 500 | | cloudsmith-example-cli | Python | 719 | 1.1.5 | 1000 | | cloudsmith-java-app | Maven | 6910 | 1.0.0 | 2500 | | cloudsmith-example-pdf | Raw | 309 | 2.0.0 | 100 | **To sort packages by size descending, your sort query would be**: ``` -size ``` Which would provide the following results - cloudsmith-java-app, Size: 6910 bytes - cloudsmith-example-cli, Size: 722 bytes - cloudsmith-example-cli, Size: 719 bytes - cloudsmith-example-pdf, Size: 309 bytes **To sort packages by format alphabetically, with version descending, your sort query would be**: ``` -format,-version ``` Which would provide the following results: - cloudsmith-java-app, Format: Maven, Version: 1.0.0 - cloudsmith-example-cli, Format: Python, Version: 1.2.0 - cloudsmith-example-cli, Format: Python, Version: 1.1.5 - cloudsmith-example-pdf, Format: Raw, Version: 1.1.0 ## Sorting combined with Searching Sorting can also be combined with querying. If you wanted to sort by most downloaded Python packages, your query and sort would be: ``` query=format:python&sort=-downloads ``` Which would provide the following results: - cloudsmith-example-cli, Format: Python, Version: 1.2.0, Downloads: 1000 - cloudsmith-example-cli, Format: Python, Version: 1.2.0, Downloads: 500 # Authentication In software, **authentication** refers to the process of verifying the identity of a user or agent. It's a crucial security measure to ensure that only authorized entities can access systems and its resources. Essentially, authentication confirms that someone or something is who or what they claim to be. In the next pages you can find more information about how to: - Set up and enforce SAML authentication - Configure and modify SAML Group Sync - Configure and modify SCIM - Enforce two-factor authentication - Create and manage OIDC provider settings Cloudsmith offers several methods for user authentication to integrate with your existing Identity Provider (IdP) and allow you to define _who_ can access _what_. ## Supported Protocols - Security Assertion Markup Language (SAML) - System for Cross-domain Identity Management (SCIM) - OpenID Connect ### SAML Workspace SAML settings is where you can enable and enforce SAML authentication. To enable SAML Authentication, you just need to either provide a URL to remote fetch your SAML XML Metadata, or provide the SAML XML Metadata directly inline using the form. #### SAML Group Sync SAML Group Sync is where you can configure automatic mapping of your SAML Groups to Cloudsmith Teams. Please see the [SAML Group Sync](/authentication/single-sign-on#saml-group-sync) documentation for further details. ### SCIM SCIM is where you can enable SCIM provisioning and de-provisioning and obtain your username and password to configure SCIM in your chosen Identity Provider. Please see the [Single Sign-On with Okta](/authentication/single-sign-on-with-okta) documentation for an example of how you can configure SCIM for an Identity Provider. ### 2FA Workspace 2FA is where you can enable Two-Factor Authentication. This will force members to set up Two-Factor Authentication for additional security. # OpenID Connect ## Glossary | Term | Description | | :----------------------------------| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **OpenID Token** | A JWT token with [a specified format](https://openid.net/specs/openid-connect-core-1_0.html#IDToken) that Cloudsmith receives from an OIDC provider which we use to enable users to authenticate as a service account | | **OpenID Connect (OIDC) Provider** | The OIDC Token provider (GitHub Actions, CircleCI, etc) | | **Claims** | Contained within the payload of the OpenID Token, used to verify the authenticity and restrict access of specific Provider requests to specific service accounts | | **ACCOUNT** | Your Cloudsmith Organization slug/identifier | ## How OIDC Works with an OIDC Identity Provider and Cloudsmith The following diagram gives an overview of how an OIDC provider integrates with your workflows and Cloudsmith. - Auth Server: OIDC provider - Client: Service requesting authentication - Implicit trust between Service and Auth Server is necessary to allow seamless OIDC token exchange and verification e.g. GitHub Actions OIDC Server (Auth Server) and its GitHub Actions Public Workers (Client). - Cloudsmith creates an OIDC trust relationship between a Cloudsmith service account and a Client that needs access to Cloudsmith. ### OIDC Flow 1. **Request token**: The client initiates the workflow by requesting a token from the auth service to authenticate with Cloudsmith. 2. **Return token**: The auth service receives the token request. After verification and validation, it generates a JWT token. The auth service and the client have implicit trust, allowing for this token exchange. 3. **Present token**: The client presents a JWT token to Cloudsmith for access to resources. 4. **Verify token**: Cloudsmith verifies the JWT token with the auth service to ensure its authenticity and validity. This step includes verifying claims. 5. **Return new token**: Upon successful validation, Cloudsmith issues a short-lived access token to the client, granting access to Cloudsmith's resources for around 90 minutes to cover the duration of the job. ## OIDC Settings OIDC Provider Settings for your organization are configured at: `https://app.cloudsmith.com/{ACCOUNT}/settings/authentication/openid` You must have the Manager or Owner role in your Cloudsmith organization to configure OIDC Provider Settings. OIDC Provider Settings can only be configured if there's at least one [Service Account](/accounts-and-teams/service-accounts) in your organization Click "Create Provider Settings" to open the **Create Provider Settings** form: To configure a provider, you must provide: |Field|Description| |:---|:---| |**Provider Name**|A Unique name for the provider| |**Provider URL**|A Provider URL. This is unique to each provider, for example for Github Actions it is: `https://token.actions.githubusercontent.com` **Note: this needs to be the root URL i.e. it has the `.well-known/openid-configuration` portion of the URL removed**.This provides the configuration for the chosen provider; we use this to verify the provider signed the OpenID Token we received| |**Required OpenID Token Claims**|The required claims must be in the received OpenID Token's payload to authenticate successfully.For example, tokens from CircleCI could specify the [`oidc.circleci.com/vcs-origin`](https://circleci.com/docs/openid-connect-tokens/#format-of-the-openid-connect-id-token) value, which would allow users to limit requests from a specific version control source repository to allow access to specified Cloudsmith Service Account(s). We **strongly** encourage people to set at least one claim, as some providers share signing keys across all orgs, meaning that without any configured claims, any OIDC request from that provider could authenticate if they knew the service account(s) to target.We support adding a wildcard (`.*` ) operator, **only** at the end of the value of the Claims.| |**Service Accounts**|The service accounts that the user wants to be able to authenticate as with the configured provider & claims combination| Changes will be applied immediately. ## Provider Documentation ### Bitbucket Pipelines [Bitbucket Documentation](https://support.atlassian.com/bitbucket-cloud/docs/integrate-pipelines-with-resource-servers-using-oidc/) [Bitbucket Request Example](#bitbucket-pipelines-example) - Provider configuration: - Provider URL: `https://api.bitbucket.org/2.0/workspaces//pipelines-config/identity/oidc` - **BITBUCKET REQUIRES CLAIMS TO BE PASSED (IT CANNOT BE EMPTY)** - Navigate to: `https://bitbucket.org///admin/pipelines/openid-connect` Copy `Audience` and `Identity provider URL` and insert below - Create a claim with the following: ``` { "aud": "ari:cloud:bitbucket::workspace/XXXX-XXXXX-XXXX-XXXXX-XXXX", "iss": "https://api.bitbucket.org/2.0/workspaces/ORG-NAME/pipelines-config/identity/oidc" } ``` ### CircleCI [CircleCI Documentation](https://circleci.com/docs/openid-connect-tokens/) [CircleCI Request Example](#circleci-example) - Provider URLs are unique per organization, the format is: `https://oidc.circleci.com/org/` ### GitHub Actions [GitHub Actions Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect) [GitHub Actions Request Example](#github-actions-example) - There is [an extra step](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#updating-your-actions-for-oidc) to generate an OpenID Token compared to CircleCI - this needs to be done before sending the token to us. - Because the keys used to verify the token are shared across all GitHub Actions for all organizations, at least one claim verification is _strongly_ encouraged so that only jobs from your GitHub organization can authenticate. If only one claim is being configured, then the audience claim (`aud`) is probably best for this as it scopes it to your GitHub organization's requests. - See [Setup GitHub Actions to authenticate to Cloudsmith using OIDC](/authentication/setup-cloudsmith-to-authenticate-with-oidc-in-github-actions) for a step-by-step guide. ### GitLab [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/index.html#id_tokens) [GitLab Request Example](#gitlab-example) - Provider URL defaults to: - `https://gitlab.com` - If you have a custom domain, the format would remain the same: `https://` ### Jenkins [Jenkins Documentation](https://plugins.jenkins.io/oic-auth/) - See [Setup Jenkins to authenticate to Cloudsmith using OIDC](/authentication/setup-jenkins-to-authenticate-to-cloudsmith-using-oidc) for a step-by-step guide. - The provider URL will be the root URL of your Jenkins master. So if you access Jenkins at `https://jenkins..com`, then your OIDC provider URL is `https://jenkins..com`. ## OIDC Requests With the settings configured, you can now have requests from your provider to use your Cloudsmith organization's OIDC endpoint to exchange the OpenID Token for a JWT token to authenticate with all Cloudsmith API endpoints. ### Token Exchange To receive the Cloudsmith JWT token, you need to make a `POST` request to our OpenID endpoint. This `POST` request must have a body of `oidc_token`, with the OpenID Token as its value and `service_slug` as the slug of the service account the request wants to attempt to authenticate as. The OIDC Endpoint for your Cloudsmith organization will be: `https://api.cloudsmith.io/openid/{ACCOUNT}/` Examples: #### Bitbucket Pipelines Example ```yaml image: atlassian/default-image:3 pipelines: default: - step: name: Cloudsmith OIDC Authentication oidc: true # Enable OIDC for this step script: # Define variables - CLOUDSMITH_ORG="YOUR-ORG" # Replace with your Cloudsmith organization - SERVICE_SLUG="SERVICE-ACCOUNT-SLUG" # Replace with your Cloudsmith service slug # Install required utilities - apt-get update && apt-get install -y curl jq # Exchange Bitbucket OIDC token for Cloudsmith token and capture the response - | OIDC_RESPONSE=$(curl -s -X POST -H "Content-Type: application/json" \ -d "{\"oidc_token\":\"$BITBUCKET_STEP_OIDC_TOKEN\", \"service_slug\":\"${SERVICE_SLUG}\"}" \ "https://api.cloudsmith.io/openid/${CLOUDSMITH_ORG}/") - | echo "OIDC Response: $OIDC_RESPONSE" # Extract the Cloudsmith token from the response - | CLOUDSMITH_TOKEN=$(echo "$OIDC_RESPONSE" | jq -r '.token') # Authenticate with Cloudsmith API - | curl --request GET \ --url "https://api.cloudsmith.io/user/self/" \ --header "X-Api-Key: Bearer ${CLOUDSMITH_TOKEN}" \ --header "accept: application/json" # Output the obtained Cloudsmith token - | echo "Cloudsmith Token: ${CLOUDSMITH_TOKEN}" ``` #### CircleCI Example Using the [Cloudsmith CircleCI orb](https://circleci.com/developer/orbs/orb/cloudsmith/cloudsmith), the `authenticate-with-oidc` command handles the token exchange automatically and exports the result as `CLOUDSMITH_API_KEY`: ```yaml version: 2.1 orbs: cloudsmith: cloudsmith/cloudsmith@2.0.0 workflows: cloudsmith_oidc_publish: jobs: - publish jobs: publish: executor: cloudsmith/default steps: - checkout - cloudsmith/authenticate-with-oidc: organization: my-org service-account: my-service-account - cloudsmith/install-cli - run: name: Build and publish Python package command: | pip install build python -m build --wheel cloudsmith push python my-org/my-repo dist/*.whl ``` Ensure OIDC is enabled in your CircleCI project settings (Project Settings → Advanced → Enable OpenID Connect Tokens). If successful, `CLOUDSMITH_API_KEY` will be set for all subsequent steps. If unsuccessful, you will receive an error message that is intentionally quite generic. This is by design so that we do not leak any information (such as whether OIDC is configured, which claim failed, whether a service account is associated with the provider, etc.). #### GitHub Actions Example ```yaml name: OpenID Connect Demo run-name: ${{ github.actor }} is testing out GitHub Actions with OpenID Connect 🚀 on: [push] permissions: id-token: write # Required for OIDC authentication contents: read # Required for actions/checkout jobs: test-openid-connect: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Authenticate with Cloudsmith via OIDC uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: 'your-oidc-workspace' # Replace with your Cloudsmith workspace oidc-service-slug: 'your-service-account-slug' # Replace with the slug of your Cloudsmith service account oidc-auth-only: 'true' # The action only performs authentication - name: Run subsequent steps run: echo "Successfully authenticated with Cloudsmith." ``` For further details on configuring Github Actions for OIDC see the guide [here](/authentication/setup-cloudsmith-to-authenticate-with-oidc-in-github-actions) or review the [documentation](https://github.com/cloudsmith-io/cloudsmith-cli-action) of the Cloudsmith CLI GitHub Action. #### GitLab Example In this example, we will install the Cloudsmith CLI using PIP and authenticate using OIDC by passing the OIDC token into the `-k` flag of `whoami` command. ```yaml variables: PYTHON_VERSION: "3.9" CLOUDSMITH_ORG: "YOUR-ORG-NAME" CLOUDSMITH_SERVICE: "YOUR-SERVICE-ACCOUNT-SLUG" CLOUDSMITH_API_URL: "https://api.cloudsmith.io" stages: - setup install_cloudsmith_cli: stage: setup image: python:${PYTHON_VERSION} id_tokens: CLOUDSMITH_ID_TOKEN: aud: "${CLOUDSMITH_API_URL}/openid/${CLOUDSMITH_ORG}" script: # Install necessary tools (jq for JSON parsing) - apt-get update && apt-get install -y jq # Install the Cloudsmith CLI - pip install --upgrade pip - pip install cloudsmith-cli - cloudsmith --version # Verify installation # Get the OIDC token generated by GitLab - echo "Requesting OIDC token from GitLab" - export OIDC_TOKEN=$CLOUDSMITH_ID_TOKEN # Use the ID token generated # Send the OIDC token to Cloudsmith API and get the Cloudsmith token - | echo "Authenticating with Cloudsmith using OIDC" CLOUDSMITH_TOKEN=$(curl -s -X POST \ -H "Content-Type: application/json" \ -d "{\"oidc_token\":\"$CLOUDSMITH_ID_TOKEN\", \"service_slug\":\"${CLOUDSMITH_SERVICE}\"}" \ ${CLOUDSMITH_API_URL}/openid/${CLOUDSMITH_ORG}/ | jq -r '.token') # Check if the Cloudsmith token was obtained if [ -z "$CLOUDSMITH_TOKEN" ]; then echo "Failed to obtain Cloudsmith token" exit 1 fi echo "Cloudsmith OIDC Token: $CLOUDSMITH_TOKEN" # Verify authentication using the Cloudsmith token - cloudsmith whoami -k $CLOUDSMITH_TOKEN ``` ## Using the JWT The JWT token generated by the OIDC endpoint can be used as an API key: `X-Api-Key` header: `"X-Api-Key: OIDC_TOKEN"` This token is active for 2 hours from the time of creation. This token works with all Cloudsmith API endpoints to manage resources and format-specific endpoints, e.g., The Ruby endpoint to get all available packages. This token will also work with the [Command-Line Interface](/developer-tools/cli) when specified as `CLOUDSMITH_API_KEY`. # SCIM with Google SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). ## Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to **Authentication → SCIM** and enable the SCIM functionality by selecting **"Allow SCIM."** After enabling SCIM, proceed to Google's Getting Started Guide to complete the setup process. Follow the detailed instructions on their support page for Identity Management Connectors: [Set up SSO with Google as your identity provider](https://support.google.com/a/topic/7556794?hl=en&ref_topic=7556686&sjid=825024795023446732-EU). # SCIM with JumpCloud SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). ## Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to **Authentication → SCIM** and enable the SCIM functionality by selecting **"Allow SCIM."** After enabling SCIM, proceed to JumpCloud's Getting Started Guide to complete the setup process. Follow the detailed instructions on their support page for Identity Management Connectors: [JumpCloud Getting Started Guide](https://jumpcloud.com/support/get-started-identity-management-connectors). # SCIM with Microsoft Entra ID (previously Azure Active Directory) SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to the Authentication Section, then click on the SCIM option. Enable the SCIM functionality by selecting "Allow SCIM." After enabling SCIM, you'll receive a bearer token from Cloudsmith. Copy this token as you'll need it for the Microsoft Entra ID configuration. ## Configuring Microsoft Entra ID Next, configure the SCIM provisioning in Microsoft Entra ID: 1. **Create a new application in Microsoft Entra ID:** - Click **+ New application** - Click **Create your own application** - In the dialog: - Name it something like "Cloudsmith SCIM" - Select "Integrate any other application you don't find in the gallery (Non-gallery)" - Click **Create** 2. **Set up provisioning:** - Click **New provisioning configuration** - Under **Tenant URL** enter: `https://api.cloudsmith.io/scim/v2` - Under **Secret token** paste your bearer token you copied from Cloudsmith. 3. **Complete the setup** by following any additional configuration steps in Microsoft Entra ID's Getting Started Guide. You can find detailed instructions on their support page for SCIM Connectors: [Microsoft Entra SCIM Connector Documentation.](https://learn.microsoft.com/en-us/entra/architecture/sync-scim). # SCIM with Okta SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to **Authentication → SCIM** and enable the SCIM functionality by selecting **"Allow SCIM."** Once SCIM functionality is allowed in Cloudsmith, you then enable SCIM for the Cloudsmith application in Okta on the "General" tab: You then use the "Provisioning" tab to configure SCIM as follows: ||| |:---|:---| |SCIM Connector base URL|https://api.cloudsmith.io/scim/v2| |Unique Identifier for users|email| |Supported provisioning actions|Push New Users PushProfile Updates| |Authentication Mode|Basic Auth| |Basic Auth Username|token| |Basic Auth Password|Please see your [Organization Account Settings](/workspaces/workspace-settings) on Cloudsmith for your SCIM password.| Then test and save the configuration. Once saved, you can then enable the "Create Users", "Update User Attributes" and "Deactivate Users" functionality via the "Provisioning" > "To App" tab: Okta is now configured to provision, update and de-provision users from your Cloudsmith organization. # SCIM with OneLogin SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). ## Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to **Authentication → SCIM** and enable the SCIM functionality by selecting **"Allow SCIM."** After enabling SCIM, proceed to OneLogin's Getting Started Guide to complete the setup process. Follow the detailed instructions in their SCIM documentation: [OneLogin SCIM Documentation](https://developers.onelogin.com/scim). # SCIM with Ping Identity SCIM, or **System for Cross-domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users, and update existing users' profile information based on changes within your Identity Provider (IdP). To begin using SCIM, you need to enable the SCIM functionality in the [Cloudsmith Workspace Settings](/workspaces/workspace-settings). ## Follow these steps: 1. Navigate to the Cloudsmith Workspace Settings. 2. Navigate to **Authentication → SCIM** and enable the SCIM functionality by selecting **"Allow SCIM."** After enabling SCIM, proceed to Ping Identity's Getting Started Guide to complete the setup process. Follow the detailed instructions in their SCIM documentation: [Ping Identity SCIM Documentation](https://docs.pingidentity.com/pingfederate/13.0/introduction_to_pingfederate/pf_scim.html). # SCIM ## Overview SCIM, or **System for Cross‑domain Identity Management**, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0‑compliant. With Cloudsmith’s support for SCIM, you can automatically provision new users, de‑provision existing users and update user profile information based on changes within your IdP. **Early Access** SCIM Groups provisioning integration is currently available in early access. Please contact us if you need this feature enabled for your organization. ## Prerequisites for enabling SCIM on Cloudsmith To configure SCIM for your organization you must meet the following requirements: 1. **Cloudsmith plan** – Your account must be on a plan that includes **Ultra** or **Enterprise**. 2. **Administrator access** – You must be the owner of a Cloudsmith organization. 3. **Domain registry** – At least one domain must be claimed in Cloudsmith - please reach out to our [support team](https://cloudsmith.com/company/contact-us). 4. **Identity provider (IdP)** – You need an IdP that supports SCIM 2.0 to use with Cloudsmith. --- ## Configuration When configuring your IdP, you’ll need the following details that apply to all SCIM integrations with Cloudsmith: - **Base URL**: `https://api.cloudsmith.io/scim/v2` - **Authentication**: Basic Auth, using a SCIM token generated from Cloudsmith. - **Supported Features**: User provisioning and de‑provisioning. **Unique User Identifier** Make sure to use the user’s e‑mail address as the unique identifier. Additionally, the following fields are mandatory: ``` - email - givenName - familyName ``` ## Supported IdP Providers Cloudsmith works with any generic SCIM 2.0‑compliant IdP, but we provide detailed documentation for the most common providers below as an example configuration: - [Google](/authentication/scim-with-google) - [JumpCloud](/authentication/scim-with-jumpcloud) - [Microsoft Entra ID](/authentication/scim-with-microsoft-entra-id) - [Okta](/authentication/scim-with-okta) - [OneLogin](/authentication/scim-with-onelogin) - [Ping Identity](/authentication/scim-with-pingidentity) If you need help with an IdP not listed above, feel free to [contact us](https://cloudsmith.com/company/contact-us). # Setup GitHub Actions to authenticate to Cloudsmith using OIDC Our workflow using GitHub Actions and Cloudsmith with OIDC will push a package to Cloudsmith from GitHub Actions using OIDC to authenticate. ## How does OIDC work with GitHub Actions and Cloudsmith? The following diagram gives an overview of how GitHub's OIDC provider integrates with your workflows and Cloudsmith: ### OIDC flow overview 1. **Request token**: GitHub Actions Worker initiates the workflow by requesting a token from the GitHub OIDC service to authenticate with Cloudsmith. 2. **Return token**: GitHub OIDC service receives the token request. After verification and validation, it generates a JWT token. The GitHub OIDC service and the GitHub Actions Worker have implicit trust, allowing for this token exchange. 3. **Present token**: GitHub Actions Worker presents a JWT token to Cloudsmith for access to resources. 4. **Verify token**: Cloudsmith verifies the JWT token with the GitHub OIDC service to ensure its authenticity and validity. This step includes verifying claims. 5. **Return new token**: Upon successful validation, Cloudsmith issues a short-lived access token to the GitHub Actions Worker, granting access to Cloudsmith's resources for around 90 minutes to cover the duration of the job. ## Step-by-Step Guide Follow the steps below for the Cloudsmith and GitHub Actions OIDC Setup: ### 1. Create a Service Account in Cloudsmith and set Access Controls 1. [Service accounts ](/accounts-and-teams/service-accounts) are a specific type of account that allows you to create a Cloudsmith API Key not tied to a specific user. 2. Navigate to the Services section on your Cloudsmith workspace's accounts page and click "Create Service". 3. In the form, give your Service a name and optionally add a description and choose any teams in your workspace that you wish to add the service to. 4. Configure the repository access controls to allow the [Service Account ](/repositories/repository-privileges#access-control) to push packages to Cloudsmith. ### 2. Configure GitHub OIDC in Cloudsmith **Requirements** NOTE: Manager or Owner role required in your Cloudsmith workspace to configure OIDC Provider Settings. 1. Navigate to the OIDC Provider Settings at: https://app.cloudsmith.com/{WORKSPACE}/settings/authentication/openid 2. Click "Create" to open the Edit Provider Settings form. 3. Fill in the OIDC form: - **Provider Name**: Set a unique name for the provider. - **Provider URL**: This URL must precisely match the iss (issuer) claim from the OIDC token generated by your GitHub environment: - **Standard GitHub.com**: The URL is typically `https://token.actions.githubusercontent.com`. - **GitHub Enterprise Cloud**: The URL will include your enterprise name, like `https://token.actions.githubusercontent.com/YOUR_ENTERPRISE_NAME`. - **Required OpenID Token Claims**: To validate the token, the cloud provider checks if the OIDC token's subject and other claims are a match for the conditions that were preconfigured on the cloud role's OIDC trust definition. - **Service Accounts**: Select the service accounts that can be authenticated with this provider and claim combination. **Claims** We strongly encourage you to set at least one claim. Without claims, any GitHub Action that knows the service account slug could gain access. In the example, we use `repository_owner`. ### 3. Update your GitHub Actions for OIDC Cloudsmith provides a native GitHub action integration, that avoids a manual exchange of tokens. 1. **Add permissions**: In your GitHub Actions YAML file, add the permissions block to allow the workflow to request an OIDC token. ```yaml permissions: id-token: write # Required for requesting the JWT contents: read # Required for actions/checkout ``` 2. **Authenticate using the Cloudsmith Action**: Add the following step to your job. It will handle the entire OIDC token exchange process automatically and configure the Cloudsmith CLI for any subsequent steps. ```yaml - name: Authenticate with Cloudsmith via OIDC uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: 'your-workspace' # Your Cloudsmith Workspace oidc-service-slug: 'your-service-account-slug' # Your Cloudsmith service account slug oidc-auth-only: 'true' # Ensures the action only performs authentication ``` The action fetches the OIDC token from GitHub, exchanges it for a Cloudsmith token, and sets the `CLOUDSMITH_API_KEY` environment variable for the rest of the job. Finally, add any extra step for your workflow and trigger a build in GitHub Actions. ## Example GitHub Actions Workflow Below is a full example of pushing a package to Cloudsmith in GitHub Actions using the modern OIDC authentication method. ```yaml name: Push a Debian package to Cloudsmith with OIDC on: push: branches: [ main ] permissions: id-token: write # Required to request the OIDC token from GitHub contents: read # Required for actions/checkout to read the repository jobs: push: runs-on: ubuntu-latest name: Push a Debian package to Cloudsmith using OIDC steps: - name: Check out code uses: actions/checkout@v4 # This step handles the entire OIDC authentication flow automatically. # It gets a token from GitHub and exchanges it for a Cloudsmith token. - name: Authenticate with Cloudsmith via OIDC uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: 'WORKSPACE' # Replace with your Cloudsmith workspace oidc-service-slug: 'SERVICE_ACCOUNT_SLUG' # Replace with your service account slug oidc-auth-only: 'false' - name: Push package to Cloudsmith run: | cloudsmith push deb WORKSPACE/REPOSITORY/DISTRO/VERSION PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH.deb ``` # Setup Jenkins to Authenticate to Cloudsmith using OIDC Cloudsmith establishes a trust relationship with Jenkins to allow authentication via OIDC tokens for secure access to Cloudsmith's services. # Using OIDC with Jenkins and Cloudsmith This guide provides a setup process for configuring Jenkins to authenticate with Cloudsmith using OpenID Connect (OIDC). By using OIDC, Jenkins can issue short-lived tokens for secure access to Cloudsmith resources. ## Prerequisites 1. **Jenkins OIDC Plugins** Install the following plugins in Jenkins: * **OpenID Connect Provider Plugin** - The OIDC Provider plugin allows Jenkins to act as an OpenID Connect Provider, issuing tokens that can be consumed by Cloudsmith for authentication. * **Credentials Binding Plugin** - Allows secure injection of credentials (e.g., OIDC tokens) into Jenkins jobs. 2. Service Account in Cloudsmith Create a Service Account in Cloudsmith for OIDC access. This will act as a proxy identity for Jenkins jobs. ## Step-by-Step Setup ### 1. Install Required Plugins 1. Go to **Manage Jenkins → Manage Plugins**. 2. Under the Available tab, search for: * **OIDC Connect Provider** https://plugins.jenkins.io/oidc-provider/ * **Credentials Binding** https://plugins.jenkins.io/credentials-binding/ 3. Install both plugins and restart Jenkins if prompted. ### 2. Create OIDC Credential in Jenkins 1. Go to "Manage Jenkins" → "Credentials" → "System" → "Global credentials" 2. Click "Add Credentials" 3. Select "Kind" → "OpenID Connect ID Token" 4. Fill in: i. Scope: "Global" ii. ID: "oidc-token-cred" iii. Audience: jenkins iv. Description: "OIDC Token for Jenkins" v. Issuer URL: Your publically assessible URL 5. Click "OK" ### 3. Host OIDC Configuration Files After creating the credential, Jenkins will show you two URLs where you can get the configuration files: 1. OpenID Configuration URL: `http://your-jenkins-instance/manage/descriptorByName/io.jenkins.plugins.oidc_provider.IdTokenStringCredentials/wellKnownOpenidConfiguration?issuer=your-public-url` 2. JWKS URL: `http://your-jenkins-instance/manage/descriptorByName/io.jenkins.plugins.oidc_provider.IdTokenStringCredentials/jwks?id=oidc-token-cred&issuer=your-public-url` Requirements for hosting these files: * Put the OpenID configuration at /.well-known/openid-configuration * Put the JWKS at /jwks * Host domain must match the issuer URL in Jenkins ### 4. Create a Service Account in Cloudsmith 1. Navigate to your *Cloudsmith Organization*. 2. Go to *Services* under *Account Settings* and click *Create Service*. 3. Provide: * A *name* and optional description. * Select any *teams* for access control if needed. 4. Save your service account. 5. Assign the service account appropriate permissions for your repository. ### 5. Configure OIDC Provider in Cloudsmith 1. Go to *OIDC Provider Settings* at: `https://cloudsmith.io/orgs/{ACCOUNT}/settings/openid-connect/` 2. Click *Create* to open the provider form and configure: * **Provider Name**: Enter a unique name for Jenkins. * **Provider URL**: Enter the Jenkins OIDC endpoint- this should match the URL in Step 2, Issuer. * **Required OpenID Token Claims**: Configure at least one claim, such as aud: jenkins. * **Service Accounts**: Select the Cloudsmith service accounts to authenticate with the provider. 3. Save the configuration. ### 6. Configure a Jenkins Job 1. Create a new Freestyle Project in Jenkins. 2. In the project configuration:: i. Under "Build Environment" → "Use secret text(s) or file(s)" ii. Add → "Secret text" iii. Variable: "OIDC_TOKEN" iv. Credentials: Select your OIDC token 3. Add build step "Shell": ```shell # Get Cloudsmith token response=$(curl -X POST -H "Content-Type: application/json" \ -d "{\"oidc_token\":$OIDC_TOKEN, \"service_slug\": $CLOUDSMITH_SERVICE_ACCOUNT_SLUG}" \ https://api.cloudsmith.io/openid/${CLOUDSMITH_ORG}/) # Get token from response token=$(echo "$response" | jq -r ".token") # Install packages using token python -m venv jenkins source ./jenkins/bin/activate PIP_INDEX_URL="https://token:$token@dl.cloudsmith.io/basic/${CLOUDSMITH_ORG}/${CLOUDSMITH_REPO}/python/simple/" pip install package-name --index-url $PIP_INDEX_URL ``` ### 7. Test the Setup 1. Trigger the Jenkins job. 2. Monitor the console output: i. Ensure the OIDC token is successfully retrieved. ii. Verify the Cloudsmith token is exchanged and used for authentication. 3. Confirm packages are pushed/pulled to/from the specified Cloudsmith repository. ### Example Jenkins Pipeline You can find an example Jenkins Pipeline that integrates with Cloudsmith via OIDC here: https://github.com/cloudsmith-iduffy/jenkins-oidc-poc ## Troubleshooting * Make sure the Cloudsmith service account has permissions to access the Cloudsmith repository. * Invalid Claims or Tokens: Ensure the claims in Jenkins and Cloudsmith match. * Plugin or Credential Issues: Verify plugin installations and credential configurations. * Network Errors: Confirm Jenkins can reach Cloudsmith APIs and vice versa. ### Error Handling If the job fails, common errors might include: * **Invalid Claims or OIDC Token**: Ensure your Jenkins OIDC provider claims match Cloudsmith’s settings. * **Missing Plugins or Credentials**: Double-check plugin installation and credential configuration. * **Network Issues**: Verify connectivity between Jenkins and Cloudsmith. # Single Sign-On with Microsoft Entra ID (formerly Azure Active Directory) This guide provides step-by-step instructions on setting up [Microsoft Entra ID](https://www.microsoft.com/en-gb/security/business/identity-access/microsoft-entra-id) as a SAML IdP for your Cloudsmith Organization. ## Adding Cloudsmith to Microsoft Entra ID Cloudsmith is not an integrated application in Microsoft Entra ID. You'll have to add Cloudsmith manually so you can configure SSO. ### Step 1 Log into the Microsoft Entra ID portal as an admin and click **Azure Active Directory** in the left menu, then **Enterprise Applications** in the menu that appears: ### Step 2 Choose + **New application** from the top menu: ### Step 3 Click **Non-gallery application** and enter "Cloudsmith" in the "Name" box: ### Step 4 Click the blue **Add** button at the bottom of the page. After a short processing delay, you'll be redirected to the overview page for your new application. ### Step 5 Select **Single sign-on** from the left menu and choose **SAML**: ### Step 6 Next, we'll configure SAML settings. Click the pencil symbol beside **Basic SAML Configuration** to begin editing: ### Step 7 To determine your **Identifier** and **Reply URL** (we use the same value for both) we use the following format: `https://cloudsmith.io/orgs/WORKSPACE/saml/acs/`, where "WORKSPACE" is replaced with your workspace's slug. Hit the **Save** button at the top of the page. ### Step 8 Next, we'll configure Azure to also send the user's first and last names during sign-in. Click the pencil icon on the **User Attributes & Claims** section: ### Step 9 Remove all attributes _except_ the **Name identifier value**, the screen should look as below: ### Step 10 Next, we'll add first and last names to the attributes sent to Cloudsmith. Click + **Add new claim** at the top of the page, and add **FirstName** as follows: ### Step 11 Repeat the process for **LastName** and hit **Save**. The attributes screen should now look as below: ### Step 12 Finally, we'll need to add any users that need to be able to access the application. Click** Users and groups** in the left sidebar and then **+Add user**. You can add as many users or groups as needed: ## Providing configuration to Cloudsmith Once configured as above, you'll need to provide metadata to Cloudsmith to connect to your newly configured IdP. Go back to the **Single sign-on** tab in the sidebar, you should see, in section **3 (SAML Signing Certificate)** a link that provides metadata for dynamic configuration, it is labelled **App Federation Metadata Url**. Copy this link and add it to your SAML configuration in your [Cloudsmith organization settings](/authentication/single-sign-on#enable-saml) ## SAML Group Sync Once SAML has been setup, you can then use Microsoft Entra ID and Cloudsmith's group mapping in order to import your teams into Cloudsmith. To get started, head into your Cloudsmith Enterprise application on Azure and select "Single sign-on", then click on "Edit" under "Attributes & Claims": Select "Add a group claim": Select "Security Groups" from the sidebar menu and click "Save", if your AD grouping is setup in a different way, you may choose the relevant fields that suit your infrastructure. You should now see a new entry in the table under "Additional Claims" called `user.groups`: Save the URL for later: `http://schemas.microsoft.com/ws/2008/06/identity/claims/groups` Now go back to your main Active Directory view and select "Groups" tab: Copy the relevant "Object ID" for the group that you wish to sync Cloudsmith with: In your Cloudsmith workspace, go to "Settings" -> "Authentication" -> "SAML Group Syc", click "Enable SAML Group Sync" then "Create Group Sync Mapping": On the pop-up form, fill in the previously saved URL as the "Attribute Key" and the Object ID of the group as the value, click on "Create Group Sync Mapping": Now when a user re-logs with SAML, they will be placed in the assigned group based on the mapping. General documentation on how SAML group sync works can be found [here](/authentication/single-sign-on#saml-group-sync). ## SCIM De-provisioning You can use SCIM to automatically de-provision users from your Cloudsmith organization whenever you remove a user assignment for the Cloudsmith application in Microsoft Entra ID. To set this up, you need to add an "Enterprise application" to your Active Directory on Azure: Click on "New Application": Click on "Create your own application": On the right side, a new tab should appear, fill in the details of the name and select the "...(Non-gallery)" option and click "Create": Once the application is created, select "Provisioning": Once the page loads, select "Provisioning" again, and select "Provisioning Mode" to "Automatic": In Cloudsmith navigate to the following link: `https://app.cloudsmith.com//settings/authentication/scim/` Enable SCIM if it's not enabled and copy your SCIM Authentication token. Go back to Microsoft Entra ID and set the Tenant URL to: `https://api.cloudsmith.io/scim/v2` and the "Secret token" to the copied token from Cloudsmith, click "Save" and "Test connection" Now under the same "Provisioning tab", select "Mappings" dropdown and select "Provision Azure Active Directory Users" Under "Target Object Actions", select only "Updated, Delete" and click "Save": Microsoft Entra ID should now have the necessary information to de-provision users from Cloudsmith. ## All wrapped up Once you've added the link to your App Federation Metadata, and enabled SAML in your Cloudsmith organization settings, you will be able to access the landing page of your organization at the following URL: `https://app.cloudsmith.com/WORKSPACE/saml/login/` Where `WORKSPACE` is your organization's slug/identifier (what you would normally see in the URL when accessing your organization within Cloudsmith). # Single Sign-On with Google --- This guide provides step-by-step instructions on setting up Google (G-Suite) as a SAML IdP for your Cloudsmith Organization. ## Adding Cloudsmith to Google (G-Suite) Cloudsmith is not yet an integrated application in [Google (G-Suite)](https://google.com). You'll have to add Cloudsmith manually so you can configure SSO. ### Step 1 Log into the Google (G-Suite) [Admin Console](https://admin.google.com) and click _Apps_: ### Step 2 Click on the _SAML apps_ panel: ### Step 3 Click the _yellow plus (+) button_ in the bottom right to add a new SAML application: You should then see a modal window pop up where we can begin to enter our application's details. ### Step 4 A new modal dialog pops up. In Step 1, _Enable SSO for SAML Application_, click on _SETUP MY OWN CUSTOM APP_ at the bottom: ### Step 5 In Step 2, _Google IdP Information_, click on the _DOWNLOAD_ button beside _IDP metadata_ and save this file, then click _NEXT_: ### Step 6 In Step 3, _Basic information for your Custom App_, enter "Cloudsmith" (without quotes) as the _Application Name_. (You can optionally add the Cloudsmith logo too for easier visibility, you can find hi-res versions of the logo [here](https://cloudsmith.com/company/brand)): ### Step 7 In Step 4, _Service Provider Details_, enter: `https://cloudsmith.io/orgs/MY_ORG_NAME/saml/acs/` as the _ACS URL_ and _Entity ID_, where "MY_ORG_NAME" is replaced with your organization's slug (i.e. what you normally see in the URL bar for your organization). Leave the _Signed Response_ checkbox _unchecked_ (this is very important). Then for the _Name ID Format_, ensure that "EMAIL" is chosen from the dropdown list. Then click on _NEXT_ on the bottom right: ### Step 8 Next, we'll configure Google (G-Suite) to also send name details to Cloudsmith. Create two mappings. The first is "FirstName" as the value, mapped from _Basic Information_ / _First Name_. The second is "LastName" as the value, mapped from _Basic Information_ / _Last Name_. Then click on _FINISH_: ### Step 9 Finally, you'll need to enable the Cloudsmith SAML application for all or some member's of your Google (G-Suite) organization. Go back to the SAML apps dashboard (as in step 2). Then, click the three dots to the right of the _Cloudsmith_ application, and select either _ON for everyone_ (to enable for all users), or _ON for some_ (to enable for a subset of users): ### Step 10 Your application is now configured on Google (G-Suite). Google states that it may take up to 24 hours before the application is active on your domain, but in our experience it takes much less than this. ## Adding SAML configuration to Cloudsmith Once configured as above, you'll need to add the IdP Metadata XML to the [SAML Settings](/authentication/single-sign-on#enable-saml) for your Cloudsmith organization. When you have added your IdP Metadata XML to your organization SAML settings and enabled SAML Authentication, you will be able to access the landing page of your organization at the following URL: `https://app.cloudsmith.com/WORKSPACE/saml/login/`. Where `WORKSPACE` is your organization's slug/identifier (what you would normally see in the URL when accessing your organization within Cloudsmith). # Single Sign-On with JumpCloud This guide provides step-by-step instructions on setting up [JumpCloud](https://jumpcloud.com/) as a SAML IdP for your Cloudsmith Organization. ## Adding Cloudsmith to JumpCloud Cloudsmith is not an integrated application in JumpCloud. You'll have to add Cloudsmith manually so you can configure SSO. ### Step 1 Log in as an administrator to JumpCloud, choose **Applications** from the sidebar and use the green **+** icon to add a new application: ### Step 2 Choose the generic SAML connector (usually first in the list, labelled **SAML**) and hit **Configure**: ### Step 3 On the configuration screen, enter the **Display Label** as "Cloudsmith", and optionally choose a colour for the application. ### Step 4 Next, you'll need to choose an **IdP entity ID**, which is just a unique string used to identify this application/connector with JumpCloud. It doesn't matter what you use, so long as it's unique within your JumpCloud account. For example purposes we use "JumpCloud-Cloudsmith": ### Step 5 For the next step, you'll need to generate a public and private key used to sign and secure communication between JumpCloud and Cloudsmith (if you don't already have them). JumpCloud have [their own docs](https://jumpcloud.com/support/saml-configuration-notes) on exactly how to generate these keys depending on your operating system. Once generated, upload the private and public keys using the next two fields in the form: ### Step 6 Next, we'll fill in **SP Entity ID** and **ACS URL** with the same value. To determine the value we use the following format: `https://cloudsmith.io/orgs/MY_ORG_NAME/saml/acs/` where "MY_ORG_NAME" is replaced with your organization's slug e.g. for the `cloudsmith` org we use `https://cloudsmith.io/orgs/cloudsmith/saml/acs/` (you can find your organization's slug in the URL bar when you are logged into Cloudsmith, or browsing to [User Settings > Workspaces](https://app.cloudsmith.com/settings/workspaces)): ### Step 7 We then need to configure the SAML Name ID attribute. We want to choose `email` and the appropriate `emailAddress` format: ### Step 8 Cloudsmith requires that users are identified by a first and last name, so we'll need to configure JumpCloud to send those too. Under **User Attributes** click **add attribute** and enter first/last name _exactly_ as follows: ### Step 9 Almost there, we need to check the box labelled **Sign Assertion**: ### Step 10 Check the box labelled **Declare redirect endpoint**: ### Step 11 And finally, choose an appropriate IdP URL, which must be unique in your account (`cloudsmith` is fine, unless you have more than one connector). ### Step 12 Hit the green **Activate** button in the bottom right to complete your configuration: ### Step 13 Your application is now configured on JumpCloud and you can add users and groups as required using the **Users** tab in the sidebar: ## Providing configuration to Cloudsmith Once configured as above, you'll need to provide metadata to Cloudsmith to connect to your newly configured IdP. At the bottom-right of the form, right beside the **Activate** button from the previous step you'll see an **Export Metadata** button. Click it and an XML file containing metadata will be downloaded: Take this file and add the XML contents to your [SAML settings](/authentication/single-sign-on#enable-saml) in your Cloudsmith organization. ## All wrapped up The Cloudsmith application should now appear on the JumpCloud portal as normal: You can then enable SAML in your Cloudsmith [SAML settings](/authentication/single-sign-on#enable-saml). You'll be able to access the landing page of your organization at the following URL: `https://app.cloudsmith.com/WORKSPACE/saml/login/`. Where _WORKSPACE_ is your organization's slug/identifier (what you would normally see in the URL when accessing your organization within Cloudsmith). # Single Sign-On with Okta This guide provides step-by-step instructions on setting up Okta as a SAML IdP for your Cloudsmith Organization. ## Adding Cloudsmith to Okta Cloudsmith is not yet an integrated application in [Okta](https://www.okta.com/). You'll have to add Cloudsmith manually so you can configure SSO. ### Step 1 Log into Okta and click "Applications" and then "Create App Integration" You will then see a modal window pop up where you define the application details ### Step 2 Select SAML 2.0 and click next: ### Step 3 On the next screen (General Settings), enter the App name as "Cloudsmith". (You can optionally add the Cloudsmith logo too for easier visibility, you can find hi-res versions of the logo [here](https://cloudsmith.com/company/brand)): ### Step 4 Next, we'll configure SAML settings. To determine your **Single sign on URL** use the following format: ``` https://cloudsmith.io/orgs/WORKSPACE/saml/acs/ ``` Where `WORKSPACE` is your Cloudsmith organization identifier (slug). We use the same URL for the **Audience URI** value. For **Name ID Format** choose "EmailAddress" For **Application username** choose "Email". ### Step 5 Next, we'll configure Okta to also send the user's first and last names during sign-in: Then click the blue **Next** button at the bottom of the page. ### Step 6 Fill out the **Feedback** section on the next page and click the blue **Finish** button: ### Step 7 Your application is now configured on Okta and you can add users and groups as required using the **Assignments** tab of your application management screen: ## Providing configuration to Cloudsmith Once configured as above, you'll need to provide metadata to Cloudsmith to connect to your newly configured IdP. In the **Sign On** tab of your application management screen, you can see a link to view the IdP metadata (under SAML Signing Certificates): Copy this metadata and add it to your Cloudsmith organization [SAML settings](/authentication/single-sign-on) as Inline XML. You can then [enable SAML authentication](/authentication/single-sign-on#enable-saml) in your Cloudsmith organization, and you can use Okta to begin logging in straight away. You'll be able to access the landing page of your organization at the following URL: ``` https://app.cloudsmith.com/WORKSPACE/saml/login/ ``` Where `WORKSPACE` is your organization's slug/identifier (what you would normally see in the URL when accessing your organization within Cloudsmith). If you're unsure what this is, please just [contact us](https://cloudsmith.com/company/contact-us). You can also optionally set up [SAML Group Sync mappings](/authentication/single-sign-on) to automatically map OKTA groups to teams in your Cloudsmith organization, and SCIM to provision, deprovision and update user profile information. ## SAML Group Sync Once SAML has been setup, group sync can then be configured in order to automatically assign teams from your OKTA directory to Cloudsmith. In order to get started, go to your Cloudsmith application in OKTA and click on the **General** tab: Under **SAML Settings** click **Edit**: Click **Next** to proceed to step 2 and under **A** section on the very bottom **(Group Attribute Statements)** insert the name of the group from Okta along with the valid filter, for example, a group called "Dev" in OKTA will look like this: The attribute for the group: Once all desired groups are inserted in Okta, you can now configure the group mapping in Cloudsmith. In your Cloudsmith workspace, go to "Settings" -> "Authentication" -> "SAML Group Sync", click "Enable SAML Group Sync" then "Create Group Sync Mapping". The name and value will be the same as the group attribute created in OKTA: To test SAML Group Sync, the user must re-log for the sync to apply. ## SCIM SCIM, or System for Cross-domain Identity Management, is an open standard designed to manage user identity information. Cloudsmith is SCIM 2.0-compliant. With Cloudsmith's support for SCIM, you can automatically provision new users, de-provision existing users and update existing users profile information based on changes within your idP. To get started using SCIM, go to "Settings" -> "Authentication" -> "SCIM" and "Allow SCIM": Once SCIM functionality is allowed in Cloudsmith, you then enable SCIM for the Cloudsmith application in Okta on the "General" tab: You then use the "Provisioning" tab to configure SCIM as follows: ||| |:---|:---| |SCIM Connector base URL|`https://api.cloudsmith.io/scim/v2`| |Unique Identifier for users|email| |Supported provisioning actions|Push New Users Push Profile Updates| |Authentication Mode|Basic Auth| |Basic Auth Username|token| |Basic Auth Password|In your Cloudsmith workspace, go to "Settings" -> "Authentication" -> "SCIM" to retrieve your password token.| Then test and save the configuration. Once saved, you can then enable the "Create Users", "Update User Attributes" and "Deactivate Users" functionality via the "Provisioning" > "To App" tab: Okta is now configured to provision, update and de-provision users from your Cloudsmith organization. # Single Sign-On with OneLogin This guide provides step-by-step instructions on setting up [OneLogin](https://www.onelogin.com/) as a SAML IdP for your Cloudsmith Organization. ## Adding Cloudsmith to OneLogin Follow the steps to add Cloudsmith and configure SSO with OneLogin. ### Step 1: Login and add a new app Log into OneLogin and click **Administration** in the top right: Once there, choose **Applications** from the top menu and there, click the blue **Add App** button in the top right: ### Step 2: Select and configure the SAML Test Connector (Advanced) app Search for "SAML Test Connector" and choose "SAML Test Connector (Advanced)": On the next screen - **Add SAML Test Connector (Advanced)** - enter the **Display Name** as "Cloudsmith". You can optionally add the Cloudsmith logo too for easier visibility, you can find hi-res versions of the logo [here](https://cloudsmith.com/company/brand). Then, click the blue **Save** button in the top right. Once saved, a number of additional options will appear in the sidebar. Click "Configuration" to configure SAML settings. 1. Set **Single sign on URL** to `https://cloudsmith.io/orgs/WORKSPACE/saml/acs/`, where "WORKSPACE" is replaced with your organization's slug. We use this URL for the **Audience**, **Recipient**, and **ACS (Consumer) URL** values in the form below. 2. For **ACS (Consumer) URL Validator** use `.*`. 3. Further down the page, for **SAML signature element** choose `Assertion`. 4. Hit the blue **Save** button at the top of the page. ### Step 3: Configure parameters: *FirstName* and *LastName* Next, we'll configure OneLogin to also send the user's first and last names during sign-in. 1. Click on **Parameters** in the sidebar, and then the small blue "+" symbol on the right: 2. Add a new parameter named `FirstName` and ensure the box labelled "Include in SAML assertion" is checked: 3. Hit the blue **Save** button. Then choose **First Name** from the drop-down presented and click **Save**. 4. Repeat the above process for the `LastName` attribute. 5. Once completed for `LastName`, click the **Save** button in the top-right to save all changes. Once configured, the parameters should appear as below: ### Step 4: Configure SAML Signature Algorithm Choose the **SSO** tab in the sidebar and change the** SAML Signature Algorithm** to "SHA-256" and click **Save**. Your application is now configured on OneLogin and you can add users groups as required using the **Users** tab in the sidebar: ## Providing configuration to Cloudsmith Once configured as above, you'll need to provide metadata to Cloudsmith to connect to your newly configured IdP. In the **SSO** tab of your configuration screen you should see a link that provides metadata for dynamic configuration. This is named **Issuer URL**: To enable it, copy this link and add it to your Cloudsmith organization [SAML settings](/authentication/single-sign-on#enable-saml). Your application should now appear on the OneLogin portal and you can use OneLogin to begin logging in straight away. You'll be able to access the landing page of your organization at the following URL: `https://app.cloudsmith.com/WORKSPACE/saml/login/`, where `WORKSPACE` is your organization's slug/identifier. # Single Sign-On with PingIdentity This guide provides step-by-step instructions on setting up [PingIdentity](https://www.pingidentity.com/) as a SAML IdP for your Cloudsmith Organization. ## Add Cloudsmith Application Login to your PingIdentity account and select the environment that you will be managing, then click "manage environment". Under the `Applications`heading on the left-hand menu, select "Applications", then the "Add application" plus icon on the top of the page. You can use the following settings: - Application name: Cloudsmith - Application Type: SAML Application Then click "Configure" and choose "Manually Enter" for `Provide Application Metadata`. Use the following details: - `ACS URLS`: `https://cloudsmith.io/orgs/WORKSPACE/saml/acs/` - `ENTITY ID`: `https://cloudsmith.io/orgs/WORKSPACE/saml/acs/` Replace the values with your `WORKSPACE` name, then click **Save**. Click on the new application and select the "Configuration" tab. Use the following details to complete configuration. - Ensure `Sign Assertion` is selected under the `Signing Key` section - Set `Subject Nameid Format` to be emailAddress, `urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress` Click Save again. Next go to the "Attribute Mappings" tab and use the following attributes: | Cloudsmith | PingOne | | :----------- | :------------ | | saml_subject | Email Address | | FirstName | Given Name | | LastName | Family Name | | groups | Group IDs | | username | Username | ## Add a group Under the `Directory` heading in the left hand menu, select "Groups" then the "Add group" plus icon at the top of the page. Enter your Group Name and click Save. Under your application, select the "Access" tab and click the edit button. Check the box next to your newly created group and click Save. ## Add your users Under the `Directory` heading in the left hand menu, select "Users" then the "Add user" plus icon at the top of the page. Enter the Given Name and Family Name of the new user. the `Username` and `Email` fields should match the details in Cloudsmith. Once completed click save. Next, click on the newly created user and browse to the "Groups" tab. Click the edit icon and check the box next to the group that you created previously. Click save and the user will now be a part of that group. # Cloudsmith configuration ## SAML authentication Under the `SAML Authentication` setup in your Cloudsmith organization settings, you will need to configure `SAML Metadata URL` and `SAML Metadata XML`. The values for these can be found in your Cloudsmith application in PingIdentity. Under the `Applications` heading in the left hand menu of PingIdentity, choose "Applications" and then click on your application, then select the Configuration tab. Copy the `IDP Metadata URL` and paste that into the `SAML Metadata URL` field in the SAML authentication configuration page of Cloudsmith. Click the `Download Metadata` button to download an XML file containing details of the application. Save this file somewhere and open it up in a text editor (or other application like Firefox that will allow you to copy the text within). Copy all of the text and paste that into the `SAML Metadata XML` field in Cloudsmith. Ensure the `Enable SAML Authentication (and Signup)` checkbox is ticked and save the configuration. ## SAML Group Sync Under the `SAML Group Sync` setup in your Cloudsmith organization settings, click on the `Create Group Sync Mapping` button. You will need to provide the `Attribute Key`, `Attribute value` and the team that this configuration will act against. The `Attribute Key` will be what was mapped for the `Group IDs` attribute in your applications attribute mappings within PingIdentity. In the example above, this is `groups`, but can be found by navigating to your application and checking the `Attribute mappings` tab. The `Attribute value` will be the `Group ID` of the group within PingIdentity. This can be found by navigating to your group and checking the `Overview` tab. Once this is configured, select the Role within Cloudsmith that this will apply to (either Member or Manager) and click save. The next time a user logs in to Cloudsmith and is attached to this group within PingIdentity, they should be automatically provisioned for the selected teams within Cloudsmith. # Single Sign-On via SAML Single Sign-On (SSO) is a user authentication method that allows users to log in once and access multiple applications. Security Assertion Markup Language (SAML) is an XML-based standard that enables secure exchange of user authentication and authorization data between parties, particularly in web-based applications, making it a common protocol for implementing SSO Cloudsmith offers support for Single Sign-On (SSO) at the workspace level using Security Assertion Markup Language (SAML). With SAML, organizations can use their existing SSO provider to manage and control access to their Cloudsmith Workspace account. 📘 **About SAML Login** SAML Login is only allowed via your [primary workspace](/workspaces#primary-workspace). If you try to SAML login into a workspace that is not the primary one, the [login will fail](#failed-saml-login). If you can't access your account to set it up as your default workspace, login with username and password to a different workspace, then access to your workspaces section, and set the deafult workspace to your SAML one. ## Failed SAML Login If a user's primary workspace is not the one configured for SAML, their login attempt will fail and they will receive the following error message: ## Getting Started Before configuring SSO with SAML, you'll need: - A SAML Identity Provider that you can connect with Cloudsmith. - Manager access to your primary Cloudsmith Workspace. ## Supported Providers Whilst Cloudsmith should work with any generic SAML IdP, we officially support and provide documentation for a number of the most common providers. Please see the below for guides for each officially supported provider: - [Microsoft Entra ID](/authentication/single-sign-on-with-entra-id) - [Google](/authentication/single-sign-on-with-google-g-suite) - [JumpCloud](/authentication/single-sign-on-with-jumpcloud) - [PingIdentity](/authentication/single-sign-on-with-pingidentity) - [Okta](/authentication/single-sign-on-with-okta) - [OneLogin](/authentication/single-sign-on-with-onelogin) Other providers may be supported if they can set up a generic SAML application. If you need help with an unlisted integration, you can still contact us. ## SAML Landing Page (Login) Once configured, you'll be able to access the SAML login page of your workspace at the following URL: `https://app.cloudsmith.com/WORKSPACE/saml/login/`. Where _WORKSPACE_ is your workspace's slug/identifier (what you would normally see in the URL when accessing your workspace within Cloudsmith). ## Enable SAML **Get in touch** To enable SAML, contact [Cloudsmith Support](https://cloudsmith.com/company/contact-us). You can enable SAML in your Cloudsmith workspace settings: [Image: SAML Authentication Setup] You just need to provide your SAML Metadata XML. You can provide the Metadata XML via a URL, or by copy/pasting the Metadata XML from a file directly inline in the form You can then enable SAML and optionally choose if you wish to enforce SAML-only authentication. If you choose to enforce SAML-only authentication all users that belong to this org will be _forced_ to authenticate via SAML-only, in order to access Cloudsmith. They will not be able to use password-based authentication or other social auth providers. This is more secure, but use caution to prevent lockouts. ## SAML Group Sync Use SAML group sync to assign users with specific roles to existing Cloudsmith teams, based on the users’ group assignment in your selected identity provider (IdP). With SAML group sync you can create a many-to-many mapping between SAML IdP groups and your teams in Cloudsmith. SAML group sync does not create groups. You have to first create a group, then create the mapping. Please do not enable SAML Group Sync before creating your group mappings, as SAML Group Sync will remove any users from a team if there is no corresponding group mapping present. SAML Group Sync is used to use map an attribute from your Identity Provider to a team in your Cloudsmith Workspace. This allows you to add users automatically to the team. For example, if the user @paul is assigned to the design team in the SAML IdP, you can use our SAML group sync to assign @paul to the design team (and then assign to him any roles assigned to this group). ### Creating a Group Mapping You first need to create mappings that will define which attributes and values will map to the respective teams in your Cloudsmith Workspace. To configure a new mapping, click the "Create Group Sync Mapping" button under "SAML Group Sync": [Image: Create Group Sync Mapping Button] You are then presented with the "Create Mapping" form: [Image: Create Mapping Form] Here you can define the following: | Field | Description | | :-------------- | :--------------------------------------------------------------------------- | | Attribute Key | The Attribute name from your Identity Provider that you use to define groups | | Attribute Value | The name of the group from your Identity Provider | | Team | The team in your Cloudsmith Workspace that you want this group mapped to | | Role | The role will be granted within the Team | Once you have configured your mappings and verified the values are correct, you can then enable the mapping functionality by clicking **Enable SAML Group Sync**. # Two-Factor Authentication Two times the security. Twice as secure. Right? Well, that only matters if the base level of security is strong, to begin with. At Cloudsmith, security is one of our most paramount concerns. We utilize our collective years across different disciplines, such as financial technology and Internet startups, to apply this to package management. You can see this in the architectural DNA of the service, such as how we process packages away from the front-end, through the utilization of front-end security techniques, such as the use of Content Security Policy (CSP), HTTP Strict Transport Security (HSTS), etc. We provide support for two-factor authentication via a TOTP (Time-based One-time Password Algorithm) device, such as Google Authenticator, LastPass Authenticator, etc.: Once you've completed enrolment (i.e. registration of your device with us), you will be challenged to authenticate via the device after social or password-based login. You do this by entering a 6-digit pin that your device presents. If you forget your 6-digit pin, we also offer a recovery service using disposable tokens. If you're a member of an organization with "Owner" permissions, you can also force **Enforce Enrolment** of Two-Factor for everyone in the organization: A flag that denotes 2fa within the organization members' list will tell you if the member has two-factor enabled or not: If you enforce enrolment and a User hasn't yet enrolled, they will not be able to access any of the pages for the organization (e.g. they can't view or manipulate packages). If you are security conscious, please consider enabling this. # API Bindings The Cloudsmith API bindings provide libraries in different languages for accessing the service programmatically. The API is fully compliant with the OpenAPI 2.0 Specification, and the clients are generated using `swagger-codegen-cli` from the Swagger Project. Please note that the generated client APIs are intended for developers and are subject to change as the upstream API evolves (especially pre-1.0 releases). If you're looking for something a little more user-friendly, please see the [Cloudsmith CLI application](/developer-tools/cli) for something that comes pre-baked. ## Releases Releases for each language (where supported) are uploaded to the [Cloudsmith API repository](https://broadcasts.cloudsmith.com/cloudsmith/api). The following languages are supported: | Language | Source | Location | | :------- | :----------------------------------------------------------------------------------------- | :--------------------------------------------------------------------- | | Go | [Source Code](https://github.com/cloudsmith-io/cloudsmith-api-go) | | | Java | [Source Code](https://github.com/cloudsmith-io/cloudsmith-api/tree/master/bindings/java) | | | Python | [Source Code](https://github.com/cloudsmith-io/cloudsmith-api/tree/master/bindings/python) | Available on [PyPi](https://pypi.org/project/cloudsmith-api/) | | Ruby | [Source Code](https://github.com/cloudsmith-io/cloudsmith-api/tree/master/bindings/ruby) | Available on [RubyGems.org](https://rubygems.org/gems/cloudsmith-api/) | If you need a different binding, contact us. ## Getting Started We use [swagger-codegen-cli](https://github.com/swagger-api/swagger-codegen/tree/master/modules/swagger-codegen-cli) to generate the bindings by using the [Cloudsmith API schema](https://api.cloudsmith.io/?format=openapi) that is compliant with the [OpenAPI 2.0 Specification](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md). Each of the supported language bindings has its own directory within the bindings directory, and three standardised scripts are used: | | | | :-------- | :------------------------------------------------------------------------------------------------- | | build.sh | Sets the swagger-codegen configuration for the language and generates the bindings into 'src'. | | test.sh | Runs the test suite for the bindings (minimal) to ensure they are "correct" (able to be compiled). | | deploy.sh | Uploads the bindings to Cloudsmith and, if appropriate, the language-specific mainline repository. | Adding a new binding involves the creating of a new directory and the implementation of these three scripts to perform the build and release. To see a list of all the available language bindings you can execute: ``` bin/swagger-codegen-cli ``` To receive configuration help (in order to setup build.sh correctly), execute: ``` bin/swagger-codegen-cli config-help -l ``` Where `` is the language you're generating bindings for (e.g. python). ## Versioning The version of the generated library bindings is automatically made to match the upstream API. So if the Cloudsmith API is 0.21.5, then the language bindings will also be 0.21.5. ## Releasing The client APIs are automatically released as part of the build and release process. See the CI configuration in the [Source Code](https://github.com/cloudsmith-io/cloudsmith-api/) for more details. # Command-Line Interface The Cloudsmith Command Line Interface (CLI) is a Py2/Py3 text-based interface to the API. This allows users, machines and other services to access and integrate smoothly with Cloudsmith without requiring explicit plugins or tools. ## Installation You can install or deploy the latest CLI application from: - [Cloudsmith](https://broadcasts.cloudsmith.com/cloudsmith/cli/) - [PyPi](https://pypi.python.org/pypi/cloudsmith-cli) - [Homebrew Tap](https://github.com/cloudsmith-io/homebrew-cloudsmith-cli) - [DockerHub](https://hub.docker.com/r/cloudsmith/cloudsmith-cli) ### Installing with pip The easiest way is to use `pip`, such as: ```shell pip install --upgrade cloudsmith-cli ``` Or you can get the latest pre-release version from Cloudsmith: ```shell pip install --upgrade cloudsmith-cli --extra-index-url=https://dl.cloudsmith.io/public/cloudsmith/cli/python/index/ ``` ### Installing with zipapp Distributing Python applications can be challenging because, as an interpreted language, Python requires an interpreter to run code, unlike compiled languages that produce standalone executables. However, for smaller, pure-Python programs, a Python Zip application offers a straightforward solution for bundling and sharing your work. This method packages all the necessary code into a single ZIP file. **Requirements** `python3` required to use this installation method. ```bash # download latest release and make it executable curl -s https://api.github.com/repos/cloudsmith-io/cloudsmith-cli/releases/latest | sed -n 's/"browser_download_url": //p' | xargs wget -qO cloudsmith.pyz chmod +x ./cloudsmith.pyz # move it into your $PATH sudo mv ./cloudsmith.pyz /usr/local/bin/cloudsmith ``` You are ready to use it: ```bash cloudsmith -h ``` ### Installing with Homebrew Tap Homebrew is a package manager that can be [installed]() and used in different operative systems (MacOS, Linux, and also Windows). To install the Cloudsmith CLI with Brew, add the tap first: ```bash brew tap cloudsmith-io/cloudsmith-cli ``` Then, install it with: ```bash brew install cloudsmith-cli ``` And you should be able to start using it. If you need to upgrade: ```bash brew upgrade cloudsmith-cli ``` For issues with the tap, please open a [GitHub issue](https://github.com/cloudsmith-io/homebrew-cloudsmith-cli/issues/new) or contact [support@cloudsmith.io](mailto:support@cloudsmith.io). ### Deploying the CLI containerized Cloudsmith maintains a [Docker image for the Cloudsmith CLI](https://hub.docker.com/r/cloudsmith/cloudsmith-cli)⁠ built for use in CI/CD pipelines and automation environments. To deploy it, replace the `API_TOKEN` with the API Token associated to the service account you want to use, specifying the Cloudsmith CLI command you want to use. For example, you can run `cloudsmith whoami`: ```bash docker run --rm \ -e CLOUDSMITH_API_KEY=API_TOKEN \ cloudsmith/cloudsmith-cli:1.8.3 \ whoami ``` Successful execution of the command above will return the member or service account associated to the API Token used: ```text Retrieving your authentication status from the API ... OK You are authenticated as: M. Bolton (slug: mbolton, email: mbolton@initech.com) ``` For example, you can use the Cloudsmith CLI container to push a python package to your workspace/repository `WORKSPACE/REPOSITORY`. In this example, the python package `PACKAGE.whl` is located in `/path_to_package/`, and the package is being mounted in the container filesystem in `/tmp/` as `my_package.whl`: ```bash docker run --rm \ -e CLOUDSMITH_API_KEY=API_TOKEN \ -v "/path_to_package/PACKAGE.whl:/tmp/my_package.whl" \ cloudsmith/cloudsmith-cli:1.8.3 \ push python WORKSPACE/REPOSITORY /tmp/my_package.whl ``` Successful execution of the command will return: ```text Checking python package upload parameters ... OK Checking PACKAGE.whl file upload parameters ... OK Requesting file upload for PACKAGE.whl ... OK Uploading PACKAGE.whl: Creating a new python package ... OK Created: WORKSPACE/REPOSITORY/PACKAGEwhl (package_slug) Synchronising PACKAGEwhl: Package synchronised successfully in 6.001236 second(s)! ``` ### Getting your API Key You'll need to authenticate Cloudsmith for any CLI actions that result in accessing private data or changing resources (such as pushing a new package to a repository). There are two ways to retrieve your API Key: **1. Via the Cloudsmith web app** Go to the [API Key](https://app.cloudsmith.com/settings/api-keys) page in your user settings to view the API Key. **2. via the Cloudsmith CLI** You can retrieve your API key using the `cloudsmith login` command: ```bash cloudsmith login Login: you@example.com Password: PASSWORD Repeat for confirmation: PASSWORD ``` **Use email for login** Please ensure you use your email for the 'Login' prompt and not your user slug/identifier. The resulting output is: ``` Retrieving API token for 'you@example.com' ... OK Your API token is: 1234567890abcdef1234567890abcdef ``` Once you have your API key, you can put it in your `credentials.ini` file, use it as an environment variable `export CLOUDSMITH_API_KEY=`, or pass it to the CLI using the `-k ` flag. For convenience, the CLI will ask you if you want to install the default configuration files, complete with your API key, if they don't already exist. Enter `y` or `yes` to create the configuration files. If the configuration files already exist, you'll have to put the API key into the configuration files manually, but the CLI will print out their locations. **SAML Single Sign On Users** SSO Users do not have a Cloudsmith password and cannot use the `cloudsmith login`command to retrieve their API-Key. SSO Users should instead use the `cloudsmith auth`command and pass their workspace identifier like: `cloudsmith auth -o my-workspace` You will then be prompted to complete the SSO login process via your web browser (if not already signed in), and 2FA if applicable. Once authentication is complete, the CLI is issued an access token for your account. To authenticate in environments without a web browser, such as a remote terminal session, set the CLOUDSMITH_API_KEY environment variable. You can set it for your current session like this: ```bash 📘 export CLOUDSMITH_API_KEY="your-api-key-here" ``` ## Configuration / Setup There are two configuration files used by the CLI: - `config.ini`: For non-credentials configuration. - `credentials.ini`: For credentials (authentication) configuration. By default, the CLI will look for these in the following locations: - The current working directory. - A directory called cloudsmith in the OS-defined application directory. For example: **Linux** - `$HOME/.config/cloudsmith` - `$HOME/.cloudsmith` **Mac OS** - `$HOME/Library/Application Support/cloudsmith` - `$HOME/.cloudsmith` **Windows** - `C:\Users\\AppData\Local\cloudsmith (Win7+, not roaming)` - `C:\Users\\AppData\Roaming\cloudsmith (Win7+, roaming)` - `C:\Documents and Settings\\Application Data\cloudsmith (WinXP, not roaming)` - `C:\Documents and Settings\\Local Settings\Application Data\cloudsmith (WinXP, roaming)` ### config.ini You can specify the following configuration options: `api_host:` The API host to connect to. `api_proxy:` The API proxy to connect through. `api_ssl_verify:` Whether or not to use SSL verification for requests. `api_user_agent:` The user agent to use for requests. The default config is: ```shell config.ini # Default configuration [default] # The API host to connect to (default: api.cloudsmith.io). api_host= # The API proxy to connect through (default: None). api_proxy= # Whether to verify SSL connection to the API (default: True) api_ssl_verify=true # The user agent to use for requests (default: calculated). api_user_agent= # Profile-based configuration # You can set as many additional profiles as you need to provide # for different configuration environments (e.g. prod vs staging). # Add your overrides in the sections and then specify one of: # * -P your-profile-name (as an argument) # * --profile your-profile-name (an an argument) # * CLOUDSMITH_PROFILE=your-profile-name (as an env variable) [profile:your-profile-name] ``` ### credentials.ini You can specify the following configuration options: `api_key:` The API key for authenticating with the API. ```shell credentials.ini # Default configuration [default] # The API key for authenticating with the API. api_key= # Profile-based configuration # You can set as many additional profiles as you need to provide # for different configuration environments (e.g. prod vs staging). # Add your overrides in the sections and then specify one of: # * -P your-profile-name (as an argument) # * --profile your-profile-name (an an argument) # * CLOUDSMITH_PROFILE=your-profile-name (as an env variable) [profile:your-profile-name] ``` ## CLI Scripting The CLI provides a powerful interface for interacting with your packages and repositories in Cloudsmith. However, some operations require additional scripting to achieve the required result. Please see the examples below: ## Copying/Moving Multiple Packages The CLI supports moving one package at a time. However, it is easy to script a solution for moving multiple packages. The following command will list all the packages returned by `YOUR-QUERY`. Extract the relevant metadata and run the copy command for each. ```bash cloudsmith ls pkg YOUR-ACCOUNT/YOUR-REPO -q 'YOUR-QUERY' -F json \ | jq '.data[] | .namespace + "/" + .repository + "/" + .slug' -r \ | xargs -Ipackage cloudsmith copy package YOUR-DEST-REPO ``` **Error message** `The destination is not qualified by a namespace (YOUR-ACCOUNT)`. This is because you can only copy/move packages from repositories within the same namespace. 📘 You can remove the query (YOUR-QUERY) to target all packages. The only downside to this approach is that it might require multiple invocations if you have more packages than the page size limit. It's possible to navigate pages using -p and increase the page size limit to 500 using -l 500 ### Example The following moves all Maven packages named cloudsmith-api with version 0.21.\* from `lskillen/test2` to `lskillen/test3` (Permissions permitting obviously). ```bash cloudsmith ls pkg lskillen/test2 -q 'format:maven AND name:cloudsmith-api AND version:^0.21' -F json | jq '.data[] | .namespace + "/" + .repository + "/" + .slug' -r | xargs -Ipackage cloudsmith copy package test3 ``` ## Waiting for a package to complete synchronizing The `wait_for_package_sync` function below will wait for a package matching the provided query to either complete syncing or fail. For example `wait_for_package_sync "name:^foo$ AND version:0.0.1"` would wait for a package named `foo` with version `0.0.1` to either successfully complete synchronization, fail or time out after 180 seconds. The full list of available search parameters can be found in [our documentation on searching and filtering](/artifact-management/search-filter-sort-packages). ```bash get_package_identifier() { local query=${1:-""} cloudsmith list packages --output-format json --query "$query" "$ORG/$REPOSITORY" 2> /dev/null | jq '.data[0].slug' | sed -e 's/^"//' -e 's/"$//' } get_package_status() { local identifier=${1:-""} cloudsmith status "$ORG/$REPOSITORY/$identifier" 2> /dev/null } wait_for_package_sync() { local query=${1:-""} local total_time_limit=${2:-180} local identifier="" local package_status="" local total_time=0 local identifier="" local package_sync_complete=1 local package_sync_failed=0 local sleep_time=10 while [[ $total_time -lt $total_time_limit ]]; do if [[ -z "$identifier" ]] || [[ "$identifier" == "null" ]]; then identifier=$(get_package_identifier "$query") if [[ -z "$identifier" ]] || [[ "$identifier" == "null" ]]; then echo "Waiting for package .. (query: $query)" > /dev/stderr total_time=$((total_time+$sleep_time)) sleep $sleep_time continue fi fi package_status=$(get_package_status "$identifier") echo "$package_status" | grep --quiet 'Completed' package_sync_complete=$? echo "$package_status" | grep --quiet 'Failed' package_sync_failed=$? if [[ $package_sync_complete -eq 0 ]] || [[ $package_sync_failed -eq 0 ]]; then break fi echo "Waiting for package status ... (identifier: $identifier)" > /dev/stderr total_time=$((total_time+$sleep_time)) sleep $sleep_time done if [[ $total_time -gt $total_time_limit ]]; then echo "Timed out after waiting $total_time seconds for package to sync" > /dev/stderr exit 1 fi if [[ $package_sync_complete -ne 0 ]]; then echo "Package failed to sync after $total_time seconds" > /dev/stderr exit 1 fi echo "Package synced successfully after $total_time seconds" > /dev/stderr } ``` # Troubleshooting When upgrading the Cloudsmith CLI, you may also need to update the Cloudsmith API using: ```shell pip install --upgrade cloudsmith-api ``` If using a proxy with self-signed / internal TLS Certificates, you may need to point to your custom certs with: ``` export REQUESTS_CA_BUNDLE=/path/to/converted/certificate.pem ``` --- title: Cloudsmith Developer Community --- # Developer Community Platforms are built and thrive upon strong communities. At Cloudsmith, we believe in the power of developer communities and want to encourage and foster community growth and participation wherever possible. We open-source as much of our work as possible and love it when our community gets involved and contributes. Contributions from the community strengthen the platform and benefit all of our users. We also believe in contributing back to the broader open-source community, whether through contributions back to the open-source software we use or participating in community events and discussions. You can find our open-source projects below, and we greatly look forward to your contributions. This is here so we can test search better: download logs. ## Open Source projects Everything external to our API is open-source. **[Command Line Interface (CLI)](https://github.com/cloudsmith-io/cloudsmith-cli)**: Command Line Interface (CLI) **[cloudsmith-api](https://github.com/cloudsmith-io/cloudsmith-api)**: API Clients (Generated) **[cloudsmith-examples](https://github.com/cloudsmith-io/cloudsmith-examples)**: Example projects for integration **[cloudsmith-terraform-provider](https://github.com/cloudsmith-io/terraform-provider-cloudsmith)**: Terraform Provider for Cloudsmith **[cloudsmith-circleci-orb](https://github.com/cloudsmith-io/orb)**: A reusable orb to integrate CircleCI **[cloudsmith-github-action](https://github.com/cloudsmith-io/action)**: Github Action to push to Cloudsmith **[docker-extension-cloudsmith](https://github.com/cloudsmith-io/docker-extension-cloudsmith)**: Extension for Docker's Desktop app **[cloudsmith-pipes-publish](/)**: Bitbucket Pipe for publishing artifacts **[GitHub Repos](https://github.com/cloudsmith-io)**: View all repos on GitHub **[cloudsmith-pipes-publish](https://bitbucket.org/cloudsmith-io/workspace/repositories/)**: Bitbucket Pipe for publishing artifacts ## Supporting Free / Open-Source Software In addition to our typical (yet awesome) free/paid public and private repository hosting, we also offer a special type of hosting for open-source projects. This is entirely free and offers 10GB of storage and 100GB of bandwidth by default. You can find more details about this on the [Open-Source Hosting Policy](/open-source-hosting-policy) page. ## Supporting Open Source Here are some projects we love and support: - [HoneyDB](https://honeydb.io) - [Shields](http://shields.io) - [Pony Language](https://www.ponylang.io) - [CloudPosse](https://github.com/cloudposse/packages) ## Community Sponsorship At Cloudsmith we love building communities. Which is why we sponsor them. Including: - [PyBelfast](https://www.meetup.com/PyBelfast/) --- title: "Terraform Provider" --- # Terraform Provider How to integrate Terraform with Cloudsmith Cloudsmith is a verified provider on the [Terraform Registry](https://registry.terraform.io/browse/providers). **[cloudsmith-terraform-provider](https://github.com/cloudsmith-io/terraform-provider-cloudsmith)**: Terraform Provider for Cloudsmith ## Installation To install the Cloudsmith Terraform Provider, add the following to the `required_providers` in your Terraform module: ``` required_providers { cloudsmith = { source = "cloudsmith-io/cloudsmith" version = "0.0.5" } } ``` You also need to add your Cloudsmith API Key, for use by the provider: ``` provider "cloudsmith" { api_key = "API-KEY" } ``` You then run `terraform init` and Terraform will automatically install the provider. To specify a particular provider version when installing providers, see the Terraform documentation on [provider versioning](https://developer.hashicorp.com/terraform/language/providers/configuration#version-deprecated). ## Data Sources ### namespace The namespace data source allows fetching of metadata about a given Cloudsmith namespace. The fetched data can be used to resolve permanent identifiers from a namespace's user-facing name. These identifiers can then be passed to other resources to allow more consistent identification as user-facing names can change. ``` data "cloudsmith_namespace" "my_namespace" { slug = "my-namespace" } ``` | Argument | Description | Required | | :------- | :---------------------------------------- | :------- | | `slug` | The slug identifies the namespace in URIs | Yes | ## Resources ### repository The repository resource allows the creation and management of package repositories within a Cloudsmith namespace. Repositories store packages and are the main entities with which Cloudsmith users interact. ``` resource "cloudsmith_repository" "my_repository" { name = "My Repository" namespace = "${data.cloudsmith_namespace.my_namespace.slug_perm}" description = "A certifiably-awesome private package repository" } ``` |Argument| Description| Required| |--|--|--| `name`| A descriptive name for the repository. |Yes `namespace`| Namespace / account to which this repository belongs. |Yes `description`| A description of the repository's purpose/contents. |No `index_files`| If checked, files contained in packages will be indexed, which increase the synchronisation time required for packages. Note that it is recommended you keep this enabled unless the synchronisation time is significantly impacted. |No `repository_type`| Private repositories are visible only to users that have been granted access. Public repositories are visible to all Cloudsmith users. |No `slug`| The slug identifies the repository in URIs. |No `storage_region`| The Cloudsmith region in which package files are stored. Valid values are: default, ie-dublin, de-frankfurt, ca-montreal, us-norcal, us-ohio, us-oregon, sg-singapore, au-sydney |No `wait_for_deletion`| If true, terraform will wait for a repository to be permanently deleted before finishing. |No See the [repository documentation](/repositories) for more information on creating and managing repositories. ### Repository privileges The repository privileges resource allows the management of privileges for a given Cloudsmith repository. Using this resource it is possible to assign users, teams, or service accounts to a repository, and define the appropriate permission level for each. Note that while users can be added to repositories in this manner, since Terraform does not manage those user accounts, you may encounter issues if the users change or are deleted outside of Terraform. ``` resource "cloudsmith_repository_privileges" "privs" { organization = data.cloudsmith_organization.my_organization.slug repository = cloudsmith_repository.my_repository.slug service { privilege = "Write" slug = cloudsmith_service.my_service.slug } team { privilege = "Write" slug = cloudsmith_team.my_team.slug } team { privilege = "Read" slug = cloudsmith_team.my_other_team.slug } user { privilege = "Read" slug = "some-user-slug" } } ``` When creating a repository via Terraform using a service account and subsequently adjusting the privileges on that same repository, ensure to include a block giving the provisioning account the 'Admin' privilege (as it would get by default by being the creator). This will ensure that the account can continue to provision resources (e.g. entitlements). Otherwise, the resource may overwrite that privilege and further provisioning on the repository when using that account would fail. | Argument | Description | Required | | :----------- | :------------------------------------------------------------------------------------- | :------- | | `organization` | Organization to which this repository belongs. | Yes | | `repository` | Repository to which these privileges apply. | Yes | | `service` | Namespace / Account to which this token belongs | No | | `privilege` | The service's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | `slug` | The slug/identifier of the service. | Yes | | `team` | Variable number of blocks containing teams that should have repository privileges. | No | | `privilege` | The team's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | `slug` | The slug/identifier of the team. | Yes | | `user` | Variable number of blocks containing users that should have repository privileges. | No | | `privilege` | The user's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | `slug` | The slug/identifier of the user. | Yes | ### Entitlement The entitlement resource allows the creation and management of Entitlement tokens for a Cloudsmith repository. Entitlement tokens grant read-only access to a repository and can be configured with a number of custom restrictions if necessary. ``` resource "cloudsmith_entitlement" "my_entitlement" { name = "Entitlement Token 1" namespace = "${cloudsmith_repository.test.namespace}" repository = "${cloudsmith_repository.test.slug_perm}" } ``` | Argument| Description | Required | |---|---|---| | `name` | A descriptive name for the token| Yes | | `namespace` | Namespace / Account to which this token belongs| Yes | | `repository` | Repository to which this token belongs| Yes | | `token` | The literal string value of the token to be created| No | | `is_active` | If true, the token will be enabled and will allow downloads based on configured restrictions (if any).| No | | `limit_date_range_from` | The starting date/time the token is allowed to be used from.| No | | `limit_date_range_to` | The ending date/time the token is allowed to be used to.| No | | `limit_num_clients` | The maximum number of unique clients allowed for the token.| No | | `limit_num_downloads` | The maximum number of downloads allowed for the token.| No | | `limit_package_query` | The package-based search query to apply to restrict downloads to. This uses the same syntax as the standard search used for repositories, and also supports boolean logic operators such as OR/AND/NOT and parentheses for grouping. This will still allow access to non-package files, such as metadata. | No | | `limit_path_query` | The path-based search query to apply to restrict downloads to. This supports boolean logic operators such as OR/AND/NOT and parentheses for grouping. The path evaluated does not include the domain name, the namespace, the entitlement code used, the package format, etc. and it always starts with a forward slash. | No | Please see the [Entitlements](/software-distribution/entitlement-tokens) documentation for more details on managing entitlement tokens and restrictions. ## Example Module A complete, but minimal, example of a Terraform module that uses the Cloudsmith Provider to create a repository and an Entitlement Token (with some basic token restrictions) is: ``` terraform { required_providers { cloudsmith = { source = "cloudsmith-io/cloudsmith" version = "0.0.5" } } } provider "cloudsmith" { api_key = "abcdefghijlkl1234567890" } data "cloudsmith_namespace" "demo-organization { slug = "demo-org" } resource "cloudsmith_repository" "terraform-demo" { description = "Example repo provisioned by Terraform" name = "Terraform Demo" namespace = "${data.cloudsmith_namespace.demo.slug_perm}" slug = "terraform-demo" repository_type = "Private" } resource "cloudsmith_entitlement" "demo-entitlement" { name = "Token 1" namespace = "${cloudsmith_repository.terraform-demo.namespace}" repository = "${cloudsmith_repository.terraform-demo.slug_perm}" limit_num_downloads = "15" limit_num_clients ="1" limit_package_query = "name:,my-package" } ``` ### More example modules Further examples can be found on github in our [terraform-provider](https://github.com/cloudsmith-io/terraform-provider-cloudsmith/tree/master/examples) # Cloudsmith Visual Studio Code Extension The Cloudsmith extension for Visual Studio Code allows you to view and manage your Cloudsmith packages directly within your VS Code IDE. **Note** [Cursor](https://cursor.com/) and [Theia](https://theia-ide.org/) IDEs support VS Code extensions. The Cloudsmith VSCode Extension can be installed using the [vsix](https://github.com/cloudsmith-io/cloudsmith-vscode-extension/releases) file (browse to the [Installation from a .vsix file](/developer-tools/vscode#from-a-vsix-file) section to learn how to install the extension using this method). ## Installation You can install the extension from the [Visual Studio Code Marketplace](https://marketplace.visualstudio.com/items?itemName=Cloudsmith.cloudsmith-vsc) or by using the `.vsix` file from [a release](https://github.com/cloudsmith-io/cloudsmith-vscode-extension/releases). We recommend installing via the Marketplace for ease of use and automatic updates. ### From the VS Code Marketplace (Recommended) 1. Open the **Extensions** view in VS Code (or press `Ctrl+Shift+X`). 2. Search for `cloudsmith` in the search bar. 3. Select the Cloudsmith extension published by Cloudsmith and click **Install**. ### From a .vsix File You can also install the extension manually from a `.vsix` file, which is available on the [GitHub repository releases page](https://github.com/cloudsmith-io/cloudsmith-vscode-extension/releases). 1. In the **Extensions** view, click the **Views and More Actions...** (`...`) button. 2. Select **Install from VSIX...** and choose the downloaded `cloudsmith-x.x.x.vsix` file. Alternatively, open your terminal and run: ```bash code --install-extension cloudsmith-x.x.x.vsix ``` ## Configuration ### Authentication After installation, connect the extension to your Cloudsmith account using an [API Key](/accounts-and-teams/api-key) or a [Service Account Token](/accounts-and-teams/service-accounts) (Entitlement tokens are not supported). 1. Navigate to the Cloudsmith view in the VS Code Activity Bar. 2. Click the **key icon** located in the view's menu. 3. Enter your Personal API Key or Service Account Token into the input box that appears and press `Enter`. 4. Click the **connect/refresh icon** to load your workspaces and repositories. ### Settings You can configure the extension's behavior by navigating to **File > Preferences > Settings** and searching for `Cloudsmith`: - **Show Packages by Group**: If enabled, the explorer will display packages as [Package Groups](/artifact-management/package-groups). By default, individual packages are shown. - **Inspect Output Target**: Choose where the `Inspect Package` command sends its output: `Output` window or a `New Document`. - **Maximum Packages Per Repository**: Set the maximum number of packages to display per repository. The value can be between 1 and 30. The default (and maximum) is 30. - **Use Legacy Web App**: If enabled, URLs opened via the `Open Package in Cloudsmith` command will point to the legacy Cloudsmith web app ([https://cloudsmith.io/](https://cloudsmith.io/)). ## Using the Cloudsmith VSCode extension The extension adds a **Cloudsmith Explorer** view to VS Code. This view provides a tree-based navigator to explore your Cloudsmith assets, including workspaces, repositories, and packages. Selecting a package or package group in the explorer will display a selection of its most important details directly in the tree view. You can right-click on any detail (e.g., Name, Version) to copy its value to the clipboard. This is a subset of the full data available via the Cloudsmith API. For the complete dataset, use the `Inspect Package` command. | Field | Description | | :-------------------- | :----------------------------------------------------------------------------------- | | **Status** | Indicates the current state of the package (e.g., Completed, Failed, Deleted, etc.). | | **Name** | The human-readable title or name of the package. | | **Slug** | A URL-friendly version of the package name, typically lowercase with hyphens for spaces. | | **Slug Perm** | A permanent, unique slug that does not change, even if the package name is updated. | | **Number of downloads** | The total count of how many times the package has been downloaded. | | **Version** | The specific release version number, often following semantic versioning (e.g., `1.2.3`). | | **Tags** | Keywords or labels used to categorize the package and make it discoverable. | | **Uploaded at date/time**| The timestamp indicating the exact date and time the package version was published. | **Package Group fields** When the **Package Groups** option is enabled, the next fields are available for each of the package groups: **Count of packages in group**, **Size**, **Number of downloads**, and **Last pushed date/time**. Right-clicking on a package or package group in the explorer opens a context menu with several commands: - **Inspect Package**: this command retrieves the raw JSON data for the selected package or group. By default, the output is sent to the VS Code `Output` window. You can change this behavior in the extension settings to send the output to a new, untitled text document instead. - **Open Package in Cloudsmith**: this command opens the selected package's page directly in the Cloudsmith web application using your default browser. # Webhooks Webhooks are great for driving automation by pushing data to your pipelines, or by integrating with your chat tools to provide slick ChatOps. Cloudsmith webhooks support events that occur in a repository, such as when packages have been uploaded, are synchronizing, have synchronized or have failed. ## Pipeline Automation For pipeline automation, you might utilise webhooks to instruct a CI/CD service, such as [CircleCI](https://circleci.com) or [Spinnaker](https://spinnaker.io), that it is time to deploy when a synchronized package is moved to your production repository. In this way, you can control the flow from development to production by limiting who (or what) has the authority to move packages to the production repository, and thus, to deploy publicly. ## ChatOps For ChatOps, you can utilise webhooks to [send a message to a chat tool such as Slack](/integrations/integrating-with-slack) when each type of event occurs. You'll format this in such a way so as to present critical information to your team, such as what the package is, where it is located, who uploaded it, and how to access it, etc. If you're really fancy, you'll have a Slack integration that lets users interact with Cloudsmith via slash commands, for super slick bi-directional DevOps goodness. ## Create a Webhook Creating a webhook is simple and only requires an endpoint to send it to. [Image: Create Webhook Button] [Image: Create Webhook Form] For testing webhooks, we recommend either [Webhook Tester](https://webhook.site) or [RequestCatcher](https://requestcatcher.com). ## Webhook Payload Formats Cloudsmith supports multiple payload formats for webhooks. Plus a templating format, using the [handlebars](https://handlebarsjs.com/) language, for the ultimate in flexibility. You can find more details on that, later in this guide. The payload formats supported are: |Payload Format|Description|Example| |:--|:--|:--| |JSON Object|The payload is encoded as a singular JSON Object at the root-level.|`{ "data": {"foo": bar"} }`| |JSON Array|The payload is encoded as a JSON Array at the root-level.|`[{ "data": {"foo": bar"} }]`| |Form Encoded JSON Object|The payload is encoded as a singular JSON Object at the root-level.|`payload=%22%7B%5C%22data%5C%22%.....`| |Handlebars Template|The payload is encoded in a format determined by the template that you create.|`data:` `- foo: bar`| See below for more information on how to construct/use handlebars templates. ## Webhook Event Types / Subscriptions Cloudsmith splits repository-level webhooks into several different event types. When you create a webhook, you subscribe to a subset of event types. When an event occurs, if your webhook is subscribed, you'll get a notification in the payload format you chose previously. Each type of webhook event occurs for a different reason and may have a varying structure to the payload. However, each `Package` event will always have the package that caused the event. The event types are: | Event | Identifier | Description | Content Type | | :------------------------ | :------------------------- | :---------------------------------------------------------------------------------------------------------------- | :----------- | | Ping | `ping` | Sent when a new webhook is created or when requested to test an endpoint. | `Ping` | | Package Created | `package.created` | Sent when a package has first been uploaded but not yet processed. | `Package` | | Package Deleted | `package.deleted` | Sent when a package has been deleted. | `Package` | | Package Failed | `package.failed` | Sent when a package has failed to process. | `Package` | | Package Quarantined | `package.quarantined` | Sent when a package is quarantined. | `Package` | | Package Released | `package.released` | Sent when a package is released from quarantine. | `Package` | | Package Restored | `package.restored` | Sent when a package is restored from deletion. | `Package` | | Package Scanned | `package.security_scanned` | Sent when a package security scan has been completed (which makes new vulnerabilities for the package available). | `Package` | | Package Synchronized | `package.synced` | Sent when a package has been fully processed (i.e., synchronized) and is available for download. | `Package` | | Package Synchronizing | `package.syncing` | Sent when a package has started to be processed (i.e., synchronizing state). | `Package` | | Package Tags Changed | `package.tags_updated` | Sent when the tags for a package have changed. | `Package` | ### Package Query Filter You can also add a package search query to webhooks. This uses the same [search syntax](/artifact-management/search-filter-sort-packages) supported elsewhere for searching. If present, packages that emit events need to match the search query provided. In this way, you could filter webhooks for only certain types of packages, in addition to specific events. For example, to send `Package Synced` events for `my-web-app` packages that have the `release` tag, you can define the query as `name:my-web-app AND tag:release`. As shown below: [Image: Using the Package Query Filter] ## Webhook Security ### Secret Header / Value If you need a way of authenticating a webhook at the receiver, and you don't want to use the HMAC-based verification outlined later below, you can use the **Secret Header** and **Secret Value** fields. These are values that will be sent with every webhook sent, that allow you to perform some form of authentication on the receiver side. Typically useful if the webhook is otherwise open to the world, but you want to authenticate before enacting on the webhook. Cloudsmith stores the **Secret Value** encrypted internally. ### Validating Webhooks If you're super paranoid about security, and you should be (of course), you can validate that webhooks originated from Cloudsmith, unaltered. To do this, each message that is sent from Cloudsmith calculates an [HMAC Digest](https://en.wikipedia.org/wiki/HMAC) of the contents. This HMAC, which stands for Hash-based Message Authentication Code, is generated using a cryptographic method for describing the contents of a message, verifiable by the receiver. To start, you'll need to either provide or make note of the **HMAC Signature Key**. This is used to generate the HMAC, and you'll need it on the receiver side to be able to validate messages. After a webhook is created, you'll be able to supply new values for the signature key, but you won't be able to retrieve the old one anymore. The value is stored encrypted in Cloudsmith. When enabled, Cloudsmith will send the `HMAC` in the `X-Cloudsmith-Signature` header of every message. The algorithm used for the calculation is `HMAC-SHA1`. With the secret, you can then verify this against the payload (contents of the message) using the following method (Python-like pseudo-code): ```python hmac_received = message_headers["X-Cloudsmith-Signature"] hmac_calculated = hmac_sha1("my-secret-key", message_body) hmac_validated = (hmac_received == hmac_calculated) if not hmac_validated: bailout() ``` Where: - `hmac_sha1` is a function that takes a secret key, and the payload, and returns an HMAC. - `message_headers` is the headers received from Cloudsmith in the HTTP request. - `message_body` is the body/payload of the webhook received from Cloudsmith in the HTTP request. - `"my-secret-key"` is the secret key from the **HMAC Signature Key** field in the webhook form. - `bailout` is a function that cancels everything, alerts your team, and stops bad things from happening. ### Verifying SSL Certificates You can choose to not verify SSL certificates when webhooks are sent. You'll need this if the destination endpoint is using a self-signed certificate. However, please be aware that it opens you to attacks where the endpoint is replaced by a malicious user. Use with care. ## Webhook Templates (Handlebars) [Handlebars](https://handlebarsjs.com/) is a minimal templating language, normally used in Javascript-based applications, for constructing messages based on variables and limited conditional flow. This means you can drive all kinds of external services, without needing an intermediate "translation" service in-between, such as [IFTTT](https://ifttt.com) or [Zapier](https://zapier.com). Of course, you can still use those to achieve some [powerfully dynamic integrations](/integrations/integrating-with-zapier). ### Template Format When creating templates, you can choose the overall **Template Format** that you'd like to emit. This primarily changes the `Content-Type` that is sent with the webhook but also determines the syntax highlighting in the editor, and the validation of the content. You can also override the exact **Content Type** you'd like to send if you want to be more specific. ### Event Templates For each of the event types, you can write a different template. If you provide a specific template for a specific event, that template will be chosen first. Otherwise, it will use the **Default** template. In this way, you can process the events in different ways, depending on which fired but send them all to the same endpoint. ### Handlebars Syntax The Handlebars [Basic Usage](https://handlebarsjs.com/guide/expressions.html#basic-usage) is a good first reference for how to construct a template. An example of some of the constructs you'll use are interpolation (getting data from the webhook payload), functions (manipulating the data), and conditionals (doing different thing depending on the data content): - Interpolation: Use `{{data.name}}`, to get `name` from the `data` object. - Functions: Use `{{concat data.name "-test"}}`, to concatenation `data.name` and `"-test"` together. - Conditionals: Use `{{#if (gt data.downloads 0)}foo{{/if}}` to output `foo` if `data.downloads` is greater-than zero. ### Payloads You will be acting upon the JSON data from a webhook payload (which you can see later on in the "Webhook Payloads") section. ### Helper Functions In addition to the [builtin helpers](https://handlebarsjs.com/guide/builtin-helpers.html#each), we have a number of additional helper functions that extend the base handlebars syntax. You can call a helper function using the Handlebars syntax of `{{func arg1 arg2}}` for emitting values, and `{{#if (func arg1 arg2)}}` in conditionals. The functions supported are shown below. Function types are either Unary (1 argument) or Binary (2 arguments): |Function|Type|Description|Example| |:---|:---|:---|:---| |`neg`|Unary|Arithmetic Negation|`{{neg 10}}` = `-10`| |`not`|Unary|Logical Negation|`{{neg false}}` = `true`| |`truth`|Unary|Truth Test|`{{truth 1}}` = `true`| |`tojson`|Unary|Convert to JSON|`{{tojson data.some.object}}`| |`add`|Binary|Arithmetic Addition|`{{add 5 5}}` = `10`| |`concat`|Binary|Concatenation|`{{concat "foo" "bar"}}` = `foobar`| |`div`|Binary|Arithmetic Floating Division|`{{div 5 2}}` = `2.5`| |`floordiv`|Binary|Arithmetic Integer Division|`{{floordiv 5 2}}` = `2`| |`and`|Binary|Logical/Bitwise And|`{{and true true}}` = `true` `{{and true false}}` = `true`| |`or`|Binary|Logical/Bitwise Or|`{{or true true}}` = `true` `{{or true false}}` = `true`| |`xor`|Binary|Logical/Bitwise Exclusive Or|`{{xor true true}}` = `true` `{{xor true false}}` = `false`| |`mod`|Binary|Arithmetic Modulo|`{{mod 5 2}}` = `1`| |`mul`|Binary|Arithmetic Multiplication|`{{mul 5 5}}` = `25`| |`sub`|Binary|Arithmetic Subtraction|`{{sub 10 5}}` = `5`| |`lt`|Binary|Relational Less-Than|`{{lt 5 10}}` = `true` `{{lt 5 5}}` = `false`| |`lte`|Binary|Relational Less-Than or Equal-To|`{{lte 5 10}}` = `true` `{{lte 5 5}}` = `true`| |`gt`|Binary|Relational Greater-Than|`{{gt 10 5}}` = `true` `{{gt 5 5}}` = `false`| |`gte`|Binary|Relational Greater-Than or Equal-To|`{{gte 10 5}}` = `true` `{{gte 5 5}}` = `true`| |`eq`|Binary|Relational Equality|`{{eq 10 5}}` = `false` `{{eq 5 5}}` = `true`| |`ne`|Binary|Relational Inequality|`{{ne 10 5}}` = `true` `{{ne 5 5}}` = `false`| |`contains`|Binary|Needle (arg1) contained in Stack (arg2)|`{{contains ""foo"" ""foobar""}} `= `true` `{{contains ""baz"" ""foobar""}}` = `false`| |`startswith`|Binary|Needle (arg1) is at start of Stack (arg2)|`{{startswith ""foo"" ""foobar""}}` = `true` `{{startswith ""bar"" ""foobar""}}` = `false`| |`endswith`|Binary|Needle (arg1) is at end of Stack (arg2)|`{{endswith ""bar"" ""foobar""}}` = `true` `{{endswith ""foo"" ""foobar""}}` = `false`| |`join`|Binary|Join Elements (arg2) by Delimiter (arg1)|`{{join ";" ["a", "b"]}}` = `a;b`| |`split`|Binary|Split String (arg2) by Delimiter (arg1)|`{{split ";" "a;b"}}` = `["a", "b"]`| ## Example Template The following is a short worked example of how you might use templates: Sheila, a Senior DevOps engineer at WhyObi Ltd, is setting up alerts for ChatOps. She'd like to get a notification for when a specific package has been synchronized (is available for download) and would like to output the download URL for it in a special Slack channel. If the package has a tag labelled "hotfix", she'd also like to call this out in bold as part of the message. To start with, Sheila creates a new webhook. This is a Slack webhook, and according to the [Slack documentation for incoming webhooks](https://api.slack.com/messaging/webhooks), it needs to have a content type of `application/json`. So Sheila enters the new webhook endpoint, chooses `Handlebars Template` as the **Payload Format**, and picks `JSON (application/json)` as the **Template Format**. She's only interested in packages that have synchronized, so she selects `Send Specific Events (choose)` in **Event Subscriptions**, and then ticks the `Package Synchronised` checkbox. Going back to the **Payload Templates**, she clicks on the `Package Synchronised` tab to start a new template for that type of event. Now, for the fun part, to meet her requirements Sheila writes out the following template using the Handlebars language, but constructs it to make a Slack-compatible payload: ```json { "username": "cloudsmith-bot", "icon_emoji": ":cloud:", "blocks": [ { "type": "section", "text": { "type": "mrkdwn", "text": "The <{{data.self_webapp_url}}|{{data.name}}{{#if (truth data.version)}} ({{data.version}}){{/if}}> {{data.format}} package is ready for download." } }, {{#each data.tags.info}} {{#if (eq this "hotfix")}} { "type": "section", "text": { "type": "mrkdwn", "text": "*HOTFIX*: Warning, this is an non-standard release." } }, {{/if}} {{/each}} {{#each data.files}} { "type": "section", "text": { "type": "mrkdwn", "text": "Download File: <{{this.cdn_url}}|{{this.filename}}>" } }{{#if (not @last)}},{{/if}} {{/each}} ] } ``` Also, to make the **Ping** event (for testing the endpoint) compatible with Slack, she also fills in the template for that too but makes it a simple one: ```json { "username": "cloudsmith-bot", "icon_emoji": ":cloud:", "text": "Ping? Ping!" } ``` Putting this together, the final form looks like: She tests the integration by uploading a package called `genesis`, at version `1.0.3`, and assigns it the `hotfix` tag. When Cloudsmith has synchronized the content, she receives a ChatOps alert in her Slack channel where she created the incoming webhook: Mission. Success! Now her team has a way of being informed directly in Slack of packages that are uploaded. Along with an easy link to the UI for the package, and a download link for the file. She's also informed if the package was labelled as a hot fix release. ## Webhook Payloads ### Common Structure All webhooks sent from Cloudsmith have the following structure (in `JSON Object` format): ```json { "context": object, "data": object, "meta": { "attempt_at": string, "event_at": string, "event_id": string, "trigger_id": string, "webhook_id": integer } } ``` With the following definitions: | Field (Path) | Type | Description | Required | Example | | :----------------- | :------ | :--------------------------------------------------------------------------------------------------------------- | :------- | :------------------------------------------------- | | `.context` | Object | An object for the event context if available (e.g. "old_tags" and "new_tags" for "Package Tags Changed" events). | No | `{ "old_tags": ["foo"], "new_tags": ["bar"] }` | | `.data` | Object | An object for the event, usually a `Package` model. | Yes | See **Example Package Payload** below. | | `.meta.attempt_at` | String | An ISO 8601 datetime string, representing when the webhook was sent at. | Yes | `"2020-07-07T17:30:34.342167+00:00"` | | `.meta.event_at` | String | | Yes | `"2020-07-07T17:30:34.296482+00:00"` | | `.meta.event_id` | String | | Yes | `"package.synced"` | | `.meta.trigger_id` | String | A **globally** (as in, all of Cloudsmith) unique identifier for the trigger. | Yes | `"c0e2b63e-3d84-4d54-bd62-ae7d0b2764a7"` | | `.meta.webhook_id` | Integer | A sequential id for the source webhook. Unique per repository. | Yes | `1` | ## Example Package Payload An example of a `Package` payload, sent for webhooks emitted by packages: ```json { "data": { "architectures": [], "cdn_url": "https://dl.cloudsmith.io/basic/my-org/my-repo/raw/files/my-file.deb", "checksum_md5": "f64c6c0eb95e8455d0d5b248e988b4ff", "checksum_sha1": "4d10412ec6c66b5e2bfd28fbda893a9d7c8fd1a4", "checksum_sha256": "82ee34b2ee3715e055f58a5825799b5d763e8517bc2d5da80a8ac5c59d9940d3", "checksum_sha512": "b6075cb64564ab9048c28426ebb1345aef52f0a65f2ff1dcf5caba1b228be85e29775c419a6dc36ac0b455888544e2948f6182ef38064bebebc349e2851027e1", "description": "My Awesome Package", "distro": null, "distro_version": null, "downloads": 0, "epoch": null, "extension": ".deb", "filename": "my-file.deb", "files": [ { "cdn_url": "https://dl.cloudsmith.io/basic/my-org/my-repo/raw/files/my-file.deb", "checksum_md5": "f64c6c0eb95e8455d0d5b248e988b4ff", "checksum_sha1": "4d10412ec6c66b5e2bfd28fbda893a9d7c8fd1a4", "checksum_sha256": "82ee34b2ee3715e055f58a5825799b5d763e8517bc2d5da80a8ac5c59d9940d3", "checksum_sha512": "b6075cb64564ab9048c28426ebb1345aef52f0a65f2ff1dcf5caba1b228be85e29775c419a6dc36ac0b455888544e2948f6182ef38064bebebc349e2851027e1", "downloads": 0, "filename": "my-file.deb", "is_downloadable": true, "is_primary": true, "is_synchronised": true, "size": 2204, "slug_perm": "8m7hRtzW7Ylq", "tag": "pkg" } ], "format": "raw", "format_url": "http://api.cloudsmith.io/formats/raw/", "identifier_perm": "xKYgfsOsboz6", "indexed": true, "is_sync_awaiting": false, "is_sync_completed": true, "is_sync_failed": false, "is_sync_in_flight": false, "is_sync_in_progress": false, "license": null, "name": "my-file.deb", "namespace": "my-org", "namespace_url": "https://api.cloudsmith.io/namespaces/my-org/", "num_files": 0, "package_type": 1, "release": null, "repository": "testo2", "repository_url": "https://api.cloudsmith.io/repos/my-org/my-repo/", "self_html_url": "https://app.cloudsmith.com/~my-org/repos/my-repo/packages/detail/my-package/package-version", "self_webapp_url": "https://app.cloudsmith.com/my-org/r/my-repo/my-package-format/my-package-name/package-version/package-id", "self_url": "https://api.cloudsmith.io/packages/my-org/my-repo/xKYgfsOsboz6/", "size": 2204, "slug": "my-package", "slug_perm": "xKYgfsOsboz6", "stage": 9, "stage_str": "Fully Synchronised", "stage_updated_at": "2020-07-06T21:00:50.635246Z", "status": 4, "status_reason": null, "status_str": "Completed", "status_updated_at": "2020-07-06T21:00:50.635221Z", "status_url": "https://api.cloudsmith.io/packages/my-org/my-repo/xKYgfsOsboz6/status/", "subtype": "file", "summary": null, "sync_finished_at": "2020-07-06T21:00:50.635239Z", "sync_progress": 100, "tags": { "version": [ "latest" ] }, "tags_immutable": {}, "type_display": "file", "uploaded_at": "2020-07-06T18:57:07.199040Z", "uploader": "my-user", "uploader_url": "https://api.cloudsmith.io/users/profile/my-user/", "version": null, "version_orig": null }, "meta": { "attempt_at": "2020-07-07T17:30:34.342167+00:00", "event_at": "2020-07-07T17:30:34.296482+00:00", "event_id": "package.synced", "trigger_id": "c0e2b63e-3d84-4d54-bd62-ae7d0b2764a7", "webhook_id": 1 } } ``` ## Webhook Origin IP Addresses The webhooks can originate from the following IP addresses: - 34.252.163.216 - 52.208.86.0 - 108.129.59.129 These are not guaranteed to remain static forever. For an updated list, please [contact us today](https://cloudsmith.com/company/contact-us). # Alpine Repository Alpine Linux is a lightweight Linux Distribution that is designed to be smaller and more resource efficient, and as such has grown popular as the basis for Docker containers. Cloudsmith supports hosting and distributing packages in the APK format, used by both **Alpine Linux** and **Wolfi**. Although both distributions use the same package format, they are distinct and incompatible — packages built for Alpine Linux cannot be used on Wolfi, and vice versa. - **Alpine Linux** is a lightweight Linux distribution popular as a Docker container base image. It uses versioned releases (e.g., v3.0, …, v3.22, v3.23, etc.) grouped by architecture e.g. `v3.23/main/aarch64/` via the `apk` package manager. - **Wolfi** is a minimal, container-optimized Linux distribution from Chainguard. Unlike Alpine, it follows a rolling release model in which packages are removed after 12 months. For more information: - [Alpine Linux](https://alpinelinux.org/) — official Alpine Linux website - [Alpine Linux Wiki](https://wiki.alpinelinux.org) — Alpine Linux documentation - [Wolfi OS](https://wolfi.dev/) — official Wolfi website ## What Cloudsmith Supports Cloudsmith repositories for the Alpine format support: - **Public and private package hosting** serve APK packages to authenticated users or publicly - **RSA signing** all repositories are signed with an RSA key that clients can verify via `apk`. Alpine uses SHA-1 and Wolfi uses SHA-256. - **Upstream proxying and caching** proxy and cache packages from Alpine Linux mirrors, Wolfi, or Chainguard (see [Upstream Proxying / Caching](#upstream-proxying--caching)) - **Both Alpine Linux and Wolfi** a single repository format handles both distributions; the distribution is selected at upload time and automatically detected at install time **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :------------------------------------------------------------------------------------------------------------------------------ | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | VERSION | Alpine distribution version, e.g. `v3.23`. Use `any-version` if your package is compatible with more than one version | | FINGERPRINT | The 8-byte fingerprint of the public RSA key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). #### Alpine Linux - Package Upload The command to upload an Alpine package via the Cloudsmith CLI is: ```shell cloudsmith push alpine OWNER/REPOSITORY/alpine/VERSION PACKAGE_NAME-PACKAGE_VERSION.apk ``` Upload a package for specific Alpine version: ```shell cloudsmith push alpine OWNER/REPOSITORY/alpine/v3.23 libjq-1.0.3.apk ``` Upload a package for any version of Alpine: ```shell cloudsmith push alpine OWNER/REPOSITORY/alpine/any-version libjq-1.0.3.apk ``` Upload a package for Wolfi: ```shell cloudsmith push alpine OWNER/REPOSITORY/wolfi/any-version libjq-1.0.3.apk ``` **Contextual Documentation** Wolfi has a single index that typically contains all versions available within the past 12 months. When uploading a Wolfi package to Cloudsmith, use the `wolfi/any-version` distribution. ### Upload via Cloudsmith Website Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package You have a choice of 3 methods to set up your Cloudsmith repository: - Automatic configuration (recommended) - Force a specific distribution/release (if your system is compatible but not identical) - Manual configuration ### Public Repositories To install Alpine packages from a public Cloudsmith repository, you can set up the repository using the following: ```shell sudo apk add --no-cache bash curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo bash ``` If you need to force a specific distribution: ```shell sudo apk add --no-cache bash curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo distro=alpine codename=VERSION bash ``` Or, you can manually configure the repository: ```shell curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/rsa/rsa.FINGERPRINT.key' > /etc/apk/keys/REPOSITORY@OWNER-FINGERPRINT.rsa.pub curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/config.alpine.txt?distro=alpine&codename=VERSION' >> /etc/apk/repositories apk update ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Secrets management** Entitlement Tokens and API-Keys should be treated as secrets. You should ensure that you do not commit them in configuration files along with source code, or expose them in any logs. To install Alpine packages from a private Cloudsmith repository, you can set up the repository using the following: ```shell Entitlement Token Auth curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (API-Key) sudo apk add --no-cache bash curl -u "USERNAME:API-KEY" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (Token) sudo apk add --no-cache bash curl -u "token:TOKEN" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo bash ``` If you need to force a specific distribution, set `distro` and `codename` (for Alpine): ```shell Alpine (Entitlement Token) sudo apk add --no-cache bash curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.alpine.sh' \ | sudo distro=alpine codename=VERSION bash ``` Or, you can manually configure the repository for Alpine: ```shell Entitlement Token Auth curl -1sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/rsa.FINGERPRINT.key' \ > /etc/apk/keys/REPOSITORY@OWNER-FINGERPRINT.rsa.pub echo "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/alpine/VERSION/main" \ > /etc/apk/repositories apk update ``` ```shell HTTP Basic Auth curl -u "token:TOKEN" -1sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rsa.FINGERPRINT.key' \ > /etc/apk/keys/REPOSITORY@OWNER-FINGERPRINT.rsa.pub echo "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/alpine/VERSION/main" \ > /etc/apk/repositories apk update ``` Alternatively, for Wolfi distributions: ```shell Manual Configuration apk add curl # Remove existing Wolfi/Chainguard keys and repository list rm -f /etc/apk/keys/chainguard-* rm -f /etc/apk/keys/wolfi-signing.rsa.pub rm -f /etc/apk/repositories # Download the Cloudsmith RSA key curl -L -o /etc/apk/keys/REPOSITORY@OWNER-FINGERPRINT.rsa.pub \ 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/rsa.FINGERPRINT.key' # Configure the repository echo "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/alpine/os" \ > /etc/apk/repositories apk update -v --no-cache apk add -v --no-cache PACKAGE_NAME ``` **No automated setup script for Wolfi** The automated setup script (`bash.alpine.sh`) does not support Wolfi. Use the manual setup for this distribution. ### Installing a Package After setting up the repository, install a package with: ```shell sudo apk add PACKAGE_NAME=PACKAGE_VERSION --update-cache ``` ### Removing Setup If you no longer want to install packages from your Cloudsmith repository, you can remove it with: ```shell $EDITOR /etc/apk/repositories ``` Remove the cloudsmith repository line, save then execute: ```shell rm -f /etc/apk/keys/REPOSITORY@cloudsmith-FINGERPRINT.rsa.pub apk update ``` ## Upstream Proxying / Caching Supported The Alpine format supports upstream proxying and caching for both Alpine Linux and Wolfi repositories. When a package is requested, Cloudsmith checks for a local copy. If missing, it fetches the package from the upstream, caches it as a first-class local package, and serves it. The key difference between Alpine and Wolfi upstreams is in their URL structures — these paths only return packages for the given distribution: - **Alpine Linux** uses a versioned path structure: `/{version}/{branch}/{arch}/` (e.g., `/v3.23/main/x86_64/`) - **Wolfi** uses a flat structure with no version hierarchy: `/{arch}/` (e.g., `/x86_64/`) ### Supported Upstream Sources | Distribution | Upstream URL | Notes | |:-------------|:-----------------------------------------|:---------------------------| | Alpine Linux | `https://dl-cdn.alpinelinux.org/alpine/` | Official Alpine CDN mirror | | Wolfi | `https://packages.wolfi.dev/os/` | Official Wolfi OS packages | ### Configure an Upstream Please see our [Upstream Proxying](/repositories/upstreams#create-a-alpine-upstream) documentation for further instructions on form fields and configuration options. ## Key Signing Support RSA Index ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Cargo Registry Cloudsmith provides public & private registries for Cargo (Rust). For more information on Rust, please see: - [Rust](https://www.rust-lang.org/): The official website for Rust language - [Cargo](https://doc.rust-lang.org/cargo/index.html): The official documentation for Cargo - the Rust package manager - [Crates Public Registry](https://crates.io/): The official Rust community's crate registry **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | REGISTRY_NAME | A name for the Cargo registry | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | | PROJECT_NAME | The name of your Rust project | ## Upload a Package ### Upload via Cargo Publish A name and URL for the registry must be added to your `.cargo/config` file as follows: ### Public Repositories ```toml [registries] REGISTRY_NAME = { index = "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cargo/index.git" } ``` ### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```toml # Entitlement Token Auth [registries] REGISTRY_NAME = { index = "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cargo/index.git" } ``` ```toml # HTTP Basic Auth [registries] REGISTRY_NAME = { index = "https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cargo/index.git" } ``` If using HTTP basic authentication, you'll need to configure Git with credentials. Git's standard authentication mechanisms are used by Cargo and can be configured in the normal way. For example, you could use git's per-user credential store as follows: ```shell # HTTP Basic Auth (User & Pass) git config --global credential.helper store echo "https://USERNAME:PASSWORD@dl.cloudsmith.io" > ~/.git-credentials ``` ```shell # HTTP Basic Auth (API-Key) git config --global credential.helper store echo "https://USERNAME:API-KEY@dl.cloudsmith.io" > ~/.git-credentials ``` ```shell # HTTP Basic Auth (Token) git config --global credential.helper store echo "https://token:TOKEN@dl.cloudsmith.io" > ~/.git-credentials ``` For further details or other configuration options, see the [Official Git Documentation](https://git-scm.com/docs/gitcredentials). In order to authenticate for publishing via cargo, you can either enter your credentials using the command: ```shell cargo login --registry REGISTRY_NAME ``` Or, add your credentials to your `.cargo/credentials` file: ```toml [registries.REGISTRY_NAME] token = API-KEY ``` ### Publish To publish a crate, you can do so from your project directory using cargo publish as follows: ```shell cargo publish --registry REGISTRY_NAME ``` If you haven't specified credentials using one of the methods above, you'll be asked to provide them using `cargo login`. Instead of using the previous authentication methods, you can also set the following environment variables to define your index and token: - `CARGO_REGISTRIES__INDEX` - `CARGO_REGISTRIES__TOKEN` Remember to rename the variable with your registry name. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a cargo crate via the Cloudsmith CLI is: ```shell cloudsmith push cargo OWNER/REPOSITORY PACKAGE_NAME.crate ``` Example: ```shell cloudsmith push cargo org/repo your-package.crate ``` ### Upload via Cloudsmith Website Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download a Package ### Configure a new Upstream Cloudsmith supports [https://index.crates.io](https://index.crates.io) as an upstream. This allows you to proxy and cache external Rust crates that are not part of your repository. To enable the upstream, browse to your repository overview page and click in the Upstreams tab. Then, click on **+ Add Upstream Proxy** and then choose the **Cargo** Upstream. In the Upstream creation menu, define a name your Upstream and in the **Proxy URL** field insert `https://index.crates.io/`. Click on **+ Create Upstream Proxy** and the upstream should become immediately available. ### Registry Setup It is easy to add a Cloudsmith-based Cargo registry. **Cargo Sparse Registry** Cargo Sparse Registries are a new addition to Cargo as of v1.68.0, and are the recommended way to interact with your Cloudsmith Repositories - they offer significant performance advantages over the old Git-based registries, such as reducing the bandwidth used and improving dependency resolution times. First, the name and config for the registry must be added to your `.cargo/config.toml` or `.cargo/config` file as follows: ### Cargo >= v1.74 (HTTP Sparse Registry) ```toml API Key Auth [registries.OWNER-REPOSITORY] index = "sparse+https://cargo.cloudsmith.io/OWNER/REPOSITORY/" token = "Token API-KEY" credential-provider = "cargo:token" ``` ```toml Entitlement Token Auth [registries.OWNER-REPOSITORY] index = "sparse+https://cargo.cloudsmith.io/OWNER/REPOSITORY/" token = "Token TOKEN" credential-provider = "cargo:token" ``` **config.toml** Once you’ve configured your credentials in the config.toml file, there’s no need to run cargo login. Cargo will automatically authenticate using the token from the configuration. ### Cargo \< v1.74 (HTTP Sparse Registry) If you are using Cargo version `< 1.74`, the only way to authenticate with a private sparse registry is using Cloudsmith's URL-based authentication. ```toml Entitlement Token Auth [registries.OWNER-REPOSITORY] index = "sparse+https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cargo/" ``` ### Cargo < v1.68 (Legacy Git Registry) ```toml HTTP Basic Auth (API-Key) [registries] OWNER-REPOSITORY = { index = "https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cargo/index.git" } ``` ```toml URL Auth (Entitlement Token) [registries] OWNER-REPOSITORY = { index = "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cargo/index.git" } ``` ### Registry Authentication You must configure Git with the proper credentials to clone the registry when using HTTP basic authentication. Git's standard authentication mechanisms are used by Cargo and can be configured normally. For example, you could use git's per-user credential store like so: ```shell git config --global credential.helper store echo "https://USERNAME:API-KEY@dl.cloudsmith.io" > ~/.git-credentials ``` When using URL-based authentication, no further configuration is required, you're all set up and ready to go. ### Install a Package Once you have configured a registry using one of the methods described above, a crate can then depend on a crate from your registry by specifying the registry key and a value of the registry's name in that dependency's entry in `Cargo.toml`: ```toml [package] name = "PROJECT_NAME" version = "0.1.0" [dependencies] PACKAGE_NAME = { version = "PACKAGE_VERSION", registry = "REGISTRY_NAME" } ``` You can also install a crate directly by specifying the registry on the command line: ```shell cargo install PACKAGE_NAME --registry REGISTRY_NAME` ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Key Signing Support Not Supported ## Known Limitations The cargo search command is not supported. ## Troubleshooting **Q. It's not working...** Cargo's cache can become corrupted. If this happens, you can often fix the issue by manually deleting the cache directories. The folders to remove are: - `~/.cargo/registry` - `~/.cargo/git` If you previously set an override for `$CARGO_HOME`, the cache may be in a non-standard location. ## Still Need Help? Contact us [here](https://support.cloudsmith.com). We're always happy to help. # Chocolatey Repository Cloudsmith provides public & private repositories for Chocolatey Chocolatey is a package manager for Windows (like apt-get but for Windows). Chocolatey is built on top of NuGet and PowerShell (the automation language for Windows). You can package and install anything on Windows using Chocolatey - if it can be automated, Chocolatey and PowerShell can install, upgrade, and uninstall it. For more information on Chocolatey, please see: - [Chocolatey](https://chocolatey.org/): The official website for Chocolatey - [Chocolatey Documentation](https://docs.chocolatey.org/en-us/): The official docs for Chocolatey - [Create a Chocolatey Package](https://docs.chocolatey.org/en-us/create/create-packages) - [Install Chocolatey](https://docs.chocolatey.org/en-us/choco/setup) - [Build/Test/Push a Chocolatey Package](https://docs.chocolatey.org/en-us/create/create-packages#build-your-package) Chocolatey packages are known as nupkg files, which is a compiled NuSpec or a fancy zip file that knows about package metadata (including dependencies and versioning). These packages are an enhanced NuGet package, they have additional metadata that is specific to Chocolatey. Chocolatey is also compatible with vanilla NuGet packages. A Chocolatey package can contain embedded software and/or automation scripts. For more information on how Chocolatey works see: [How Does Chocolatey Work](https://docs.chocolatey.org/en-us/getting-started#how-does-chocolatey-work) **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Prerequisites [Prerequisites](https://docs.chocolatey.org/en-us/create/create-packages-quick-start#prerequisites) to creating your Choco package. Documentation on the Chocolatey CLI including a full list of commands can be found [here](https://docs.chocolatey.org/en-us/choco/). The Chocolatey Windows package manager uses the same infrastructure as NuGet. Therefore packages are based on the same principles. One of those is a package description (specification) in xml format, known as the [Nuspec file](https://docs.chocolatey.org/en-us/create/create-packages#nuspec). The following are the main elements of a Chocolatey package. Only the Nuspec is required (#1 below). 1. Nuspec 2. `chocolateyInstall.ps1` is triggered on install and upgrade. 3. Any application files to include (it is highly suggested that you are the author in this case or you have the right to distribute files). EXE files in the package/downloaded to package folder from `chocolateyInstall.ps1` will get a link to the command line. 4. `chocolateyUninstall.ps1`, for uninstalling your package. 5. `chocolateyBeforeModify.ps1` for upgrades. ## Upload a package The endpoint for the chocolatey packages API is: ``` https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 ``` ### Create a Chocolatey Package using the Chocolatey CLI Open a command line in the directory where the nuspec is and type ```shell choco pack ``` This generates a nupkg (`.nupkg`) file like `your-package-1.2.3.nupkg` that you can upload. ### Test your Package To test the package you just built, open a command line shell and navigate to the directory where the \*.nupkg file is located. Then type: ```shell choco install packageName -dv -s . ``` ### Upload a package via native Chocolatey tooling You can upload your package using the Chocolatey CLI. You can publish a nupkg file that you've generated from your project, using [NuGet](https://www.nuget.org/downloads). Then you can publish your package using: ```shell choco push PACKAGE_NAME-PACKAGE_VERSION.nupkg -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 -ApiKey API-KEY ``` **NuGet V3 feeds** As of Chocolatey V 2.0.0, Chocolatey can use NuGet V3 feeds as sources. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command-Line Interface](/developer-tools/cli) . The command to upload a Chocolatey package via the Cloudsmith CLI is: ```shell cloudsmith push nuget OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.nupkg ``` Example: ```shell cloudsmith push nuget your-account/your-repo your-package-1.2.3.nupkg ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package As a shortcut, you can set up the source (upstream) ahead of time, using `choco source`: ```shell Entitlement Token choco source add -n example-repo -s https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 -u "token" -p "TOKEN" ``` ```shell Username & Password choco source add -n example-repo -s https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 -u "USERNAME" -p "PASSWORD" ``` ```shell Username & API-Key choco sources add -n example-repo -s https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 -u "USERNAME" -p "API-KEY" ``` **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ### Install a Package Then to install a package, you use `choco install` ```shell choco install PACKAGE_NAME -Source example-repo ``` ## Nuget and Cloudsmith CLI See, [NuGet Feed](/formats/nuget-feed), for how to create chocolatey packages using Nuget or the Cloudsmith CLI. # CocoaPods Repository Cloudsmith provides public & private repositories for CocoaPods (Swift & Objective-C) CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. For more information on Cocoapods, please see: - [Cocoapods.org](https://cocoapods.org/): Official Cocoapods public repository and documentation **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command-Line Interface](/developer-tools/cli) The command to upload an (Objective-C or Swift) CocoaPods package via the Cloudsmith CLI is: ```shell cloudsmith push cocoapods OWNER/REPOSITORY PACKAGE_NAME.tar.gz ``` Example: ```shell cloudsmith push cocoapods org/repo your-package.tar.gz ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package Before you can install packages from your Cloudsmith repository you'll need to add your repository to your project's Podfile. This can be done using the `source` keyword in your project's Podfile. The `source` keyword is a URL to a git repository with the metadata index of all of your repository's private pods. To add the repository to your project's Podfile, add the following: ### Public Repositories ```shell source 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cocoapods/index.git' ``` ### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. **Warning** When using HTTP basic authentication, you'll need to configure Git with the proper credentials. Git's standard authentication mechanisms are used by CocoaPods and can be configured in the normal way. For example, you could use git's per-user credential store like so: `git config --global credential.helper store` `echo "https://USERNAME:PASSWORD@dl.cloudsmith.io" > ~/.git-credentials` ```shell Entitlement Token Auth source 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cocoapods/index.git' ``` ```shell HTTP Basic Auth (User & Pass) source 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cocoapods/index.git' ``` ```shell HTTP Basic Auth (API Key) source 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cocoapods/index.git' ``` ```shell HTTP Basic Auth (Token) source 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cocoapods/index.git' ``` ### Specifying Dependencies Add the latest version of a package to your Podfile using the syntax outlined by [CocoaPods](https://guides.cocoapods.org/using/the-podfile.html): #### Public Repositories ```shell source 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cocoapods/index.git' target 'MyApp' do pod 'PACKAGE_NAME', '~> PACKAGE_VERSION' end ``` #### Private Repositories ```shell Entitlement Token Auth source 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cocoapods/index.git' target 'MyApp' do pod 'PACKAGE_NAME', '~> PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth source 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cocoapods/index.git' target 'MyApp' do pod 'PACKAGE_NAME', '~> PACKAGE_VERSION' end ``` ### Installing a Package Once added, the cocoapods tool will download the dependency to the Pods directory in your project, and update the `Podfile.lock` after running following command: ```shell pod install ``` **Warning** When a dependency is added to the Podfile and retrieved via `pod install`, it gets the new dependency, along with any of its transitive dependencies. However, the cocoapods tool won’t change the versions of any already-acquired dependencies unless that’s necessary to get the new dependency. The cocoapods tool also stores a cache of the index on your machine, so in order to receive the most up to date version of the index it may be necessary to use the `--repo-update` flag to force a repository update before installing your project's pods. ### Configuration Example Here is a complete (but minimal) `Podfile` using entitlement token authentication: ```shell source 'https://dl.cloudsmith.io/abcedef1234567/org/repo/cocoapods/index.git' workspace 'cocoapods-install-example.xcworkspace' target 'cocoapods-install-example' do # Comment the next line if you don't want to use dynamic frameworks use_frameworks! # Pods for cocoapods-install-example pod 'cloudsmith-cocoapods-example', '~> 1.0.158207047536418' end ``` ## Upstream Proxying / Caching Not Supported ## Key Signing Support Not Supported ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Composer Repository Cloudsmith provides public & private repositories for Composer (PHP) Composer is a dependency management tool for PHP. For more information on Composer, please see: - [Composer](https://getcomposer.org/): The official website for Composer - [Composer Documentation](https://getcomposer.org/doc/): The official docs for Composer **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Composer package via the Cloudsmith CLI is: ```shell cloudsmith push composer OWNER/REPOSITORY PACKAGE_NAME.phar ``` Example: ```shell cloudsmith push composer org/repo your-package.phar ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package To enable the retrieval of Cloudsmith hosted packages via Composer, the first step is to add your repository to the repositories section of your `composer.json` file. How you add your repository to your `composer.json` file differs if the repository is private or public. ### Public Repositories Add the following JSON snippet to your `composer.json` file: ```json "repositories": [ { "type": "composer", "url": "https://composer.cloudsmith.io/OWNER/REPOSITORY/", "options": { } } ] ``` ### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. Add one of the following JSON snippets to your project `composer.json` file: ```json Entitlement Token Auth "repositories": [ { "type": "composer", "url": "https://composer.cloudsmith.io/TOKEN/OWNER/REPOSITORY/", "options": { } } ] ``` ```json HTTP Basic Auth "repositories": [ { "type": "composer", "url": "https://composer.cloudsmith.io/basic/OWNER/REPOSITORY/", "options": { } } ] ``` When using HTTP Basic Authentication you'll probably want to keep your credentials separately in your `auth.json` file instead of within the `composer.json` file. When you have your credentials ready, add one of the following JSON snippets to your `auth.json` file: ```json Username + Password { "http-basic": { "composer.cloudsmith.io": { "username": "USERNAME", "password": "PASSWORD" } } } ``` ```json Username + API-Key { "http-basic": { "composer.cloudsmith.io": { "username": "USERNAME", "password": "API-KEY" } } } ``` ```json Entitlement Token { "http-basic": { "composer.cloudsmith.io": { "username": "token", "password": "TOKEN" } } } ``` ### Specifying Dependencies After the repository is added to your `composer.json` file, you then specify the dependency in the **require** section of your `composer.json` file or using the `composer require` command: #### via composer.json file Add the following JSON snippet to your project `composer.json` file: ```json { "require": { "PACKAGE_NAME": "PACKAGE_VERSION" } } ``` #### via composer require Use the following command: ```shell composer require PACKAGE_NAME:PACKAGE_VERSION ``` ### Installing a Package To install the dependencies listed in your `composer.json` file use the command: ```shell composer install ``` Composer will then resolve all dependencies listed in your `composer.json` file and download them into the `vendor` directory in your project. ### Configuration Example The following is an example of a complete (but minimal) `composer.json` file with a public cloudsmith repository and a single dependency: ```json { "repositories": [ { "type": "composer", "url": "https://composer.cloudsmith.io/OWNER/REPOSITORY/", "options": {} } ], "require": { "cloudsmith/cloudsmith-composer-example": "1.0.157954077030330" } } ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Supported ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Conan Repository Cloudsmith provides public & private repositories for Conan (C & C++) Conan is a dependency and package manager for C and C++. It is designed to help manage the development and Continuous Integration of C and C++ projects. Cloudsmith supports both Conan v1 and v2. **Conan Compatibility** Conan v1 and v2 packages are not compatible with each other. If you have v1 packages uploaded in our platform these won't be visible when using the Conan v2 CLI and have to be re-uploaded with the proper recipe changes. Please refer to the [Conan v2 Migration Guide](https://docs.conan.io/1/conan_v2.html) on how to. For more information please see: - [Conan](https://conan.io/): The official Conan website - [Conan Documentation](https://docs.conan.io/en/latest/index.html): The official docs for Conan In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package ### Upload via Conan The endpoint for the native Conan API is: ``` https://conan.cloudsmith.io/OWNER/REPOSITORY/ ``` First, you need to set up the Repository as a Conan remote, with your Cloudsmith API-Key and Username for authentication: ```shell # Shell - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p API-KEY -r OWNER-REPOSITORY USERNAME ``` ```shell # Shell - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p API-KEY OWNER-REPOSITORY USERNAME ``` You can then upload natively using [Conan](https://docs.conan.io/en/latest/reference/commands/creator/export.html) with the command: ```shell # Shell - Conan v1 conan upload PACKAGE_NAME/PACKAGE_VERSION --all -r OWNER-REPOSITORY ``` ```shell # Shell - Conan v2 conan upload PACKAGE_NAME/PACKAGE_VERSION -r OWNER-REPOSITORY ``` An optional USER/CHANNEL can also be provided when uploading a package; this enables multiple versions of the same package to coexist using the user/channel to create a unique version of the package version: ```shell # Shell - Conan v1 conan upload PACKAGE_NAME/PACKAGE_VERSION@USER/CHANNEL --all -r OWNER-REPOSITORY ``` ```shell # Shell - Conan v2 conan upload PACKAGE_NAME/PACKAGE_VERSION@USER/CHANNEL -r OWNER-REPOSITORY ``` The user/channel will also be used as a tag on the package which will allow for filtering within the UI. ### Upload via the Cloudsmith CLI or Website **Conan v2** Uploading using the Cloudsmith CLI or Website is only supported for Conan v1 packages. For v2 packages please use the Conan CLI tooling. ### Create a Package To upload via the Cloudsmith API/CLI, you'll need to generate a package first. The following commands can be used to produce a package and collect the additional files required for upload (otherwise handled by `conan upload`). **Conan build native documentation** Please see the [Conan documentation](https://docs.conan.io/en/latest/reference/commands/creator/create.html) on creating packages for more information on building your own packages. You can build a package with Conan by creating a new example project using new, create, and export: ```shell # Shell - Conan v1 conan new PACKAGE_NAME/PACKAGE_VERSION -t conan create . conan install . conan build . conan package . conan export . ``` To create a package, files need to be extracted from the Conan home directory: ```shell export CLOUDSMITH_NAMESPACE=OWNER export CLOUDSMITH_REPOSITORY=REPOSITORY ``` use `_` when user/channel is treated ```shell path=/home/circleci/.conan/data/example-conan-package/0.0.1/_/_ ``` List files in build directory (using wildcard to skip over autogenerated id): ```shell info_file=$(ls -d $path/package/*/conaninfo.txt) metadata_file=( "$path/export/conanfile.py" ) manifest_file=( "$path/export/conanmanifest.txt" ) package_directory=( "$path/export" ) ``` Create an archive in the current directory for upload (metadata only): ```shell tar -czf conan_package.tgz --absolute-names "$package_directory" ``` **External network requests in conanfile.py** When extracting metadata from the `conanfile.py`, Cloudsmith restricts all network access, meaning that any attempt to reach external services from within your `conanfile.py` will fail, potentially leading package synchronization to fail. Cloudsmith recommends keeping your `conanfile.py` as simple as possible to ensure your upload does not encounter any issues. #### Upload via Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The package and additional files can then be uploaded via the Cloudsmith CLI. The following example shows an upload via the CLI.: ```shell cloudsmith push conan "$CLOUDSMITH_NAMESPACE/$CLOUDSMITH_REPOSITORY" conan_package.tgz \ --metadata-file "$metadata_file" \ --info-file "$info_file" \ --manifest-file "$manifest_file" ``` #### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package First, the remote repository must be created and then credentials can be added for a user as follows: ### Public Repositories ```shell # Shell - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p API-KEY -r OWNER-REPOSITORY USERNAME ``` ```shell # Shell - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p API-KEY OWNER-REPOSITORY USERNAME ``` ### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```shell # Entitlement Token Auth - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p TOKEN -r OWNER-REPOSITORY OWNER/REPOSITORY ``` ```shell # HTTP Basic Auth (User & Pass) - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p PASSWORD -r OWNER-REPOSITORY USERNAME ``` ```shell # HTTP Basic Auth (API Key) - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p API-KEY -r OWNER-REPOSITORY USERNAME ``` ```shell # HTTP Basic Auth (Token) - Conan v1 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan user -p TOKEN -r OWNER-REPOSITORY token ``` ```shell # Entitlement Token Auth - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p TOKEN OWNER-REPOSITORY OWNER/REPOSITORY ``` ```shell # HTTP Basic Auth (User & Pass) - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p PASSWORD OWNER-REPOSITORY USERNAME ``` ```shell # HTTP Basic Auth (API Key) - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p API-KEY OWNER-REPOSITORY USERNAME ``` ```shell # HTTP Basic Auth (Token) - Conan v2 conan remote add OWNER-REPOSITORY https://conan.cloudsmith.io/OWNER/REPOSITORY/ conan remote login -p TOKEN OWNER-REPOSITORY token ``` ### Install a Package Once set up, Conan can download the package by running: ```shell # Shell - Conan v1/v2 conan download PACKAGE_NAME/PACKAGE_VERSION@ -r OWNER-REPOSITORY ``` ## Conan Revisions **Support for Revisions** Package revisions uploads are only supported using Conan CLI. Uploading a v1 package with the Cloudsmith CLI or Website doesn't create a revision. Cloudsmith Conan repositories support [Package Revisions](https://docs.conan.io/1/versioning/revisions.html). By default all Conan v2 uploads use revisions. ### Enable revisions in Conan v1 When using the Conan v1 CLI you have to explicitly enable revisions by either: - Adding `revisions_enabled=1` in the `[general]` section of your conan.conf file (preferred) - Setting the `CONAN_REVISIONS_ENABLED=1` environment variable. After enabling revisions, the `conan upload` command will upload revisions instead. ## Upstream Proxying / Caching Not Supported ## Key Signing Support Not Supported by format ## Troubleshooting ## Q. Why can my conanfile not access external services? When extracting metadata from the `conanfile.py`, Cloudsmith restricts all network access, meaning that any attempt to reach external services from within your `conanfile.py` will fail, potentially leading to package synchronization to fail. We recommend keeping your `conanfile.py` as simple as possible to ensure your upload has no issues. ## Q. Why is my recipe failing to synchronize? Similar to the previous point, when extracting metadata from the `conanfile.py`, Cloudsmith runs your recipe in an isolated secure environment. Things like: - Reading or writing files to the local filesystem - Reaching out to external services (with the exception of Conan Central), like GitHub. - Utilization of third-party python libraries - Dynamically setting metadata values like version, license, description, url, author, topics from external sources to the recipe file. Will potentially lead to a synchronization failure. We recommend keeping your `conanfile.py` as simple as possible to ensure the upload has no issues. If you or your organization have a use-case for any of the previous points please [reach out to us](https://cloudsmith.com/company/contact-us). # Conda Repository Cloudsmith provides public & private repositories for Conda Conda is a cross-platform, language-agnostic binary package manager. It is the package manager used by Anaconda installations. For more information on Conda, please see: - [Conda Documentation](https://docs.conda.io/) - [Anaconda](https://www.anaconda.com/) **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package ### Creating Conda Packages To upload via the Cloudsmith API/CLI, you'll need to generate a package first. You can do this with [conda-build](https://docs.conda.io/projects/conda-build/en/latest/index.html): ``` conda build . ``` This generates a `.tar.bz2` file like `your-package-0.1.0.tar.bz2` that you can upload. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload Conda package via the Cloudsmith CLI is: ```shell cloudsmith push conda OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.tar.bz2 ``` Example: ```shell cloudsmith push conda my-org/my-repo your-package-0.1.0.tar.bz2 ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package ### Public Repositories Adding a new channel can be accomplished by using the `conda` CLI: ```shell conda config --add channels https://conda.cloudsmith.io/OWNER/REPOSITORY/ ``` Alternatively, the channel can be added directly to your `.condarc`: ```yaml channels: - https://conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults ``` Or if you prefer, directly in your `environment.yml` file: ```yaml name: env-name channels: - https://conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults dependencies: - python=3.7 - codecov ``` To confirm the channel exists, you can view the channels configured for Conda by using: ```shell conda config --show ``` ### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. Adding a new channel can be accomplished by using the `conda` CLI: HTTP Basic Auth (User & Pass) HTTP Basic Auth (API-Key) Entitlement Token Auth {`conda config --add channels https://USERNAME:PASSWORD@conda.cloudsmith.io/OWNER/REPOSITORY/`} {`conda config --add channels https://USERNAME:API-KEY@conda.cloudsmith.io/OWNER/REPOSITORY/`} {`conda config --add channels https://token:TOKEN@conda.cloudsmith.io/OWNER/REPOSITORY/`} Alternatively, the channel can be added directly to your `.condarc`: HTTP Basic Auth (User & Pass) HTTP Basic Auth (API-Key) Entitlement Token Auth {`channels: - https://USERNAME:PASSWORD@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults`} {`channels: - https://USERNAME:API-KEY@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults`} {`channels: - https://token:TOKEN@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults`} Or if you prefer, directly in your `environment.yml` file: HTTP Basic Auth (User & Pass) HTTP Basic Auth (API-Key) Entitlement Token Auth {`name: env-name channels: - https://USERNAME:PASSWORD@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults dependencies: - python=3.7 - codecov`} {`name: env-name channels: - https://USERNAME:API-KEY@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults dependencies: - python=3.7 - codecov`} {`name: env-name channels: - https://token:TOKEN@conda.cloudsmith.io/OWNER/REPOSITORY/ - defaults dependencies: - python=3.7 - codecov`} ### Install a Package Once configured, you can install packages from your channel using the `conda` CLI: ```shell conda install your-package=1.2.3 ``` Or you can search for packages via blended channels (multiple upstreams configured in your Cloudsmith repository) for any package: ```shell conda search your-package ``` ## Conda repodata patching In certain scenarios, Conda package metadata may need to be patched. Patching is often required when a package dependency is updated, and backwards compatibility is broken, as in the following example: - `package-a`, version `3.5.0` depends on `package-b` with a version constraint of `package-b >= 2.0.0`. - A new major version of `package-b` is released (`3.0.0`). This introduces breaking changes, making it incompatible with all previous versions of `package-a`. Repodata patching can be used to resolve the above scenario. For example, the dependency constraint can be amended (patched) to the more specific: `package-b >= 2.0.0, <3`. This approach does not alter the original package artifacts, and means no requirement is placed on package maintainers to re-release corrected versions of any existing (broken) packages. ### Applying repodata patching Patching is accomplished by applying patching instructions to a channel's repodata file (`repodata.json`). This file format is used by Conda channels to provide an index of all packages and their respective metadata within a specific subdirectory (subdir) of the channel. When a request is made for the relevant repodata.json file, the patch instructions are merged with the repodata, and the patched (modified) repodata is returned. An overview of this process is available [here](https://bioconda.github.io/developer/repodata_patching.html). Cloudsmith provides a dedicated [cloudsmith-conda-repodata-patches](https://github.com/cloudsmith-io/cloudsmith-conda-repodata-patches) repository to facilitate the easy creation of patching instructions. This repository provides relevant scripts and instructions on how to add, generate and submit your patch instructions to Cloudsmith. **Note** Please note that when repodata patching is applied, either through patch instructions uploaded to the repository or from the upstream, the Cloudsmith repository version takes precedence. Patch instructions are applied on a per-package basis. ## Upstream Proxying / Caching Configurable ProxyingCaching --- Please see [Upstream Proxying](/repositories/upstreams) for more details. Upstreams provide a way to _blend_ multiple Conda repositories into a single repository. Cloudsmith supports proxying and caching from upstream repositories for any available channels. This means you can connect not just the canonical upstreams, but also other popular channels. For example, some of the largest upstream channels include: - [Anaconda](https://repo.anaconda.com/pkgs/main) - [Conda-Forge](https://conda.anaconda.org/conda-forge) - [Bioconda](https://conda.anaconda.org/bioconda) # CRAN Repository Cloudsmith provides public & private repositories for R packages R is a free software environment for statistical computing and graphics. CRAN or Comprehensive R Archive Network is the repository for R packages. For more information on R, please see: - [R](https://www.r-project.org): The official website for R - [CRAN](https://cran.r-project.org/mirrors.html): The official list of mirrors public repositories - [CRAN Package Repository](https://cran.r-project.org/web/packages/index.html): The official package repository **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package To upload, you will need to generate a package first. Hadley Wickham's [R Packages](http://r-pkgs.had.co.nz/) book provides the best overview of what is required to do so. We highly recommend following the best practices outlined in the book (it's available free online). Cloudsmith CRAN repositories support uploading both source and binary packages. The package type is determined via the package file extension: - `.tar.gz` for source packages. - `.zip`for Windows binary packages. - `.tgz`for macOS binary packages. CRAN binaries are built for a specific R version, and if it's a macOS binary, an architecture too. These can be easily specified when uploading via the UI, API, or CLI. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a R/CRAN package via the Cloudsmith CLI is: ```shell cloudsmith push cran OWNER/REPOSITORY PACKAGE_NAME_PACKAGE_VERSION.tar.gz ``` For a source package: ```shell cloudsmith push cran org/repo your-package_0.1.0.tar.gz ``` For a Windows binary package: ```shell cloudsmith push cran org/repo your-package_0.1.0.zip --r-version 4.3 ``` For a macOS binary package: ```shell cloudsmith push cran org/repo your-package_0.1.0.tgz --r-version 4.3 --architecture arm64 ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Upload via API Please see [Create a new CRAN package](/api/packages/upload/cran) for details of how to upload via the API. ## Download / Install a Package To install a package, use the `install.packages` method directly in your R session. Including `type = "binary"` within the method will instruct CRAN to install the binary. With this, R will not check that a source package is available and of a higher version. For binary installations, the type must be set if no matching source package exists within your repository: ### Public Repositories ```r install.packages( "PACKAGE_NAME", repos = c(cloudsmith = "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cran/") ) ``` ### Private Repositories **Private repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```r Entitlement Token Auth install.packages( "PACKAGE_NAME", repos = c(cloudsmith = "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cran/") ) ``` ```r HTTP Basic Auth (User & Pass) install.packages( "PACKAGE_NAME", repos = c(cloudsmith = "https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/") ) ``` ```r HTTP Basic Auth (API-Key) install.packages( "PACKAGE-NAME", repos = c(cloudsmith = "https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/") ) ``` ```r HTTP Basic Auth (Token) install.packages( "PACKAGE-NAME", repos = c(cloudsmith = "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/") ) ``` For most use cases, users will probably want to persist their repository settings and not specify them every time. To set the repository and avoid having to specify this during every package installation, create the R startup command file `.Rprofile` in your home directory and add the following R code to it: ### Public Repositories ```r print("Configuring CRAN repositories") r = getOption("repos") r["cloudsmith"] = "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cran/" r["CRAN"] = "https://cloud.r-project.org" options(repos = r) rm(r) ``` ### Private Repositories ```r Entitlement Token Auth print("Configuring CRAN repositories") r = getOption("repos") r["cloudsmith"] = "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cran/" r["CRAN"] = "https://cloud.r-project.org" options(repos = r) rm(r) ``` ```r HTTP Basic Auth (User & Pass) print("Configuring CRAN repositories") r = getOption("repos") r["cloudsmith"] = "https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/" r["CRAN"] = "https://cloud.r-project.org" options(repos = r) rm(r) ``` ```r HTTP Basic Auth (API-Key) print("Configuring CRAN repositories") r = getOption("repos") r["cloudsmith"] = "https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/" r["CRAN"] = "https://cloud.r-project.org" options(repos = r) rm(r) ``` ```r HTTP Basic Auth (Token) print("Configuring CRAN repositories") r = getOption("repos") r["cloudsmith"] = "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cran/" r["CRAN"] = "https://cloud.r-project.org" options(repos = r) rm(r) ``` ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream CRAN repositories that you wish to use for R packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Dart Repository Cloudsmith provides public & private repositories for Dart packages **Dart SDK Versions** From February 28th 2025, Cloudsmith will only support Dart SDK version `2.15.1` or greater. After this date support for SDK versions `<=2.14`, and associated workflows, will no longer be supported. Dart is a client-optimized programming language developed by Google. For more information on Dart, please see: - [Dart.dev](https://dart.dev/): The official website for Dart - [Pub.dev](https://pub.dev/): The official public repository for Dart packages **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | | DEPENDENCY_NAME | A name for a dependency in a pubspec file | ## Upload a Package ## Publish via Dart **Dart SDK Version** Please note that Dart SDK >= 2.15 is required to use Dart Repositories. The endpoint for the native Dart API is: ``` https://dart.cloudsmith.io/OWNER/REPOSITORY/ ``` Your project `pubspec.yaml` defines the location that artifacts should be published to. Add the following to the root of this file: ```yaml publish_to: https://dart.cloudsmith.io/OWNER/REPOSITORY/ ``` In order to authenticate for native publishing, you'll need run `dart pub token`, as follows: ```shell echo 'API-KEY' | dart pub token add https://dart.cloudsmith.io/OWNER/REPOSITORY/ Enter secret token: Requests to "https://dart.cloudsmith.io/OWNER/REPOSITORY/" will now be authenticated using the secret token. ``` You can then publish from your project directory using `dart pub publish`: ``` dart pub publish Uploading... Upload complete. Package will available following synchronization. ``` **Limitation** When publishing via the native API package sizes are limited to 200.0 MB per file. If this is an issue, please use the Cloudsmith CLI or API (which support up to 5GB for single-part uploads and beyond for multi-part), or contact us if that's not an option. ### Upload via the Cloudsmith CLI **Building Dart packages** Please see Dart's [documentation on creating packages](https://dart.dev/guides/libraries/create-library-packages) for more information on building your own packages. #### Creating Dart Packages To upload via the Cloudsmith API/CLI, you'll need to generate a package first. You can build a package with standard command-line tooling like `tar`. To illustrate the process we'll use [cli_util](https://github.com/dart-lang/cli_util) as an example: First, check out the version of cli_util we want to pack (**v0.1.3** for example purposes): ```shell git clone https://github.com/dart-lang/cli_util@v0.1.3 cd cli_util ``` ```shell tar --exclude='.dart_tool' -czvf cli_util_0.1.3.tar.gz ./* ``` For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Dart package via the Cloudsmith CLI is: ```shell cloudsmith push dart OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.tgz ``` Example: ```shell cloudsmith push dart org/repo your-package-1.0.0.tgz ``` ### Upload via Cloudsmith Website Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. *** ## Download / Install a Package **Public Repositories** No further setup is required with public repositories, you can add a dependency from this repository to your package `pubspec.yaml` automatically with `dart pub`. See [Installing a Package](/formats/dart-repository) **Private Repositories** In order to authenticate, you need to run `dart pub token`, as follows: ```shell Entitlement Token Auth echo 'TOKEN' | dart pub token add https://dart.cloudsmith.io/OWNER/REPOSITORY/ Enter secret token: Requests to "https://dart.cloudsmith.io/OWNER/REPOSITORY/" will now be authenticated using the secret token. ``` ```yaml HTTP Basic Auth (User & Pass) echo 'PASSWORD' | dart pub token add https://dart.cloudsmith.io/OWNER/REPOSITORY/ Enter secret token: Requests to "https://dart.cloudsmith.io/OWNER/REPOSITORY/" will now be authenticated using the secret token. ``` ```yaml HTTP Basic Auth (API-Key) echo 'API-KEY' | dart pub token add https://dart.cloudsmith.io/OWNER/REPOSITORY/ Enter secret token: Requests to "https://dart.cloudsmith.io/OWNER/REPOSITORY/" will now be authenticated using the secret token. ``` ```yaml HTTP Basic Auth (Token) echo 'TOKEN' | dart pub token add https://dart.cloudsmith.io/OWNER/REPOSITORY/ Enter secret token: Requests to "https://dart.cloudsmith.io/OWNER/REPOSITORY/" will now be authenticated using the secret token. ``` ## Installing a package `dart pub` is capable of adding a dependency from this repository to your package `pubspec.yaml` automatically: **Public Or Private Repositories** ```shell dart pub add PACKAGE_NAME:PACKAGE_VERSION --hosted-url https://dart.cloudsmith.io/OWNER/REPOSITORY/ Resolving dependencies... + your-package 1.2.3 Downloading your-package 1.2.3... Changed 1 dependency! ``` Please also see the [pubspec docs](https://dart.dev/tools/pub/pubspec) for further examples of pubspec files. ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Dart repositories that you wish to use for packages that are not available in your Cloudsmith repository. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support Not Supported by Format ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Debian Repository Cloudsmith provides public & private repositories for Debian packages For more information on Debian, please see: - [Debian](https://www.debian.org/): The official website for Debian - [Debian Packages Documentation](https://www.debian.org/distrib/packages): The official docs for Debian Packages **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :---------------- | :------------------------------------------------------------------------------------------------------------------------------------ | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | DISTRO | Your distribution (i.e debian, ubuntu). You can also use "any-distro" if your package is compatible with more than more distribution | | VERSION | Your version name (i.e xenial, buster). You can also use "any-version" if your package is compatible with more than more version | | FINGERPRINT | The 8 Byte fingerprint of the Public GPG key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | | PACKAGE_ARCH | The architecture of your package (e.g. `x86_64`) | | PACKAGE_COMPONENT | The release component (channel) of your package (e.g. `unstable`) | | SOURCES_FILENAME | The filename of the Debian sources, for a Debian source (`.dsc`) package. | ## Upload a Package You can upload either source packages (`.dsc`) or binary packages (`.deb`) To upload a binary package, you will need to generate your package first. We highly recommend [fpm](https://fpm.readthedocs.io/en/latest/) for simplifying this. With fpm, you can build a package from a directory that represents the layout on the target system installation using: ```shell fpm -f -s dir -t deb -n PACKAGE_NAME -v PACKAGE_VERSION . ``` ### Upload via the Cloudsmith CLI For full details of how to install and set up the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). #### Binary Packages The command to upload a Debian binary package via the Cloudsmith CLI is: ```shell cloudsmith push deb OWNER/REPOSITORY/DISTRO/VERSION PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH.deb ``` Example: Upload a binary package for Ubuntu Xenial ```shell cloudsmith push deb org/repo/ubuntu/xenial libxml2-2.9.4-2.x86_64.deb ``` Example: Upload a binary package for any version of Ubuntu ```shell cloudsmith push deb org/repo/ubuntu/any-version libxml2-2.9.4-2.x86_64.deb ``` Example: Upload a binary package for any version of Ubuntu, within the `unstable` release component (channel) ```shell cloudsmith push deb org/repo/ubuntu/any-version libxml2-2.9.4-2.x86_64.deb --component unstable ``` Example: Upload a binary package for any version of any distribution ```shell cloudsmith push deb org/repo/any-distro/any-version libxml2-2.9.4-2.x86_64.deb ``` #### Source Packages A source package is one that provides you with all of the necessary files to compile or otherwise, build the desired piece of software. In other words, you can get the sources used to create a binary package, and compile them yourself; perhaps to customise the package, or build it for a different operating environment. When uploading a DSC package, the following files are involved: - The DSC package file is a metadata file that describes the other sources/changes files. (Required) - The sources file contains the actual source files used to build the software. (Required) - The changes file usually contains the patch applied, or to be applied, to the sources file if different from the original. (Optional) The command to upload a Debian **source** package via the Cloudsmith CLI is: ```shell cloudsmith push deb OWNER/REPOSITORY/DISTRO/VERSION PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH.dsc --sources-file SOURCES_FILENAME ``` Example: Upload a source package for Ubuntu Xenial ```shell cloudsmith push deb org/repo/ubuntu/xenial mypackage-1.1.2-2.x86_64.src --sources-file mypackage-1.1.2-2.x86_64.tar.xz ``` Example: Upload a source package for any version of Ubuntu ```shell cloudsmith push deb org/repo/ubuntu/any-version mypackage-1.1.2-2.x86_64.src --sources-file mypackage-1.1.2-2.x86_64.tar.xz ``` Example: Upload a source package for any version of any distribution ```shell cloudsmith push deb org/repo/any-distro/any-version mypackage-1.1.2-2.x86_64.src --sources-file mypackage-1.1.2-2.x86_64.tar.xz ``` ##### Automatically detecting changes and sources archives for upload The script below will loop over all `.dsc` files in `$WORKING_DIR` and parse it to identify the source and changes files to be passed to the `--source-file` and `--changes-file` CLI options. This relies on the [`dcmd`](https://manpages.debian.org/buster/devscripts/dcmd.1.en.html) tool being installed and the `.dsc` file and the associated changes and source archives being present in `$WORKING_DIR`. ```shell for DESCRIPTION_FILE in ${WORKING_DIR}/*.dsc; do cloudsmith push deb ${REPO}/any-distro/any-version "$DESCRIPTION_FILE" --source-file=$(dcmd --orig "$DESCRIPTION_FILE") --changes-file=$(dcmd --debtar "$DESCRIPTION_FILE") done ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload#upload-via-website-ui) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package **Recommended APT Version** We recommend using `apt` v1.2.0 or newer so that you are able to fetch repository indexes using **acquire-by-hash**. This will avoid race conditions when updating your repository whilst simultaneously deploying from it. ### Setup You have a choice of 3 methods to setup your Cloudsmith repository: - Automatic configuration (**recommended**) - Force a specific distribution/release (if your system is compatible but not identical) - Manual configuration #### Public Repositories To install Debian packages from a public Cloudsmith repository, you can quickly set up the repository automatically: ```shell curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo bash ``` If you need to force a specific distribution: ```shell curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` Or, you can manually configure the repository ```shell apt-get install -y debian-keyring # debian only apt-get install -y debian-archive-keyring # debian only apt-get install -y apt-transport-https # For Debian Stretch, Ubuntu 16.04 and later keyring_location=/usr/share/keyrings/OWNER-REPOSITORY-archive-keyring.gpg # For Debian Jessie, Ubuntu 15.10 and earlier keyring_location=/etc/apt/trusted.gpg.d/OWNER-REPOSITORY.gpg curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/gpg.FINGERPRINT.key' | gpg --dearmor > ${keyring_location} curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/config.deb.txt?distro=DISTRO&codename=VERSION' > /etc/apt/sources.list.d/OWNER-REPOSITORY.list ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To install Debian packages from a private Cloudsmith repository, you can quickly set up the repository automatically: ```shell Entitlement Token Auth curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (User & Pass) sudo apk add --no-cache bash curl -u "USERNAME:PASSWORD" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (API-Key) sudo apk add --no-cache bash curl -u "USERNAME:API-KEY" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (Token) sudo apk add --no-cache bash curl -u "token:TOKEN" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo bash ``` If you need to force a specific distribution: ```shell Entitlement Token Auth curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (User & Pass) curl -u "USERNAME:PASSWORD" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (API-Key) curl -u "USERNAME:API-KEY" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (Token) curl -u "token:TOKEN" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.deb.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` Or, you can manually configure the repository: ```shell Entitlement Token Auth apt-get install -y debian-keyring # debian only apt-get install -y debian-archive-keyring # debian only apt-get install -y apt-transport-https curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' | apt-key add - curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/config.deb.txt?distro=DISTRO&codename=VERSION'> /etc/apt/sources.list.d/OWNER-REPOSITORY.list apt-get update ``` ```shell HTTP Basic Auth (User & Pass) apt-get install -y debian-keyring # debian only apt-get install -y debian-archive-keyring # debian only apt-get install -y apt-transport-https curl -u "USERNAME:PASSWORD" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' | apt-key add - curl -u "USERNAME:PASSWORD" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.deb.txt?distro=DISTRO&codename=VERSION'> /etc/apt/sources.list.d/OWNER-REPOSITORY.list apt-get update ``` ```shell HTTP Basic Auth (API-Key) apt-get install -y debian-keyring # debian only apt-get install -y debian-archive-keyring # debian only apt-get install -y apt-transport-https curl -u "USERNAME:API-KEY" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' | apt-key add - curl -u "USERNAME:API-KEY" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.deb.txt?distro=DISTRO&codename=VERSION'> /etc/apt/sources.list.d/OWNER-REPOSITORY.list apt-get update ``` ```shell HTTP Basic Auth (Token) apt-get install -y debian-keyring # debian only apt-get install -y debian-archive-keyring # debian only apt-get install -y apt-transport-https curl -u "token:TOKEN" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' | apt-key add - curl -u "token:TOKEN" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.deb.txt?distro=DISTRO&codename=VERSION'> /etc/apt/sources.list.d/OWNER-REPOSITORY.list apt-get update ``` ### Installing a package After you have set up the repository, to install a package you do: ```shell sudo apt-get install PACKAGE_NAME=PACKAGE_VERSION ``` ### Removing Setup If you no longer want to install packages from your Cloudsmith repository, you can remove it with: ```shell rm /etc/apt/sources.list.d/OWNER-REPOSITORY.list apt-get clean rm -rf /var/lib/apt/lists/* apt-get update ``` ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Debian repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Index ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Docker Registry Cloudsmith provides public & private registries for Docker images. **OCI Support** Cloudsmith provides public & private registries for OCI artifacts, to learn more about it visit our [OCI Repository ](/formats/oci-repository) documentation. **Docker DHI Support** Cloudsmith provides support for Docker Hardened Images via upstreams. To learn how to configure an upstream for this registry, visit our [Docker Hardened Images](/integrations/integrating-with-docker-hardened-images) documentation. [Docker](https://www.docker.com/) is an ecosystem used to run packaged software known as containers. A container image is bundled with all of the software and configured to run as an independent process (or collection of processes), and an executing container is isolated from other containers and processes. Docker provides the concept of the registry as an artifact data service for storing and distributing image containers. The official one of which is [Docker Hub](https://hub.docker.com), which is owned and operated by Docker itself. Cloudsmith provides a fully fledged Docker registry that is fully compatible with current and future versions of the Docker engine and container image formats. With Cloudsmith, you can push, pull, inspect, and manage container images privately and publicly. This is provided with the standard functionality and features offered in the Cloudsmith platform, such as collaboration, advanced permissions, white-labeled distribution, multi-tenancy with other packaging formats, etc. For more information on Docker, please see: - [Docker](https://www.docker.com): The official website for Docker (the company and product). - [Docker Hub](https://hub.docker.com): The official public registry for Docker repositories. - [Docker Article](https://en.wikipedia.org/wiki/Docker_(software)): The Wikipedia article on Docker. **Storage Deduplication** Cloudsmith's Docker repositories benefit from native Docker [image layer storage deduplication](https://docs.docker.com/get-started/docker-concepts/building-images/understanding-image-layers/) technology. This means that if you upload several images sharing some of their composing layers, those layers will be reused. This reduce the amount of artifact data and package delivery required to distribute the images across all repositories within a workspace. ## Differences from Docker For clarity, it's important to note some of the differences between a registry such as Docker Hub, and a Cloudsmith Docker registry. These are both naming and functional in nature. ### Naming Differences Docker defines the following names (this is not the official wording): - **Layer**: A blob (big object of bytes) containing software and configuration. - **Image**: A collection of Docker layers plus metadata that represent an application. - **Container**: A running instance of a Docker image in-memory. - **Repository**: A collection of Docker images, separated by hashref and version tags. - **Registry**: A collection of all Docker repositories, separated by namespaces. For comparison purposes, where terms differ from Cloudsmith: - **Package**: A specific identifiable and versionable artifact. - **Repository**: A collection of versionable artifacts, with multiple allowed per account. Therefore, based on the above, the following terms are equivalent: | Docker Term | Cloudsmith Term | | :---------- | :-------------- | | Image | Package | | Registry | Repository | For consistency, the Docker terms will be used within all of the Docker-related documentation but please be aware of the differences if looking at documentation elsewhere. ### Functionality Differences We have attempted to achieve absolute compatibility at the wire (API) level with Docker Hub and the [Docker registration specification (2.0)](https://docs.docker.com/registry/spec/api/). The functionality differs in how the registry itself is treated. Referring to the naming differences section above, every Cloudsmith account can have multiple Cloudsmith repositories. Each Cloudsmith repository is an individual Docker registry in its own right and will require a different login in order to push/pull images for it. As such, it is also not possible to have a global registry of all images on Cloudsmith. This may change in the future as we introduce different forms of Cloudsmith repositories and groupings. **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :--------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organisation name (namespace) | | REGISTRY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | IMAGE_NAME | The name of your Docker image | | TAG | A tag for your Docker image | *** ## Upload an Image **Custom Domains** If you have added a [Custom Domain](/workspaces/custom-domains) for Docker, you must use it to authenticate and push. Please replace `docker.cloudsmith.io` in the following instructions with the Docker custom domain you have created. ### Publish via Docker The endpoint for the native Docker API is: ``` https://docker.cloudsmith.io/v2/OWNER/REGISTRY/ ``` Or if you're referring to it from `docker` commands: ``` docker.cloudsmith.io/OWNER/REGISTRY ``` In order to authenticate for native publishing, you'll need use `docker login`: ```shell docker login docker.cloudsmith.io ``` You will be prompted for your Username and Password. Enter your Cloudsmith username and your Cloudsmith API Key. To publish an image to a Cloudsmith-based Docker registry, you first need to tag your image: ```shell docker tag IMAGE_NAME:TAG docker.cloudsmith.io/OWNER/REGISTRY/IMAGE_NAME:TAG ``` **About 'latest' tags** Docker images are not automatically tagged as 'latest' based on upload date / time. In order to ensure that you have an image tagged as 'latest' you need to explicitly tag the image. For example: ```shell docker tag your-image:latest docker.cloudsmith.io/org/repo/your-image:latest ``` You can then publish the tagged image using `docker push`: ```shell docker push docker.cloudsmith.io/OWNER/REGISTRY/IMAGE_NAME:TAG ``` ### Upload via the Cloudsmith CLI or Website To upload via the Cloudsmith CLI or Website. you need to export your Docker image first. You can do this with: ```shell docker save -o IMAGE_NAME.docker IMAGE_NAME:TAG ``` This exports the full contents of the image, including all metadata and layers. #### Upload via Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Docker image via the Cloudsmith CLI is: ```shell cloudsmith push docker OWNER/REGISTRY IMAGE_NAME.docker ``` Example: ```shell cloudsmith push docker org/repo your-image.docker ``` #### Upload via Cloudsmith Website Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. *** ## Download / Pull an Image ### Setup #### Public Registries For public registries, no further setup is needed as authentication is not required. #### Private Registries **Private Repositories** Private Registries require authentication. You can choose between three types of authentication, Entitlement Token Authentication, Service Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, Service Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. You need to authenticate via `docker login` to pull images: ```shell # Entitlement Token Auth docker login docker.cloudsmith.io Username: OWNER/REGISTRY Password: TOKEN Login Succeeded ``` ```shell # Service Token Auth docker login docker.cloudsmith.io Username: SERVICE_NAME Password: TOKEN Login Succeeded ``` ```shell # HTTP Basic Auth (User & Pass) docker login docker.cloudsmith.io Username: OWNER/REGISTRY Password: PASSWORD Login Succeeded ``` ```shell # HTTP Basic Auth (API-Key) docker login docker.cloudsmith.io Username: OWNER/REGISTRY Password: API-KEY Login Succeeded ``` ```shell # HTTP Basic Auth (Token) docker login docker.cloudsmith.io Username: OWNER/REGISTRY Password: TOKEN Login Succeeded ``` ### Pull an Image Pulling (downloading) an image from the Cloudsmith Docker registry can be done using the standard `docker pull` command: ```shell docker pull docker.cloudsmith.io/OWNER/REGISTRY/IMAGE_NAME:TAG ``` To refer to this image after pulling in a Dockerfile, specify the following: ``` FROM docker.cloudsmith.io/OWNER/REGISTRY/IMAGE_NAME:TAG ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Current Limitations The Cloudsmith Docker registry implementation has the following limitations: - Remote layer mounting (for non-distributable layers, such as Windows) is not supported. - Digests for images will not match those pushed to Docker Hub (but layers will match). - Digests for offline image uploads (via `docker save`) will not match those from `docker push`. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Docker registries you wish to use for images. In addition, you can also choose to cache any requested images for future use. **Warning** If you have a Docker upstream configured, please be aware of the following behaviours observed when making requests against the Docker API v2 endpoints: - [Tags/List](https://distribution.github.io/distribution/spec/api/#listing-image-tags): All tags from the upstream and not just local Cloudsmith tags are returned. - [Manifests](https://distribution.github.io/distribution/spec/api/#pulling-an-image-manifest): This will trigger a cache of the referenced image to your Cloudsmith repository. Iterating through a list of tags to fetch manifests will trigger automatic caching for every image. This can result in hundreds or thousands of cached images, significantly increasing artifact storage consumption and associated costs. ## Key Signing Support RSA Index ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Generic Repository Cloudsmith provides public & private repositories for Generic files. Cloudsmith supports Generic files, a flexible format for managing any file or binary that relies on specific file paths rather than semantic versioning. **HTML-based upstreams** Generic upstreams work with simple HTML-based sources. The upstream must consist of simple, unstyled HTML pages with no JavaScript. Each page should contain only links to package files or subdirectories. Examples include Gradle distributions, Apache releases, and Node distributions. **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo pre-configured). In the following examples: | Identifier | Description | | :------------- | :------------------------------------------------------------------------------------------------------------------ | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | API-KEY | Your Cloudsmith API Key | | UPSTREAM_PREFIX | Upstream prefix (Provided without a trailing slash) used as a path for files cached from upstreams | **Substitute placeholder values** Where you see uppercase values such as `OWNER`, `REPOSITORY`, `TOKEN`, `API-KEY`, or `UPSTREAM_PREFIX`, substitute these with your own Cloudsmith namespace, repository, token, API key, and upstream prefix as appropriate. ## Why Generic Format? The Generic format is designed for artifacts that rely on specific file paths rather than semantic versions. Key benefits include: - **Path-Based Identification**: Uses the filepath as the unique identifier, enabling exact replication of upstream directory structures - **Upstream Proxying and Caching**: Proxy and cache files from simple HTML-based upstream sources - **Supply Chain Management**: Bring custom scripts, installers, and binaries into your managed supply chain - **Consistent URLs**: Internal URLs remain consistent with upstream sources ### Generic vs Raw Format The Generic format is optimized for path-based file distribution and upstream proxying. It relies strictly on the filepath as the unique identifier and does not support: - Package naming (separate from filepath) - Package versioning (semantic versions) - End User License Agreements (EULA) If you need package naming, semantic versioning, or EULA support, use the [Raw format](/formats/raw-repository) instead. The Raw format provides traditional package management features including distinct package names, version tracking, and license acceptance workflows. **File-Based Storage** Generic packages are treated as files and binaries. Unlike native package formats (Python, npm, Maven), Generic packages do not have metadata, version information, or vulnerability scanning capabilities. They are stored and served as-is without package-specific processing. ## Upload a File ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli) The command to upload via the Cloudsmith CLI is: ```shell cloudsmith push generic OWNER/REPOSITORY ~/downloads/file.tar.gz --filepath path/to/file.tar.gz ``` Example: ```shell cloudsmith push generic OWNER/REPOSITORY ~/downloads/file.tar.gz --filepath distributions/latest/file.tar.gz ``` **Filepath parameter** The `--filepath` parameter specifies where the file will be stored in the repository structure. This path becomes the unique identifier for the package and determines its download URL. ### Upload via Cloudsmith web app Please see [upload a package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download a Package ### Download via Cloudsmith web app #### Public Repositories When logged into Cloudsmith via a Web Browser, use the Download button located on the package details page to download a generic package. #### Private Repositories To download from a private repository via the Cloudsmith web app, use the Use Package button to download a generic package using the default Entitlement Token for the repository. Use the dropdown arrow within the pop-up window to select a different authentication method. ### Download via Command Line To download a Generic package, fetch uploaded files using a well-crafted URL. The package URL format is based on the filepath used when uploading. #### Public Repositories Example `curl` commands to download a generic package from a public repository: ```shell curl -sLf -O "https://generic.cloudsmith.io/OWNER/REPOSITORY/UPSTREAM_PREFIX/path/to/file.tar.gz" ``` #### Private Repositories **Private Repositories** Private repositories require authentication. Use an Entitlement Token or API Key depending on the authentication method. The following authentication methods do not work when browsing the HTML index in a web browser. For browser access, use HTTP Basic Authentication when prompted—enter `token` as the username and your entitlement token as the password. **HTTP Basic Authentication (generic.cloudsmith.io):** Use an entitlement token with HTTP Basic Auth to download files: ```shell curl -sLf -u "token:TOKEN" -O "https://generic.cloudsmith.io/OWNER/REPOSITORY/UPSTREAM_PREFIX/path/to/file.tar.gz" ``` Alternatively, embed HTTP Basic Auth credentials in the URL as user information: ```shell curl -sLf -O "https://token:TOKEN@generic.cloudsmith.io/OWNER/REPOSITORY/UPSTREAM_PREFIX/path/to/file.tar.gz" ``` **API Key Authentication (generic.cloudsmith.io):** Use an API Key with the `X-Api-Key` header to download files. An Entitlement Token cannot be used with this method. ```shell curl -sLf -H "X-Api-Key: API-KEY" -O "https://generic.cloudsmith.io/OWNER/REPOSITORY/UPSTREAM_PREFIX/path/to/file.tar.gz" ``` **Authentication when using the Download domain (dl.cloudsmith.io):** The `dl.cloudsmith.io` domain most commonly uses path-based authentication with entitlement tokens embedded in the URL path. There are some exceptions: HTTP Basic Auth with credentials in the userinfo portion of the URL (e.g., `token:TOKEN@dl.cloudsmith.io`) is only supported for the `/basic/...` download route on this domain and is not supported for the default `/TOKEN/...` path-based URLs. ```shell curl -sLf -O "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/generic/download/UPSTREAM_PREFIX/path/to/file.gz" ``` Include `generic/download/` in the URL path before the upstream prefix to ensure requests are routed correctly. ```shell curl -sLf -O "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/generic/download/UPSTREAM_PREFIX/path/to/file.tar.gz" ``` **Supported Authentication by Domain:** A URL follows the structure: `scheme://userinfo@host/path` - **Userinfo** (`token:TOKEN@`): Credentials placed before the host. - **Path** (`/TOKEN/...`): Token embedded after the domain in the URL path. This only works on `dl.cloudsmith.io`. | Domain | Authentication Method | Token Location | | :----- | :-------------------- | :------------- | | `generic.cloudsmith.io` | Basic Auth | Userinfo | | `generic.cloudsmith.io` | API Key | HTTP header | | `dl.cloudsmith.io` | Path-based | Path | | `dl.cloudsmith.io` | Basic Auth | Userinfo | Note: Not all forms of Basic Auth are supported on this domain. See the documented example, HTTP Basic Auth with credentials in the userinfo portion of the URL. **When to use the download domain** Use `dl.cloudsmith.io` instead of `generic.cloudsmith.io` when: - **Your HTTP library doesn't follow redirects**: The `generic.cloudsmith.io` domain redirects to `dl.cloudsmith.io`. Some libraries require manual redirect handling. - **Your tooling doesn't support credentials in the URL (userinfo)**: Some tools cannot pass credentials in the userinfo portion of URLs (`token:TOKEN@host`), but can construct URLs with tokens in the path. - **You want consistency with other formats**: Other Cloudsmith package formats use `dl.cloudsmith.io` with path-based authentication. Using the same pattern simplifies configuration across formats. - **You need direct download URLs**: Path-based URLs on `dl.cloudsmith.io` can be shared or embedded without requiring separate authentication configuration. ### Download via the HTML Index Cloudsmith supports serving Apache-style HTML formatted indexes of generic package files in the repository. **Disabled by Default** HTML index generation is disabled by default. Enable it in [Repository Settings](/repositories/repository-settings) under the Generic section. When the HTML index is disabled (by default), the repository only supports direct file downloads. Requests for specific files can be configured to proxy (if the file exists on the upstream) or served from the repository cache. When the HTML index is enabled, requests to directory paths (without specifying a file) will return an HTML view showing the index of available files and subdirectories. When enabled, the HTML index is available at: `https://generic.cloudsmith.io/OWNER/REPOSITORY/` #### Public Repositories Browse the HTML index to view and download packages directly in your browser. #### Private Repositories When accessing the HTML index in a browser for private repositories, you will be prompted for authentication. Enter `token` as the username and your entitlement token or API key as the password to view the index. [Image: Generic Upstream Index - Prompt for Password] After entering valid credentials, you should then be able to view the HTML index. [Image: Generic Upstream Index - List of Files] ## Upstream Proxying / Caching Supported The Generic format supports upstream proxying and caching for simple HTML-based sources and select non-HTML sources. When a file is requested, Cloudsmith checks for a local copy. If missing, it fetches the file from the upstream, caches it as a first-class local package, and serves it. ### Supported Upstream Sources The Generic format accepts any files or binaries, not just traditional software packages. The following upstream sources are explicitly supported and work out of the box: - **Gradle** - Gradle distributions - **Node** - Node.js distributions - **Hashicorp** - Hashicorp releases - **Fluentbit** - Fluentbit distributions - **GitHub** - GitHub Releases - **HTML-based sources** - Simple, unstyled HTML pages with no JavaScript, where each page contains only links to files or subdirectories (e.g., Apache releases) **Enterprise Repository Support** Enterprise repository managers like Artifactory and Sonatype Nexus require additional configuration and will not work out of the box. These sources also have practical size limits—repositories with millions of packages may not be processable. [Contact support](https://cloudsmith.com/company/contact-us) to discuss setup for these upstream sources. ### Configure an Upstream To configure an upstream for your Generic repository, browse to your repository overview page and click the **Upstreams** tab. Then, click **+ Add Upstream Proxy** and choose the **Generic** format upstream. In the upstream creation menu, define a name for your upstream and enter the upstream URL. You must specify an **Upstream Prefix** to prevent file collisions when using multiple upstreams. The prefix becomes part of the file's permanent path in the repository, ensuring files from different upstreams never collide, even with identical filenames. [Image: Generic Upstream Configuration] Please see our [Upstream Proxying](/repositories/upstreams#create-a-generic-upstream) documentation for further instructions on configuring upstreams. ### Examples #### Node.js Distributions This example demonstrates how upstream URLs, prefixes, and file paths are applied when working with Node.js distributions. **Upstream configuration:** - **Upstream URL**: `https://nodejs.org/dist/` - **Upstream Prefix**: `node_distributions` **How paths are resolved:** The upstream URL includes the base path (`/dist/`), so file paths are relative to that location. To download `https://nodejs.org/dist/latest/node-v25.6.1.tar.gz`, request the path `latest/node-v25.6.1.tar.gz` from Cloudsmith: | Component | Value | | :-------- | :---- | | Upstream URL | `https://nodejs.org/dist/` | | File path on upstream | `latest/node-v25.6.1.tar.gz` | | Full upstream URL | `https://nodejs.org/dist/latest/node-v25.6.1.tar.gz` | | Upstream prefix | `node_distributions` | | Cloudsmith URL | `https://generic.cloudsmith.io/OWNER/REPOSITORY/node_distributions/latest/node-v25.6.1.tar.gz` | **Download command:** ```shell curl -sLf -O 'https://generic.cloudsmith.io/OWNER/REPOSITORY/node_distributions/latest/node-v25.6.1.tar.gz' ``` The upstream prefix (`node_distributions`) ensures files from Node.js don't collide with files from other upstreams, even if they share identical file paths. #### Github Releases This example demonstrates how upstream URLs, prefixes, and file paths are applied when working with GitHub Releases. **Upstream configuration:** - **Upstream URL**: `https://api.github.com/repos/cloudsmith-io/cloudsmith-cli/releases` - **Upstream Prefix**: `github_releases` **How paths are resolved:** The upstream URL includes the base path (`/releases/`), so file paths are relative to that location. To download `https://github.com/cloudsmith-io/cloudsmith-cli/releases/download/v1.16.0/cloudsmith.pyz`, request the path `download/v1.16.0/cloudsmith.pyz` from Cloudsmith: | Component | Value | | :-------- | :---- | | Upstream URL | `https://api.github.com/repos/cloudsmith-io/cloudsmith-cli/releases` | | File path on upstream | `download/v1.16.0/cloudsmith.pyz` | | Full upstream URL | `https://github.com/cloudsmith-io/cloudsmith-cli/releases/download/v1.16.0/cloudsmith.pyz` | | Upstream prefix | `github_releases` | | Cloudsmith URL | `https://generic.cloudsmith.io/OWNER/REPOSITORY/github_releases/download/v1.16.0/cloudsmith.pyz` | **Download command:** ```shell curl -sLf -O 'https://generic.cloudsmith.io/OWNER/REPOSITORY/github_releases/download/v1.16.0/cloudsmith.pyz' ``` ### Upstream Priority Upstream priority behaves differently for Generic repositories due to the nature of upstream prefixes. All upstreams within a prefix are given a priority of 1. Changing the priority setting does not impact package blending, as each upstream is uniquely namespaced within its own prefix. Local packages (directly uploaded to the repository) always take precedence over upstream packages. When viewing the HTML index, cached packages display with a different icon than packages that will be proxied on demand. ## Key Signing Support Not Supported Generic files are stored and served as-is without package signing. If you need signed packages, consider using the [Raw format](/formats/raw-repository), which supports GPG signatures. ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Go Registry Cloudsmith provides public & private registries for Go. Go, also known as Golang, is an open source programming language designed at Google. For more information on Go, please see: - [Go](https://www.golang.org): The official website for Go language - [Go Packages](https://golang.org/pkg): Officially supported Go packages - [GoDoc](https://godoc.org): Documentation/search for Community Go packages **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package **Raw GO binaries** Raw GO binaries are not supported and need to be uploaded as a RAW package format instead. To upload, you need to generate a go module first. You can build a module with standard command-line tooling like `zip` and `git`. **Go module folder structure** Go modules must be packaged with a single top-level folder named `@/` (for example, `github.com/foo/bar@v1.2.3/`). This structure is required for Go to resolve module paths correctly when downloading modules. To illustrate the process we'll use [logrus](https://github.com/sirupsen/logrus) as an example: 1. First, we'll create the correct directory structure and check out the version of `logrus` we want to pack (`v1.4.2`): ```shell mkdir -p github.com/sirupsen/logrus@v1.4.2 git clone git@github.com:sirupsen/logrus.git github.com/sirupsen/logrus@v1.4.2 cd github.com/sirupsen/logrus@v1.4.2 git checkout v1.4.2 ``` 2. Finally, clean up and pack the module. Use `find` to include only folders with files in them: ```shell rm -rf .git/ cd ../../../ find -type f | while read f; do zip v1.4.2.zip "$f"; done ``` Now, we have a go module ready to be uploaded to a Cloudsmith repository. **Building Go modules** For a full overview on building your own modules, please refer to Russ Cox's [Go Modules](https://research.swtch.com/vgo-module.pdf). ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Go module via the Cloudsmith CLI is: ```shell cloudsmith push go OWNER/REPOSITORY PACKAGE_NAME.zip ``` Example: ```shell cloudsmith push go org/repo v1.4.2.zip ``` **Go Modules format** Older Go projects may not be in the Go Modules format, and you may experience an error when pushing these to Cloudsmith. You can add module support using `go mod init` and `go mod tidy`, then commit the new `go.mod` and `go.sum` files and add a new tag. ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). ## Download a Package ### Setup #### Configure a new Upstream Cloudsmith supports [https://proxy.golang.org](https://proxy.golang.org) as an upstream. This allows you to **proxy** and **cache** modules not yet in your repository through Cloudsmith for your Golang projects. To enable it for your repository, browse to your repository overview page and click in the Upstreams tab. Then, click on **+ Add Upstream Proxy** and then choose the **Go** format Upstream. In the Upstream creation menu, define a name your Upstream and in the **Proxy URL** field insert `https://proxy.golang.org`. Click on **+ Create Upstream Proxy** and the upstream should become immediately available. **Go Standard Library packages** Packages which are part of the [Go Standard Library](https://pkg.go.dev/std) are bundled directly with the Go compiler. They are not available for direct download and cannot be downloaded and cached through the Cloudsmith upstreams. Cloudsmith's Go repositories are designed to proxy and cache Go Modules (downloadable from the internet). - **Modules:** `github.com/gin-gonic/gin`, `golang.org/x/text` — Cached in Cloudsmith. - **Standard Library:** `strconv`, `fmt`, `net/http` — Bundled into the Go installer. Before you can install modules from your Cloudsmith repository, you'll need to configure your environment for access. The configuration is defined using the `GOPROXY` environment variable as explained below. #### Public Repositories To define the `GOPROXY` environment variable for a public Cloudsmith repository: **Warning** Cloudsmith provides a dedicated endpoint for Go artifacts: `https://golang.cloudsmith.io`. This endpoint supports native Golang Upstreams so you can access modules from [https://proxy.golang.org](https://proxy.golang.org). While the legacy endpoint ([https://dl.cloudsmith.io](https://dl.cloudsmith.io)) is still maintained, we recommend to use the new one. **Multiple servers in GOPROXY** `GOPROXY` allows the concatenation of multiple servers separated by commas. For example, for `GOPROXY=https://dl.cloudsmith.io/public/OWNER/REPOSITORY/go/,https://proxy.golang.org,direct`, all requests not satisfied by the first server will fallback the the next one (`https://proxy.golang.org`). ##### **Linux / Mac** ```shell export GOPROXY=https://golang.cloudsmith.io/OWNER/REPOSITORY/ ``` ##### **Windows (cmd)** ```shell set GOPROXY=https://golang.cloudsmith.io/OWNER/REPOSITORY/ ``` ##### **Windows (Powershell)** ```shell $env:GOPROXY=https://golang.cloudsmith.io/OWNER/REPOSITORY/ ``` #### Private Repositories Private Cloudsmith repositories require authentication. GOPROXY protocol only supports authentication via HTTP Basic Authentication: `/`. It you want to authenticate via Entitlement Token, use the literal string `token` in the user field: `token:`. **Secrets management** Entitlement Tokens, User Credentials, and API-Keys should be treated as secrets. You should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To define the `GOPROXY` environment variable for a private Cloudsmith repository: ##### **Linux / Mac** ```shell # HTTP Basic Auth (with Entitlement Token) export GOPROXY=https://token:TOKEN@golang.cloudsmith.io/OWNER/REPOSITORY/ # HTTP Basic Auth (API-Key) export GOPROXY=https://USERNAME:API-KEY@golang.cloudsmith.io/OWNER/REPOSITORY/ ``` ##### **Windows (cmd)** ```shell # HTTP Basic Auth (with Entitlement Token) set GOPROXY=https://token:TOKEN@golang.cloudsmith.io/OWNER/REPOSITORY/ # HTTP Basic Auth (API-Key) set GOPROXY=https://USERNAME:API-KEY@golang.cloudsmith.io/OWNER/REPOSITORY/ ``` ##### **Windows (Powershell)** ```powershell # HTTP Basic Auth (with Entitlement Token) $env:GOPROXY=https://token:TOKEN@golang.cloudsmith.io/OWNER/REPOSITORY/ # HTTP Basic Auth (API-Key) $env:GOPROXY=https://USERNAME:API-KEY@golang.cloudsmith.io/OWNER/REPOSITORY/ ``` **About Go modules authentication** The Go sumdb cannot record the hash value of a private repository and this will cause the local Go command to fail the verification after downloading. You can see more details running the `go mod download` with the `-x` option. It is recommended to use the environment variable `GONOSUMDB` and set its value to `cloudsmith.io`. This will skip verification for modules in your Cloudsmith repositories with a module name beginning with `cloudsmith.io`. Verification is performed against the global checksum database [sum.golang.org](https://sum.golang.org) (this service provides an auditable checksum database service used by the go command to authenticate modules). `GONOSUMDB` should be a [list of module path prefixes](https://go.dev/ref/mod#module-proxy:~:text=GONOSUMDB%20%E2%80%94%20list%20of%20glob%20patterns%20of%20module%20path%20prefixes%20that%20should%20not%20be%20checked%20using%20the%20public%20checksum%20database%2C%20sum.golang.org.), like for example: `GONOSUMDB=demo-docs/awesome-repo,cloudsmith-test/acme2`. While verifying the checksum of downloaded go modules is a critical step, there's general agreement against having separate checksum databases (for example, having a Cloudsmith checksum database). See for example: [https://github.com/golang/go/issues/44936](https://github.com/golang/go/issues/44936) and [https://github.com/gomods/athens/issues/1572](https://github.com/gomods/athens/issues/1572). {/* Cloudsmith adheres to this recommendations and will proxy checksum verification requests to the global checksum database. */} ### Installing a Package You can install the latest version of a package with: ```shell go get PACKAGE_NAME ``` Or install a specific version of a package with: ```shell go get PACKAGE_NAME@PACKAGE_VERSION ``` ## Security Scanning Cloudsmith supports security scanning for Go modules. Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information about this capability. ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Gradle Repository Cloudsmith provides public & private repositories for Gradle Gradle is an open-source build-automation tool that harnesses the power of Maven via a Groovy-based domain-specific language making it easier to define projects and their dependencies. For more information on Gradle, please see: - [Gradle](https://gradle.org/): The official website for Gradle - [Gradle Docs](https://docs.gradle.org/): The official documentation for Gradle **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :------------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_VERSION | The version number of your package | | GROUP-ID | A unique Maven identifier for your project across all projects i.e "com.companyname.project" | | ARTIFACT_ID | The name of the jar without version i.e "project" | **Kotlin syntax** These examples use the Groovy-based syntax for Gradle. For the Kotlin syntax, please refer to the [official documentation](https://docs.gradle.org/current/userguide/declaring_dependencies.html) for declaring dependencies. ## Upload a Package ### Upload via gradle publish The endpoint for native Gradle API (Maven-based) is: ``` https://maven.cloudsmith.io/ ``` For Maven-based publishing you'll need to enable the maven-publish plugin: ```groovy plugins { id 'maven-publish' } ``` Next, configure a repositories block to point to Cloudsmith as follows: ```groovy publishing { repositories { maven { name = "cloudsmith" url = "https://maven.cloudsmith.io/OWNER/REPOSITORY/" def releasesRepoUrl = "https://maven.cloudsmith.io/OWNER/REPOSITORY/" def snapshotsRepoUrl = "https://maven.cloudsmith.io/OWNER/REPOSITORY/" url = version.endsWith('SNAPSHOT') ? **snapshotsRepoUrl **: releasesRepoUrl credentials { username = 'USERNAME' password = 'API-KEY' } } } } ``` A bare minimum publications section is required: ```groovy publishing { publications { maven(MavenPublication) { // [snip] } } } ``` You can now publish to the native API with: ```shell gradle publish ``` ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload via the Cloudsmith CLI is: ```shell cloudsmith push maven OWNER/REPOSITORY ARTIFACT_ID-PACKAGE_VERSION.jar --pom-file=ARTIFACT_ID-PACKAGE_VERSION.pom ``` Example: ```shell cloudsmith push maven org/repo validation-api-1.0.0.GA.jar --pom-file=validation-api-1.0.0.GA.pom ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Installing ### Setup To enable the retrieval of Cloudsmith hosted packages via Gradle, the first step is to add your repository to the `build.gradle` file. #### Public Repositories Add the following, at any location, to your `build.gradle` file: ```kotlin repositories { maven { url "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/maven/" } } ``` After the repository is added to the `build.gradle` file, all that is left is to specify the dependency in the dependencies section of the project `build.gradle` file. To do this add the below to your `build.gradle` file: ```kotlin dependencies { implementation 'GROUP_ID:ARTIFACT_ID:PACKAGE_VERSION' } ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To enable the retrieval of packages from a private Cloudsmith repository via Gradle, add your repository your `build.gradle` file as follows: ```groovy Entitlement Token Authentication repositories { maven { url "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/maven/" } } ``` ```groovy HTTP Basic Authentication repositories { maven { url "https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/maven/" credentials { username "$repositoryUser" password "$repositoryPassword" } } } ``` When using Entitlement Token Authentication, no further setup is required. If using HTTP Basic Authentication, you can provide one following three types of credentials: - Cloudsmith Username and Password - Cloudsmith API Key - An Entitlement Token When using HTTP Basic Authentication you'll probably want to keep your credentials separately in your `~/.gradle/gradle.properties` file instead of within the `build.gradle` file. Once you have decided which credentials you wish to use, setup your `~/.gradle/gradle.properties` file as follows: ```groovy Username & Password repositoryUser=USERNAME repositoryPassword=PASSWORD ``` ```groovy API Key repositoryUser=USERNAME repositoryPassword=API-KEY ``` ```groovy Entitlement Token repositoryUser=token repositoryPassword=TOKEN ``` For more details on authentication in Gradle, please refer to the official [Gradle documentation](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.repositories.AuthenticationSupported.html) ### Specifying Dependencies After the repository is added to the `build.gradle` file and your credentials have been added to your `~/.gradle/gradle.properties` file (required for private repositories if using HTTP Basic Authentication), all that is left is to specify the dependency in the dependencies section of the project `build.gradle` file. To do this add the following to your build.gradle file: ```groovy dependencies { implementation 'GROUP_ID:ARTIFACT_ID:PACKAGE_VERSION' } ``` ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Index Packages ## Troubleshooting **Occasional Publish Failures** Gradle + Java8 can experience issues with SNI (Server Name Indicator) endpoints which we use to power our CDN, like `https://maven.cloudsmith.io`. If you experience occasional or intermittent publish failures, resulting in a 443 error, then please update your endpoint to `https://api-g.cloudsmith.io/maven`. Contact us [here](https://support.cloudsmith.com). We're always happy to help. # Helm Chart Repository Cloudsmith provides public & private repositories for Helm Charts 🚧 **Support for OCI and non-OCI Helm Registries** While Cloudsmith supports both OCI and non-OCI Helm registries, please note that for Helm OCI support the endpoint is: `helm.oci.cloudsmith.io`. For support on OCI Helm Charts, please refer to our OCI Repository documentation [here](/formats/oci-repository). [Helm](https://helm.sh) is a package manager for Kubernetes that allows development and operations teams to easily manage and deploy these increasingly complex cloud native applications to their Kubernetes clusters. Helm allows you to manage applications on your Kubernetes cluster in much the same way as you’d manage applications on your Linux server with `apt` or `yum`. Helm works by packaging up a set of YAML definitions along with the necessary configuration to quickly stand up all components of an application in a repeatable way. A single chart can be as simple or complex as necessary, deploying anything from a single container to a full distributed application. Helm combines these application definitions with user-provided configuration to allow simple overriding of configuration where needed, allowing users to concentrate on shipping software and not on the nitty-gritty of configuring every application they need to run. Helm packages are known as “Charts” and are stored in a “Chart Repository”. By default, Helm comes bundled with the “stable” chart repository, hosted for free by Google. Most public charts are hosted here, mostly provided by vendors packaging their own software for use by others. For more information on Helm, please see: - [Helm](https://helm.sh/): The official website for Helm - [Helm Hub](https://hub.helm.sh): The official public repository for Helm Charts - [Helm Documentation](https://helm.sh/docs): Helm Documentation - [Kubernetes Documentation](https://kubernetes.io/docs): Kubernetes Documentation **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :------------ | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | NAME | A name for your repository in your helm configuration | | CHART_NAME | The name of your chart | | CHART_VERSION | The version number of your chart | ## Upload a Chart To upload, you will need to generate your chart first. You can do this with the [helm CLI](https://helm.sh/docs/helm/#helm-package): ```shell helm package . ``` This generates a chart package file (.tgz) like `CHART_NAME-CHART_VERSION.tgz` that you can upload. **Building Charts** Please see the [official Chart development](https://helm.sh/docs/developing_charts/) guide for more information on building your own Charts for distribution. **Early Access** OCI support is in Early Access. Helm Charts uploaded via OCI will show as "Docker Packages" on Cloudsmith. Helm Charts uploaded via OCI aren't available to download via the classic `Chart.yaml` repository method. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Helm chart via the Cloudsmith CLI is: ```shell cloudsmith push helm OWNER/REPOSITORY CHART_NAME-CHART_VERSION.tgz ``` Example: ```shell cloudsmith push helm org/repo your-chart-1.0.0.tgz ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). We'll supplement these with more detailed guidance later but otherwise, just ask: we're here to help. ## Download / Install a Chart ### Setup Assuming you have helm already installed, it is straight-forward to add a Cloudsmith-based chart repository. You use the `helm repo add` and `helm repo update` commands as follows: #### Public Repositories ```shell helm repo add NAME \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/helm/charts/' helm repo update ``` #### Private Repositories ```shell Entitlement Token Auth helm repo add NAME \ 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/helm/charts/' helm repo update ``` ```shell HTTP Basic Auth (User & Pass) helm repo add NAME \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/helm/charts/' \ --username USERNAME \ --password PASSWORD helm repo update ``` ```shell HTTP Basic Auth (API-Key) helm repo add NAME \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/helm/charts/' \ --username USERNAME \ --password API-KEY helm repo update ``` ```shell HTTP Basic Auth (Token) helm repo add NAME \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/helm/charts/' \ --username token \ --password TOKEN helm repo update ``` **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ### Installing a Chart To install/use a specific version of a chart: ```shell helm install NAME/CHART_NAME --version CHART_VERSION ``` To install the latest version of a chart: ```shell helm install NAME/CHART_NAME ``` Or you can upgrade to the most recent version of a chart if you've already installed: ```shell helm upgrade NAME/CHART_NAME ``` If you've got a `requirements.yaml` file in your chart, you can specify this as a dependency: ```yaml dependencies: - name: CHART_NAME version: CHART_VERSION repository: NAME ``` ### Removing Setup Helm provides a very clean method of removing a chart repository, simply run the following command: ```shell helm repo remove NAME ``` ### Logging out of Registry Helm provides a command to logout of registries ```shell helm registry logout docker.cloudsmith.io ``` ## Provenance Provenance files allow for verification of both the integrity and source of a Helm chart. Cloudsmith fully supports both verification and generation of Helm provenance files. ### Building Provenance Files Provenance files can be built whilst packaging helm charts, by passing in the `--sign` and `--key` switches: ```shell helm package --sign --key "Your Key Name" . ``` The Helm client will prompt for the secret key used to sign your Chart. One complete, a `.prov` file will coexist alongside your chart tarball. ### Uploading Provenance Files Provenance files may be uploaded on the UI as above. ### Upload via the Cloudsmith CLI Provenance files can be passed to the Cloudsmith CLI during a `push`. Please note this requires version `2.0.4` (or later) of the `cloudsmith-api` Python package: ```shell cloudsmith push helm OWNER/REPOSITORY --provenance-file CHART_NAME-CHART_VERSION.tgz.prov CHART_NAME-CHART_VERSION.tgz ``` ### Automated Generation of Provenance Files Where a provenance file has not been provided at upload, Cloudsmith will automatically generate one during synchronization. Generated provenance files will be signed using the current repository GPG signing key. ### Verifying a Chart Charts and provenance files may verified by passing the `--verify` switch to `helm install`: ```shell helm install --verify NAME/CHART_NAME ``` Note that this will require the GPG public key of the repository to be installed to the keyring of the target system. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Helm repositories that you wish to use for charts that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested charts for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Packages ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Hex Repository Cloudsmith provides public and private repositories for Hex Hex is a package manager for the BEAM ecosystem, any language that compiles to run on the BEAM VM, such as [elixir](https://elixir-lang.org/) and [Erlang](https://www.erlang.org/), can be used to build Hex packages. With Hex, developers can create, share, and manage packages for their projects. For more information on Hex, please see: - [Hex](https://hex.pm/): Public repository for hosting and managing Hex packages - [HexDocs](https://hexdocs.pm/): HexDocs is where Hex packages host their documentation. - [Publishing a Hex package](https://hex.pm/docs/publish): How to Publish a Hex package. **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package Publishing a Hex package to Cloudsmith involves several steps, including adding Cloudsmith to your project's repository list, configuring metadata in the mix.exs file, and finally, using the Mix client to publish the package. **Mix** Mix is a build tool that ships with Elixir. For an introduction on Elixir and Mix, see [introduction to Mix](https://hexdocs.pm/elixir/introduction-to-mix.html). You can also read about how hex packages work with mix at [Hex Mix Usage](https://hex.pm/docs/usage). Assuming you have mix already installed, it is straightforward to add a Cloudsmith-based hex repository. ### Add and Configure a Cloudsmith Repository First, you'll need to set some environment variables to assist in the process, replacing OWNER with your Cloudsmith account's organization and REPOSITORY with the name of your Cloudsmith repository: ```shell export CLOUDSMITH_API_KEY='YOUR-API-KEY' export CLOUDSMITH_API_HOST='https://api.cloudsmith.io' export CLOUDSMITH_NAMESPACE='OWNER' export CLOUDSMITH_REPOSITORY='REPOSITORY' ``` While documented as optional, it's necessary to obtain the repository's fingerprint via Cloudsmith's API. This fingerprint ensures secure communication between your project and Cloudsmith. Use the command below to Cloudsmith's API to [retrieve the active RSA key](/api/repos/rsa/list) (aka its fingerprint) for the Cloudsmith Repository: ```shell export FINGERPRINT=$(curl -s -H "X-Api-Key: ${CLOUDSMITH_API_KEY}" ${CLOUDSMITH_API_HOST}/v1/repos/${CLOUDSMITH_NAMESPACE}/${CLOUDSMITH_REPOSITORY}/rsa/ | jq -r '.ssh_fingerprint') ``` Next, use Mix's `hex.repo add` command to add your Cloudsmith repository: ```shell mix hex.repo add REPOSITORY https://hex.cloudsmith.io/${CLOUDSMITH_NAMESPACE}/${CLOUDSMITH_REPOSITORY} --auth-key $CLOUDSMITH_API_KEY --fetch-public-key SHA256:${FINGERPRINT} ``` Replace REPOSITORY with the name of your Cloudsmith repository. **Install hex task** Note: If you get the error 'The task "hex.repo" could not be found' you will first need to install the hex task in mix. (For more information, see the Hex usage documentation.) Run: `mix local.hex` **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ### Configure the HEX Package To configure your Hex package for publishing to your Cloudsmith repository, follow these steps: 1. Open `mix.exs`: Locate and open the `mix.exs` file of the Hex package you intend to publish to Cloudsmith. 2. Add defp Function: Within the `mix.exs` file, add a `defp` function to define the Cloudsmith repository. This function specifies the repository URL and authentication details. 3. Here's an example of how to add the `defp` function: ```shell defp hex do [ api_url: "https://hex.cloudsmith.io/OWNER/REPOSITORY", api_key: "API-KEY" ] end ``` Replace OWNER with your Cloudsmith account's organization and REPOSITORY with the name of your Cloudsmith repository. Additionally, replace API-KEY with your Cloudsmith API key. Next, integrate with Mix Tasks: After adding the defp function, the project function needs to have the hex and project variables defined. 1. Open `mix.exs`: Locate and open the `mix.exs` file of the Hex package you intend to publish to Cloudsmith. 2. Locate the `project` function and populate the hex and package variables. Here's an example of what to add to the project function: ```shell def project do [ ... package: package(), hex: hex() ... ] end ``` ### Publish your HEX package Once you've configured your Hex package to publish to your Cloudsmith repository, you can proceed with the publishing process. Follow these steps to publish your HEX package: 1. Navigate to Your Package Folder: Open a terminal or command prompt and navigate to the top-level directory of your package. ```shell cd /path/to/your/package ``` 2. Execute the Publish Command: Use the mix hex.publish package command to initiate the publishing process. This command automatically packages your project and publishes it to the configured Cloudsmith repository. ```shell mix hex.publish package ``` 3. Follow On-screen Instructions: Follow any on-screen prompts or instructions to complete the publishing process. These prompts may include confirming the package details and agreeing to publish. By following these steps, you'll successfully publish your HEX package to your Cloudsmith repository. Remember to review the published package on your Cloudsmith dashboard to verify its availability and correctness. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Hex package via the Cloudsmith CLI is: ```shell cloudsmith push hex OWNER/REPOSITORY PACKAGE_NAME.tar ``` Replace OWNER with your Cloudsmith account's organization and REPOSITORY with the name of your Cloudsmith repository. ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). We'll supplement these with more detailed guidance later but otherwise, just ask - we're here to help. ## Download or Installing a Package ### Setup Before you can install modules from your Cloudsmith repository you'll need to add the repository. Follow the steps above "Add and Configure a Cloudsmith Repository". Your entitlement token can be used as the API key. ### Download a Package You can download a package by using mix's fetch command: ```shell mix hex.package fetch PACKAGE-NAME PACKAGE-VERSION --repo REPOSITORY ``` Replace REPOSITORY with the name of your Cloudsmith repository. For example, to download version 1.0.0 of the 'jason' module: ```shell mix hex.package fetch jason 1.0.0 --repo REPOSITORY ``` Replace REPOSITORY with the name of your Cloudsmith repository. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ### Adding a Cloudsmith Package as a Dependency To incorporate a package hosted on Cloudsmith into your project, you need to modify your `mix.exs` file. Follow these steps to add the Cloudsmith repository as a dependency: 1. Locate and open the mix.exs file of your Hex project in a text editor. 2. Edit the Dependencies Function: Within the deps function of mix.exs, add an entry for your Cloudsmith package. Ensure to replace PACKAGE_NAME, VERSION, and REPOSITORY with your specific details. ```shell defp deps do [ {:PACKAGE_NAME, "~> VERSION", repo: "REPOSITORY"}, ] end ``` 3. Run the mix publish command and it will pull in the dependencies ```shell mix hex.publish package ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Hex repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. **Hex upstreams** Hex upstreams are in Early Access. ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Hugging Face Repository Hugging Face is an open-source platform and ecosystem for building, sharing, and deploying machine learning models, especially in the field of natural language processing (NLP) and generative AI. ## Repository Types The Hugging Face Hub has three different types of repositories: 1. Model Repositories: These are used to store and distribute **machine learning models**, including weights, configs, tokenizer files, and model cards. They are commonly used with the [`transformers` library](https://huggingface.co/docs/transformers/models). 2. Dataset Repositories: These are used to store **datasets**, scripts for loading them, and metadata. They are commonly used with the [`datasets` library](https://huggingface.co/docs/datasets/en/stream). 3. Space Repositories: These are used to host interactive apps, usually built with Gradio or Streamlit. They are like web demos for models or data. **Note** Spaces are not supported in Cloudsmith Hugging Face repositories. Models and Datasets are supported and identified in the `` field as `models` and `datasets`. Cloudsmith supports the main [`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/index) python library to work with `models` and `datasets`. For more information on Hugging Face, see: - **[Hugging Face](https://huggingface.co/)**: The official website for the company and its ecosystem. - **[Hugging Face Hub](https://huggingface.co/)**: The official public platform for models, datasets, and demo apps. **Storage Deduplication** Cloudsmith's Hugging Face repositories benefit from native Hugging Face [file storage deduplication](https://huggingface.co/docs/hub/en/storage-backends) technology. This means that if you upload the same model or dataset multiple times (even across different repositories), Cloudsmith will only store a single copy of the data, saving you storage space and associated artifact data cost across all repositories within a workspace. ## Differences from Hugging Face Here you can see a general overview of the main naming equivalences: | Hugging Face Term | Cloudsmith Term | | :---------------- | :-------------- | | Repository | Package | | Commit / Revision | Version | | Tag | Package Tag | Keep reading to learn more about them. ### Repository as a Package In Cloudsmith, a Hugging Face repository is treated as a single package. This creates a simple and direct one-to-one relationship, so a package name in your Cloudsmith repository will be identical to the Hugging Face repository name, including its namespace. For example, a Hugging Face repository named `workspace/sentiment-analyzer` will correspond to a Cloudsmith package named `workspace/sentiment-analyzer`. ### Commits and versions In the Hugging Face ecosystem, a commit represents a complete snapshot of a repository at a specific point in time. This concept is fundamental to achieving reproducibility in your AI/ML builds, as each commit provides a precise, traceable record of your model, dataset, or space. To align with this structure, Cloudsmith maps Hugging Face repositories and their commits directly to packages and versions within Cloudsmith: - **Hugging Face Repository as a Package**: the name of a Hugging Face repository (e.g., org-name/model-name) corresponds directly to the package name in Cloudsmith. You don't need to create a separate object to represent the repository itself, or create multiple repositories within Cloudsmith. - **Commit as a package version**: each unique commit pushed to the Hugging Face repository becomes a new, distinct version of that package in Cloudsmith. The version string is typically the commit hash (e.g., `e243b1b`). This approach creates an intuitive mapping: commit-version. Each of your Hugging Face repositories is treated as a package in Cloudsmith, with the package name matching the repository name (e.g., `workspace/my-awesome-model`). A new version of the package is created for every commit you push. The version is identified by the commit hash. The initial push of a commit will create both the package and its first version simultaneously. For instance, pushing a commit with the hash `a1b2c3d` creates the version `a1b2c3d` within the `workspace/my-awesome-model` package. This ensures that every change is versioned and fully reproducible, and guarantees a similar workflow within Cloudsmith to the ones you are used to. ### Tags In Hugging Face, a tag is a user-friendly alias to a specific commit (version). Instead of using a long commit hash like e243b1b, you can use a human-readable tag like `v1.0` or `latest` to refer to that same snapshot. At Cloudsmith: - Uniqueness: just as tags are unique within a Hugging Face repository, Cloudsmith tags are unique per package - Flexibility: unlimited number of tags pointing to a single version (commit). For example, the tags latest, v2.1, and stable can all point to the same commit hash. This mapping allows you to request a package version using a simple tag, similarly to what you can do with Docker. For instance, you can fetch the files associated with the main branch by requesting the package version tagged as main, and Cloudsmith will resolve it to the correct underlying commit automatically. **Note** Hugging Face tags included in the model card (i.e. in the README metadata), are automatically parsed and added to the Cloudsmith artifact during the sync process. Additionally, tags can be created and managed via the UI/CLI. --- In the following examples: | Identifier | Description | | :--------- | :--------------------------------------------------------------- | | `WORKSPACE` | Your Cloudsmith workspace name. | | `REGISTRY` | Your Cloudsmith Repository name (also called "slug"). | | `TOKEN` | Your Cloudsmith API Token | | `API-KEY` | Your Cloudsmith API Key. | | `repo_id` | The name of your HF artifact. | | `repo_type` | The type of artifact: `model` or `dataset`. | | `TAG` | A tag for your HF artifact. | ## Uploading Packages You can easily upload your models and datasets to Cloudsmith using the tools you already know. Cloudsmith supports uploads via the `huggingface_hub` Python library, which is the underlying engine for popular libraries like `transformers` and `datasets`. ### Using `huggingface_hub` Before starting, update the `HF_ENDPOINT` and `HF_TOKEN` variables with your Cloudsmith endpoint and API Token. This is the most direct method. You can upload any folder containing your model or dataset files using the `upload_folder` function. ```python from huggingface_hub import HfApi HF_ENDPOINT = "https://huggingface.cloudsmith.io/WORKSPACE/REPOSITORY" HF_TOKEN = "CSA_API_TOKEN" MODEL_NAME = "acme-corp/text-summarizer" LOCAL_DIR = "/path/to/your/model_files/snapshot/id" api = HfApi(endpoint=HF_ENDPOINT, token=HF_TOKEN) api.upload_folder( folder_path=LOCAL_DIR, repo_id=MODEL_NAME, repo_type="model", token=HF_TOKEN, commit_message="Initial upload via CS", revision="main" ) ``` ### Using `hf cli` The Hugging Face CLI provides a convenient way to upload models and datasets directly from your terminal. Before starting, set `HF_ENDPOINT` and `HF_TOKEN` as environment variables with your Cloudsmith endpoint and API Token: ```bash export HF_ENDPOINT="https://huggingface.cloudsmith.io/WORKSPACE/REPOSITORY" export HF_TOKEN="CSA_API_TOKEN" ``` Upload any folder containing your model or dataset files using the `upload` command: ```bash hf upload --repo-type --commit-message "Commit via CLI" --revision ``` For example, to upload a model called `acme-corp/text-summarizer` stored locally at `./hugging-face-models/acme-corp/text-summarizer`: ```bash hf upload acme-corp/text-summarizer ./hugging-face-models/acme-corp/text-summarizer --repo-type model --commit-message "Commit via CLI" --revision "1.2.3" ``` Execute the command, and you will see the upload progress in your terminal: You will see the package appear in your Cloudsmith repository shortly after the upload completes: ## Downloading Packages Cloudsmith supports the `huggingface_hub` Python library to download packages using its `snapshot_download` function. ### Using `huggingface_hub` Before starting, set the `HF_ENDPOINT` and `HF_TOKEN` variables with your Cloudsmith endpoint and API Token. This is the most direct way to download all the files for a specific package version (commit) from your Cloudsmith repository. ```python from huggingface_hub import HfApi HF_ENDPOINT = "https://huggingface.cloudsmith.io/WORKSPACE/REPOSITORY" HF_TOKEN = "CSA_API_TOKEN" MODEL_NAME = "acme-corp/text-summarizer" api = HfApi( endpoint=HF_ENDPOINT, token=HF_TOKEN, ) local_dir = api.snapshot_download( repo_id=MODEL_NAME, repo_type="model", revision="main" ) ``` To download a dataset instead, set `repo_type` to `"dataset"` and provide the dataset name as the `repo_id`: ```python from huggingface_hub import HfApi HF_ENDPOINT = "https://huggingface.cloudsmith.io/WORKSPACE/REPOSITORY" HF_TOKEN = "CSA_API_TOKEN" DATASET_NAME = "my-org/sentiment-data" api = HfApi( endpoint=HF_ENDPOINT, token=HF_TOKEN, ) local_dir = api.snapshot_download( repo_id=DATASET_NAME, repo_type="dataset", revision="main" ) ``` **Revision** The `revision` parameter defaults to `"main"` if omitted. The value can be: - A **branch name**, e.g., `revision="main"` or `revision="dev"` - A **full commit SHA**, e.g., `revision="a1b2c3d4e5f67890abcdef1234567890abcdef12"` - A **version tag**, e.g., `revision="v1.0"` or `revision="1.2.3"` For example, to download a specific commit: ```python local_dir = api.snapshot_download( repo_id=MODEL_NAME, repo_type="model", revision="a1b2c3d4e5f67890abcdef1234567890abcdef12" ) ``` ### Using `hf cli` The Hugging Face CLI provides a convenient way to download models and datasets directly from your terminal. Before starting, set `HF_ENDPOINT` and `HF_TOKEN` as environment variables with your Cloudsmith endpoint and API Token: ```bash export HF_ENDPOINT="https://huggingface.cloudsmith.io/WORKSPACE/REPOSITORY" export HF_TOKEN="CSA_API_TOKEN" ``` Download a model or dataset using the `download` command: ```bash hf download --repo-type --revision ``` For example, to download version `1.2.3` of the `acme-corp/text-summarizer` model: ```bash hf download acme-corp/text-summarizer --repo-type model --revision 1.2.3 ``` To download a dataset, set `--repo-type` to `dataset`: ```bash hf download my-org/sentiment-data --repo-type dataset --revision 2.0.0 ``` **Revision** If `--revision` is omitted, the CLI defaults to `main`. The value can be: - A **branch name**, e.g., `--revision main` or `--revision dev` - A **full commit SHA**, e.g., `--revision a1b2c3d4e5f67890abcdef1234567890abcdef12` - A **version tag**, e.g., `--revision v1.0` or `--revision 1.2.3` For example, to download a specific commit by SHA: ```bash hf download acme-corp/text-summarizer --repo-type model --revision a1b2c3d4e5f67890abcdef1234567890abcdef12 ``` ## Upstream Proxying / Caching Supported You can configure your Cloudsmith repository to act as a proxy for the public **Hugging Face Hub**. This allows you to cache models and datasets from the public Hub into your private Cloudsmith repository for faster, more reliable access, at the same time as you manage your own private models and datasets. # LuaRocks Repository Cloudsmith provides public & private repositories for Lua Modules LuaRocks is the package manager for Lua modules. It allows you to create and install Lua modules as self-contained packages called rocks. Cloudsmith is proud to support fully-featured repositories for managing your own private and public Lua rocks. For more information on Lua, please see: - [Lua](https://www.lua.org/): The official website for Lua - [LuaRocks](https://luarocks.org/): The official website for LuaRocks **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | MODULE_NAME | The name of your Lua Module | | MODULE_VERSION | The version number of your Lua Module | ## Upload a Module To upload, you need to generate a module first (rockspec, source and binary can all be uploaded). You can do this with the [luarocks CLI](https://github.com/luarocks/luarocks/wiki/pack): ```shell luarocks pack { | []} ``` This generates a `.rock` file like `MODULE_NAME-MODULE_VERSION.src.rock` that you can upload. It is also possible to upload a `.rockspec` file directly, without packing. In this case, luarocks will pull the sources from the upstream location and build a package automatically at install time. Please see the [official LuaRocks documentation](https://github.com/luarocks/luarocks/wiki/Creating-a-rock) for more information on building your own rocks. **Alternative Upload Methods** The `luarocks upload` command only supports uploading modules to the official public rocks repository. To upload your modules to Cloudsmith, you can use the Web UI, the Cloudsmith CLI or the Cloudsmith API. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Lua module via the Cloudsmith CLI is: ```shell cloudsmith push luarocks OWNER/REPOSITORY MODULE_NAME-MODULE_VERSION.src.rock ``` Example: ```shell cloudsmith push luarocks org/repo your-module-1.0.0-1.src.rock ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). ## Download / Install a Module You can install modules directly by using the `--server` command-line flag when executing a luarocks command. #### Public Repositories ```shell luarocks install MODULE_NAME MODULE_VERSION --server https://dl.cloudsmith.io/public/OWNER/REPOSITORY/luarocks/ ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```shell Entitlement Token Auth luarocks install MODULE_NAME MODULE_VERSION --server https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/luarocks/ ``` ```shell HTTP Basic Auth (User & Pass) luarocks install MODULE_NAME MODULE_VERSION --server https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/ ``` ```shell HTTP Basic Auth (API-Key) luarocks install MODULE_NAME MODULE_VERSION --server https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/ ``` ```shell HTTP Basic Auth (Token) luarocks install MODULE_NAME MODULE_VERSION --server https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/ ``` **Enforcing Server-only Search** To search only your Cloudsmith repository for modules use the `--only-server` command-line flag. This will force luarocks to search only the Cloudsmith repository and will result in luarocks not being able to install public modules that your private module may depend on. For most use cases, users will probably want to persist their repository settings and not specify them every time. luarocks provides a configuration file that can be modified to persist settings, see the [luarocks config file documentation](https://github.com/luarocks/luarocks/wiki/Config-file-format) for full details of available options and the location of the file for your platform. To add your private repository, adjust the `rocks_servers` section of your config file. **Note** - if you still want to be able to install packages from [luarocks.org](https://luarocks.org/) you should leave the default value in place and **add** your repository, otherwise you can replace the value entirely: #### Public Repositories ```json rocks_servers = { "http://luarocks.org/repositories/rocks", "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/luarocks/" } ``` #### Private Repositories ```json Entitlement Token Auth rocks_servers = { "http://luarocks.org/repositories/rocks", "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/luarocks/" } ``` ```json HTTP Basic Auth (User & Pass) rocks_servers = { "http://luarocks.org/repositories/rocks", "https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/" } ``` ```json HTTP Basic Auth (API-Key) rocks_servers = { "http://luarocks.org/repositories/rocks", "https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/" } ``` ```json HTTP Basic Auth (Token) rocks_servers = { "http://luarocks.org/repositories/rocks", "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/luarocks/" } ``` ## Upstream Proxying / Caching Not Supported ## Troubleshooting Please see the [Troubleshooting Overview](/troubleshooting) page for further help and information. # Maven Repository Cloudsmith provides public & private repositories for Maven Maven is a build automation tool primarily associated with the Java programming language. Developed by the Apache Software Foundation and released in 2004 it provides a standardized way to describe how a software project is built. A Maven Repository or Maven Repo is a registry of packaged files, stored, indexed, and made accessible to projects that depend on them. Each package has a unique name and version allowing for repeatable continuous integration and continuous delivery (or continuous deployment) tasks. The Maven repository index stores metadata about each package; that the Maven tooling looks up at build time enabling pulling in of dependency projects and extensions. For more information on Maven, please see: - [Maven](https://maven.apache.org/): The official website for Apache Maven - [Maven Central](https://maven.org/): Popular public repository for Maven artifacts If using Gradle - please see our [Gradle documentation](/formats/gradle-repository) If using sbt - please see our [sbt documentation](/formats/sbt-repository) **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_VERSION | The semantic version number of your package | | GROUP_ID | A unique Maven identifier for your project across all projects and usually takes the form of a reverse domain i.e "com.companyname" | | ARTIFACT_ID | The name of the jar without version i.e "project" | ## Upload a Package To upload, you need to generate your package first. You can do this with: ```shell mvn package ``` This generates a Maven package file (`.jar` or similar) like `your-package-1.2.3.jar` that you can upload. **Required Files** You will always need at least the package file and the POM file for uploading. #### What is a POM? A POM, the Project Object Model, is the XML file that describes all the aspects of your project that relate to building and packaging the source code into a package file. Typically a jar (java archive). The metadata held within the pom.xml that is typically stored within the jar itself allows Maven to index the package into a Maven Repository for easy distribution. #### What is a Fat Jar? A Fat Jar, is also referred to as an Uber Jar, is a Java Archive library that contains all classes, including all the classes of its dependencies. This allows the Jar to be run standalone without requiring any further code available on the Class Path. The disadvantage of creating an all-in-one jar mean that you have to deploy everything (a potentially large file) each time. If you split the Fat Jar into components you can separately test, version and release code enabling faster deployments and your developers to cherry-pick components for inclusion in other projects. ### Upload via Maven The endpoint for the native Maven API is: ``` https://maven.cloudsmith.io/OWNER/REPOSITORY/ ``` The distribution repositories define where to push your artifacts. In this case it will be a single repository, but you can configure alternatives. Add the following to your project `pom.xml` file: ```xml NAME https://maven.cloudsmith.io/OWNER/REPOSITORY/ NAME https://maven.cloudsmith.io/OWNER/REPOSITORY/ ``` **Snapshot and Releases Repositories** You can configure different repositories for snapshots and releases, and you can replace **NAME** with your own identifier (but make sure they match settings elsewhere). You then can configure your `~/.m2/settings.xml` file with the API key of the uploading user: ```xml NAME USERNAME API-KEY ``` You can now publish to the native API with: ``` mvn deploy ``` **Official Maven Documentation** You can find out more about Maven publishing in the official [Maven documentation](https://maven.apache.org/plugins/maven-deploy-plugin/usage.html). ### Upload via the Cloudsmith CLI The command to upload a Maven package via the Cloudsmith CLI is: ```shell cloudsmith push maven OWNER/REPOSITORY ARTIFACT_ID-PACKAGE_VERSION.jar --pom-file=ARTIFACT_ID-PACKAGE_VERSION.pom ``` Example: ```shell cloudsmith push maven org/repo validation-api-1.0.0.GA.jar --pom-file=validation-api-1.0.0.GA.pom ``` ### Uploading of extra files via the Cloudsmith CLI Customers can take advantage of the new extra-files option available for the Maven push command in cloudsmith-cli as of version 1.7.0. Here’s an example of how to use the command. You can either provide a comma-separated list or pass multiple `--extra-files` parameters. Using `extra-files` as a comma-separated list: ```shell cloudsmith push maven cloudsmith-test/cli-upload example-maven-project-1.0.0.jar \ --javadoc-file example-maven-project-1.0.0-javadoc.jar \ --sources-file example-maven-project-1.0.0-sources.jar \ --pom-file example-maven-project-1.0.0.pom \ --version 1.0.0 \ --group-id com.example \ --artifact-id maven-cli-push-test \ --extra-files build-script.sh,random-file.xml ``` Or sing multiple `extra-files` parameters: ```shell cloudsmith push maven cloudsmith-test/cli-upload example-maven-project-1.0.0.jar \ --javadoc-file example-maven-project-1.0.0-javadoc.jar \ --sources-file example-maven-project-1.0.0-sources.jar \ --pom-file example-maven-project-1.0.0.pom \ --version 1.0.0 \ --group-id com.example \ --artifact-id maven-cli-push-test \ --extra-files build-script.sh,random-file.xml \ --extra-files readme.md ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). We'll supplement these with more detailed guidance later, but otherwise just ask, we're here to help. *** ## Download / Install a Package ### Setup To enable the retrieval of Cloudsmith hosted packages via Maven, the first step is to add your repository to the [dependencyManagement](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management) section of your `pom.xml` file. To do this add one of the following XML examples to your project `pom.xml` file: *** #### Public Repositories ```xml NAME https://dl.cloudsmith.io/public/OWNER/REPOSITORY/maven/ true always true always ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```xml Entitlement Token Authentication NAME https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/maven/ true always true always ``` ```xml HTTP Basic Authentication NAME https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/maven/ true always true always ``` If using HTTP Basic Authentication, you need to provide one following three types of credentials: - Cloudsmith Username and Password - Cloudsmith API Key - An Entitlement Token When using HTTP Basic Authentication you'll probably want to keep your credentials separately in your `settings.xml` file instead of within the `pom.xml` file. once you have decided which credentials you wish to use, setup your `settings.xml` file as follows: ```xml Username & Password NAME USERNAME PASSWORD ``` ```xml API Key NAME USERNAME API-KEY ``` ```xml Entitlement Token NAME token TOKEN ``` **Credential Encryption** We would highly advise that you encrypt your credentials using something like `mvn encrypt-password`, of which you can refer to the mini [encryption guide](http://maven.apache.org/guides/mini/guide-encryption.html) for more detailed help. For more details on authentication in Maven, please refer to the official [Maven documentation](https://maven.apache.org/settings.html). ### Specifying Dependencies After the repository is added to the `pom.xml` file, and credentials are added to the `settings.xml` file (if using HTTP Basic Authentication), all that is left is to specify the dependency in the dependencies section of the project pom.xml file. To do this add the following XML to your project `pom.xml` file: ```xml GROUP_ID ARTIFACT_ID PACKAGE_VERSION ``` *** ### Install a Package To download all the dependencies specified in your `pom.xml` file and build your project you just need to run: ```shell mvn install ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Maven repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. If you’ve configured your Maven upstream for caching, Cloudsmith will cache all files fetched from your upstream, including arbitrary files. Our supported upstreams for arbitrary file support are: - Maven Central - Artifactory - Gradle - Nexus - WSO2/JBOSS - Google - Confluent - Shibbolenth - Atlassian Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Index Packages ## Troubleshooting **Q. Maven maintains plain text passwords in settings.xml and credential files. How do we prevent exposure of such passwords?** The Maven tooling chain supports encrypted credentials. You can also use interpolation from environment variables instead so that you don't store them in configuration files. We also support authentication by de-privileged [entitlement tokens](/software-distribution/entitlement-tokens) (i.e. read-only specific access) to minimise exposure. *** **Q. Why does Cloudsmith keep all snapshot versions of a .jar, and not only the last one? Usually, we only want the last version.** Mainly, it is because some people do it one way, and others do it another. Some people utilise or want to keep the chain of snapshots, so we keep them to facilitate those customers. For others, they'll need to delete them manually (or automatically with a retention policy) *** **Q. My uploaded Maven artifact version was renamed from `X.X.X-SNAPSHOT` to `X.X.X-20190815.205452-1`. Can I set it so that it doesn’t rename?** Snapshots are handled differently from release artifacts because there can be many snapshots of the same version, which is why it got renamed. If you uploaded `1.0.0-SNAPSHOT` multiple times, it would let you, and it would sync as `1.0.0--1` , then `1.0.0.--2` and so on. It isn't possible to disable this behaviour. However, despite what the UI says, you should still be able to refer to your package in dependencies as `1.0.0-SNAPSHOT` and it will pick up the latest version of `1.0.0-SNAPSHOT` uploaded. This is pretty standard for Maven-based repositories but might be surprising unless you know about it. ## Still Need Help? Contact us [here](https://support.cloudsmith.com). We're always happy to help. # npm Registry Cloudsmith provides public & private registries for npm (JavaScript) npm is an extremely popular package manager for [JavaScript](https://en.wikipedia.org/wiki/JavaScript), and is used for creating and using packaged [Node.js](https://en.wikipedia.org/wiki/Node.js) modules. A public index of packages is available from npm, inc. on [npmjs.org](https://npmjs.org). npm, Inc. also develop and maintain the official `npm` client, ecosystem and tooling. For more information on npm, please see: - [npm](https://npmjs.org): The official website for npm Cloudsmith is proud to support fully-featured registries for managing your own private and public npm packages. We provide a high-level of compatibility with the official npmjs API meaning you can use the [official CLI client](https://docs.npmjs.com/cli/npm) - `npm` - for installing, managing, and publishing npm packages to Cloudsmith. Or if you prefer you can use the Cloudsmith UI, API or CLI - `cloudsmith`. The Cloudsmith npm registry has been fully tested with the following: - `npm` CLI version: v6.4.1 - `node` version: v6.11.3 - `yarn` version: v1.9.4 It is likely that it will work for other environments, including older and more recent versions. If you encounter any issues please [let us know](https://cloudsmith.com/company/contact-us). **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | | TAG | The name of an optional npm distribution tag | # Upload a Package ## Publish via npm You can publish your npm packages to a Cloudsmith-based npm registry via the native npm tooling. ### Setup The endpoint for the native npm API is: ``` https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` To authenticate for native publishing, you can either use `npm login` or create an `npmrc` file (in your **$HOME** or in the project directory) **Authentication types from npm v9** npm Version 9 introduced a change which consolidated the legacy authentication types. Please see the official NPM release notes **[here](https://github.blog/changelog/2022-10-24-npm-v9-0-0-released/)** for further details. Use `npm login`: ```shell NPM \< v9 npm login --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ Username: USERNAME Password: API-KEY Email: YOUR-EMAIL-ADDRESS ``` ```shell NPM v9 + npm login --auth-type=legacy --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ Username: USERNAME Password: API-KEY Email: YOUR-EMAIL-ADDRESS ``` Or create an `.npmrc` file with the following: ``` registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:_authToken=API-KEY ``` ### Publish Once you have set up the registry, you can then publish from your project directory using `npm publish`: ```shell npm publish --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` For more details on `npm publish` see the official [npm documentation](https://docs.npmjs.com/cli/publish). (external link) ## Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). To upload via the Cloudsmith CLI / API, you'll need to generate your package first. You can do this with: ```shell npm pack ``` This will generate a tarball file (.tgz) like `your-package-1.2.3.tgz` that you can upload. **Required package.json** This assumes that you've created a `package.json` file for your project. Please see the official npmjs [package.json reference](https://docs.npmjs.com/files/package.json) (external link) for more information. The command to upload an npm package via the Cloudsmith CLI is: ```shell cloudsmith push npm OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.tgz ``` Example: ```shell cloudsmith push npm your-account/your-repo cloudsmithjs-1.0.0.tgz ``` ## Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload#upload-via-website-ui) for details of how to upload via the Cloudsmith web app. ## Scoped Packages Using a package scope provides a different namespace to other similarly named packages to differentiate them. To scope packages when publishing, add the scope to the name in your `package.json` file: ```json { "name": "@SCOPE/PACKAGE-NAME" } ``` **@SCOPE** Replace `@SCOPE` with your own scope name You can then [publish the package](/formats/npm-registry#upload-a-package) directly using `npm publish`, or generate the package with `npm pack` and then push via the Cloudsmith CLI or Cloudsmith web app You can find further information in the npm documentation on [scoped packages](https://docs.npmjs.com/misc/scope) (external link) ## Distribution Tags Distribution tags allow npm packages to be tagged with a mnemonic that is associated with a specific package version. Cloudsmith has full support for distribution tags and (mostly) follows the same rules for them as on npmjs.com: 1. A specific tag can point at one version of a package only. 2. A package version may have multiple unique tags. 3. Unless specified otherwise, the default tag for the last package published is latest. 4. When a package that is **latest** is deleted, the tag is moved to the next applicable version by [semver](https://semver.npmjs.com/). 5. When a package is copied/moved to another repository, its tags are carried with it. 6. If the **latest** package is moved/deleted, then existing packages are sorted via SemVer to determine the next latest. You can inspect a package to see what tags it has: ```shell npm dist-tags ls PACKAGE_NAME --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` You can add tags to a package: ```shell npm dist-tags add PACKAGE_NAME@PACKAGE_VERSION TAG - -registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` You can also remove tags from a package: ```shell npm dist-tags rm PACKAGE_NAME TAG --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` You can find out more about [distribution tags](https://docs.npmjs.com/adding-dist-tags-to-packages) in the npm documentation (external link). # Download / Install a Package ## Setup You can configure npm to use a Cloudsmith-based npm registry in one of two ways: 1. Specify the registry per user, as the global default or per-project 2. Provide the registry URL when executing `npm` commands ### Public Registries To use/set the registry as the default for your user, execute the following: ```shell npm config set registry https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` You can set it globally (with permissions) by using the `-g` argument. **Global configuration** Setting the registry globally will impact **all** npm commands unless they explicitly override the registry. You can also add the registry directly to your user or project `.npmrc` file as follows: ```text registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` Alternatively, you can specify the registry each time you execute `npm` commands, such as: ```shell npm install PACKAGE_NAME --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` ### Private Registries **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To set up the registry with authentication, add the one of the following to your user or project `.npmrc` file: ```text Entitlement Token Auth registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:_authToken=TOKEN ``` ```shell HTTP Basic Auth (User & Pass) registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:username=USERNAME //npm.cloudsmith.io/OWNER/REPOSITORY/:_password=PASSWORD ``` ```shell HTTP Basic Auth (API-Key) registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:_authToken=API-KEY ``` ```shell HTTP Basic Auth (Token) registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:username=token //npm.cloudsmith.io/OWNER/REPOSITORY/:_password=TOKEN ``` **base64 Encoding** If using HTTP Basic Authentication with your Cloudsmith username and password or with a Cloudsmith Entitlement Token, you must encode your password or token in base64 ## Install a Package Once you have an authentication method configured, you can then install packages using: ```shell npm install PACKAGE_NAME ``` If you have added tags to the package, then these tags can be used as an alternative to the package version when installing packages, such as: ```shell npm install PACKAGE_NAME@TAG --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` ### Scoped Registries Using a registry scope tells npm to route installs for packages in that scope to Cloudsmith. You can set it via the command-line using: ```shell npm config set '@SCOPE:registry' https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` You can also set it directly in your user or project `.npmrc` file: ``` @SCOPE:registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` Installing packages with a scope then requires putting the scope before the name: ```shell npm install @SCOPE/PACKAGE_NAME ``` ### Using Yarn with Cloudsmith To ensure Yarn compatibility, enable **always-auth** by adding it to your user or project `.npmrc` or `.yarnrc.yml` file as shared below. 🔹 Yarn v1 Yarn v1 uses the same `.npmrc` format as npm: ```ini registry=https://npm.cloudsmith.io/ORG/REPO/ //npm.cloudsmith.io/ORG/REPO/:_authToken=KEY always-auth=true ``` 🔹 Yarn v2 and later (Berry) Yarn 2+ (*also known as Berry*) manages registries through `.yarnrc.yml` instead of `.npmrc`: ```yaml npmRegistryServer: "https://npm.cloudsmith.io/ORG/REPO/" npmAlwaysAuth: true npmAuthIdent: "AUTHKEY" ``` The Auth key should be in base64, which you can get by running: ```shell echo -n 'User:APIKEY' | base64 ``` ## Upstream Proxying / Caching Configurable Proxying Caching. Please see [Upstream Proxying](/repositories/upstreams) for more details. Upstreams provide a way to blend multiple external repositories into a single repository. This allows your single Cloudsmith repository to serve packages from multiple 'upstream' repositories. ## Security Auditing Cloudsmith supports proxying of npm audit requests to detect vulnerabilities in dependencies, you just need to execute: ```shell npm audit ``` **Authentication** To use `npm audit`, you must authenticate using a Cloudsmith [API-Key](/developer-tools/cli#getting-your-api-key), not an [Entitlement Token](/software-distribution/entitlement-tokens). You can find out more about [security auditing](https://docs.npmjs.com/getting-started/running-a-security-audit) in the npm documentation (external link). # Current Limitations The Cloudsmith npm registry implementation has the following limitations: - The maximum size per-package file is limited to 100MiB (100 megabytes), but only when utilising the native npm-cli for publishing. If uploading using the cloudsmith-cli, then the absolute maximum size per-package file limit will be the standard 5GiB. Cloudsmith is unlikely to support the following (out-of-scope): - Profile, user, team or org commands; use the `cloudsmith-cli` instead. - `npm access` and visibility; packages follow repository visibility. - Viewing and changing collaborators of packages via 'npm owner'. - Creating tokens via `npm token`; use the `cloudsmith-cli` or UI instead. - Changes stream to implement followers; webhooks are a functional alternative. Additionally, any search terms used are not passed to upstream repositories and are handled by Cloudsmith. Search results from upstream repositories are not blended into results; only Cloudsmith results are shown. ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream NPM repositories you wish to use as additional package sources for your Cloudsmith repository. You can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # NuGet Feed Cloudsmith provides public & private feeds for NuGet. NuGet is an open-source package manager designed for the Microsoft development technologies. The NuGet repository support at Cloudsmith is compatible with [Chocolatey](https://chocolatey.org/), so if you're looking to manage packages on Windows, that's our recommended approach. **NuGet Symbol Server** Visit the [NuGet Symbol Server](/formats/nuget-symbol-server) documentation page for more information about NuGet debugging symbols (PDB files) and source files for packages published. For more information on NuGet, please see: - [NuGet](https://www.nuget.org): The official website for NuGet - [NuGet Packages](https://www.nuget.org/packages): The official package repository guide for NuGet - [Introduction to NuGet](https://docs.microsoft.com/en-us/nuget/what-is-nuget): Official guide to getting started with NuGet **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package Before you can upload, you need to create your NuGet package using the NuGet CLI or the .NET Core CLI: **Required project file** This assumes that you've created a compatible `project.csproj` file for your project. ```text NuGet CLI nuget pack ``` ```text .NET Core CLI dotnet pack ``` This generates a nupkg (`.nupkg`) file like `your-package-1.2.3.nupkg` that you can upload. ### Upload via native NuGet Tooling The endpoint for the native NuGet API is: ``` https://nuget.cloudsmith.io/OWNER/REPOSITORY/ ``` You can upload your package using the NuGet CLI or the .NET Core CLI. **NuGet CLI** You can publish a nupkg file that you've generated from your project, using [nuget](https://www.nuget.org/downloads). As a shortcut, you can set up the source (upstream) ahead of time, using `nuget source`: ```shell nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` Then you can publish your package using: ```shell nuget push PACKAGE_NAME-PACKAGE_VERSION.nupkg -Source example-repo -ApiKey API-KEY ``` **.NET Core CLI** You can publish a nupkg file that you've generated from your project, using [dotnet](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet?tabs=netcore21): ```shell dotnet nuget push PACKAGE_NAME-VERSION.nupkg -k API-KEY -s https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a NuGet package via the Cloudsmith CLI is: ```shell cloudsmith push nuget OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.nupkg ``` Example: ```shell cloudsmith push nuget your-account/your-repo your-package-1.0.0.nupkg ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). We'll supplement these with more detailed guidance later, but otherwise just ask, we're here to help. ## Download / Install a Package ### Setup To consume packages in NuGet from a Cloudsmith NuGet Feed , you'll need to configure it as a source: #### Public Repositories **NuGet CLI** ```shell nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` **.NET Core CLI** When specifying the source in commands via `-s`, use the following URL: ``` https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` **Paket CLI** You can add the source to your `paket.dependencies` file: ``` source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` If you don't want to check your username into source control, you can use environment variables instead **VS Package Manager (PM)** When specifying the source in commands via `-Source`, use the following URL: ``` https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. **NuGet CLI** ```shell Entitlement Token Auth nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json -Username token -Password TOKEN -StorePasswordInClearText ``` ```shell HTTP Basic Auth (User & Pass) nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json -Username USERNAME -Password PASSWORD -StorePasswordInClearText ``` ```shell HTTP Basic Auth (API-Key) nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json -Username USERNAME -Password API-KEY -StorePasswordInClearText ``` ```shell HTTP Basic Auth (Token) nuget sources add -Name example-repo -Source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json -Username token -Password TOKEN -StorePasswordInClearText ``` **.NET Core CLI** When specifying the source in commands via `-s`, use the following URL: ```shell Entitlement Token Auth https://token:TOKEN@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (User & Pass) https://USERNAME:PASSWORD@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (API-Key) https://USERNAME:API-KEY@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (Token) https://token:TOKEN@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` **Paket CLI** You can add the source to your `paket.dependencies` file: ```shell Entitlement Token Auth source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json username: "token" password: "TOKEN" authtype: "basic" ``` ```shell HTTP Basic Auth (User & Pass) source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json username: "USERNAME" password: "PASSWORD" authtype: "basic" ``` ```shell HTTP Basic Auth (API-Key) source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json username: "USERNAME" password: "API-KEY" authtype: "basic" ``` ```shell HTTP Basic Auth (Token) source https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json username: "token" password: "TOKEN" authtype: "basic" ``` If you don't want to check your username into source control, you can use environment variables instead. **VS Package Manager (PM)** When specifying the source in commands via `-Source`, use the following URL: ```shell Entitlement Token Auth https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (User & Pass) https://USERNAME:PASSWORD@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (API-Key) https://USERNAME:API-KEY@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```shell HTTP Basic Auth (Token) https://token:TOKEN@nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ### Install a Package To install the latest version of a package you would use: **NuGet CLI** ```shell nuget install PACKAGE_NAME -Source example-repo -DependencyVersion Highest ``` **.NET Core CLI** ``` dotnet add package PACKAGE_NAME -s https://nuget.cloudsmith.io/OWNER/REPOSITOY/v3/index.json ``` **Paket CLI** ``` paket add nuget PACKAGE_NAME ``` **VS Package Manager (PM)** ``` Install-Package PACKAGE_NAME -Source example-repo ``` ## Current Limitations The Cloudsmith NuGet feed implementation has the following limitations: - The maximum size per-package file is limited to 200MiB (~210 megabytes), but only when utilising the native nuget-cli for publishing. If uploading using the cloudsmith-cli, then the absolute maximum size per-package file limit will be the standard 5GiB. ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream NuGet feeds that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Signing NuGet Packages Cloudsmith supports natively signing all NuGet packages using an X.509 certificate, enabling consumers to verify package repository signatures in native tooling or the NuGet CLI command, ensuring the integrity and authenticity of the packages. To get started, navigate to Repository Settings >> Miscellaneous, and check the box `"NuGet Native Signing Enabled?"`: **Signing of existing packages** Packages uploaded prior to enabling NuGet Native Signing will not be signed or have a certificate in the index. Once this setting is enabled, you will need to resync existing NuGet packages to sign them. ### How it works When native NuGet signing is enabled for a Cloudsmith repository, a unique X.509 certificate is issued for that repository. When a NuGet package is uploaded or resynced to that repository, Cloudsmith will create a repository signature. The certificate will be available in the `RepositorySignatures` resource in the [service index](https://learn.microsoft.com/en-us/nuget/api/service-index). If a NuGet package contains an author signature, Cloudsmith will countersign the package. If the repository upstream NuGet repositories configured, Cloudsmith will index the RepositorySignature endpoint from the Nuget service index. The upstream repository's signing certificates will then be available for client-side verification as well. #### Client-side verification To enable client-side verification, the trusted certificates need to be added to the consumer's machine. Cloudsmith issues signing certificates using our own Certificate Authority. The Certificate Authority chain will need to be added to NuGet's trusted roots bundle. #### Linux and MacOS For Linux and MacOS, this is located at: `/usr/local/share/dotnet/sdk/\/trustedroots/codesignctl.pem` #### Windows For Windows, this is managed by the operating system. Please reference Microsoft's [Trusted Root Certification Authorities Certificate Store](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/trusted-root-certification-authorities-certificate-store) for more information. You can download Cloudsmith’s root Certificate Authority chain by going to the Key Management tab in your repository or by calling the [x509-rsa API endpoint](/api/repos/x509/rsa/list). ## Troubleshooting **Q. I'm unable to authenticate using HTTP Basic Authentication with a password or with an entitlement token. I'm using dotnet core SDK 2.2** This is the error: ```text Unable to get repository signature information for source https://nuget.cloudsmith.io/YOUR-ACCOUNT/YOUR-REPO/v3-index/repository-signatures/X.X.X/index.json error : Response status code does not indicate success: 401 (Unauthorized) ``` *** It looks like this (not being able to use username/password directly on a source) might be an accepted bug in dotnet 2.2, possibly fixed later, as referenced in the following GitHub issues: - https://github.com/dotnet/cli/issues/10356 - https://github.com/NuGet/Home/issues/4668 **SOLUTION** Add the source using `nuget sources` as follows first:: ``` nuget sources add -Name NAME -source https://nuget.cloudsmith.io/YOUR-ACCOUNT/YOUR-REPO/v3/index.json -Username YOUR-USERNAME -Password YOUR-API-KEY-OR-PASSWORD -StorePasswordInClearText ``` Then do `dotnet restore` like: ``` dotnet restore -s https://nuget.cloudsmith.io/YOUR-ACCOUNT/YOUR-REPO/v3/index.json --configfile ~/config/NuGet/NuGet.Config ``` If you want to use an Entitlement Token instead of a username and password/API-Key, use "token" for the username and an Entitlement Token as the password. ## Still Need Help? Contact us [here](https://support.cloudsmith.com). We're always happy to help. # NuGet Symbol Server Cloudsmith provides public & private symbol server support for NuGet. The NuGet Symbol Server is a repository that stores and serves debugging symbols (PDB files) and source files for packages published. By using the Symbol Server, developers can step through and debug their code more easily, even when working with compiled libraries or packages, as it provides access to the necessary debugging information. For more information on Symbol Server, please see: - [NuGet](https://docs.microsoft.com/en-us/nuget/): The official website for NuGet Documentation - [Symbols for Windows Debugging](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/symbols): Official documentation on Symbols for windows - [Symbols in the Visual Studio Debugger](https://docs.microsoft.com/en-us/visualstudio/debugger/specify-symbol-dot-pdb-and-source-files-in-the-visual-studio-debugger): Official guide on configuring a Symbol Server on Visual Studio **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :----------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | ## Configuring Cloudsmith Repository as Symbol Server **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. Each individual repository in Cloudsmith can be configured as a Symbol Server in Visual Studio. Only symbols within that repository will be served back to the debugger. In Visual Studio, go to `Tools > Options` Inside Options, on the menu to the left navigate to `Debugging > Symbols`, using the plus icon on the top-right, enter the URL of your repository, as follows: ```text Nuget - Symbol Server URL https://nuget.cloudsmith.io/OWNER/REPOSITORY/download/symbols ``` [Image: npm Proxy Settings] If your repository is private, at this stage you might be asked for credentials, you can use either an entitlement token or your personal api key. ```text Entitlement Token Username: token Password: TOKEN ``` ```text Api Key Username: USERNAME Password: API-KEY ``` By default the debugger will only display and step into user code only, ignoring system code and other code that is optimized or does not have debugging symbols available locally. To allow Visual Studio to go fetch PDB symbols for imported nuget packages, the `Just My Code` option has to be disabled. For this in the same Options screen, navigate to `Debugging > General` and disable `Just My Code` [Image: npm Proxy Settings] You can now start a debug session, the first time you do it Visual Studio, if your repository is private, it will prompt you for credentials: [Image: npm Proxy Settings] You can use either an entitlement token or your personal api key. ```text Entitlement Token Username: token Password: TOKEN ``` ```text Api Key Username: USERNAME Password: API-KEY ``` ## Troubleshooting ### I have snupkg symbols in my repository but Visual Studio can't find them If you had `snupkg` symbols uploaded from before this feature was released, it's possible that the `pdb` files are not indexed. Check in the UI in the `Files` tab if you see `pdb` files attached to your package If you don't see one or more `pdb` files here, then do a `Resync` of your package to force a re-index. If you do see `pdb` files then it's possible that Visual Studio is configured to not load the module you want to debug. To fix this in Visual Studio, head to `Tools > Options`, and navigate to `Debugging > Symbols`, on the bottom of the screen there's an option titled `Automatic symbol loading preference` Set this to `Load all modules, unless excluded` # OCI Repository Cloudsmith provides public & private registries for OCI artifacts [The OpenContainer Initiative (OCI)](https://opencontainers.org/) is a lightweight, open governance structure project, formed under the auspices of the Linux Foundation, for the express purpose of creating open industry standards around container formats and runtimes. OCI provides the concept of the _registry_ as a storage service for storing and distributing artifacts for use. Cloudsmith is OCI-compliant and provides a fully-fledged registry with full OCI v1.1 compatibility. With Cloudsmith you'll be able to push, pull, inspect and manage OCI artifacts, privately and publicly. All of this is provided with the standard functionality and features that are offered in the Cloudsmith platform, such as collaboration, advanced permissioning, white-labelled distribution, multi-tenancy with other packaging formats, etc. For more information on OCI, please see: - [OCI](https://opencontainers.org/): The official website for the OpenContainer Initiative. - [OCI Distribution Spec](https://github.com/opencontainers/distribution-spec): The official Github repository containing the distribution specification. - [OCI Article](https://en.wikipedia.org/wiki/Open_Container_Initiative): The Wikipedia article on OCI. ## Differences from the OCI Image and Distribution Specification For clarity, it's important to note some of the differences between a registry such as Docker Hub, and a Cloudsmith OCI registry. These are both naming and functional in nature. ### Naming Differences Docker defines the following names (this is not the official wording): - **Layer**: A blob (big object of bytes) containing software and configuration. - **Image**: A collection of Docker layers plus metadata that represent an application. - **Container**: A running instance of a Docker image in-memory. - **Repository**: A collection of Docker images, separated by hashref and version tags. - **Registry**: A collection of all Docker repositories, separated by namespaces. For comparison purposes, where terms differ from Cloudsmith: - **Package**: A specific identifiable and versionable artifact. - **Repository**: A collection of versionable artifacts, with multiple allowed per account. Therefore, based on the above, the following terms are equivalent: | OCI Term | Cloudsmith Term | | :------- | :-------------- | | Image | Package | | Registry | Repository | For consistency, the terms will be used within all of the OCI-related documentation but please be aware of the differences if looking at documentation elsewhere. ## Upload an Generic Artifact with ORAS **Docker Custom Domain** If you have added a [Custom Domain](/workspaces/custom-domains) for Docker, you must use it to authenticate and push. Please replace `docker.cloudsmith.io` in the following instructions with the Docker custom domain you have created. [ORAS (OCI Registry As Storage)](https://oras.land/) is an open-source tool that enables users to push, pull, and manage non-container artifacts in OCI-compliant registries. It extends the OCI specification beyond container images, turning registries into a general-purpose storage system for diverse artifact types. ### Setup **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code or expose them in any logs You need to authenticate via `oras login` to push/pull artifacts: ```shell HTTP Basic Auth (API-Key) oras login docker.cloudsmith.io Username: USERNAME Password: API-KEY Login Succeeded ``` ### Push an artifact After authentication you can push the artifact by doing: ```shell Shell $ oras push docker.cloudsmith.io/OWNER/REPOSITORY/ARTIFACT_NAME:TAG ./FILENAME_PATH ``` ORAS allows you to specify the type of the artifact by using the `--artifact-type` parameter, you can use it in the following way: ```shell Shell $ oras push docker.cloudsmith.io/OWNER/REPOSITORY/ARTIFACT_NAME:TAG --artifact-type application/vnd.cloudsmith.v1 ./FILENAME_PATH ``` For more information on how to define a unique artifact type check the [official documentation](https://github.com/opencontainers/artifacts/blob/main/artifact-authors.md#defining-a-unique-artifact-type). ## Attach file to an existing Artifact **Warning** The `attach` [command](https://oras.land/docs/commands/oras_attach) in ORAS is in preview and still under development As Cloudsmith is fully OCI v1.1 compliant, this includes support for the Referrers API. Referrers allow artifacts to reference related objects, enabling a parent artifact (like a container image) to have associated metadata or attachments (e.g., SBOMs, signatures, or provenance files). This is critical for securely and efficiently managing relationships between artifacts, such as attaching additional context or verifying authenticity without modifying the original artifact. To attach a file to an existing artifact, execute: ```shell Shell $ oras attach --artifact-type application/vnd.cloudsmith.attachment.v1 docker.cloudsmith.io/OWNER/REPOSITORY/ARTIFACT_NAME:TAG ./FILE_TO_ATTACH ``` ## Download / Pull an Generic Artifact with ORAS ### Setup #### Public Registries For public registries, no further setup is needed as authentication is not required. #### Private Registries **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. You need to authenticate via `oras login` to pull images: ```shell Entitlement Token Auth oras login docker.cloudsmith.io Username: token Password: TOKEN Login Succeeded ``` ```shell HTTP Basic Auth (API-Key) oras login docker.cloudsmith.io Username: USERNAME Password: API-KEY Login Succeeded ``` ### Pull an Artifact Pulling (downloading) an artifact from the Cloudsmith OCI registry can be done using the standard `oras pull` command: ```shell $ oras pull docker.cloudsmith.io/OWNER/REPOSITORY/ARTIFACT_NAME:TAG ``` ## Helm v3 charts via OCI Starting Helm v3.8.0 OCI-compliant registries can now be used to store and share chart packages. For full detail see [Helm Docs](https://helm.sh/docs/topics/registries/) . 🚧 **Early Access Feature** While Cloudsmith OCI Registries are GA, our Helm OCI registry is in Early Access, so some Cloudsmith features like Helm OCI Upstreams aren't yet available. ### Setup Before you can push your chart you need to login to Cloudsmith's registry. ```shell Entitlement Token Auth helm registry login helm.oci.cloudsmith.io -u OWNER/REPOSITORY -p TOKEN ``` ```shell HTTP Basic Auth (User & Pass) helm registry login helm.oci.cloudsmith.io -u USERNAME -p PASSWORD ``` ```shell HTTP Basic Auth (API-Key) helm registry login helm.oci.cloudsmith.io -u USERNAME -p API-KEY ``` With public repositories, you don't need any sort of authentication and you can jump straight to the Installing a Chart section below. ### Push your chart The command to upload a Helm Chart via OCI using the Helm CLI is: ```shell helm push CHART_NAME-CHART_VERSION.tgz oci://helm.oci.cloudsmith.io/OWNER/REPOSITORY ``` ### Installing a Chart To install/use a specific version of a chart: ```shell helm install oci://docker.cloudsmith.io/OWNER/REPOSITORY/CHART_NAME --version CHART_VERSION ``` To install the latest version of a chart: ```shell helm install oci://docker.cloudsmith.io/OWNER/REPOSITORY/CHART_NAME ``` Or you can upgrade to the most recent version of a chart if you've already installed: ```shell helm upgrade oci://docker.cloudsmith.io/OWNER/REPOSITORY/CHART_NAME ``` If you've got a `requirements.yaml` file in your chart, you can specify this as a dependency: ```yaml dependencies: - name: CHART_NAME version: CHART_VERSION repository: NAME ``` ### Logging out of Registry Helm provides a command to logout of OCI-based registries ```shell helm registry logout docker.cloudsmith.io ``` ## Provenance Provenance files allow for verification of both the integrity and source of a Helm chart. 🚧 Provenance files aren't supported for OCI chart uploads yet, but this feature is coming soon. Cloudsmith fully supports both verification and generation of Helm provenance files for non-OCI Helm Charts. ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream OCI registries you wish to use by using our Docker Upstream feature. In addition, you can also choose to cache any requested artifacts for future use. ## Key Signing Support RSA Index ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # PowerShell Modules Repository Cloudsmith provides public & private repositories for PowerShell Modules A PowerShell module is a package that contains PowerShell members, such as cmdlets, providers, functions, workflows, variables, and aliases. The members of this package can be implemented in a PowerShell script, a compiled DLL, or a combination of both. For more information on PowerShell modules, please see: - [PowerShell Documentation](https://docs.microsoft.com/en-gb/powershell/): The official website for PowerShell - [PowerShell Modules Documentation](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_modules?view=powershell-7.1): The official documentation for PowerShell Modules - [PowerShell Gallery](https://www.powershellgallery.com/): Central Repository for public PowerShell Modules - [PowerShell Module Browser](https://docs.microsoft.com/en-us/powershell/module/): Browser for Microsoft PowerShell Modules **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | MODULE_NAME | The name of your module | | MODULE_VERSION | The version number of your module | *** PowerShell modules use the NuGet package format, and Cloudsmith NuGet Feeds are fully compatible as a PowerShell Module Repository. ## Upload a package The endpoint for the PowerShell Modules API is: ``` https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 ``` ### Create a PowerShell Module PowerShell Modules combine several PowerShell functions into a single reusable and easily sharable resource. You can put many files, such as .DLLs and tools, in a module but the absolute minimum is a `.psm1` file. #### PowerShell Module Manifest A PowerShell Module manifest is a PowerShell data file (`.psd1`) that contains information about the module like the version number and exported variables. **Manifests required** PowerShell Module manifests are required to publish a module to a repository. To create a PowerShell Module Manifest, use the `New-ModuleManifest` command: ```powershell New-ModuleManifest -Path TARGET_PATH -Author AUTHOR_NAME -Description "MODULE_DESCRIPTION" ``` ### Upload a package via native PowerShell Module tooling You can upload your package using `Publish-Module`. PowerShell requires that publish locations be explicitly added (with authentication credentials) via `Register-PSRepository` prior to publishing: ```powershell $credential = New-Object PSCredential('USERNAME',$(ConvertTo-SecureString 'API-KEY' -AsPlainText -Force)) ``` ```powershell Register-PSRepository -Name 'NAME' -SourceLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -PublishLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -InstallationPolicy 'trusted' -Credential $credential ``` Then you can publish a module with: ```powershell Publish-Module -Path 'path/to/Your.Module' -Repository 'NAME' -NugetApiKey 'API-KEY' ``` ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command-Line Interface](/developer-tools/cli) . The command to upload a PowerShell Module via the Cloudsmith CLI is: ```shell cloudsmith push nuget OWNER/REPOSITORY MODULE_NAME-MODULE_VERSION.nupkg ``` Example: ```shell cloudsmith push nuget demo/examples-repo your-module-1.2.3.nupkg ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload#upload-via-website-ui) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package ### Setup You can set up the source ahead of time, using ` Register-PackageSource` and `Register-PSRepository`: #### Public Repositories ```powershell Register-PackageSource -Name 'NAME' -Location 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -Trusted -ProviderName NuGet ``` ```powershell Register-PSRepository -Name 'NAME' -SourceLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -InstallationPolicy 'trusted' ``` For example: ```powershell Register-PackageSource -Name 'cloudsmith' -Location 'https://nuget.cloudsmith.io/demo/examples-repo/v2/' -Trusted -ProviderName NuGet ``` ```powershell Register-PSRepository -Name "cloudsmith" -SourceLocation "https://nuget.cloudsmith.io/demo/examples-repo-public/v2" -InstallationPolicy 'trusted' ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. In order to authenticate to a private Cloudsmith repository, you must first create a `PSCredential` object that you then use with `Register-PackageSource` and `Register-PSRepository`: ```powershell Entitlement Token $credential = New-Object PSCredential('token',$(ConvertTo-SecureString 'TOKEN' -AsPlainText -Force)) Register-PackageSource -Name 'NAME' -Location 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -Trusted -ProviderName NuGet -Credential $credential Register-PSRepository -Name 'NAME' -SourceLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -InstallationPolicy 'trusted' -Credential $credential ``` ```powershell Username & Password $credential = New-Object PSCredential('USERNAME',$(ConvertTo-SecureString 'PASSWORD' -AsPlainText -Force)) Register-PackageSource -Name 'NAME' -Location 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -Trusted -ProviderName NuGet -Credential $credential Register-PSRepository -Name 'NAME' -SourceLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -InstallationPolicy 'trusted' -Credential $credential ``` ```powershell Username & API-Key $credential = New-Object PSCredential('USERNAME',$(ConvertTo-SecureString 'API-KEY' -AsPlainText -Force)) Register-PackageSource -Name 'NAME' -Location 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -Trusted -ProviderName NuGet -Credential $credential Register-PSRepository -Name 'NAME' -SourceLocation 'https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2/' -InstallationPolicy 'trusted' -Credential $credential ``` ### Install a Package Then to install a package, you use `Install-Module` #### Public Repositories ```powershell Install-Module -Name "MODULE_NAME" -Repository NAME ``` For Example: ```powershell Install-Module -Name "MyModule" -Repository cloudsmith ``` #### Private Repositories ```powershell Install-Module -Name 'MODULE_NAME' -Repository NAME -Credential $credential ``` For Example: ```powershell Install-Module -Name "MyModule" -Repository cloudsmith -Credential $credential ``` # Python Repository Cloudsmith provides public & private repositories for Python packages Python is an awesome general-purpose programming language. Cloudsmith is proud to support fully-featured registries for managing your own public and private python packages. For more information on Python, please see: - [Python](https://python.org): The official website for Python - [PyPi](https://pypi.org): The Python Package Index **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | **UV Support** The main examples in this docs use `pip` or the Cloudsmith CLI. If you use [UV](https://docs.astral.sh/uv/) and wants to learn how to integrate it with Cloudsmith, jump to the [UV Support section](#uv-support). ## Upload a Package To upload, you need to generate your package first. You can do this with: ```shell python setup.py bdist_wheel --universal ``` This generates a wheel package file (`.whl`) like `your-package-1.2.3.whl` that you can upload. **setup.py required** This assumes that you've created a `setup.py` file for your project. Please see the [official PyPi](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py) packaging guide on how to create a `setup.py` for more information. There are also different types of distributions that you might be interested in, such as a source distribution, tarball distribution, etc. ### Upload via native Python tooling The endpoint for the native Python API is: ``` https://python.cloudsmith.io/OWNER/REPOSITORY/ ``` In order to authenticate for native publishing, you'll need to create a `.pypirc` file (in your `$HOME` or project directory), with the following: ```ini [distutils] index-servers = pypi cloudsmith [cloudsmith] repository: https://python.cloudsmith.io/OWNER/REPOSITORY/ username: USERNAME password: API-KEY ``` You can then publish from your project directory using [twine](https://pypi.org/project/twine/): ```shell twine upload -r cloudsmith dist/PACKAGE_NAME-PACKAGE_VERSION.whl ``` ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Python package via the Cloudsmith CLI is: ```shell cloudsmith push python OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.whl ``` Example: ```shell cloudsmith push python org/repo boto3-1.4.4.py2.p3-none-any.whl ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload#upload-via-website-ui) for details of how to upload via the Cloudsmith web app. ## Download / Install a package ### Setup You have a choice of 2 methods to set up your Cloudsmith repository: - Python set up via command line - Python set up via Pip #### Public Repositories ##### Set up via command line Tell pip the location of your Cloudsmith repository using the the `--index-url` option. ```shell pip install PACKAGE_NAME==PACKAGE_VERSION --index-url https://dl.cloudsmith.io/public/OWNER/REPOSITORY/python/simple/ ``` ##### Set up via Pip Similar to set up via command-line, pip needs to be passed the `--index-url` configuration option. To do this add `--index-url` to the top of your `requirements.txt` (or similar) file: ``` --index-url https://dl.cloudsmith.io/public/OWNER/REPOSITORY/python/simple/ PACKAGE_NAME==PACKAGE_VERSION ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ##### Set up via command line ```shell Entitlement Token Auth pip install PACKAGE_NAME==PACKAGE_VERSION --index-url https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/python/simple/ ``` ```shell HTTP Basic Auth (User & Pass) pip install PACKAGE_NAME==PACKAGE_VERSION --index-url https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ ``` ```shell HTTP Basic Auth (API-Key) pip install PACKAGE_NAME==PACKAGE_VERSION --index-url https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ ``` ```shell HTTP Basic Auth (Token) pip install PACKAGE_NAME==PACKAGE_VERSION --index-url https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ ``` ##### Set up via Pip Similar to set up via command-line, pip needs to be passed the `--index-url` configuration option. To do this add `--index-url` to the top of your `requirements.txt` (or similar) file: ```text Entitlement Token Auth --index-url https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/python/simple/ PACKAGE_NAME==PACKAGE_VERSION ``` ```text HTTP Basic Auth (User & Pass) --index-url https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ PACKAGE_NAME==PACKAGE_VERSION ``` ```text HTTP Basic Auth (API-Key) --index-url https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ PACKAGE_NAME==PACKAGE_VERSION ``` ```text HTTP Basic Auth (Token) --index-url https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/ PACKAGE_NAME==PACKAGE_VERSION ``` _**Private Repository Credential Security**_ As private repositories require authentication in order to access the repository content, when specifying a private repository in a `requirements.txt` file please bear in mind that the URL will contain the credentials (especially important if the `requirements.txt` file is shared.) You could choose to encrypt your `requirements.txt` file via something like [git-crypt](https://github.com/AGWA/git-crypt) (if you're using git or GitHub, for example). ### Removing Setup If you no longer want to install packages from the repository, remove the `--index-url` line from your `requirements.txt` file. ### Extra index url When using pip to access your packages, there are two parameter options to ensure pip searches your repository - they are [`--index-url`](https://pip.pypa.io/en/stable/cli/pip_wheel/?highlight=--index-url#cmdoption-i) and [`--extra-index-url`](https://pip.pypa.io/en/stable/cli/pip_wheel/?highlight=--index-url#cmdoption-extra-index-url). There is an important distinction to be made between these parameters, especially from a security perspective. Specifying `--index-url` will override pip's default repository and _only_ search the specified repository. This is the recommended approach from Cloudsmith. This improves your security posture as it reduces the risk of malicious public packages being installed in place of your own. **Dependency confusion** An attack known as [dependency confusion](https://cloudsmith.com/blog/dependency-confusion-attacks): where an attacker can claim the package on the public repository in a way that will ensure it gets chosen over the private package. If you still wish to access third-party repositories, like `pypi.org`, paid-for Cloudsmith plans include upstream proxying. This allows your repository to call out to other python repositories whenever a package cannot be found in your repository. See the Upstream Proxying section below. If your Cloudsmith plan does not include upstream proxying and some of your dependencies live outside your Cloudsmith repository, then you can also also specify extra index urls to pip. This is done by specifying `--extra-index-url`. When pip is supplied with extra index urls, it has a _list_ of repositories it searches for packages (the extra urls plus the index url). Note, _this list is not ordered_. All repositories are considered equal and pip will simply search for the _best_ package match according to [PEP 440](https://peps.python.org/pep-0440/). Using the `--extra-index-url` increases your exposure to dependency confusion attacks. See [pip install](https://pip.pypa.io/en/stable/cli/pip_install/#pip-install) for more information. **Indexing external repositories** To search your Cloudsmith repository for packages use the `--index-url` Pip configuration argument. Using the `--index-url` configuration option will force pip to search only the Cloudsmith repository and will result in pip not being able to install public (PyPi) packages that your private package may depend on. This reduces your exposure to dependency confusion attacks. ## UV Support `uv` is quickly gaining traction as a Python package and project manager. It stands out for its ultra-fast dependency resolution, native lock-file support (which enables reproducible builds, as we'll discuss later), and a built-in publisher. In the following sections, you'll learn how to install and publish your Python packages with uv in your private Cloudsmith repositories. ### Project setup #### Requirements In case you don't have it yet, use `pip` to install `uv` and `build`. ```bash pip install uv --upgrade uv pip install build ``` #### Create project assets Let's start defining a new `pyproject.toml` file for your project. Replace `OWNER`, `REPOSITORY` with your own Cloudsmith Workspace and Repository names: ```toml [project] name = "sample-flask-app" version = "0.1.0" requires-python = ">=3.10" dependencies = [ "requests==2.32.3" ] [[tool.uv.index]] name = "cloudsmith" url = "https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/python/simple/" # optional, but recommended for publish: publish-url = "https://python.cloudsmith.io/OWNER/REPOSITORY/" ``` Then, define the next environmental variables for `uv` to perform basic HTTP Auth. Replace `API-KEY` with your user API Token: ```bash # install / resolution export UV_INDEX_CLOUDSMITH_USERNAME=token # literal string export UV_INDEX_CLOUDSMITH_PASSWORD=API-KEY # optional: forbid fallback to PyPI export UV_PIP_NO_INDEX=1 # publishing export UV_PUBLISH_USERNAME=token export UV_PUBLISH_PASSWORD= ``` `uv` will use them to authenticate against your Cloudsmith repository. Now, execute the next command to fetch and install all the required dependencies in your active virtual environment: ```bash # create a venv first: uv venv && source .venv/bin/activate # install dependencies uv pip install --system -r pyproject.toml ``` You'll find information about the dependencies being fetched: This will result in all required packages being fetched directly from your Cloudsmith repository (please note that an upstream needs to be configured). Once installed, just run the next command to build your wheel package and the source distribution (.tar.gz) file, including your Python "source code" and everything required to run it: ```bash python -m build && ls dist/ ``` Once those assets are ready, just publish both artifacts to Cloudsmith. Remember to specify your `cloudsmith` index as defined in the `[[tool.uv.index.name]]` field in your project definition: ```bash uv publish --index cloudsmith dist/* ``` Browse to your Cloudsmith repository to find your new application packages. **Integration with other CI/CD tooling** This example explain how to run it in your local machine, but instructions can be adjusted to run this process within any of the existing CI/CD tooling you might use: GitHub Actions, GitLab CI, Jenkins, TeamCity, etc. In this case, the only requirement is that the `uv` environmental variables are exported first, so you can push your artifacts into the private Cloudsmith repository. You can find a sample [GH Action workflow](https://github.com/ciaracarey/python-sample-vscode-flask-tutorial/blob/644b230e14bea3763b4c7b4a3f6500c1f664ee25/.github/workflows/python-app.yml) reproducing a similar workflow. ### Reproducible builds via `uv compile` A software compiling process is reproducible when it ensures that it will always output the same binary. `uv compile` allows you to lock the exact versions so the environment can be reproduced in later builds. Without locking, the versions of dependencies could change over time, when using a different tool, or across platforms. For example, to freeze the dependencies of the previous `pyproject.toml` project and generate a deterministic lock‑file, execute: ```bash uv pip compile pyproject.toml -o requirements.lock ``` This will generate a `requirements.lock` file like the following: ``` Resolved 5 packages in 581ms # This file was autogenerated by uv via the following command: # uv pip compile pyproject.toml -o requirements.lock certifi==2025.6.15 # via requests charset-normalizer==3.4.2 # via requests idna==3.10 # via requests requests==2.32.3 # via sample-flask-app (pyproject.toml) urllib3==2.5.0 # via requests ``` This file specifies the version for each of the packages used and the its source. Adding the `uv` dependency freezing capabilities to Cloudsmith Upstreams caching will allow you to guarantee that the software you distribute remain stable and reproducible. Once you have the file, you just need to use it to install your project and guarantee all remains unchanged: ```bash uv pip install --system --strict -r requirements.lock ``` To learn more about other dependency declaration formats supported, visit the [official documentation](https://docs.astral.sh/uv/pip/compile/#locking-environments). ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable ProxyingCaching Please see [Upstream Proxying](/repositories/upstreams) for more details. Upstreams provide a way to _blend_ multiple Python repositories into a single repository. This allows your single Cloudsmith repository to serve packages from multiple 'upstream' repositories (like PyPi.org, Artifactory, DevPi etc). Please note, blended upstreams can be a source of dependency confusion attacks. ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Raw Repository Cloudsmith provides public & private repositories for Raw files Cloudsmith supports "Raw" files. Any file, any extension, suitable for datasets, images or whatever you want to throw at it. You get all the benefits of a Cloudsmith repository such as CLI uploads for automation, fine-grained access controls and logging/statistics (among others) - but for any file type at all. **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). **Plans** RAW file support is not available on the Core (Free) Plan. Please see our [pricing page](https://cloudsmith.com/product/pricing) for full details of features included at each plan level. In the following examples: | Identifier | Description | | :--------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | FILENAME | The name of your file (i.e file.zip or file.txt etc) | | VERSION_NO | The optional version number for a package | | NAME | The optional package name | ## Upload a File **Package name and filename** When you upload raw package, you can specify a package name with `--name` via the CLI, or via the Web UI upload form. If you specify a name that exactly matches the package filename, this is treated as if a name was not specified and the resulting [download links](/formats/raw-repository#download-via-command-line) will follow the format for a package that was uploaded without a name. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli) The command to upload via the Cloudsmith CLI is: ```shell cloudsmith push raw OWNER/REPOSITORY FILENAME ``` Example: ```shell cloudsmith push raw your-account/your-repo file.zip ``` ### Upload via Cloudsmith web app Please see [upload a package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download a Package ### Download via Cloudsmith web app #### Public Repositories When logged into Cloudsmith via a Web Browser, there's a "Download" button that's located within the traffic light button dropdown that provides a link to download a raw package on the package details page: #### Private Repositories For downloading from a private repository via the Cloudsmith web app, you can use the "Use Package" button to download a raw package using the default Entitlement Token for the repository. You can also use the dropdown arrow within the pop-up window to pick a different way of authenticating for the download. ### Download via Command Line To download a Raw package, you'll need to fetch uploaded files using a well-crafted URL. You can find the specific URL for a raw package from the Cloudsmith CLI, with the `cloudsmith list packages` command and adding `-F pretty_json`. The package URL is listed in the JSON output as `cdn_url`: This URL is in the following formats; it varies based on if you uploaded the raw package with a package name, a version, both or none: #### Public Repositories ```text Package Version https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/versions/VERSION_NO/FILENAME ``` ```text Package Name and Version https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/names/NAME/versions/VERSION_NO/FILENAME ``` ```text Package Name https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/names/NAME/files/FILENAME ``` ```text None https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/files/FILENAME ``` Some example `curl` commands to download a raw package from a public repository: A package with no package name or version specified at upload ```shell curl -sLf -O 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/files/FILENAME' ``` A specific version of a package ```shell curl -sLf -O 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/versions/VERSION_NO/FILENAME' ``` The latest version of the package ```shell curl -sLf -O 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/versions/latest/FILENAME' ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. Raw Package uploaded with a version number: ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/versions/VERSION_NO/FILENAME ``` ```shell HTTP Basic Auth https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/versions/VERSION_NO/FILENAME' ``` Raw Package uploaded with a package name and version number: ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/names/NAME/versions/VERSION_NO/FILENAME ``` ```shell HTTP Basic Auth https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/names/NAME/versions/VERSION_NO/FILENAME' ``` Raw Package uploaded with a package name ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/names/NAME/files/FILENAME ``` ```shell HTTP Basic Auth https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/names/NAME/files/FILENAME ``` Raw Package uploaded without a package name or version number ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/files/FILENAME ``` ```shell HTTP Basic Auth https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/files/FILENAME ``` Some example `curl` commands to download a raw package from a private repository: A package with no package name or version specified at upload using HTTP basic authentication: ```shell curl -sLf -u "USERNAME:PASSWORD" -O 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/files/FILENAME' ``` A specific version of a package using Entitlement Token authentication: ```shell curl -sLf -O 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/versions/VERSION_NO/FILENAME' ``` ## HTML / JSON Indexes Cloudsmith supports serving Apache-style HTML and JSON formatted indexes of raw package files in the repository. You can enable index generation in the [Repository Settings](/repositories/repository-settings). When enabled, the HTML index is available at: #### Public Repositories ``` https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/ ``` #### Private Repositories ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/ ``` ```shell HTTP Basic Auth https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/ ``` The JSON index is available at: #### Public Repositories ``` https://dl.cloudsmith.io/public/OWNER/REPOSITORY/raw/index.json ``` #### Private Repositories ```shell Entitlement Token Auth https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/index.json ``` ```shell HTTP Basic Auth (User & Pass) https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/raw/index.json ``` *** ## Upstream Proxying / Caching Not Supported ## Key Signing Support GPG ### Viewing Package Signatures Package signatures for raw files are presented in three ways: - As a separate entry in the HTML index. - As an object keyed by `signature` on each file entry with the JSON index. - Directly, by appending `.asc` to a file URL. #### HTML Index HTML Indexes will output a separate `
  • ` entry containing the URL for the signature for each file. Additionally, the checksum is available under the `data-checksum-sha256` attribute on the element: #### JSON Index JSON indexes will attach the package signature URL and checksum values under the `checksum` key for each package object: ```json { "packages" : [ { "name" : "PACKAGE_NAME", "signature" : { "checksum_sha256" : "SIGNATURE_CHECKSUM", "url" : "https://dl.cloudsmith.io/TOKEN/OWNER/REPO/PACKAGE_IDENTIFIER/gpg.FILE_IDENTIFIER.asc" }, ... } ] } ``` #### File URL For convenience, each of the file URLs listed above in 'Download a Package' can also map to the signature, by appending `.asc` to the URL. For example: ```shell FILENAME="foo.zip" # Retrieve a file curl -sLf -O https://dl.cloudsmith.io/TOKEN/OWNER/REPO/raw/files/$FILENAME # Retrieve the signature for the file curl -sLf -O https://dl.cloudsmith.io/TOKEN/OWNER/REPO/raw/files/$FILENAME.asc ``` ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # RedHat Repository Cloudsmith provides public & private repositories for RPM packages (RedHat, Fedora, SUSE) RedHat is a popular Enterprise focused flavor of Linux. RedHat Package Manager (RPM) is a free and open source packaging format. For more information on RedHat, please see: - [RedHat](https://www.redhat.com): The official website for RedHat - [RPM](https://rpm.org): The official website for RedHat Package Manager - [RPM Documentation](https://rpm.org/documentation.html): The official docs for RPM **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :------------------------------------------------------------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | DISTRO | Your distribution (i.e el, fedora or opensuse). You can also use "any-distro" if your package is compatible with more than one distribution | | VERSION | Your version name (i.e 7, 29). You can also use "any-version" if your package is compatible with more than one version | | FINGERPRINT | The 8 Byte fingerprint of the Public GPG key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | | PACKAGE_ARCH | The architecture of your package (i.e x86_64) | ## Upload a Package To upload, you will need to generate your package first. We highly recommend [fpm](https://fpm.readthedocs.io/en/latest/) for simplifying this. With fpm, you can build a package from a directory that represents the layout on the target system installation using: ``` fpm -f -s dir -t rpm -n PACKAGE_NAME -v PACKAGE_VERSION` ``` This generates a RedHat package file (`.rpm`) like `your-package-1.2.3.rpm` that you can upload. **Existing Packages from other formats** You can also build a package from a tarball (`.tgz`) or even Python, Ruby or npm packages. In addition to that, you can provide additional information during the packaging, such as other metadata like packaging dependencies, authorship, etc. Please refer to the [official fpm documentation](https://fpm.readthedocs.io/en/latest/) for more information. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a RedHat package via the Cloudsmith CLI is: ```shell cloudsmith push rpm OWNER/REPOSITORY/DISTRO/VERSION PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH.rpm ``` Example: Upload a package for RedHat 5 ```shell cloudsmith push rpm org/repo/el/5 libxml2-2.9.4-2.el5.x86_64.rpm ``` Example: Upload a package for any RedHat version ```shell cloudsmith push rpm org/repo/el/any-version libxml2-2.9.4-2.x86_64.rpm ``` Example: Upload a package for any version of any distribution ```shell cloudsmith push rpm org/repo/any-distro/any-version libxml2-2.9.4-2.x86_64.rpm ``` ### Upload via Cloudsmith Website Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download/ Install a Package ### Setup You have a choice of 3 methods to setup your Cloudsmith repository: - Automatic configuration (**recommended**) - Force a specific distribution/release (if your system is compatible but not identical) - Manual configuration #### Public Repositories To install RPM packages from a public Cloudsmith repository, you can quickly set up the repository automatically: **Yum (RedHat, CentOS, Amazon), Dnf (Fedora) or Zypper (Suse)** ```shell curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo bash ``` Alternatively, if you need to force a specific distribution: ```shell curl -sLf \ 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` Or, you can manually configure the repository: **Yum (RedHat, CentOS, Amazon)** ```shell yum install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=CODENAME' > /tmp/OWNER-REPOSITORY.repo yum-config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' yum -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' ``` **Dnf (Fedora)** ```shell dnf install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=CODENAME' > /tmp/OWNER-REPOSITORY.repo dnf config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' dnf -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' --enablerepo='OWNER-REPOSITORY-source' ``` **Zypper (Suse)** ```shell curl -sLf 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=CODENAME' > /tmp/OWNER-REPOSITORY.repo zypper ar -f '/tmp/OWNER-REPOSITORY.repo' zypper --gpg-auto-import-keys refresh OWNER-REPOSITORY OWNER-REPOSITORY-source ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To install RPM packages from a private Cloudsmith repository, you can quickly set up the repository automatically: **Yum (RedHat, CentOS, Amazon), Dnf (Fedora) or Zypper (Suse)** ```shell Entitlement Token Auth curl -sLf \ 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (User & Pass) curl -u "USERNAME:PASSWORD" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (API-Key) curl -u "USERNAME:API-KEY" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo bash ``` ```shell HTTP Basic Auth (Token) curl -u "token:TOKEN" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo bash ``` Alternatively, if you need to force a specific distribution: ```shell Entitlement Token Auth curl -sLf \ 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (User & Pass) curl -u "USERNAME:PASSWORD" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (API-Key) curl -u "USERNAME:API-KEY" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` ```shell HTTP Basic Auth (Token) curl -u "token:TOKEN" -sLf \ 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/bash.rpm.sh' \ | sudo distro=DISTRO codename=VERSION bash ``` Or, you can manually configure the repository: **Yum (RedHat, CentOS, Amazon)** ```shell Entitlement Token Auth yum install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo yum-config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' yum -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' ``` ```shell HTTP Basic Auth (User & Pass) yum install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "USERNAME:PASSWORD" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo yum-config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' yum -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' ``` ```shell HTTP Basic Auth (API-Key) yum install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "USERNAME:API-KEY" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo yum-config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' yum -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' ``` ```shell HTTP Basic Auth (Token) yum install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "token:TOKEN" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo yum-config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' yum -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' ``` **Dnf (Fedora)** ```shell Entitlement Token Auth dnf install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo dnf config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' dnf -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' --enablerepo='OWNER-REPOSITORY-source' ``` ```shell HTTP Basic Auth (User & Pass) dnf install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "USERNAME:PASSWORD" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo dnf config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' dnf -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' --enablerepo='OWNER-REPOSITORY-source' ``` ```shell HTTP Basic Auth (API-Key) dnf install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "USERNAME:API-KEY" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo dnf config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' dnf -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' --enablerepo='OWNER-REPOSITORY-source' ``` ```shell HTTP Basic Auth (Token) dnf install yum-utils pygpgme rpm --import 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' curl -u "token:TOKEN" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo dnf config-manager --add-repo '/tmp/OWNER-REPOSITORY.repo' dnf -q makecache -y --disablerepo='*' --enablerepo='OWNER-REPOSITORY' --enablerepo='OWNER-REPOSITORY-source' ``` **Zypper (Suse)** ```shell Entitlement Token Auth curl -sLf 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo zypper ar -f '/tmp/OWNER-REPOSITORY.repo' zypper --gpg-auto-import-keys refresh OWNER-REPOSITORY OWNER-REPOSITORY-source ``` ```shell HTTP Basic Auth (User & Pass) curl -u "USERNAME:PASSWORD" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo zypper ar -f '/tmp/OWNER-REPOSITORY.repo' zypper --gpg-auto-import-keys refresh OWNER-REPOSITORY OWNER-REPOSITORY-source ``` ```shell HTTP Basic Auth (API-Key) curl -u "USERNAME:API-KEY" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo zypper ar -f '/tmp/OWNER-REPOSITORY.repo' zypper --gpg-auto-import-keys refresh OWNER-REPOSITORY OWNER-REPOSITORY-source ``` ```shell HTTP Basic Auth (Token) curl -u "token:TOKEN" -sLf 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/setup/config.rpm.txt?distro=DISTRO&codename=VERSION' > /tmp/OWNER-REPOSITORY.repo zypper ar -f '/tmp/OWNER-REPOSITORY.repo' zypper --gpg-auto-import-keys refresh OWNER-REPOSITORY OWNER-REPOSITORY-source ``` ### Installing a Package Once you have setup the repository using your chosen authentication method, you can then install a package using: **Yum (RedHat, CentOS, Amazon)** ```shell sudo yum install PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH ``` **Dnf (Fedora)** ```shell sudo dnf install PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH ``` **Zypper (Suse)** ```shell sudo zypper install PACKAGE_NAME-PACKAGE_VERSION.PACKAGE_ARCH ``` ### Removing Setup If you no longer want to install packages from your Cloudsmith repository, you can remove it with: **Yum (RedHat, CentOS, Amazon) and Dnf (Fedora)** ```shell rm /etc/yum.repos.d/OWNER-REPOSITORY.repo rm /etc/yum.repos.d/OWNER-REPOSITORY-source.repo ``` **Zypper (Suse)** ```shell zypper rr OWNER-REPOSITORY zypper rr OWNER-REPOSITORY-source ``` ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream RedHat repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Index Packages **Automatic GPG Key Generation** As RPM packages are signed with a GPG key upon upload, this results in a new checksum being generated for the package. ## Troubleshooting **Q. I don't see packages that are available in my repository when I do `yum list`, even after clearing my yum cache and updating?** It may be possible that you already have a package installed with the same version as that which is available from the repository. In this case, you need to do `yum list available --showduplicates` to see all available packages. **Q. I'm having trouble pulling `noarch` packages?** The noarch index in RedHat is separate, unlike in Debian where it is blended into all arch-specific repositories. The reason for this is that it follows the standard layout for RedHat repositories. If you use our automated repository setup scripts, they configure two repositories when setting up a machine, one for the arch-specific and another for noarch, which will enable you to install `noarch` packages. If you are manually setting up a Cloudsmith RedHat repository, or using something like Ansible, please be sure to configure the`noarch` repository also. Contact us [here](https://support.cloudsmith.com). We're always happy to help. # Ruby Repository Cloudsmith provides public & private repositories for Ruby gems Ruby Gems is the packaging format for Ruby language. Cloudsmith is proud to support fully-featured registries for managing your own private and public Gem packages. For more information on Ruby, please see: - [Ruby Lang](https://www.ruby-lang.org): The official website for Ruby language - [Ruby Gem Community](https://rubygems.org): The official public repository for Ruby Gems - [Ruby Gem Documentation](https://guides.rubygems.org): The official docs for Ruby Gems **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | PACKAGE_VERSION | The version number of your package | ## Upload a Package To upload, you need to generate your Ruby gem package first. You can do this with: ```shell gem build PACKAGE_NAME.gemspec ``` This generates a gem package file (.gem) like `PACKAGE_NAME-PACKAGE_VERSION.gem` that you can upload. **.gemspec required** This assumes that you've created a `.gemspec` file for your project. Please see the official [Rubygems guide](https://guides.rubygems.org/make-your-own-gem/) on how to make your own gem for more information. (external link) ### Upload via gem push The endpoint for the native Ruby API is: ``` https://ruby.cloudsmith.io/OWNER/REPOSITORY/ ``` In order to authenticate for native publishing, you can either enter your credentials during the `gem push` command or add your credentials to your `$HOME/.gem/credentials` file: ``` :rubygems_api_key: API-KEY ``` You can also replace `:rubygems_api_key:` with an alternative, such as `:cloudsmith:`, and then specify `--key 'cloudsmith'` during the gem push command. **Warning** These credentials are not encrypted. It is recommended to log in on the first `gem push` \`instead. To publish a gem, you can do so from your project directory using `gem push`: ```shell gem push PACKAGE_NAME-VERSION.gem \ --host 'https://ruby.cloudsmith.io/OWNER/REPOSITORY' ``` If you haven't specified credentials above, you'll be asked for them during the push. **envvar alternative** You can also set the `RUBYGEMS_HOST` environment variable instead of `--host`: ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Ruby gem package via the Cloudsmith CLI is: ```shell cloudsmith push ruby OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.gem ``` Example: ```shell cloudsmith push ruby your-account/your-repo safe_yaml-1.0.4.gem ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ### Example Project For examples of what your project should look like for packaging and publishing/uploading, please have a look at our [examples repository](http://github.com/cloudsmith-io/cloudsmith-examples) (on GitHub). We'll supplement these with more detailed guidance later, but otherwise, just ask - we're here to help. ## Download / Install a Package ### Setup As stated by [Bundler](https://bundler.io/), "Bundler provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed." Bundler from version 1.7 supports scoped sources, so you can install a gem from Cloudsmith using the following declaration in your Gemfile: #### Public Repositories ```shell source 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE-NAME', '~> PACKAGE_VERSION' end ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```shell Entitlement Token Auth source 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE-NAME', '~> PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (User & Pass) source 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE-NAME', '~> PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (API-Key) source 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE-NAME', '~> PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (Token) source 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE-NAME', '~> PACKAGE_VERSION' end ``` #### Setup with Ruby CLI You can also add Cloudsmith as a source for the Ruby CLI instead: #### Public Repositories ```shell gem sources --add 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/ruby/' ``` #### Private Repositories The Ruby CLI setup method will differ depending on what authentication type you choose to use. ```shell Entitlement Token Auth gem sources --add 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/ruby/' ``` ```shell HTTP Basic Auth (User & Pass) gem sources --add 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' ``` ```shell HTTP Basic Auth (API-Key) gem sources --add 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' ``` ```shell HTTP Basic Auth (Token) gem sources --add 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' ``` ### Specifying Dependencies If you have a project **Gemfile** you can specify your Ruby gem package as a dependency: #### Public Repositories ```shell source 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE_NAME', 'PACKAGE_VERSION' end ``` #### Private Repositories ```shell Entitlement Token Auth source 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE_NAME', 'PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (User & Pass) source 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE_NAME', 'PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (API-Key) source 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE_NAME', 'PACKAGE_VERSION' end ``` ```shell HTTP Basic Auth (Token) source 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/ruby/' do gem 'PACKAGE_NAME', 'PACKAGE_VERSION' end ``` When specifying a private repository in a **Gemfile**, please bear in mind that the URL will contain the credentials (especially important if the **Gemfile** shared.) Our recommendation is to specify the authentication credentials via environment variables, but you could also choose to encrypt your **Gemfile** via something like [git-crypt](https://github.com/AGWA/git-crypt) (if you're using git or GitHub, for example). ### Installing a Package Once setup, you can install your Ruby gem package using `gem install` as follows. To install a specific version of a package: ```shell gem install 'PACKAGE_NAME:PACKAGE_VERSION' ``` To install the latest version of a package: ```shell gem install 'PACKAGE_NAME' ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream Ruby repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # sbt Repository Cloudsmith provides public & private repositories for sbt packages (Scala) sbt is an open-source build tool for Scala and Java projects. Its main features are native support for compiling Scala code and integrating with many Scala test frameworks. For more information on sbt, please see: - [scala-sbt](https://www.scala-sbt.org/): The official sbt website **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :------------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE_VERSION | The version number of your package | | GROUP-ID | A unique Maven identifier for your project across all projects i.e "com.companyname.project" | | ARTIFACT_ID | The name of the jar without version i.e "project" | ## Upload a Package ### Upload via sbt Publish The endpoint for the native Maven API is: ``` https://maven.cloudsmith.io/OWNER/REPOSITORY/ ``` By default, Sbt 1.x uses Maven-style publishing and no other plugins are required to publish to Cloudsmith. If you're using Sbt 0.x then you'll need to use the Maven Wagon integration approach (see the integrations tab within the repository on the Cloudsmith website for further details). Include a `publishTo` stanza in your `publish.sbt` file as follows: ```scala publishTo := { Some("Cloudsmith API" at "https://maven.cloudsmith.io/OWNER/REPOSITORY/") } pomIncludeRepository := { x => false } ``` You then configure a `~/.sbt/.credentials` file with the authentication details of the uploading user as follows: ``` realm=Cloudsmith API host=maven.cloudsmith.io user=OWNER password=API-KEY ``` Then refer to the above in your `publish.sbt` file with: ```scala credentials += Credentials(Path.userHome / ".sbt" / ".credentials") ``` **envvar alternative** You can also encode your credentials directly (and use an environment variable to pull in the API key). You can now publish to the native API with: ```shell sbt publish ``` You can find out additional information about Sbt publishing in the [official Sbt documentation](https://www.scala-sbt.org/1.x/docs/Publishing.html). #### Publishing an sbt plugin Cloudsmith supports publishing sbt plugins via the Maven publish style. To publish an sbt plugin you need to add the following to your `build.sbt`: ```scala lazy val root = (project in file(".")) .enablePlugins(SbtPlugin) .settings( name := "PLUGIN_NAME", sbtPlugin := true, publishMavenStyle := true, ) ``` ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload via the Cloudsmith CLI is: ```shell cloudsmith push maven OWNER/REPOSITORY ARTIFACT_ID-PACKAGE_VERSION.jar --pom-file=ARTIFACT_ID-PACKAGE_VERSION.pom ``` Example: ```shell cloudsmith push maven org/repo validation-api-1.0.0.GA.jar --pom-file=validation-api-1.0.0.GA.pom ``` ### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Package ### Setup #### Public Repositories To enable the retrieval of packages from a public Cloudsmith repository via sbt, add your repository your `build.sbt` file as follows: ```scala resolvers += "NAME" at "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/maven/" ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. To enable the retrieval of packages from a private Cloudsmith repository via sbt, add your repository your `build.sbt` file as follows: ```scala Entitlement Token Authentication resolvers += "NAME" at "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/maven/" ``` ```scala HTTP Basic Authentication resolvers += "NAME" at "https://dl.cloudsmith.io/basic/OWNER/REPOSITIORY/maven/" ``` When using Entitlement Token Authentication, no further setup is required. If using HTTP Basic Authentication, you can provide one following three types of credentials: - Cloudsmith Username and Password - Cloudsmith API Key - An Entitlement Token You should keep these credentials separately in your `~/.sbt/.credentials` file instead of within the `build.sbt` file. When you have your credentials ready, setup your `~/.sbt/.credentials` file as follows: ```text Username & Password realm=Private Repository host= user=USERNAME password=PASSWORD ``` ```text API Key realm=Private Repository host= user=USERNAME password=API-KEY ``` ```text Entitlement Token realm=Private Repository host= user=token password=TOKEN ``` Then add the following to your `build.sbt` file: ```scala credentials += Credentials(Path.userHome / ".sbt" / ".credentials") ``` ### Specifying Dependencies After the repository is added to the `build.sbt` file, and your credentials are added to the `/.sbt/.credentials` file (required for private repositories if using HTTP Basic Authentication), all that is left is to [specify the dependency](https://www.scala-sbt.org/1.x/docs/Library-Dependencies.html) in the dependencies section of the project `build.sbt` file. ```scala libraryDependencies += "GROUP_ID" % "ARTIFACT_ID" % "PACKAGE_VERSION" ``` *** **sbt 0.13.x** In sbt 0.13.x (not sbt 1.x or above) an extension point in the dependency resolution to use Maven-style resolvers is required. To enable this plugin add the following to `project/maven.sbt` (or `project/plugin.sbt`): ```scala addMavenResolverPlugin ``` ## Configuring Cloudsmith as the only source for sbt packages In this section, we cover how to configure sbt to block incoming packages from any repository outside of your Cloudsmith repository. Note that these steps might vary depending on the platform you use. We recommend reviewing the [official sbt documentation](https://www.scala-sbt.org/1.x/docs/Proxy-Repositories.html) to learn about any specifics of your platform. 1. In your `sbtopts` file in `~/.config/sbt/sbtopts`, set `Dsbt.override.build.repos` to `true`: ``` -Dsbt.override.build.repos=true -Dsbt.repository.config=/home//.sbt/repositories -Dsbt.boot.credentials=/home//.sbt/.cloudsmith.credentials ``` 2. Then, if you haven't done yet, create a `~/.sbt/.cloudsmith.credentials` file to set your credentials to access your Cloudsmith repo: ``` realm=Private Repository: / host=dl.cloudsmith.io user= password= ``` 3. Configure your global resolvers to use your Cloudsmith repository and credentials in `~/.sbt/1.0/global.sbt`: ``` resolvers := Seq(“" at "https://dl.cloudsmith.io/basic/WORKSPACE/REPOSITORY/maven/") credentials += Credentials(Path.userHome / ".sbt" / ".cloudsmith.credentials") resolvers += “" at "" externalResolvers := resolvers.value ``` Before proceeding to the next step, add the same configuration to `~/.sbt/1.0/plugins/plugins.sbt`. Finally, configure your repositories file in `~/.sbt/repositories` to use the resolver configured in the previous step: ``` [repositories] local : https://dl.cloudsmith.io/basic///maven/ : ``` ## Upstream Proxying / Caching Configurable Proxying Caching You can configure upstream repositories that you wish to use for packages that are not available in your Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support GPG Index Packages ## Troubleshooting **Q. sbt maintains plain text passwords in settings.xml and credential files. How do we prevent exposure of such passwords?** The underlying Maven toolchain supports encrypted credentials. You can also use interpolation from environment variables instead so that you don't store them in configuration files. We also support authentication by de-privileged [entitlement tokens](/software-distribution/entitlement-tokens) (i.e. read-only specific access) to minimize exposure. ## Still Need Help? Contact us [here](https://support.cloudsmith.com). We're always happy to help. # Signing Swift Packages **Early Access** Swift Package signing is currently in Early Access. To use this feature please contact us. ## How it works Swift packages are signed using an ECDSA private key and X.509 Certificate combination, with the certificate containing the private key’s corresponding public key. When Swift package signing is enabled for a repository, an ECDSA key pair and X.509 certificate are created, and all Swift packages uploaded or resynchronized to the repository will be signed automatically. ## Note on signing existing packages **Signing of existing packages** Signing a Swift package mutates its state, and may cause builds employing the package as a dependency to break. This can be rectified by removing the stored fingerprint(s) for the relevant manifest and source archive. Signing a Swift package mutates the package's state, as package manifest signatures are added to the manifest files themselves, modifying the checksum of the overall source archive. As Swift performs checksum Trust On First Use (TOFU), signing a Swift package may cause builds to break where those packages have already been consumed locally as dependencies. Consequently, to sign packages already used as dependencies, the stored fingerprint(s) for the relevant manifest and source archive must be removed. These can be viewed and modified in the ~/.swiftpm/security/fingerprint directory. ## Configuring the Swift CLI to verify packages When the Swift CLI consumes a signed package, it validates the signature for both the manifest and the source archive. These signatures are in Cryptographic Message Syntax (CMS) format and are base64 encoded. The signatures vary slightly: * A manifest signature is an attached signature; it is included directly within the manifest file (see [here](https://github.com/swiftlang/swift-package-manager/blob/main/Documentation/PackageRegistry/PackageRegistryUsage.md#package-manifests) for example). * A source archive signature is a detached signature; it is included within the metadata response when fetching a package for a specific scope/name/version. Swift determines whether the package is signed by inspecting the signing attribute in the metadata response. For more information see [here](https://github.com/swiftlang/swift-package-manager/blob/main/Documentation/PackageRegistry/Registry.md). The source archive signature is generated after the manifest(s) are signed. The Swift CLI registry security configuration is specified at the user level in a registries.json file. For more information on this configuration see [here](https://github.com/swiftlang/swift-package-manager/blob/main/Documentation/PackageRegistry/PackageRegistryUsage.md#security-configuration). ## Adding a Trusted Certificate A certificate is trusted by Swift if it is chained to any root in SwiftPM’s trusted store (which can be in two locations). To ensure that the certificate used by Cloudsmith to sign Swift your packages is trusted you’ll need to perform the following steps, which will fetch the intermediate CA certificate used to sign the repository-level certificate, and place it in Swift’s default trust store. 1. Fetch the certificate chain for the signing certificate and assign it to a variable. The chain contains both the root CA certificate and intermediate CA certificate (it does not include the actual signing certificate): ``` CERTIFICATE_CHAIN="$(curl --location '//x509-ecdsa/' \\ --header 'Accept: application/json' \\ --header 'X-Api-Key: ' \\ | jq -r .certificate_chain)" ``` 2. Write the chain to a .pem file. ``` echo $CERTIFICATE_CHAIN >> cert_chain.pem ``` 3. Swift requires that the certificate be in .der (Distinguished Encoding Rules) format. To do that, convert the .pem file to .der. Note that a certificate chain can’t be represented in the DER format, however only the intermediate CA certificate is required. This can be achieved by the following: ``` openssl x509 -in cert_chain.pem -out cert.der -outform DER ``` 4. Now place this trusted certificate in the following path: ``` ~/.swiftpm/security/trusted-root-certs/ ``` 5. Add the trustedRootCertificatesPath to your security configuration, as shown [here](https://github.com/swiftlang/swift-package-manager/blob/main/Documentation/PackageRegistry/PackageRegistryUsage.md#security-configuration). Confusingly, the CLI doesn’t seem to like the relative path in the example shown, so you might have to change it to an absolute path. Once the above steps are complete, you will be able to consume fully verifiable and signed swift packages from Cloudsmith. # Swift Registry Cloudsmith provides public & private registries for Swift The Swift Package Manager is a tool for managing the distribution of Swift code. It's integrated with the Swift build system to automate the process of downloading, compiling, and linking dependencies. Cloudsmith is fully compatible as a Swift registry. With Swift, developers can create, share, and manage packages for their projects. For more information on Swift, please see: - [Swift](https://www.swift.org/): The official website for Swift - The [Swift registry](https://swiftpackageregistry.com/)  is a HTML catalog of Swift Packages which helps direct users to third-party Git repositories. It does not host swift packages. - The [Swift package manager](https://www.swift.org/documentation/package-manager/) is a tool for managing the distribution of Swift code. It’s integrated with the Swift build system to automate the process of downloading, compiling, and linking dependencies. - [Creating a Swift package](https://www.swift.org/getting-started/library-swiftpm/): Creating a Swift package In the following examples: | Identifier | Description | | :----------------- | :---------------------------------------------------------------------------------------- | | WORKSPACE | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | GIT_REPOSITORY_URL | GIT repository URL | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-TOKEN | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your package | | APP_NAME | The name of your application | | VERSION | The version number of your package | | SCOPE | Namespaces for package names | # Configure Cloudsmith as your Swift Registry ## Create a Cloudsmith Repository If you haven't already, [create a repository](/repositories/create-a-repository) in Cloudsmith where you'll publish your package. Note down the repository URL, which should resemble: `https://swift.cloudsmith.io/WORKSPACE/REPOSITORY` Replace `WORKSPACE` with your Cloudsmith Workspace name and `REPOSITORY` with the name of your repository. ## Creating a Swift package If you don't have a sample package to test this, use `swift package init` to create a new one: ```shell mkdir swift_sample cd swift_sample swift package init --name PACKAGE_NAME --type executable ``` A successful run will output: ```shell Creating executable package: your-package Creating Package.swift Creating .gitignore Creating Sources/ Creating Sources/main.swift ``` Now, package its assets using the `swift package archive-source` command: ```shell swift package archive-source ``` The terminal will output `Created PACKAGE_NAME.zip` if successful. ## Configure Cloudsmith as your Swift Registry Use Swift version 5.9 or above (verify your version with `swift -version`) to publish Swift package to Cloudsmith using the Swift Package Manager. Then, set your Cloudsmith repository for the project: ```shell swift package-registry set https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/ ``` This creates a configuration file at `.swiftpm/configuration/registries.json`that looks like this: ```json { "authentication" : { }, "registries" : { "[default]" : { "supportsAvailability" : false, "url" : "https://swift.cloudsmith.io/cloudsmith-test/buildkite-demo" } }, "version" : 1 ``` 📘 Instead of configuring the registry per-project, you can set it globally using the `--global` flag. This is useful for configuring a build machine or your main development environment. ```bash swift package-registry set --global "https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/" ``` Then, [get your API_TOKEN](/accounts-and-teams/api-key#getting-your-api-key) and login to your private Cloudsmith Swift registry with: ```shell swift package-registry login https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/ --username USERNAME --password API_TOKEN ``` ## Upload a Package ### Upload using Swift Package Manager A Swift package consists of Swift source files and a manifest file. The manifest file, called `Package.swift`, defines the package’s name and its contents using the `PackageDescription` module. Navigate to the Swift project directory that contains the `Package.swift` file for your package. Replace `SCOPE`, `PACKAGE_NAME` and `VERSION` with the scope of the package, package name and package version respectively. ```shell swift package-registry publish SCOPE.PACKAGE_NAME VERSION ``` **Swift scopes** In Swift, scope are namespaces for package names. You can publish any scope to your Cloudsmith repository. Package scopes are case-insensitive (for example, mona ≍ MONA). **Version numbers** should follow semantic versioning. #### Upload Verification After publishing, You should receive a confirmation message similar to the following: ``` SCOPE.PACKAGE_NAME version VERSION was successfully published to https://swift.cloudsmith.io/WORKSPACE/REPOSITORY and is available at 'https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/SCOPE/PACKAGE_NAME/VERSION' ``` You can verify that your package is available in your Cloudsmith repository by navigating to the repository URL in your browser or using the Cloudsmith CLI. ### Alternative example: from source code In the next steps we cover how to package an existing swift project from its source code in GitHub, and then upload it to a Cloudsmith repository. For this example, we'll be using **Alamofire**: a popular HTTP networking library written in Swift. Download its source code and checkout version `5.9.1`: ```bash cd .. # get back to the previous dir git clone https://github.com/Alamofire/Alamofire.git --depth 1 cd Alamofire git fetch --depth 1 origin tag 5.9.1 git checkout 5.9.1 ``` To publish a package, you need to create a `package-metadata.json` file. This file provides essential details about the package. Create this file in the root of the `Alamofire` directory and paste in the following content: ```json { "author": { "name": "Alamofire Software Foundation", "email": "info@alamofire.org", "organization": { "name": "Alamofire Software Foundation" } }, "description": "Elegant HTTP Networking in Swift", "licenseURL": "https://github.com/Alamofire/Alamofire/blob/master/LICENSE", "readmeURL": "https://github.com/Alamofire/Alamofire/blob/master/README.md", "repositoryURLs": [ "https://github.com/Alamofire/Alamofire.git" ] } ``` You are ready to publish the package to your Cloudsmith Registry. Follow the previous steps to configure and login to your registry (no need to run it again if you executed the previous example) and replace `WORKSPACE` and `REPOSITORY` with your own: ```bash # swift package-registry set "https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/" # swift package-registry login https://swift.cloudsmith.io/WORKSPACE/REPOSITORY/ --username USERNAME --password API-TOKEN swift package-registry publish alamofire.alamofire 5.9.1 --metadata-path ./package-metadata.json ``` **Done**. You've successfully published your Swift package to a Cloudsmith registry. You can now view the package in your Cloudsmith web app. ## Upload via the Cloudsmith CLI **CLI Install and Auth** For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli) documentation: - [Installation](/developer-tools/cli#installation). - [Authentication](/developer-tools/cli#getting-your-api-key). To upload via the Cloudsmith CLI (or via API), you'll need to generate your package first: ```shell swift package archive-source ``` This will generate a zip file named `Alamofire.zip` that you can upload. Use the previous command to upload your new swift package via the Cloudsmith CLI and remember to use a different version number (for example `5.9.2`): ```shell cloudsmith push swift WORKSPACE/REPOSITORY PACKAGE_NAME.zip --name PACKAGE_NAME --version VERSION --scope SCOPE ``` The next output will be displayed: ```text Checking swift package upload parameters ... OK Checking Alamofire.zip file upload parameters ... OK Requesting file upload for Alamofire.zip ... OK Uploading Alamofire.zip: [####################################] 100% Creating a new swift package ... OK Created: demo-docs/awesome-repo/alamofirezip-z54v (DIElCIDoFtvd) Synchronising alamofirezip-z54v: [####################################] 100% Completed / Fully Synchronised Package synchronised successfully in 51.71964 second(s)! ``` ## Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload#upload-via-website-ui) for details of how to upload via the Cloudsmith web app. Generate your package first using `swift package archive-source`. ## Download / Install a Package Now, let's see how to use the package from your registry in a new project: 1. Create a new directory for a sample project and `cd` into it. ```bash cd .. mkdir MySwiftApp && cd MySwiftApp swift package init --type executable ``` 2. **Configure the Registry for the New Project**: following the steps described above to configure Swift to use your Cloudsmith registry URL and then login to it. 3. **Add your project dependencies**: **Swift dependencies file** Package dependencies are declared in the `Package.swift` file. Edit the `Package.swift` file in your application project folder to update the package dependencies to be used by your project: 1. In the dependencies section of `Package.swift`, add the package you want to use by adding its package identifier. The package identifier consists of the scope and package name separated by a period. See the code snippet following a later step for an example. 2. In the targets section of `Package.swift`, add the targets that will need to use the dependency. The following is an example showing configured dependencies and targets sections in a `Package.swift` file to include Alamofire as a dependency: ``` // swift-tools-version: 5.9 import PackageDescription let package = Package( name: "MySwiftApp", dependencies: [ .package(id: "alamofire.alamofire", from: "5.9.1") ], targets: [ .executableTarget( name: "MySwiftApp", dependencies: [ .product(name: "Alamofire", package: "alamofire.alamofire") ] ), ] ) ``` After this, use the `--replace-scm-with-registry` flag to prioritize fetching from your registry instead of Git. ```bash swift package --replace-scm-with-registry resolve ``` You will see output indicating that Swift is downloading `alamofire.alamofire` directly from your Cloudsmith registry. You should see the following printed out when resolving dependencies: ``` Computing version for alamofire.alamofire Computed alamofire.alamofire at 5.9.2 (1.84s) Fetching alamofire.alamofire warning: 'alamofire.alamofire': alamofire.alamofire version 5.9.2 source archive from https://swift.cloudsmith.io/demo-docs/awesome-repo is not signed Fetched alamofire.alamofire from cache (4.07s) ``` **Swift registry resolve** 📘 Using `id:` in your `Package.swift` is powerful. When you run `resolve` with the flag, Swift Package Manager will first try to find a matching version in your configured registry. If it fails, it can fall back to the Git repository specified in the `package-metadata.json` file. This provides a robust, hybrid approach. Downloading a large repository like Alamofire can take a significant amount of time, as you're cloning the entire history. Pulling a package from a registry is much faster. You only download a lightweight source archive (`.zip`) for the specific version you need, drastically reducing download times and CI/CD pipeline durations. For a more detailed overview of Package Registries usage with Swift, visit their [docs](https://github.com/swiftlang/swift-package-manager/blob/main/Documentation/PackageRegistry/PackageRegistryUsage.md). ## Resetting your registry configuration To reset your configuration: ```shell swift package-registry unset ``` Or simply delete the `registries.json` files from your project and/or home directory: ```bash rm -f ./.swiftpm/configuration/registries.json ``` To force Swift to re-resolve all your dependencies from scratch the next time you build or run your project and clear out the global Swift Package Manager cache on your machine, run: ```bash swift package reset swift package purge-cache ``` ## Security Scanning Supported Please see our [Security Scanning](/policy-management/vulnerability-policy) documentation for further information. ## Upstream Proxying / Caching Configurable Proxying Caching You can configure to proxy a private or public swift registry you wish to use for packages that are not available in your current Cloudsmith repository e.g. you can point to another Cloudsmith repository. In addition, you can also choose to cache any requested packages for future use. **Public upstreams** There is currently no public swift repository. HTML catalogs of Swift Packages such as [Swift Package Registry](https://swiftpackageregistry.com/) that point to third-party Git repositories do not host any actual packages and so can't be added as an upstream. Please see our [Upstream Proxying](/repositories/upstreams) documentation for further instructions. ## Key Signing Support Supported ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Terraform Modules Repository Cloudsmith provides public & private repositories for Terraform Modules Terraform is an infrastructure-as-code tool to provision and manage any cloud, infrastructure, or service by the awesome folks over at [Hashicorp](https://www.hashicorp.com/). For more information on Terraform, please see: - [Terraform](https://developer.hashicorp.com/terraform): The official website for Terraform - [Terraform Documentation](https://developer.hashicorp.com/terraform/docs): The official docs for Terraform **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your workspace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | WORKSPACE | Your Cloudsmith workspace name | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | MODULE_PROVIDER | The provider for your Terraform Module | | MODULE_NAME | The name of your Terraform Module | | MODULE_VERSION | The version number of your Terraform Module | ## Upload a Module ### Upload via the Cloudsmith CLI or Website To upload via the Cloudsmith API/CLI, you'll need to generate a module first. You can build a package with standard command-line tooling like `tar.` To illustrate the process we'll use the [terraform vault module](https://github.com/hashicorp/terraform-aws-vault) for AWS as an example: First, check out the version of the module we want to pack (0.13.6 for example purposes): ```shell git clone git@github.com:hashicorp/terraform-aws-vault.git cd terraform-aws-vault git checkout 0.13.6 ``` Next, create an archive using: ```shell tar --exclude='.terraform' --exclude='*.tfstate*' --exclude='*_override.tf*' -czvf terraform-aws-vault-0.13.6.tar.gz . ``` **Supported characters** Only lowercase alphanumeric characters, underscores and hyphens are supported for the module name. Please see the official [Terraform documentation](https://developer.hashicorp.com/terraform/docs) for more information on building your own modules. #### Upload via Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload a Terraform Module via the Cloudsmith CLI is: ```shell cloudsmith push terraform WORKSPACE/REPOSITORY terraform-MODULE_PROVIDER-MODULE_NAME-MODULE_VERSION.tar.gz ``` Example: ```shell cloudsmith push terraform demo/examples-repo terraform-aws-vault-0.13.6.tar.gz ``` #### Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Module ### Setup Assuming you have Terraform already installed (if not, see the official docs), it is straightforward to add a Cloudsmith-based Terraform module. First, the workspace, repository and credentials must be added to your `.terraformrc` or `terraform.rc` file. The token must contain the name of the organization which owns the module, the repository containing the module and the credentials required to authenticate with the API, delimited by a `/`: ```shell Entitlement Token Auth credentials "terraform.cloudsmith.io" { token = "WORKSPACE/REPOSITORY/TOKEN" } ``` ```shell API Key Auth credentials "terraform.cloudsmith.io" { token = "WORKSPACE/REPOSITORY/API-KEY" } ``` **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code or expose them in any logs. ### Install a Module Once configured as above, your module can then depend on a module from your registry by specifying the module's source in your Terraform file(s) using the syntax as outlined by [Terraform](https://developer.hashicorp.com/terraform#private-registry-module-sources): ``` module "my_module" { source = "terraform.cloudsmith.io/REPOSITORY/MODULE_NAME/local" version = "MODULE_VERSION" } ``` Example: ``` module "my_module" { source = "terraform.cloudsmith.io/examples-repo/terraform-aws-vault-v0136targz/local" version = "v0.13.6" } ``` Once added, terraform will download the module to your project's `.terraform` directory after running: ```shell terraform init ``` You can upgrade to the most recent version of this module matching your version constraints by running: ```shell terraform init -upgrade=true ``` ## Upstream Proxying / Caching Not Supported ## Key Signing Support Not Supported by Format ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Unity Registry Cloudsmith provides public & private repositories for the Unity Development Platform For more information on Unity, please see: - [Unity](https://unity.com/): The official website for Unity - [Unity Docs](https://docs.unity3d.com/Manual/index.html): The official documentation for Unity - [Unity UPM](https://docs.unity3d.com/Manual/Packages.html): Documentation for Unity's Package Manager - [Unity UPM Scoped Registry](https://docs.unity3d.com/Manual/upm-scoped.html): Documentation to setup a Scoped Package Registry **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :-------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | API-KEY | Your Cloudsmith API Key | | PACKAGE_NAME | The name of your Unity package | | PACKAGE_VERSION | The version number of your package | # Upload a Package ## Publish via npm As Unity packages are compatible with npm registries, you can publish your Unity packages to a Cloudsmith npm registry via the native npm tooling. ### Setup The endpoint for the native npm API is: ``` https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` In order to authenticate for native publishing, you can either use `npm login` or create an `npmrc` file (in your **$HOME** or in the project directory) Use `npm login`: ```shell npm login --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ Username: USERNAME Password: API-KEY Email: YOUR-EMAIL-ADDRESS ``` Or create an `.npmrc` file with the following: ``` registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ //npm.cloudsmith.io/OWNER/REPOSITORY/:_authToken=API-KEY ``` ### Publish Once you have set up the registry, you can then publish from your project directory using `npm publish`: ```shell npm publish --registry=https://npm.cloudsmith.io/OWNER/REPOSITORY/ ``` For more details on `npm publish` see the official [npm documentation](https://docs.npmjs.com/cli/publish). (external link) ## Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). To upload via the Cloudsmith CLI / API, you'll need to generate your package first. You can do this with: ```shell npm pack ``` This will generate a tarball file (.tgz) like `your-package-1.2.3.tgz` that you can upload. **project.json required** This assumes that you've created a `project.json` file for your project. Please see the official npmjs [package.json reference](https://docs.npmjs.com/files/package.json) (external link) for more information. The command to upload a Unity package via the Cloudsmith CLI is: ```shell cloudsmith push npm OWNER/REPOSITORY PACKAGE_NAME-PACKAGE_VERSION.tgz ``` Example: ```shell cloudsmith push npm your-account/your-repo com.cloudsmith.assets--1.0.0.tgz ``` ## Upload via Cloudsmith web app Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. # Download / Install a Package ## Public Registries To download/install a Unity package, you need to add your Cloudsmith registry to your Unity project's `manifest.json` file as a scoped registry: ```json "scopedRegistries": [ { "name": "NAME", "url": "https://npm.cloudsmith.io/OWNER/REPOSITORY/", "scopes": [ "SCOPE" ] } ], ``` You then add the packages that you wish to download and install to the dependencies section of the project `manifest.json` file: ```json "dependencies": { "PACKAGE_NAME": "PACKAGE_VERSION" } } ``` For example: ```json { "scopedRegistries": [ { "name": "MyRegistry", "url": "https://npm.cloudsmith.io/demo/examples-repo/", "scopes": [ "com.cloudsmith", ] } ], "dependencies": { "com.cloudsmith.assets": 1.1.2" } } ``` For further information, please see the Unity documentation on [Scoped Registries](https://docs.unity3d.com/Manual/upm-scoped.html). ## Private Registries To download/install a Unity package from a private Cloudsmith registry: 1. Add the registry and dependencies to your project's `manifest.json` as per the instructions for public registries 2. Locate the `upmconfig.toml` file and add the one of the following, depending on which authentication method you wish to use: ```toml Entitlement Token Auth [npmAuth."https://npm.cloudsmith.io/OWNER/REPOSITORY/"] token = "TOKEN" email = "EMAIL" OPTIONAL alwaysAuth = true OPTIONAL ``` ```toml API Key Auth [npmAuth."https://npm.cloudsmith.io/OWNER/REPOSITORY/"] token = "API-KEY" email = "EMAIL" OPTIONAL alwaysAuth = true OPTIONAL ``` If the file does not already exist, create an empty text file. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code or expose them in any logs. Unity Package Manager Global configuration file location: | Environment | Location | | :-------------- | :----------------------------------------------- | | Windows | %ALLUSERSPROFILE%\\Unity\\config\\upmconfig.toml | | macOS and Linux | /etc/upmconfig.toml | Unity Package Manager User configuration file location: | Environment | Location | | :-------------- | :---------------------------- | | Windows | %USERPROFILE%\.upmconfig.toml | | macOS and Linux | ~/.upmconfig.toml | For further information, please see the Unity documentation on [scoped registry authentication](https://docs.unity3d.com/Manual/upm-config-scoped.html) ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Vagrant Repository Cloudsmith provides public & private repositories for Vagrant boxes Vagrant is an automation tool for building and managing virtual machine environments. Developed by the wizards at Hashicorp it makes setting up environments easier to manage. For more information on Vagrant, please see: - [Vagrant](https://developer.hashicorp.com/vagrant): The official website for Vagrant - [Vagrant Docs](https://developer.hashicorp.com/vagrant/docs): The official docs for Vagrant - [Public Vagrant Repository](https://portal.cloud.hashicorp.com/vagrant/discover): The official public repository for Vagrant boxes **Contextual Documentation** The examples in this document are generic. Cloudsmith provides contextual setup instructions within each repository, complete with copy and paste snippets (with your namespace/repo/rsa-key pre-configured). In the following examples: | Identifier | Description | | :------------ | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith account name or organization name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | BOX_NAME | The name of your Vagrant box | | BOX_VERSION | The version number of your Vagrant box | | PROVIDER_NAME | The name of the Vagrant provider (i.e virtualbox, hyperv, vmware_desktop etc) | ## Upload a Box To upload, you need to generate your Vagrant box first. You can do this with: ```shell vagrant package --output BOX_NAME.box ``` This generates a box file (.box) like `your-package-1.2.3.box` that you can upload. **Project Vagrantfile required** This assumes that you've created a Vagrantfile file for your project and that you have used `vagrant up` at least once. Please see the official [Vagrant docs](https://developer.hashicorp.com/vagrant/tutorials/get-started/setup-project?in=vagrant%2Fgetting-started) for more information. ### Upload via the Cloudsmith CLI For full details of how to install and setup the Cloudsmith CLI, see [Command Line Interface](/developer-tools/cli). The command to upload an (Objective-C or Swift) CocoaPods package via the Cloudsmith CLI is: ```shell cloudsmith push vagrant OWNER/REPOSITORY BOX_NAME.box --provider PROVIDER_NAME --name BOX_NAME --version BOX_VERSION ``` Example: ```shell cloudsmith push vagrant your-account/your-repo awesome.box --provider virtualbox --name awesome --version 1.0 ``` ### Upload via Cloudsmith UI Please see [Upload a Package](/artifact-management/package-upload) for details of how to upload via the Cloudsmith web app. ## Download / Install a Box To enable the retrieval of Cloudsmith hosted Vagrant boxes, the box can either be added directly via Vagrant's CLI or the project's Vagrantfile can be updated. ### Setup To add the box directly via Vagrant's CLI, execute the following: #### Public Repositories ```shell vagrant box add 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json' \ --name 'BOX_NAME' \ --box-version 'BOX_VERSION' \ --provider 'PROVIDER_NAME' ``` #### Private Repositories **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. ```shell Entitlement Token Auth vagrant box add 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json' \ --name 'BOX_NAME' \ --box-version 'BOX_VERSION' \ --provider 'PROVIDER_NAME' ``` ```shell HTTP Basic Auth (User & Pass) vagrant box add 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json' \ --name 'BOX_NAME' \ --box-version 'BOX_VERSION' \ --provider 'PROVIDER_NAME' ``` ```shell HTTP Basic Auth (API Key) vagrant box add 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json' \ --name 'BOX_NAME' \ --box-version 'BOX_VERSION' \ --provider 'PROVIDER_NAME' ``` ```shell HTTP Basic Auth (Token) vagrant box add 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json' \ --name 'BOX_NAME' \ --box-version 'BOX_VERSION' \ --provider 'PROVIDER_NAME' ``` To add the box without having to specify the URL each time, the following must be added to the project's Vagrantfile: #### Public Repositories ```shell config.vm.box = "BOX_NAME" config.vm.box_url = "https://dl.cloudsmith.io/public/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json" ``` #### Private Repositories ```shell Entitlement Token Auth config.vm.box = "BOX_NAME" config.vm.box_url = "https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json" ``` ```shell HTTP Basic Auth (User & Pass) config.vm.box = "BOX_NAME" config.vm.box_url = "https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json" ``` ```shell HTTP Basic Auth (API Key) config.vm.box = "BOX_NAME" config.vm.box_url = "https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json" ``` ```shell HTTP Basic Auth (Token) config.vm.box = "BOX_NAME" config.vm.box_url = "https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/vagrant/BOX_NAME/metadata.json" ``` ### Install a Box After you have added a box, Vagrants CLI tool can now be used as normal to install and start a box: ```shell vagrant up ``` ## Upstream Proxying / Caching Not Supported ## Key Signing Support GPG ## Troubleshooting Please see the [Troubleshooting](/troubleshooting) page for further help and information. # Integrations At Cloudsmith, our goal is to make it as easy as possible to get your packages into our system from where you can manage and distribute them according to your need. Cloudsmith comes with an awesome [CLI](/developer-tools/cli) (Command Line Interface) tool, a discoverable first-class [API](https://api.cloudsmith.io), and [bindings](developer-tools/api-bindings) allowing you to publish and download your packages and containers using practically any leading DevOps tool/service. We also provide support for [MCP](integrations/integrating-with-mcp) via the Cloudsmith CLI. **Service Accounts** For CI/CD, we would usually recommend that you create a Service Account in your organization. See [Service Accounts](/accounts-and-teams/service-accounts) for more details. # Integrating with Aikido _How to integrate Aikido with Cloudsmith._ --- Enhance your security posture by integrating Cloudsmith’s container registry with Aikido for comprehensive vulnerability scanning. This integration enables Cloudsmith users to scan their container images for known vulnerabilities directly within Aikido, helping you stay secure and compliant - [Aikido](https://www.aikido.dev/): Aikido Website - [Aikido Docs](https://help.aikido.dev/doc): Official Aikido documentation - [Install Cloudsmith Integration](https://integrations.aikido.dev/integrations/cloudsmith): Official Cloudsmith x Aikido installation ## Overview The Cloudsmith-Aikido integration provides the following capabilities: - **Vulnerability Scanning**: Scan your container images stored in Cloudsmith for known vulnerabilities with Aikido’s advanced scanning tools. - **Enhanced Analysis (Optional)**: Link containers to a code repository in Aikido to reduce false positives and better deduplicate findings. - **Free Forever Version**: Aikido offers a free plan that enables Cloudsmith users to scan container images for vulnerabilities. For users who require additional features or advanced functionality, paid plans are also available. ## Setup Instructions ### Step 1: Enable the Aikido Integration for Cloudsmith 1. Sign in to your Aikido account or sign up if you don’t already have an account. 2. Navigate to the Integrations section and search for “Cloudsmith.” 3. Select the Cloudsmith integration to start the configuration. ### Step 2: Connect Your Cloudsmith Container Registry to Aikido 1. In Aikido, you will be prompted to enter the API credentials for Cloudsmith. 2. Enter your Cloudsmith Username, API key along with the Namespace to authenticate. 3. Once authenticated, select the specific Cloudsmith repositories you want to integrate with Aikido for vulnerability scanning. **Permissions** Ensure you use API keys or Entitlement token with appropriate permissions for accessing and scanning container images. ### Step 3: Select Container Repositories 1. Once authenticated, select the Cloudsmith container repositories that you’d like to monitor. 2. (Optional) Link each container images to its relevant code repository for better analysis and deduplication. [Image: aikido_linkImages] ### Step 4: Start Scanning Your Containers Aikido will now automatically scan your selected repositories for vulnerabilities. You can view detailed reports, including severity levels and remediation steps. ## Best Practices and Recommendations - **Enable Regular Scans**: Schedule regular scans within Aikido to ensure continuous monitoring of your container images. - **Use Code Linking for Enhanced Deduplication**: By linking your images to the relevant code repositories, you can reduce duplicate findings and gain a more accurate vulnerability report. - **Review Permissions Carefully**: Ensure that API keys and other credentials have the appropriate access levels to allow for vulnerability scanning and reporting. ## Troubleshooting If you experience any issues, consult the [Aikido Help Documentation ](https://help.aikido.dev/container-image-scanning/standalone-registries/cloudsmith-container-registry)or contact Aikido support. Integrating Aikido with Cloudsmith ensures that your container images remain secure and up-to-date with the latest vulnerability assessments. Get started today to enhance your container security strategy with Aikido. # Integrating Ansible *How to integrate Ansible with Cloudsmith.* --- Ansible is open-source software for provisioning, configuration management, and application deployment. - [Ansible](https://www.ansible.com/): Ansible Website - [Ansible Docs](https://docs.ansible.com/): Official Ansible documentation In the following examples: | Identifier | Description | | :---------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository identifier (also called "slug") | | DISTRO | Your distribution (i.e el, fedora, debian etc) | | VERSION | Your version name (i.e 7, 29, hardy, buster etc) | | FINGERPRINT | The 8 Byte fingerprint of the Public GPG key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | ## Adding a RPM repository To add a Cloudsmith repository for RPM packages using Ansible, you would use the Ansible [yum_repository](https://docs.ansible.com/ansible/latest/modules/yum_repository_module.html) module. The `yum_repository` module can add or remove YUM repositories in RPM-based Linux distributions. Example `yum_repository` task: ### Public Repository ```yaml - name: Add Cloudsmith Repository yum_repository: name: cloudsmith description: Cloudsmith file: cloudsmith baseurl: https://dl.cloudsmith.io/public/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch repo_gpgcheck: yes gpgkey: https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key enabled: yes become: true ``` ### Private Repository Entitlement Token Auth HTTP Basic Auth (User & Pass) HTTP Basic Auth (API-Key) HTTP Basic Auth (Token) {`- name: Add Cloudsmith Repository yum_repository: name: cloudsmith description: Cloudsmith file: cloudsmith baseurl: https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch repo_gpgcheck: yes gpgkey: https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key enabled: yes become: true`} {`- name: Add Cloudsmith Repository yum_repository: name: cloudsmith description: Cloudsmith file: cloudsmith baseurl: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch repo_gpgcheck: yes gpgkey: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key username: USERNAME password: PASSWORD enabled: yes become: true`} {`- name: Add Cloudsmith Repository yum_repository: name: cloudsmith description: Cloudsmith file: cloudsmith baseurl: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch repo_gpgcheck: yes gpgkey: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key username: USERNAME password: API-KEY enabled: yes become: true`} {`- name: Add Cloudsmith Repository yum_repository: name: cloudsmith description: Cloudsmith file: cloudsmith baseurl: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch repo_gpgcheck: yes gpgkey: https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key username: token password: TOKEN enabled: yes become: true`} ### Install a package To install a RPM package via Ansible, you use the Ansible [yum module](https://docs.ansible.com/ansible/2.3/yum_module.html): ```yaml - name: Install a package yum: name: PACKAGE_NAME state: present update_cache: yes become: true ``` ## Adding a Debian repository To add a Cloudsmith repository for Debian packages using Ansible, you would use the Ansible [apt_key](https://docs.ansible.com/ansible/latest/modules/apt_key_module.html) module and the [apt_repository](https://docs.ansible.com/ansible/latest/modules/apt_repository_module.html) module. Example `apt_key` and `apt_repository` tasks: ### Public Repository ```yaml - name: Import Cloudsmith Repository GPG key apt_key: url: https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key state: present become: true - name: Add Cloudsmith Repository apt_repository: repo: deb https://dl.cloudsmith.io/public/OWNER/REPOSITORY/deb/DISTRO VERSION main state: present become: true ``` ### Private Repository Entitlement Token Auth HTTP Basic Auth (User & Pass) HTTP Basic Auth (API-Key) HTTP Basic Auth (Token) {`- name: Import Cloudsmith Repository GPG key apt_key: url: https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key state: present become: true - name: Add Cloudsmith Repository apt_repository: repo: deb https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/deb/DISTRO VERSION main state: present become: true`} {`- name: Import Cloudsmith Repository GPG key apt_key: url: https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key state: present become: true - name: Add Cloudsmith Repository apt_repository: repo: deb https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO VERSION main state: present become: true`} {`- name: Import Cloudsmith Repository GPG key apt_key: url: https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key state: present become: true - name: Add Cloudsmith Repository apt_repository: repo: deb https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO VERSION main state: present become: true`} {`- name: Import Cloudsmith Repository GPG key apt_key: url: https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key state: present become: true - name: Add Cloudsmith Repository apt_repository: repo: deb https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO VERSION main state: present become: true`} ### Install a package To install a Debian package via Ansible, you use the Ansible [apt module](https://docs.ansible.com/ansible/2.3/apt_module.html): ```yaml - name: Install a package apt: name: PACKAGE_NAME state: present update_cache: yes become: true ``` # ArgoCD You can integrate Cloudsmith with ArgoCD to automate the deployment of your Kubernetes applications using Helm charts and Docker images securely stored in Cloudsmith repositories. This guide covers the steps to set up both Helm charts and Docker images with Cloudsmith in ArgoCD, including secure authentication for private repositories. This integration supports pulling: - [Docker](/formats/docker-registry) - [Helm](/formats/helm-chart-repository) ## With this integration, you can - Use Cloudsmith to securely store and manage Helm charts and container images. - Automate deployments of applications to Kubernetes clusters using ArgoCD. - Maintain version control of your deployment artifacts. ## What you’ll need - A Cloudsmith account with a repository to store your Helm charts or container images. - An ArgoCD installation configured to deploy applications to your Kubernetes cluster. - API keys or OIDC tokens for secure authentication with private Cloudsmith repositories. *** ## Getting Started ### Step 1: Tag and Push the Docker Image to Cloudsmith ```text bash docker tag your-existing-image:latest docker.cloudsmith.io/your-namespace/your-repo/image-name:latest docker push docker.cloudsmith.io/your-namespace/your-repo/image-name:latest ``` **Update pull string** Replace `your-existing-image:latest` with the name of your Docker image, and ensure that the target repository details (your-namespace/your-repo) match your Cloudsmith setup. This command uploads your Docker image to the specified Cloudsmith repository, where it will be available for deployment. ### Step 2: Handling Image Pull Secrets If your Docker image is stored in a private Cloudsmith repository, you’ll need to create an image pull secret for Kubernetes to pull the image. **Create the Docker Registry Secret for Image Pull** ```text bash kubectl create secret docker-registry cloudsmith-regcred \ --docker-server=docker.cloudsmith.io \ --docker-username=your-username \ --docker-password=your-api-key \ --docker-email=your-email@example.com \ -n default ``` This secret allows your Kubernetes cluster to authenticate with Cloudsmith to pull the Docker image securely. **Reference the Image Pull Secret in Deployment** Ensure that the Helm chart or deployment configuration references the image pull secret: ```yaml yaml spec: template: spec: imagePullSecrets: - name: cloudsmith-regcred ``` ### Step 3: Publish Helm Chart to Cloudsmith If you have a Helm chart that defines your application’s deployment, you can push it to Cloudsmith as follows: ```scss bash helm package ./my-test-app cloudsmith push helm your-namespace/your-repo ./my-test-app-0.1.0.tgz ``` ### Step 4: Add Cloudsmith Helm Repository to ArgoCD ArgoCD requires that you add the Helm repository from Cloudsmith to access the chart during deployments. You can do this using the ArgoCD CLI or the Web UI. **Adding the Helm Repository via ArgoCD CLI** ```text bash argocd repo add --name cloudsmith-helm --type helm https://dl.cloudsmith.io/basic/WORKSPACE/REPOSITORY/helm/charts/ \ --username your-username \ --password your-api-key ``` - dl.cloudsmith.io/basic/WORKSPACE/REPOSITORY/helm/charts/: Replace with the correct URL of your Cloudsmith Helm repository. - \--username and --password: Please use your Cloudsmith username with the API key for authentication, as using username and password is deprecated. **Entitlement Tokens** Where possible, use Entitlement tokens instead of API keys for more secure and flexible authentication when integrating with ArgoCD. Entitlement tokens are easier to manage and can be rotated more frequently for enhanced security. In case of Entitlement token the command for adding the helm repository via ArgoCD CLI will be: ```text bash argocd repo add --name cloudsmith-helm --type helm \ https://dl.cloudsmith.io/TOKEN/WORKSPACE/REPOSITORY/helm/charts/ ``` ### Step 5: Create an ArgoCD Application Manifest Configure your ArgoCD application to deploy the Helm chart from Cloudsmith to your Kubernetes cluster using the following manifest: ```yaml argocd-application.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-cloudsmith-app namespace: argocd spec: project: default source: repoURL: "https://dl.cloudsmith.io/TOKEN/WORKSPACE/REPOSITORY/helm/charts/" targetRevision: "0.1.0" chart: my-test-app destination: server: https://kubernetes.default.svc namespace: default syncPolicy: automated: prune: true selfHeal: true ``` **Explanation** - **repoURL**: Points to the Helm chart repository hosted on Cloudsmith. - **targetRevision**: Specifies the version of the Helm chart you want to deploy. - **chart**: The name of the chart in the Cloudsmith repository. ### Step 6: Deploy the Application with ArgoCD 1. Apply the Application manifest to ArgoCD: ```text bash kubectl apply -f argocd-application.yaml ``` 2. Sync the application to start the deployment process: ```shell bash argocd app sync my-cloudsmith-app ``` 3. Sync the application to start the deployment process: ```shell bash argocd app get my-cloudsmith-app ``` ## Best Practices - **Use Entitlement token Authentication**: As ArgoCD only pulls images, use Entitlement tokens for authentication to avoid using long-lived API keys. - **Secure Secrets**: Store sensitive information like API keys/Entitlement tokens in Kubernetes secrets instead of hardcoding them in your deployment manifests. - **Enable Auto-Sync**: Configure ArgoCD’s auto-sync feature to ensure that your Kubernetes clusters always have the latest artifacts. ## Troubleshooting - **Image Pull Issues**: Verify that the image pull secret is correctly configured in the workspace where your application is running. - **Helm Chart Errors**: Ensure that the Helm chart version specified in the ArgoCD manifest exists in the Cloudsmith repository. # Integrating AWS CodeBuild How to integrate AWS CodeBuild with Cloudsmith Cloudsmith can be used as a target for all the assets created using AWS CodeBuild. This guide shows you how to use the Cloudsmith CLI to upload/push a Debian package to your Cloudsmith repo. ## Getting Started Integrating Cloudsmith as part of your AWS CodeBuild project is as simple as installing the Cloudsmith CLI during your build and then using the `cloudsmith push` command to upload the artifacts from your build process to your Cloudsmith repository. ### Adding your API Key to AWS In order to use the Cloudsmith CLI with AWS CodeBuild, we would recommend that you add your Cloudsmith API-Key as an environment variable for the project. You can do this using AWS Secrets Manager. We do not recommend adding the API-Key itself directly into the `buildspec.yaml` file for the AWS CodeBuild project as it will then be revealed in any resulting logs from the build. To reference a secret stored in AWS Secrets Manager in your AWS CodeBuild `buildspec.yaml` file you would use the following syntax: ```yaml env: secrets-manager: CLOUDSMITH_API_KEY: CodeBuild/CloudsmithAPI:CLOUDSMITH_API_KEY ``` For further details on obtaining your Cloudsmith API-Key see: - [Retrieve your Cloudsmith API Key](/accounts-and-teams/api-key) For further details on using AWS Secrets Manager with AWS CodeBuild see: - [AWS CodeBuild Environment Variables](https://docs.aws.amazon.com/codebuild/latest/APIReference/API_EnvironmentVariable.html) - [AWS CodeBuild env/secretsmanager](https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#secrets-manager-build-spec) ### Adding the Cloudsmith CLI to your CodeBuild Project To add the Cloudsmith CLI to your AWS CodeBuild Project, add the following command to the `install` phase of your `buildspec.yaml` file: ```yaml install: commands: - pip install cloudsmith-cli ``` ### Uploading a built artifact to Cloudsmith To upload an artifact from a build to a Cloudsmith repository, add the `cloudsmith push` command to the `post build` phase of your `buildspec.yaml` file: ```yaml post_build: commands: - cloudsmith push FORMAT OWNER/REPOSITORY FILENAME ``` Please see the [Cloudsmith CLI](/developer-tools/cli) documentation for more details of the syntax of the `cloudsmith push` command and the [Supported Formats](/formats) page for examples of the `cloudsmith push` command for each supported format. ## Example buildspec.yaml The following is an example of a complete (but minimal) `buildspec.yaml` file (as used in our demo video) that will build a Debian package from a source GitHub repository specified in the AWS console project settings, and then upload it to a Cloudsmith repository: ```yaml version: 0.2 env: secrets-manager: CLOUDSMITH_API_KEY: CodeBuild/CloudsmithAPI:CLOUDSMITH_API_KEY phases: install: runtime-versions: python: 3.x commands: - apt update - apt-get install ruby ruby-dev rubygems build-essential -y - gem install --no-document fpm - pip install cloudsmith-cli build: commands: - make - fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-codebuild-test . post_build: commands: - echo "Uploading deb package to Cloudsmith repo" - cloudsmith push deb demo/codebuild-demo/debian/buster cloudsmith-codebuild-test_1.0.1_amd64.deb ``` # Integrating AWS SageMaker How to integrate AWS SageMaker with Cloudsmith Cloudsmith can be used as a secure source for your machine learning models and dependencies within AWS SageMaker workflows. This guide demonstrates how to integrate Cloudsmith with SageMaker for training and inference jobs. For a complete working example, please refer to our [Cloudsmith + SageMaker Demo Repository](https://github.com/cloudsmith-io/cloudsmith-sagemaker-demo). ## Uploading Models In your SageMaker training script, after model training is complete, you can use Cloudsmith's Hugging Face-compatible endpoint to upload models to your Cloudsmith repository. ```python from huggingface_hub import HfApi import os ws = os.environ['CLOUDSMITH_WORKSPACE'] repo = os.environ['CLOUDSMITH_REPO'] token = os.environ['CLOUDSMITH_API_KEY'] # or fetched from Secrets Manager endpoint = f"https://huggingface.cloudsmith.io/{ws}/{repo}" api = HfApi(token=token, endpoint=endpoint) api.upload_folder( folder_path="/opt/ml/model", # directory with model files repo_id="distilbert-base-uncased-finetuned", repo_type="model", token=token, ) ``` ## Downloading Models You can download models in your inference container or any client using the same endpoint. ```python from huggingface_hub import HfApi import os ws = os.environ['CLOUDSMITH_WORKSPACE'] repo = os.environ['CLOUDSMITH_REPO'] token = os.environ['CLOUDSMITH_API_KEY'] endpoint = f"https://huggingface.cloudsmith.io/{ws}/{repo}" api = HfApi(token=token, endpoint=endpoint) local_dir = "/opt/ml/model" api.snapshot_download( repo_id="distilbert-base-uncased-finetuned", # or base model name repo_type="model", revision="main", # or a specific uploaded revision hash/tag local_dir=local_dir, token=token, ) ``` ## Notes ### VPC Configuration SageMaker must reach the private Cloudsmith registry (container + Hugging Face endpoints) over the public internet. When you use a private image, you launch the training job / endpoint inside a VPC so the underlying instance has: 1. Private subnets where the job runs. 2. An egress path (NAT) so those subnets can reach `docker.cloudsmith.io` and `huggingface.cloudsmith.io`. 3. A security group that allows outbound HTTPS. ### ECR (For inference) AWS does not currently authenticate to private third-party registries during SageMaker CreateModel for real-time inference; only ECR (or public/no-auth images) are supported. # Azure DevOps _Effortlessly integrate Cloudsmith with Azure DevOps to securely manage artifacts and automate deployments._ ## Introduction Cloudsmith integrates directly with **Azure DevOps** using **OpenID Connect (OIDC)** — enabling secure, short-lived authentication without Azure Entra app registrations or long-lived credentials. This guide explains how to configure Cloudsmith and Azure DevOps to authenticate pipelines and access Cloudsmith repositories seamlessly. --- ## What You’ll Need Before getting started, ensure you have: - An **Azure DevOps** organization and project with permissions to create pipelines. - A **Cloudsmith workspace** (organization) with access to manage service accounts and OIDC providers. - The [**Cloudsmith CLI Setup & Authenticate**](https://marketplace.visualstudio.com/items?itemName=Cloudsmith.CloudsmithCliSetupAndAuthenticate) Azure DevOps extension installed. --- ## Integration Features - **Native OIDC Authentication** – Azure DevOps issues short-lived tokens that Cloudsmith exchanges for temporary API keys (~90-minute lifetime). - **Automatic CLI Setup** – Optionally installs and authenticates the Cloudsmith CLI for use in your pipeline. - **Cross-Platform** – Works on Linux, Windows, and macOS agents. - **No Entra App Needed** – Uses Azure DevOps’ native OIDC provider directly. --- ## Step 1: Create an OIDC Provider in Cloudsmith 1. In your Cloudsmith workspace, go to **Settings → Authentication → OpenID Connect → Create Provider**. 2. Fill in the following details: - **Provider Name:** `azure-devops` - **OIDC Provider URL:** ``` https://vstoken.dev.azure.com/ ``` Replace `` with your Azure DevOps **organization ID (GUID)**. You can find it at [https://app.vssps.visualstudio.com/_apis/accounts](https://app.vssps.visualstudio.com/_apis/accounts) — look for the `"AccountId"` value that corresponds to your organization, for example: ```json {"AccountId": "12345678-abcd-4321-ef00-987654321000", "AccountName": "acme-corp"} ``` - **Recommended Token Claims:** We strongly recommend including at least one claim to control which Azure DevOps pipelines can authenticate with Cloudsmith. Claims help ensure that only specific projects or repositories within your organization can access a given Cloudsmith service account. Example: ```json { "sub": "p://acme-corp/cloudsmith-integration/.*", "prj_id": "a1b2c3d4-e5f6-7890-abcd-1234567890ef" } ``` **Explanation:** - `sub` — Identifies the Azure DevOps organization and project. In this example, only pipelines in the `cloudsmith-integration` project of the `acme-corp` organization can authenticate. The `.*` wildcard allows all pipelines in that project to use the same Cloudsmith OIDC mapping. - `prj_id` — The Azure DevOps project ID (GUID). You can use this for tighter scoping if needed. You can view your Azure DevOps organization and project IDs using: [https://app.vssps.visualstudio.com/_apis/accounts](https://app.vssps.visualstudio.com/_apis/accounts) — this lists your organization GUIDs and other identifiers that also appear as claims in your Azure DevOps OIDC tokens. **Wildcard usage** Wildcards are only supported at the **end** of a claim value. For example: `p://acme-corp/cloudsmith-integration/*` - **Service Accounts:** Select the Cloudsmith service account(s) that this provider will authenticate (for example, `ado-builds-service`). 3. Click **Create Settings** to save. **Token lifetime** Azure DevOps OIDC tokens last roughly **90 minutes**. The Cloudsmith Azure DevOps task automatically exchanges these for temporary Cloudsmith API keys during each pipeline run. [Image: Cloudsmith Azure OIDC provider configuration] --- ## Step 2: Add the Cloudsmith Task to Your Azure Pipeline Install the [Cloudsmith CLI Setup & Authenticate](https://marketplace.visualstudio.com/items?itemName=Cloudsmith.CloudsmithCliSetupAndAuthenticate) Azure DevOps extension. It handles both OIDC authentication and optional CLI installation. Example pipeline using native Azure DevOps OIDC: ```yaml # azure-pipelines.yml trigger: - main pool: vmImage: 'ubuntu-latest' variables: # ---- Cloudsmith settings ---- CS_WORKSPACE: 'example-workspace' CS_REPO: 'backend-app' CS_SERVICE_SLUG: 'ado_build_service' # Cloudsmith service account slug steps: - checkout: self # Authenticate to Cloudsmith via OIDC (no Entra setup required) - task: CloudsmithCliSetupAndAuthenticate@1.1.1 displayName: "Cloudsmith: Authenticate via OIDC" inputs: authMethod: 'oidc' oidcAuthOnly: false # install CLI so 'cloudsmith whoami' works pipInstall: false cliVersion: '' oidcNamespace: '$(CS_WORKSPACE)' oidcServiceSlug: '$(CS_SERVICE_SLUG)' # Example: install dependencies from a Cloudsmith-hosted Python repo - task: UsePythonVersion@0 inputs: versionSpec: '3.12' displayName: "Use Python 3.12" - bash: | set -eo pipefail python -m pip install --upgrade pip pip install -r requirements.txt \ --index-url "https://${CS_SERVICE_SLUG}:${CLOUDSMITH_API_KEY}@dl.cloudsmith.io/basic/${CS_WORKSPACE}/${CS_REPO}/python/simple/" displayName: "pip install from Cloudsmith" env: CLOUDSMITH_API_KEY: $(CLOUDSMITH_API_KEY) ## Step 3: Verify CLI Installation and Authentication Save and run your pipeline to complete the setup. After running the pipeline, the extension will install the Cloudsmith CLI, authenticate based on the chosen method, and validate the setup by running: ```bash cloudsmith whoami ``` You can check the pipeline logs to confirm successful authentication. --- ## Step 4: Start Using the Cloudsmith CLI Seamlessly in Your Pipeline The following example demonstrates how to use the Cloudsmith CLI extension with OIDC authentication to push a package to Cloudsmith directly from an Azure DevOps pipeline: ```yaml trigger: branches: include: - main # Trigger pipeline on changes to the main branch pool: vmImage: 'ubuntu-latest' steps: - task: CloudsmithCliSetupAndAuthenticate@1.1.1 displayName: "Authenticate with Cloudsmith via OIDC" inputs: authMethod: 'oidc' oidcNamespace: '$(example-workspace)' oidcServiceSlug: '$(ado_build_service)' cliVersion: 'latest' # Example task: push a raw package to Cloudsmith - script: | cloudsmith push raw $(example-workspace)/$(backend-app) ./dist/my-package.zip displayName: "Push package to Cloudsmith" ``` Once authenticated, you can perform any Cloudsmith CLI operation — such as pushing, pulling, or managing packages — directly within your Azure DevOps pipeline. **Pipeline steps** Ensure that all pipeline steps that call the Cloudsmith CLI run on the same pipeline runner as the Cloudsmith authentication task. --- ## Best Practices - **Use OIDC**: Use OIDC authentication instead of API keys for enhanced security. - **Use Secure Variables**: Store sensitive values (e.g., `CLOUDSMITH_API_KEY`, workspace/repo names) as Azure DevOps secrets rather than hard-coding them in YAML. - **Use Latest CLI Versions**: Keep the Cloudsmith CLI updated to leverage the latest features and security patches. If specific versions are required, specify them explicitly in your configuration. ## Troubleshooting - **Authentication Issues**: - Ensure all authentication variables are correctly set and valid for either API Key or OIDC. - Verify that the correct service account is selected in Cloudsmith for the OIDC provider. - **Permission Errors**: Verify that the Azure DevOps agent has permission to download and install the CLI (for self-hosted agents). - **PATH Issues with Cloudsmith CLI**: Ensure you’re using the same runner for all steps, as switching runners may lose CLI installation and configuration. - **OIDC authentication failures**: Ensure the Provider URL exactly matches `https://vstoken.dev.azure.com/` for your organization. If you added `sub` or other claims, confirm they match the token your pipeline issues. # Integrating Bitbucket Pipelines How to integrate Bitbucket Pipelines with Cloudsmith Cloudsmith provides first-class support for Bitbucket Pipelines with our official **Cloudsmith Publish** pipe. Using the pipe, Bitbucket users can easily integrate publishing to Cloudsmith with their existing pipeline workflows. **[cloudsmith-pipes-publish](https://bitbucket.org/cloudsmith-io/publish/)**: Bitbucket Pipe for publishing artifacts Content below first appeared on the [Atlassian Community Blog](https://community.atlassian.com/t5/Marketplace-Apps-Integrations/Cloudsmith-Bitbucket/ba-p/1148691). ## Using Pipelines The Cloudsmith pipe is included in Bitbucket's collection of officially maintained pipes and is available by default for any user that wishes to include it in their pipeline. To use the pipe you'll first need an application or library that can be packaged into one of the formats that Cloudsmith support. You can see examples of Python and Javascript libraries in [Cloudsmith's Bitbucket account](https://bitbucket.org/cloudsmith-io/?_ga=2.35066210.2132496434.1565627316-1859711499.1556636245). For the purposes of this example we'll assume you're using the Python example from the link above. First, ensure that you can package your code using a Bitbucket pipeline by editing bitbucket-pipelines.yml in your repository. You can do so either via Bitbucket's web UI or using your normal editor and Git workflow. For the example library, a basic pipeline that builds a package when a new tag is created looks like so: ```text image: name: atlassian/default-image:2 build: &build step: name: Build Python Package image: python:3.7 script: - python setup.py sdist artifacts: - dist/** pipelines: tags: release-*: - <<: *build ``` This pipeline will build a new package each time you push an appropriate tag you your Bitbucket repository. The built package is then discarded as we have not yet added further instructions to tell Bitbucket what to do with it. ## Configuring the Pipe Next we'll add publish the package to Cloudsmith using our official pipe. We'll need to provide an API Key that the pipe can use to authenticate with Cloudsmith. You can find the official documentation for pipeline variables in the [Bitbucket documentation](https://confluence.atlassian.com/bitbucket/variables-in-pipelines-794502608.html?_ga=2.261427278.2132496434.1565627316-1859711499.1556636245). The API Key should be added as a "secure" variable (so it doesn't leak into logs) with the name `CLOUDSMITH_API_KEY`. Once authentication is configured, you can add the pipe configuration to your pipeline. If using the web UI to configure your pipeline, you can select the Cloudsmith pipe right from your browser, from within the list of supported pipes as in the image below: If using your local editor, you can follow the instructions in the [official README on Bitbucket](https://bitbucket.org/cloudsmith-io/publish?_ga=2.189666536.2132496434.1565627316-1859711499.1556636245). Once added, your pipeline YAML should look something like so: ```text image: name: atlassian/default-image:2 build: &build step: name: Build Python Package image: python:3.7 script: - python setup.py sdist artifacts: - dist/** publish: &publish step: name: Publish Python Package script: - pipe: cloudsmith-io/publish:0.1.1 variables: CLOUDSMITH_REPOSITORY: 'cloudsmith/examples' CLOUDSMITH_API_KEY: $CLOUDSMITH_API_KEY PACKAGE_FORMAT: 'python' PACKAGE_PATH: 'dist/*.tar.gz' pipelines: tags: release-*: - <<: *build - <<: *publish ``` This configuration instructs the pipe to push artifacts to the [cloudsmith/examples](https://app.cloudsmith.com/cloudsmith/examples) repository, with the API key we stored earlier. We're uploading a python package which is located at dist/\*.tar.gz. ## Using the Pipe Your pipe is now configured and ready to run. You should be able to push a new tag to Bitbucket and see your pipe run. Bitbucket's pipeline UI provides a great overview of the status of your jobs: Once pushed, you should see your new package in the Cloudsmith UI, ready for download and install using your preferred tools: ## Conclusion Cloudsmith's Bitbucket pipe provides the easiest and simplest way for Bitbucket users to push their assets to [Cloudsmith](https://cloudsmith.com/). Get started now. **Officially Maintained** Our official pipe is maintained by the Cloudsmith team and you can be sure it'll be kept up to date with all changes and features as we release them. # Integrating Buildkite How to integrate Buildkite with Cloudsmith Buildkite is a platform for running fast, secure, and scalable continuous integration pipelines on your own infrastructure. Use Buildkite to orchestrate and manage your own fleet of build hosts, from containers, to cloud instances, to bare metal servers. ## Getting Started Integrating Cloudsmith into your Buildkite pipeline is as simple as installing the Cloudsmith CLI during one of your pipeline steps and then using the `cloudsmith push` command to upload the artifacts from your pipeline to your Cloudsmith repository. ### Adding your API Key to your Buildkite environment For details on obtaining your Cloudsmith API-Key see: - [Retrieve your Cloudsmith API Key](/accounts-and-teams/api-key) In order to use the Cloudsmith CLI within a Buildkite pipeline, you'll need to configure the environment on your build host(s) with your Cloudsmith API Key. Your Cloudsmith API Key should be considered as a secret and should not be committed into any source repositories or stored as a plain text variable in the Buildkite pipeline. One of the easiest ways to set up your environment is to use Buildkite environment hooks, the environment hooks are scripts on a build host that run before each command in a pipeline. A simple example to set up your Cloudsmith API key would be: ```shell set -euo pipefail if [[ "$BUILDKITE_PIPELINE_SLUG" == "your-pipeline-name" ]]; then export CLOUDSMITH_API_KEY="abcdefghijklmnop1234567890" fi ``` For more information on hooks, please see the Buildkite documentation [here](https://buildkite.com/docs/agent/v3/hooks). There are several other methods of handling secrets in a Buildkite pipeline, you can find more details [here](https://buildkite.com/docs/pipelines/secrets) ### Adding the Cloudsmith CLI to your Buildkite pipeline To add the Cloudsmith CLI to your pipeline, just add the following command to one of your pipeline steps in your `pipeline.yml` file: ```yaml commands: - pip install cloudsmith-cli ``` An example of a setup step that installs Ruby, some tooling, and the Cloudsmith CLI would look like: ```yaml steps: - label: "PreBuild" commands: - sudo apt update - sudo apt-get install ruby ruby-dev rubygems build-essential python-pip -y - sudo gem install --no-document fpm - pip install cloudsmith-cli ``` You can also add the `pip install cloudsmith-cli` command to a pipeline step using the Buildkite Web UI, if not using a `pipeline.yml` file: ### Pushing a build artifact to a Cloudsmith repository To push an artifact from your Buildkite pipeline to a Cloudsmith repository, you use the `cloudsmith push` command in one of your pipeline steps: ```yaml commands: - cloudsmith push FORMAT OWNER/REPOSITORY FILENAME ``` Please see the [Cloudsmith CLI](/developer-tools/cli) documentation for more details of the syntax of the `cloudsmith push` command and the [Supported Formats](/formats) page for examples of the `cloudsmith push` command for each supported format. An example of this command in a step that builds a Debian package would look like: ```yaml - label: "BuildAndPushPackage" commands: - make - fpm -f -s dir -t deb -v 1.0.0 -n cloudsmith-buildkite-test . - cloudsmith push deb demo/buildkite-demo/debian/buster cloudsmith-buildkite-test_1.0.0_amd64.deb ``` Again, you can also add the `cloudsmith push` command to a pipeline step using the Buildkite Web UI, if not using a `pipeline.yml` file: **Buildkite Pipeline Steps** Pipeline steps in Buildkite are stateless and as a result, if you have a fleet of agents then each step is not guaranteed to run on the same agent. This means that if you build an artifact in one step, and want to push it to a Cloudsmith repository in a subsequent step, the artifact will need to be stored temporarily and then retrieved by the push step. Buildkite provides temporary artifact storage that you can use for this purpose (see [here](https://buildkite.com/docs/pipelines/artifacts) for more details). # Integrating with Chainguard Images How to integrate with Chainguard Images using Cloudsmith [Chainguard](https://www.chainguard.dev), a Docker Verified Publisher, offers Chainguard Images which are a collection of minimal, hardened Docker container images featuring: - Zero CVEs - Includes SBOMs and signatures - Minimal, containing only the application and its runtime dependencies You can retrieve these images through Cloudsmith by enabling the upstream to Chainguard. Chainguard offers both a Public Registry (`cgr.dev/chainguard`) containing developer images and a Private/Dedicated Registry (`cgr.dev/chainguard-private`) which includes all versioned tags of an image and special images not available in the public registry (including FIPS images and other custom builds). ## Adding Chainguard as an Upstream Here's how you can integrate the Chainguard Registry into your Cloudsmith account: 1. **Configure Upstream Proxying** In your Cloudsmith repository, go to the Upstream Proxying settings. Click the green "Create Upstream" button and select the Docker format. Provide a descriptive name for the upstream, e.g., Chainguard Public, and specify the URL for the Chainguard Registry. Enter the Chainguard Registry URL: - For Chainguard's public images: `https://cgr.dev/chainguard` - For Chainguard's Private/Dedicated Registry: `https://cgr.dev` Set the desired priority. Select Cache and Proxy. 2. **Configure SSL Certificate Verification** Ensure SSL certificates are verified for added security, especially for public sources. 3. **Authentication and Headers** If you are using the private URL, Chainguard requires authentication or additional headers; provide them in the respective fields. ## Pull a Chainguard Image with Docker Native Tooling Here's an example of how you would pull the nginx Chainguard Docker image into Cloudsmith after you've configured your Cloudsmith upstream for Chainguard: 1. **Configure your Cloudsmith upstream** for Chainguard using the instructions above. 2. **Ensure Docker is installed** on your system. If not, go [here](https://docs.docker.com/engine/install/) to get started with Docker. 3. **Open a terminal.** 4. **Login to Docker** with your Cloudsmith username and token, with the command: docker login docker.cloudsmith.io 5. **Pull the latest Chainguard nginx image** by running: docker pull docker.cloudsmith.io/ORGANIZATION/REPOSITORY/chainguard/nginx:latest Note: Replace ORGANIZATION and REPOSITORY with your Cloudsmith organization and repository, respectively. 6. **Check your Cloudsmith repository** to find the newly added Chainguard nginx image. # Integrating Chef How to integrate Chef with Cloudsmith Chef is a configuration management tool written in Ruby and Erlang. It uses a Ruby DSL for writing "recipes". Chef recipes are then used automate and maintain system configurations. - [Chef](https://www.chef.io/) : The Chef website. - [Chef Docs](https://docs.chef.io/) : The Official Chef Documentation In the following examples: | Identifier | Description | | :---------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository identifier (also called "slug") | | DISTRO | Your distribution (i.e el, fedora, debian etc) | | VERSION | Your version name (i.e 7, 29, hardy, buster etc) | | ARCH | The architecture (i.e x86_64) | | FINGERPRINT | The 8 Byte fingerprint of the Public GPG key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | ## Adding a RPM repository To configure a Cloudsmith repository for rpm packages using Chef, you use the Chef [yum_repository](https://docs.chef.io/resources/yum_repository/) resource. Example `yum_repository` resource configurations: **Public Repository** ```ruby yum_repository 'Cloudsmith' do description 'Cloudsmith' baseurl 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch' gpgcheck 'true' gpgkey 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' action :create end ``` **Private Repository** ```ruby Entitlement Token Auth yum_repository 'Cloudsmith' do description 'Cloudsmith' baseurl 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch' gpgcheck 'true' gpgkey 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' action :create end ``` ```ruby HTTP Basic Auth (User & Pass) yum_repository 'Cloudsmith' do description 'Cloudsmith' baseurl 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch' gpgcheck 'true' gpgkey 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' username 'USERNAME' password 'PASSWORD' action :create end ``` ```ruby HTTP Basic Auth (API-Key) yum_repository 'Cloudsmith' do description 'Cloudsmith' baseurl 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch' gpgcheck 'true' gpgkey 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' username 'USERNAME' password 'API-KEY' action :create end ``` ```ruby HTTP Basic Auth (Token) yum_repository 'Cloudsmith' do description 'Cloudsmith' baseurl 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/rpm/DISTRO/VERSION/$basearch' gpgcheck 'true' gpgkey 'https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' username 'token' password 'TOKEN' action :create end ``` ## Adding a deb repository To configure a Cloudsmith repository for deb packages using Chef, you use the Chef [apt_repository](https://docs.chef.io/resources/apt_repository/) resource. Example `apt_repository` resource configurations: **Public Repository** ```ruby apt_repository 'Cloudsmith' do uri 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/deb/DISTRO' arch 'ARCH' distribution 'VERSION' components ['main'] key 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' end ``` **Private Repository** ```ruby Entitlement Token Auth apt_repository 'Cloudsmith' do uri 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/deb/DISTRO' arch 'ARCH' distribution 'VERSION' components ['main'] key 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' end ``` ```ruby HTTP Basic Auth (User & Pass) apt_repository 'Cloudsmith' do uri 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO' arch 'ARCH' distribution 'VERSION' components ['main'] key 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' end ``` ```ruby HTTP Basic Auth (API-Key) apt_repository 'Cloudsmith' do uri 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO' arch 'ARCH' distribution 'VERSION' components ['main'] key 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' end ``` ```ruby HTTP Basic Auth (Token) apt_repository 'Cloudsmith' do uri 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO' arch 'ARCH' distribution 'VERSION' components ['main'] key 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key' end ``` # Integrating CircleCI How to integrate CircleCI with Cloudsmith Cloudsmith provides first-class support for CircleCI with our official [orb](https://circleci.com/orbs/registry/orb/cloudsmith/cloudsmith). Using the orb, users can easily integrate publishing to Cloudsmith with their existing CircleCI workflows. Full reference documentation for the orb can be found on the [CircleCI website](https://circleci.com/orbs/registry/orb/cloudsmith/cloudsmith). This documentation is automatically generated from the orb itself and so is guaranteed to always be up to date with the latest release of the orb. **[cloudsmith-circleci-orb](https://github.com/cloudsmith-io/orb)**: A reusable orb to integrate CircleCI ## Orb usage example To use the orb you must first ensure you are using Circle version 2.1. At the top of your `.circleci/config.yml` file you should add: ```yaml version: 2.1 ``` And then include the orb: ```yaml orbs: cloudsmith: cloudsmith/cloudsmith@2.0.0 ``` Note that you can check the [releases page on GitHub](https://github.com/cloudsmith-io/orb/releases) for our orb, [or the orb page](https://circleci.com/developer/orbs/orb/cloudsmith/cloudsmith) on CircleCI itself to find the latest version to use. ### OIDC Authentication (recommended) The recommended approach is to authenticate via OIDC, install the CLI, and invoke it directly in your run steps. This uses short-lived tokens and avoids storing long-lived API keys: ```yaml version: 2.1 orbs: cloudsmith: cloudsmith/cloudsmith@2.0.0 workflows: cloudsmith_oidc_publish: jobs: - publish jobs: publish: executor: cloudsmith/default steps: - checkout - cloudsmith/authenticate-with-oidc: organization: my-org service-account: my-service-account - cloudsmith/install-cli - run: name: Build and publish Python package command: | pip install build python -m build --wheel cloudsmith push python my-org/my-repo dist/*.whl ``` The `authenticate-with-oidc` command exchanges a CircleCI OIDC token for a short-lived Cloudsmith API token, exported as `CLOUDSMITH_API_KEY`. Ensure OIDC is enabled in your CircleCI project settings (Project Settings → Advanced → Enable OpenID Connect Tokens). If you only need the API token and don't need to install the CLI, you can use `authenticate-with-oidc` on its own: ```yaml jobs: authenticate: executor: cloudsmith/default steps: - checkout - cloudsmith/authenticate-with-oidc: organization: my-org service-account: my-service-account - run: name: Call Cloudsmith API command: | curl -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ https://api.cloudsmith.io/user/self/ ``` ### API Key Authentication If OIDC is not available in your environment, you can authenticate with a static API key. Add an environment variable named `CLOUDSMITH_API_KEY` in your CircleCI project settings: [Image: Screenshot of CircleCI project environment variable settings showing CLOUDSMITH_API_KEY configured.] Then use `ensure-api-key` to validate the key is set before running CLI commands: ```yaml version: 2.1 orbs: cloudsmith: cloudsmith/cloudsmith@2.0.0 workflows: cloudsmith_publish: jobs: - publish jobs: publish: executor: cloudsmith/default steps: - checkout - cloudsmith/ensure-api-key - cloudsmith/install-cli - run: name: Build and publish Python package command: | pip install build python -m build --wheel cloudsmith push python my-org/my-repo dist/*.whl ``` The `cloudsmith/default` executor uses the `cimg/python` convenience image. If you need a different base image, you can specify your own executor and the orb commands will work in any environment that has the required dependencies (Python, curl, jq). ## Support As always, if you have any questions about integration or would like some general advice, please [contact support](https://cloudsmith.com/company/contact-us). # Integrating Codefresh How to integrate Codefresh CI/CD with Cloudsmith # Cloudsmith Integration with Codefresh [Codefresh](https://codefresh.io/) is a CI/CD platform designed for Kubernetes and modern microservices, offering streamlined workflows for building, testing, and deploying applications with integrated Docker, Helm, and Kubernetes support. Cloudsmith can seamlessly work with Codefresh to authenticate, consume, and publish artifacts from Cloudsmith in your Codefresh CI/CD pipelines. ## Prerequisites - **Cloudsmith Account**: Ensure you have an active Cloudsmith account with a repository where your artifacts will be stored. - **Codefresh Account**: Ensure you have an active Codefresh account. - **Kubernetes Cluster**: A Kubernetes cluster where you can deploy your applications. ## Artifact management with Cloudsmith and Codefresh You can manage artifacts between Cloudsmith and Codefresh in several ways: - **Uploading Packages**: - Use the Cloudsmith [Command-Line Interface](/developer-tools/cli). - Use native package management tools such as Docker, pip gem or cargo. - Use Codefresh's native Docker build and push steps for containerized workflows. - **Deploying Artifacts**: Deploy Docker images and Helm charts stored in Cloudsmith repositories directly from Codefresh pipelines. ## Authentication Options - API Key: You can authenticate to Cloudsmith using an API key stored in Codefresh. - OIDC Authentication: Alternatively, set up Codefresh as an OIDC provider to authenticate dynamically without manual credential management. ### Adding Your Cloudsmith API Key in Codefresh Steps to [add encrypted variables](https://codefresh.io/docs/docs/pipelines/variables/) in your Codefresh pipelines: 1. Open your Codefresh pipeline and go to the Settings tab. 2. Click on Environment Variables and select Add New Variable. 3. Provide a name (e.g., `CLOUDSMITH_API_KEY`), input the value, and toggle Encrypt Value to ensure it's secure. 4. You can reference the secret in your pipeline YAML like this: `${{CLOUDSMITH_API_KEY}}`. For more security options, Codefresh offers a Vault plugin from the Step Marketplace to handle key-value pairs dynamically. Refer to [Vault Secrets](https://codefresh.io/docs/docs/example-catalog/ci-examples/vault-secrets-in-the-pipeline/) in the Pipeline for more details. ### Setup OIDC To authenticate securely, set up **Codefresh as an OIDC provider** for Cloudsmith. **OIDC Authentication** Codefresh does not support OIDC for Docker integrations natively. You can use OIDC for [freestyle steps](https://codefresh.io/docs/docs/pipelines/steps/freestyle/) when using the Hybrid Runtime, which allows access to the Docker Daemon. From there, you can manually log in to Docker, build your image, and push it to Cloudsmith. However, this means you will not be able to use the built-in build and push steps provided by Codefresh. Instead, you'll need to handle the authentication and image operations within freestyle steps. #### Add Codefresh as OIDC identity provider in Cloudsmith Set up **Codefresh as an OIDC provider** for Cloudsmith. This allows your Codefresh pipelines to request Cloudsmith API tokens dynamically, without manual credential management. 1. **Create a [Service Account](/accounts-and-teams/service-accounts)** in Cloudsmith (required for OIDC to work with Cloudsmith). 2. **Configure [OpenID Connect](/authentication/openid-connect) in Cloudsmith**: - Provider URL: `https://oidc.codefresh.io` - Claims: For example add your **Codefresh account ID** to restrict access. For more information on claims consult the [Codefresh documentation](https://codefresh.io/docs/docs/integrations/oidc-pipelines/#claims--conditions). - Assign the **service account** you created earlier. #### Obtain OIDC ID token from OIDC provider Obtain the ID token from the Codefresh OIDC provider by using the `obtain-oidc-id-token` Marketplace step. Add the step to your Codefresh pipeline's workflow. ```yaml version: '1.0' steps: obtain_id_token: title: Obtain ID Token type: obtain-oidc-id-token ``` For more details, refer to the [Cloudsmith OIDC documentation](/authentication/openid-connect) and [Codefresh OIDC documentation](https://codefresh.io/docs/docs/integrations/oidc-pipelines/). #### Example Pipeline: Publish a Python Package to Cloudsmith This example demonstrates how to authenticate and publish a Python package to Cloudsmith using OIDC in a Codefresh pipeline:: ```yaml version: "1.0" steps: obtain_id_token: title: "Obtain ID Token" type: obtain-oidc-id-token authenticate_with_cloudsmith: title: "Authenticate with Cloudsmith" image: "curlimages/curl:7.82.0" commands: - echo "Installing jq" - apk add --no-cache jq - | cloudsmith_token=$(curl -X POST -H "Content-Type: application/json" \ -d "{\"oidc_token\":\"$ID_TOKEN\", \"service_slug\":\"${{CS_SERVICE_USER_NAME}}\"}" \ https://api.cloudsmith.io/openid/${{CS_ORG}}/ | jq -r '.token') echo "Setting PIP_INDEX_URL with the obtained Cloudsmith token" export PIP_INDEX_URL="https://token:$cloudsmith_token@dl.cloudsmith.io/basic/${{CS_ORG}}/${{CS_REPO}}/python/simple/" cf_export PIP_INDEX_URL=$PIP_INDEX_URL ``` Replace `CS_SERVICE_USER_NAME`, `CS_ORG`, and `CS_REPO` with your service account, organization, and repository details. You can store these values as environment variables within your Codefresh pipeline. ## Adding Cloudsmith as a Docker Registry in Codefresh To add Cloudsmith as a Docker registry to Codefresh: - Go to settings and select Pipeline Integrations - Select Docker Registries and then click Add Registry Provider. - Select Other Registries - Define the following: - Registry name: A unique name for this configuration. - Username: Your Cloudsmith username. - Password: Your Cloudsmith API Key or password. - Domain: docker.cloudsmith.io - Prefix: For a Cloudsmith organization followed by the repository e.g.`CS_ORGANIZATION/CS_REPOSITORY` For more information consult the [Docker Registry](/formats/docker-registry) or [Codefresh documentation](https://codefresh.io/docs/docs/integrations/docker-registries/other-registries/). ## Adding Cloudsmith Helm Repository in Codefresh - Navigate to Helm Charts in CodeFresh: - In Codefresh, go to Artifacts > Helm Charts. - Add Existing Helm Repository: - Click on Add Existing Helm Repository. - Repository Name: Enter a unique name (e.g., Cloudsmith Helm). - Repository URL: `https://dl.cloudsmith.io/basic/WORKSPACE/REPOSITORY/helm/charts/` - Replace `WORKSPACE` and `REPOSITORY` with your Cloudsmith workspace and repository. - `HELMREPO_PASSWORD`: Add your Cloudsmith API token. - `HELMREPO_USERNAME`: Add your Cloudsmith username. - Save the Repository. # Deploying Artifacts from Cloudsmith Using Codefresh Codefresh offers several ways to deploy your Docker images and Helm charts to your Kubernetes cluster using artifacts stored in Cloudsmith: - Using the [Codefresh GUI ](https://codefresh.io/docs/docs/deployments/kubernetes/manual-deployment/)to deploy to Kubernetes on demand. - Select Docker images from your connected Cloudsmith Docker registry. - Deploy Helm charts from your added Cloudsmith Helm repository. - Deploying to [Kubernetes from a Codefresh pipeline](https://codefresh.io/docs/docs/pipelines/steps/deploy/). - Reference Docker images stored in Cloudsmith in your pipeline steps. - Use the Helm step in your pipeline to deploy charts from your Cloudsmith Helm repository. - Using the [Kubernetes templating cf-deploy-kubernetes](https://codefresh.io/docs/docs/ci-cd-guides/kubernetes-templating/). - Using [custom kubectl commands](https://codefresh.io/docs/docs/ci-cd-guides/kubernetes-templating/) in your Codefresh pipelines. - Execute kubectl commands within your pipeline, referencing artifacts from Cloudsmith. - Using [Helm deployment to Kubernetes](https://codefresh.io/docs/docs/quick-start/ci-quick-start/deploy-with-helm/). - Utilizing GitOps with ArgoCD: Integrate ArgoCD with Codefresh for advanced GitOps deployments, using Cloudsmith as your artifact source. For detailed instructions, refer to the [Codefresh documentation](https://codefresh.io/docs/docs/deployments/kubernetes/) on deployments. # Best Practices - Use OIDC: For authentication, prefer OIDC over API keys for better security. - Secure Credentials: Store sensitive information securely using Codefresh's encrypted variables or secret management features. - Automate Deployments: Leverage Codefresh pipelines and GitOps practices to automate deployment steps, reducing manual intervention. - Monitor Deployments: Use Codefresh's dashboards and integration with ArgoCD to monitor deployment status and health. - Follow GitOps Principles: Maintain your deployment manifests in Git repositories for version control and traceability. # Additional Resources Cloudsmith Documentation: - [Helm Chart Repository](/formats/helm-chart-repository) - [Docker Registry](/formats/docker-registry) - [OpenID Connect](/authentication/openid-connect) - [ArgoCD](/integrations/integrating-with-argocd) - [Octopus Deploy](/integrations/integrating-with-octopus-deploy) # Datadog [Datadog]() is an observability service for cloud-scale apps, providing monitoring of servers, databases, tools, and services through a SaaS-based data analytics platform. At Cloudsmith, we're big fans of Datadog and use it to monitor and visualize how our system is performing across a range of services and tools. The Datadog Cloudsmith Integration is a simple and effective visualization tool for your workspace artifact usage, package delivery, audit logs, and all of the security and compliance details of your assets. This integration also helps you monitor [consumption and usage](https://cloudsmith.com/blog/track-your-bandwidth-storage-limits-with-our-quota-api) within Cloudsmith, along with all of the audit logs and security scanning information we provide to help you spot if any suspicious activity is going on in your workspace. ## Metrics available The Datadog Cloudsmith integration tracks the following metrics organized in four main areas: - Artifact data and package delivery - Token Visibility - Workspace members - Events and Security Below you can find more information of each of them with some samples of how they look like. ### Artifact data and package delivery Monitor percentage usage and raw values (in bytes or gigabytes) for both artifact data (_storage_) and package delivery (_bandwidth_). Usage equal to 75% and 85% trigger a critical alerts, based on your plan limits. | Metric Name | Metric Type | Description | | :------------------------------------------------ | :---------- | :---------------------------------------------------------------------------------------------------------------------------- | | `cloudsmith.storage_used` | percent | The percentage of artifact data allowance used. | | `cloudsmith.storage_used_bytes` | byte | The amount of storage used in bytes. | | `cloudsmith.storage_used_gb` | byte | The storage used in gigabytes. | | `cloudsmith.storage_plan_limit_bytes` | byte | The storage limit in bytes defined by the plan. | | `cloudsmith.storage_plan_limit_gb` | byte | The storage limit in gigabytes defined by the plan. | | `cloudsmith.bandwidth_used` | percent | The percentage of package delivery allowance used. | | `cloudsmith.bandwidth_used_bytes` | byte | The amount of bandwidth used in bytes. | | `cloudsmith.bandwidth_used_gb` | byte | The bandwidth used in gigabytes. | | `cloudsmith.bandwidth_plan_limit_bytes` | byte | The bandwidth limit in bytes defined by the plan. | | `cloudsmith.bandwidth_plan_limit_gb` | byte | The bandwidth limit in gigabytes defined by the plan. | ### Token visibility Track the number of entitlement tokens, total downloads, and bandwidth consumed. | Metric Name | Metric Type | Description | | :------------------------------------------------ | :---------- | :---------------------------------------------------------------------------------------------------------------------------- | | `cloudsmith.token_count` | item | The number of tokens in an workspace. | | `cloudsmith.token_bandwidth_total` | byte | The total package delivery (bandwidth) consumed by tokens. | | `cloudsmith.token_download_total` | item | The total downloads consumed by tokens. | ### Workspace members activity & security Monitor active users, login methods (SAML or password), and 2FA status. Track members by their role breakdown (Owner, Manager, Admin, ReadOnly). | Metric Name | Metric Type | Description | | :------------------------------------------------ | :---------- | :---------------------------------------------------------------------------------------------------------------------------- | | `cloudsmith.cloudsmith.license_policy_violation.count` | item | The number of license policy violations. | | `cloudsmith.cloudsmith.vulnerability_policy_violation.count` | item | The number of vulnerability policy violations. | | `cloudsmith.cloudsmith.member.count` | item | The number of active members in the workspace. | | `cloudsmith.cloudsmith.member.has_2fa.count` | item | The number of members with 2FA enabled. | | `cloudsmith.cloudsmith.member.saml.count` | item | The number of members who logged in with SAML. | | `cloudsmith.cloudsmith.member.password.count` | item | The number of members who logged in with password. | | `cloudsmith.cloudsmith.member.owner.count` | item | The number of members with role Owner. | | `cloudsmith.cloudsmith.member.manager.count` | item | The number of members with role Manager. | | `cloudsmith.cloudsmith.member.readonly.count` | item | The number of members with role ReadOnly. | | `cloudsmith.cloudsmith.member.admin.count` | item | The number of members with role Admin. | | `cloudsmith.cloudsmith.member.active` | item | Whether a Cloudsmith user is active (1 or 0). | ### Events & Security Insights These events offer auditability and policy enforcement tracking. | Metric Name | Metric Type | Description | | :------------------------------------------------ | :---------- | :---------------------------------------------------------------------------------------------------------------------------- | | `cloudsmith.audit_log` | event | Captures workspace-wide audit log entries (for example: token creation, user changes, repo settings). | | `cloudsmith.vulnerabilities` | event | Represents security scan results for packages flagged with known vulnerabilities. | | `cloudsmith.vulnerability_policy_violation` | event | Indicates violations of vulnerability policies. | | `cloudsmith.license_policy_violation` | event | Indicates violations of license compliance policies. | | `cloudsmith.org_member_summary` | event | Summarizes all active members in the workspace including role, 2FA status, and last login method. | | `cloudsmith.quota_summary` | event | Summarizes quota usage (storage & bandwidth) in both percentage and GB with threshold status. | ## Install the Cloudsmith Datadog Integration The Cloudsmith Datadog Integration is a Datadog [Community Integration](https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=agentv721v621). Follow these steps to install and configure it. ### Requirements: Datadog Agent First things first, you need to install the Datadog Agent. The Datadog agent is software that runs on your hosts. It collects events and metrics from hosts and sends them to Datadog, where you can analyze your monitoring and performance data. To install the Datadog agent, follow the steps [here](https://docs.datadoghq.com/agent/). ### Cloudsmith Datadog Integration Installation The Cloudsmith check is a [Community Integration](https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=agentv721v621) and isn't included in the [Datadog Agent](https://app.datadoghq.com/account/login?next=%2Faccount%2Fsettings#agent) package, so you need to install it. For Agent versions `v7.21+ / v6.21+`, follow the instructions below to install the Cloudsmith check on your host. Run the following command to install the Cloudsmith Agent integration: ```bash datadog-agent integration install -t datadog-cloudsmith==1.0.0 ``` **Latest version** Visit the Datadog [integration page](https://docs.datadoghq.com/integrations/cloudsmith/) to find the latest version ### Configuration Once installed, configure your integration similar to core integrations. There are 3 configuration values: | Config Name | Config Description | | :----------------- | :----------------------------------------------------------------------------------------- | | `url` | The Cloudsmith API URL (https://api.cloudsmith.io) | | `cloudsmith_api_key` | Your Cloudsmith API key | | `organization` | The Cloudsmith [Workspace](/workspaces) (Workspace) that you are monitoring | | `min_collection_interval` | Collection interval of the check. **We recommend to set it's value to 180**. | Follow these steps to configure your Cloudsmith Datadog Integration: 1. Edit the `cloudsmith.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your Cloudsmith performance data. See the [sample](https://github.com/DataDog/integrations-extras/blob/master/cloudsmith/datadog_checks/cloudsmith/data/conf.yaml.example) **_cloudsmith.d/conf.yaml_** for all available configuration options. 2. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/?tab=agentv6v7#start-stop-and-restart-the-agent). ### Validation [Run the Agent's status subcommand](https://docs.datadoghq.com/agent/guide/agent-commands/?tab=agentv6v7#agent-status-and-information) and look for `cloudsmith` under the Checks section. ### Activate the Cloudsmith Integration in Datadog Log into [Datadog](https://www.datadoghq.com/) and from the side bar select **_Integrations->integrations_** and select the Cloudsmith integration. Open the Configuration tab and select **_Install Integration_**. After these steps, you are ready to start using this integration. ## Visualizing your Cloudsmith data in Datadog Datadog dashboards enable you to efficiently monitor your infrastructure and integrations by displaying and tracking key metrics. Once the Agent is installed with the Cloudsmith check, you can use the metrics in any Datadog dashboard. The Cloudsmith Datadog Integration comes with its own dashboard allowing you to monitor your Cloudsmith workspaces out of the box. 1. From Datadog, select **_Dashboards->Dashboard_** list to view your dashboards. 2. Select your installed Cloudsmith dashboard, listed as **_Cloudsmith Overview_**. Once completed, you can created a custom Dashboard or just use the default dashboard as displayed in the image below. **Dashboard Widgets** Each widget in the dashboard reflects real-time data from the Cloudsmith API and is grouped by functional area — ensuring clear operational visibility for your teams. ## To sum it up The Datadog Cloudsmith Integration is a simple and effective visualization tool for monitoring your token, artifact data, package delivery, audit logs, and vulnerability scanning. Manage and forecast your monthly usage. If you identify a limit that's approaching a threshold, then you can quickly and easily adjust your [limits](/workspaces/workspace-settings#usage-limits) at any time within the Cloudsmith UI. If you already use Datadog to monitor your tools and services, this tool is for you. # Integrating Dependabot How to integrate Github Dependabot with Cloudsmith [Github Dependabot](https://docs.github.com/en/code-security/getting-started/dependabot-quickstart-guide) can be used to automatically check for newer versions of your dependencies and update vulnerable dependencies. This guide will walk you through the process of integrating Github Dependabot with Cloudsmith. For this example, we will set it up for a Maven project, but the instructions will work for all supported formats. ## Add Cloudsmith API Token to GitHub Secrets 1. Copy your API key from Cloudsmith using the instructions [here](/accounts-and-teams/api-key). 2. Navigate to your repository on GitHub. 3. Select `Settings`. 4. Select `Secrets and variables`. 5. Select `Dependabot`. You will see the UI below. 6. Add your Cloudsmith API key details here: 1. `Cloudsmith API Key`. For this example we use`CLOUDSMITH_API_KEY `. 2. `Cloudsmith Username`. For this example we use`CLOUDSMITH_USER_NAME`. ## Enable Dependabot in GitHub To receive Dependabot alerts, you must first enable Dependabot alerts in this repository’s settings. 1. Navigate to your repository on GitHub. 2. Click on`Security` from the repository menu. 3. Select the`Enable` button in the `Dependabot alerts` section. 4. Once Dependabot has been enabled, click on `Create a config file`. ​​​​​​​This will create a `./github/dependabot.yml` for your repository (Learn more about [configuring a Dependabot configuration file here](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuring-dependabot-version-updates)). ## Configure the Dependabot Configuration file For Dependabot to connect to Cloudsmith, you will need to specify the Cloudsmith connection details in `./github/dependabot.yml`. Open the ./github/dependabot.yml file and configure it as follows: - `package-ecosystem` Specify the[ package ecosystem](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem) you are using. For example, if you are using Maven, set the `package-ecosystem` to `maven`. - `registries` You will need to include the `registries` setting in 2 places in the `dependabot.yml file`. - At the top level, add the `registries` section to define the Cloudsmith repository you are using. This section should include the following: - `type`: The type of repository e.g. maven-repository or - `url` The URL of the Cloudsmith repository. - `username` and `password`: Cloudsmith supports authentication using username and password. These credentials should be stored in the `Dependabot` tab of your repositories `Secrets and variables`settings (see above for more details). - `replaces-base`. This setting is optional and works hand in hand with Cloudsmith upstream [Upstream Proxying](/repositories/upstreams). If the `replaces-base` setting is set to `true`, Dependabot will use the specified Cloudsmith URL as the primary source for dependencies instead of the default public repository for that package ecosystem. This means you should configure a corresponding Cloudsmith upstream to ensure Dependabot checks Cloudsmith first for dependencies. - Within the updates blocks, where you can use `registries: "*"` to tell Dependabot to use any or all of the registries you defined at the top level. - `directory` specify the directory where your package manifest is located. This is usually the root directory of your project. - `schedule`Configure the schedule to define how often Dependabot should check for updates. You can set the interval to daily, weekly, or monthly. Here is a complete example with a maven project: Replace `YOUR-ORG/YOUR-REPO` with the name of your organization and repository. ```text shell version: 2 registries: cloudsmith: type: maven-repository url: https://dl.cloudsmith.io/basic/YOUR-ORG/YOUR-REPO/maven/ username: "${{ secrets.CLOUDSMITH_USER_NAME }}" password: "${{ secrets.CLOUDSMITH_API_KEY }}" replaces-base: true updates: - package-ecosystem: "maven" directory: "/" schedule: interval: "daily" registries: - cloudsmith commit-message: prefix: "deps" open-pull-requests-limit: 10 ``` More detailed information on the Dependabot configuration file can be found [here](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file). ## Verify the Connection ​​​​​To verify that Dependabot can successfully connect to Cloudsmith: 1. Navigate to your repository’s settings. 2. Go to `Insights->Dependency Graph -> Dependabot` 3. Check for updates. You will find any errors here. By following these steps, you can ensure that Dependabot will check for and update dependencies via Cloudsmith, ensuring your dependencies are always up-to-date and secure. # Integrating with Docker Hardened Images How to integrate with Docker Hardened Images using Cloudsmith [Docker Hardened Images (DHI)](https://hub.docker.com/hardened-images/catalog), are minimal, secure, and production-ready container base and application images maintained by Docker. Designed to reduce vulnerabilities and simplify compliance, DHI integrates easily into your existing Docker-based workflows with little to no retooling required. You can retrieve these images through Cloudsmith by enabling an upstream to DHI. Docker offers both a free tier registry (`dhi.io`) containing developer images and a DHI Enterprise tier which provides features like FIPS or STIG compliance variants, customization capabilities, or SLA-backed support. DHI Enterprise allows for images to be mirrored to a private organization. ## Adding DHI as an Upstream Here's how you can integrate the DHI Registry into your Cloudsmith account: 1. **Configure Upstream Proxying** In your Cloudsmith repository, go to the Upstream Proxying settings. Click the green "Create Upstream" button and select the Docker format. Provide a descriptive name for the upstream, e.g., DHI Registry, and specify the URL for the Docker Registry. Enter the DHI Registry URL: - For free tier DHI images: `https://dhi.io`. - For mirrored DHI Enterprise images, you can pull through both `https://dhi.io` or a standard [DockerHub integration](/integrations/integrating-with-docker-hub) by specifying your Docker organization in the image path (mydockerorg/dhi-node:latest). Set the desired priority. Select Cache and Proxy. 2. **Configure SSL Certificate Verification** Ensure SSL certificates are verified for added security, especially for public sources. 3. **Authentication and Headers** DHI requires authentication or additional headers; provide them in the respective fields. ## Pull a DHI Image with Docker Native Tooling Here's an example of how you would pull the node Docker Hardened image into Cloudsmith after you've configured your Cloudsmith upstream for DHI: 1. **Configure your Cloudsmith upstream** for DHI using the instructions above. 2. **Ensure Docker is installed** on your system. If not, go [here](https://docs.docker.com/engine/install/) to get started with Docker. 3. **Open a terminal.** 4. **Login to Docker** with your Cloudsmith username and token, with the command: `docker login docker.cloudsmith.io` 5. **Pull the DHI node image** by running: `docker pull docker.cloudsmith.io/WORKSPACE/REPOSITORY/node:20-alpine3.22` Note: Replace WORKSPACE and REPOSITORY with your Cloudsmith workspace and repository, respectively. 6. **Check your Cloudsmith repository** to find the newly added DHI node image. # Integrating with Docker Hub Retrieve Container Images from Docker Hub using Cloudsmith [Docker Hub](https://hub.docker.com/) is the official registry (provided by [Docker, Inc.](https://www.docker.com/)) that allows you to find and share Docker-format container images. > It's the world's largest repository of container images with an array of content sources including container community developers, open source projects, and independent software vendors (ISV) building and distributing their code in containers. You can retrieve these images through Cloudsmith by enabling an upstream to Docker Hub to proxy and cache images. ## Adding Docker Hub as an Upstream Here's how you can integrate the Docker Hub Registry into your Cloudsmith account to begin proxying and caching images from Docker Hub: 1. **Add a new Upstream Proxy** In your Cloudsmith repository, go to "Upstreams" and select "Add Upstream Proxy". 2. **Configure the Upstream URL and mode** Provide a descriptive name for the upstream, e.g., Docker Hub, and specify the URL for Docker Hub: https://index.docker.io. Set the desired priority. Select Cache and Proxy. 3. **Provide authentication credentials** Docker Hub requires authentication. Please select a Method of "Username and Password" and add your Docker Hub account credentials. If you need help creating a Docker Hub account, see [here](https://docs.docker.com/docker-id/). **Note**: For Docker Hub, Username & Password are required, not optional. 3. **Configure SSL Certificate Verification** Ensure SSL certificates are verified for added security, especially for public sources. Once configured, your account is now set up to proxy and cache images from Docker Hub through Cloudsmith. **Warning** If you have a Docker upstream configured, please be aware of the following behaviours observed when making requests against the Docker API v2 endpoints: - [Tags/List](https://distribution.github.io/distribution/spec/api/#listing-image-tags): All tags from the upstream and not just local Cloudsmith tags are returned. - [Manifests](https://distribution.github.io/distribution/spec/api/#pulling-an-image-manifest): This will trigger a cache of the referenced image to your Cloudsmith repository. Iterating through a list of tags to fetch manifests will trigger automatic caching for every image. This can result in hundreds or thousands of cached images, significantly increasing artifact storage consumption and associated costs. # Integrating with Drone CI Drone is a Continuous Integration platform that allows automation of build, test, and release workflows using a powerful, cloud-native pipeline engine. - [Drone.io](https://www.drone.io/): Drone Website - [Drone Docs](https://docs.drone.io/): Official Drone Documentation ## API Key Configuration You need to add your Cloudsmith [API Key](https://app.cloudsmith.com/settings/api-keys) within Drone CI. We recommend storing your Cloudsmith API Key as a [per-repository or per-organization secret](https://docs.drone.io/secret/) in the Drone Server and then injecting the `CLOUDSMITH_API_KEY` environment variable into your build jobs like: ```yaml environment: CLOUDSMITH_API_KEY: from_secret: CLOUDSMITH_API_KEY ``` ## Examples In the following examples: | Identifier | Description | | :----------- | :------------------------------------------------------------ | | OWNER | Your Cloudsmith account name or organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | FORMAT | The format of the package, i.e "deb", "maven", "npm" etc | | PACKAGE_FILE | The filename of the package | ### Build Step Example To push an artifact from a build step, you just need to add the commands to install the Cloudsmith CLI and use the `cloudsmith push` command: ```yaml steps: - pip install cloudsmith-cli - cloudsmith push FORMAT OWNER/REPOSITORY/PACKAGE_FILE ``` ### Docker Pipeline Example Drone supports different types of [pipelines](https://docs.drone.io/pipeline/overview/), each optimized for different use cases and runtime environments. For example, you can use a Docker pipeline with your own Docker image that includes the Cloudsmith CLI, negating the need to install the CLI in a Build Step. The following example configuration specifies a Docker image from a Cloudsmith repository: ```yaml steps: - name: Build and Push Package image: docker.cloudsmith.io/OWNER/REPOSITORY/IMAGE_NAME:latest environment: CLOUDSMITH_API_KEY: from_secret: CLOUDSMITH_API_KEY commands: - ....... - ....... - cloudsmith push FORMAT OWNER/REPOSITORY/PACKAGE_FILENAME image_pull_secrets: - dockerconfigjson ``` As the Docker image used for the pipeline in this example is hosted in a private Cloudsmith repository, you need to authenticate to the repository to pull the image for use. You can add the authentication credentials as a [per-repository or per-organization secret](https://docs.drone.io/secret/) in Drone CI. You can obtain these credentials from your `~/.docker/.config.json` file after you do a normal `docker login` to your Cloudsmith repository You then use `image_pull_secrets`, with the name of the secrets file you created in Drone CI to enable your pipeline to authenticate to your Cloudsmith repository. The push command will vary with the package format, an example of the push command for a debian package would look like: `cloudsmith push deb my-org/my-repo/ubuntu/xenial foo-1.0.deb`. Please see the [Cloudsmith CLI](/developer-tools/cli) for full details of the push command for other formats and additional help. # Integrating Cloudsmith with GitHub Actions Use the Cloudsmith CLI in your workflows via the official Cloudsmith CLI Install Action. This action can: - Authenticate with Cloudsmith using either an API key or OIDC (recommended for CI) - Install the Cloudsmith CLI (zipapp download or optional `pip` install) - Optionally only perform OIDC auth (skip CLI install) for lightweight API interactions **[Cloudsmith CLI Install Action](https://github.com/cloudsmith-io/cloudsmith-cli-action)**: Official GitHub Action to authenticate and install the Cloudsmith CLI ## Action Overview This GitHub Action installs the Cloudsmith CLI and pre-authenticates it using OIDC or API Key. ## OIDC Authentication (Recommended) Add the `id-token: write` permission so the action can request an identity token. Cloudsmith exchanges this for a short-lived JWT which is exported as `CLOUDSMITH_API_KEY`. ```yaml permissions: id-token: write contents: read steps: - uses: actions/checkout@v4 - name: Authenticate & Install Cloudsmith CLI (OIDC) uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: your-oidc-namespace oidc-service-slug: your-service-account-slug ``` **OIDC Permissions & Token** Ensure `permissions: id-token: write` is present. The obtained JWT is automatically exported as `CLOUDSMITH_API_KEY` environment variable. ### OIDC Authentication Only (Skip CLI Install) Use when you only need the token for API calls: ```yaml steps: - uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: your-oidc-namespace oidc-service-slug: your-service-account-slug oidc-auth-only: 'true' - name: Validate token run: curl -H "X-Api-Key: $CLOUDSMITH_API_KEY" https://api.cloudsmith.io/v1/user/self/ ``` ## API Key Authentication Use for quick tests or legacy setups when OIDC isn’t available. ```yaml steps: - uses: actions/checkout@v4 - name: Install Cloudsmith CLI (API Key) uses: cloudsmith-io/cloudsmith-cli-action@v2 with: api-key: ${{ secrets.CLOUDSMITH_API_KEY }} ``` ## Action Output When OIDC is used the action: - Exports the token as `CLOUDSMITH_API_KEY` (env var) - Sets an output `oidc-token` ## Installing the CLI By default the action downloads the latest Cloudsmith CLI zipapp release. You can instead install via `pip`. ```yaml with: oidc-namespace: your-oidc-namespace oidc-service-slug: your-service-account-slug pip-install: 'true' cli-version: 1.3.0 ``` The CLI binary (zipapp) is placed at the `executable-path` (defaults to `bin/cloudsmith`) and added to `PATH`. On Windows, a `cloudsmith.bat` wrapper is created. ## Example: Publish a Python Package ```yaml name: Publish Python Package on: push: branches: [ main ] permissions: id-token: write contents: read jobs: publish: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: '3.11' - name: Build distribution run: | python -m pip install build python -m build - name: Install Cloudsmith CLI (OIDC) uses: cloudsmith-io/cloudsmith-cli-action@v2 with: oidc-namespace: your-oidc-namespace oidc-service-slug: your-service-account-slug - name: Push to Cloudsmith run: cloudsmith push python your-namespace/your-repository dist/*.tar.gz ``` ## Secrets and Variables Setup 1. In your GitHub repository go to Settings → Secrets and variables → Actions. 2. Add secret `CLOUDSMITH_API_KEY` if using API key auth. 3. For OIDC create a Cloudsmith service account and note its namespace and slug. 4. Ensure workflow `permissions: id-token: write`. Refer to Cloudsmith docs for [Service Accounts](/accounts-and-teams/service-accounts) and [OIDC](/authentication/openid-connect). ## Supported Package Formats As this action directly uses the Cloudsmith CLI, it supports all formats available in the CLI: - Cloudsmith [Supported Formats](/formats) - Cloudsmith CLI [Features](https://github.com/cloudsmith-io/cloudsmith-cli#features) ## Support If you need help, open an issue on the action's GitHub repository or visit [support.cloudsmith.com](https://support.cloudsmith.com/). ## See Also - [Action Repository](https://github.com/cloudsmith-io/cloudsmith-cli-action) - [Cloudsmith CLI](https://github.com/cloudsmith-io/cloudsmith-cli) - [OIDC Docs](/authentication/openid-connect) - [Service Accounts](/accounts-and-teams/service-accounts) - [Formats](/formats) # GitLab CI/CD How to integrate GitLab CI/CD with Cloudsmith. GitLab is an open-source DevOps platform originally built as a Git-based source code management tool but has since expanded into many areas including CI/CD. **Cloudsmith CLI** The Cloudsmith CLI gives you full control when connecting to any CI/CD process; allowing you to upload any of our support formats or query your repositories. Just configure your API Key, install the CLI, and you'll be all set. To upload a package to a Cloudsmith repository using a Gitlab CI/CD pipeline, you can either use the [Cloudsmith CLI](/developer-tools/cli) or the native package management tooling, such as `gem push` or `cargo publish` (where supported) In either case, you will need to add your Cloudsmith API Key to your GitLab CI/CD environment ## Adding your API Key to GitLab Retrieve your [Cloudsmith API Key](/accounts-and-teams/api-key). ### API Key use with the Cloudsmith CLI To use the Cloudsmith CLI with GitLab CI/CD Pipelines, you will need to add your Cloudsmith API Key to your GitLab repository as a CI/CD environment variable named `CLOUDSMITH_API_KEY`. Please see the [GitLab CI/CD environment variables](https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui) documentation for further details. ### API Key use with native package managers If using native package management tooling to upload your package, you will need to add your Cloudsmith API Key to the credentials file or location that the native tools require. Please see the documentation for your specific package management tool for further details. You can then use [GitLab CI/CD file variables](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables-of-type-file) to securely store your Cloudsmith API Key and create the required credentials file from this file variable during a pipeline run. ## Examples ### Push a Ruby Package. To push a Ruby package via `gem push` using GitLab CI/CD, you will need to add the following 1. An environment variable, called `RUBYGEMS_HOST`, containing the URL for your Cloudsmith repository to your GitLab CD/CD pipeline `.gitlab-ci.yml` file: ```yaml variables: RUBYGEMS_HOST: "https://ruby.cloudsmith.io/OWNER/REPOSITORY" ``` 2. A file variable, called `CLOUDSMITH_API_KEY`, to your repository CI/CD settings: ``` --- :rubygem_api_key: Token ``` 3. A push stage in your GitLab CI/CD pipeline `.gitlab-ci.yml` file, which will create the required `~/.gem/credentials` file, copy in the contents of the `CLOUDSMITH_API_KEY` file variable and then run the `gem push` command to upload your Ruby package to your Cloudsmith repository ```yaml push: stage: push script: - mkdir -p ~/.gem - mv $CLOUDSMITH_API_KEY ~/.gem/credentials && chmod 0600 ~/.gem/credentials - gem push .gem ``` # Integrating Harness CD How to integrate Harness CD with Cloudsmith Harness is a self-service Continuous Delivery platform that allows engineers and DevOps to build, test, deploy, and verify software, on-demand. - [Harness.io](https://harness.io/): Harness Website - [Harness Docs](https://docs.harness.io/): Official Harness Documentation Harness also provides Continuous Integration through [Drone CI](/integrations/integrating-with-drone-ci) ## Adding authentication to Harness CD To authenticate to a private Cloudsmith repository, you first need to add either your Cloudsmith API Key or a Cloudsmith Entitlement Token in Harness as an encrypted secret in a Secrets Manager. Harness includes a built-in Secrets Manager (Harness Vault) and also supports many third-party Secrets Managers. Please refer to the [Harness documentation](https://developer.harness.io/docs/platform/secrets/secrets-management/harness-secret-manager-overview/) for details on how to configure your chosen Secrets Manager. ### Cloudsmith API Key You can obtain your [Cloudsmith API Key](/accounts-and-teams/api-key) from using the Cloudsmith CLI or via the [API Settings](https://app.cloudsmith.com/settings/api-keys) page when logged in to the Cloudsmith Website. ### Cloudsmith Entitlement Token Please see the [Entitlements](/software-distribution/entitlement-tokens) documentation for details of how to create and retrieve Entitlement Tokens via the Cloudsmith CLI or the Cloudsmith Website. ## Deploying a Docker Image with Harness CD ### Add a Cloudsmith Repository as a Harness Artifact Server Cloudsmith Repositories are added to Harness as an Artifact Server. At a minimum you need to provide the following information to configure an Artifact Server: | Field | Description | | :------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Type | The type of Artifact Server, select "Docker Registry". | | Display Name | A name for the Artifact Server within Harness. You can use any name that you wish here. | | Docker Registry URL | The URL for your Cloudsmith Repository. Enter `https://docker.cloudsmith.io/v2/` or your own docker [Custom Domain](/workspaces/custom-domains) (if enabled in your Cloudsmith Account) | | Username | The Cloudsmith Username, or an Entitlement Token name that you wish to authenticate with Please see the Cloudsmith [Docker Registry](/formats/docker-registry) documentation for details of authentication methods. | | Select Encrypted Password | The Encrypted Secret that you wish to use for Authentication. Please see the [Harness documentation](https://developer.harness.io/docs/platform/secrets/secrets-management/harness-secret-manager-overview/) for further information on Encrypted Secrets. | ### Declaring a Docker Image to be deployed A [Harness Service](https://docs.harness.io/article/4o7oqwih6h-harness-key-concepts#services) is where you specify the artifact that you want to deploy. Add the following to specify a Docker image : |Field|Description| |:-|:-| |Display Name|A name within Harness for the artifact.| |Source Server|The "Display Name" specified when configuring the Artifact Server.| |Docker Image Name|The name of the Docker image. In the format OWNER/REPOSITORY/IMAGE-NAME, where OWNER is your Cloudsmith Organization / Account name, REPOSITORY is your Cloudsmith Repository identifier, and IMAGE-NAME your the Docker image name in the repository.| # Integrating Jenkins How to integrate Jenkins with Cloudsmith **Contextual Documentation** These instructions are generic examples. Cloudsmith provides contextual setup details for Jenkins, with copy and paste snippets (with your namespace/repo pre-configured) in the documentation within a repository. ## API Key Configuration If you haven't already done so you'll need to expose your Cloudsmith [API Key](https://app.cloudsmith.com/settings/api-keys) within Jenkins. We recommend using the [credentials plugin](https://plugins.jenkins.io/credentials/) then injecting the `CLOUDSMITH_API_KEY` environment variable into your build jobs. ## Examples In the following examples: | Identifier | Description | | :----------- | :--------------------------------------------------------- | | OWNER | Your Cloudsmith organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository identifier (also called "slug") | | FORMAT | The format of the package, i.e "deb", "maven", "npm" etc | | PACKAGE_FILE | The filename of the package | ### Build Steps Example Add the following as a build step to perform the push to Cloudsmith: ```bash # Jenkins Build Steps using CLI pip install cloudsmith-cli cloudsmith push FORMAT OWNER/REPOSITORY PACKAGE_FILE ``` ### Pipelines Example Add the following as a build stage to perform the push to Cloudsmith: ```groovy stage "Deploy" sh('pip install cloudsmith-cli') sh('cloudsmith push FORMAT OWNER/REPOSITORY PACKAGE_FILE') ``` The push command will vary with the package format, an example of the push command for a debian package would look like: ```bash cloudsmith push deb my-org/my-repo/ubuntu/xenial foo-1.0.deb ``` Please see the [Cloudsmith CLI](/developer-tools/cli) for full details of the push command for other formats and additional help. # MCP Server Support for Cloudsmith CLI How to integrate MCP with Cloudsmith The Cloudsmith CLI now includes support for the Model Context Protocol (MCP), allowing AI assistants like Claude to interact with your Cloudsmith repositories through natural language. ## What's New The MCP server enables AI assistants to programmatically manage Cloudsmith repositories, packages, and artifacts by automatically generating tools from Cloudsmith's OpenAPI specification. ## Requirements **Python 3.10 or later** is required to use the Cloudsmith CLI. ## Getting Started ### Auto-Configure MCP Clients The CLI provides a command to facilitate the configuration of MCP clients: ```bash # Configure for default profile $ cloudsmith mcp configure # Configure for specific profile $ cloudsmith mcp configure -P my-profile ``` Currently supported MCP clients for auto-configuration: **Claude Desktop**, **Cursor**, **VS Code (Copilot)**, and **Gemini CLI**. ### Manual Configuration Alternatively, you can manually configure the MCP server by modifying the JSON configuration file for your LLM client. For example, with **Claude Desktop on macOS**, the configuration file is located at: ``` ~/Library/Application Support/Claude/claude_desktop_config.json ``` Example configuration: ```json { "mcpServers": { "cloudsmith": { "command": "cloudsmith", "args": [ "mcp", "start" ] } } } ``` **Note:** The MCP server will be started automatically by your LLM client. You don't need to run `cloudsmith mcp start` manually. ### Managing Available Tools Due to the large number of tools exposed by the MCP server, a limited subset is available by default to avoid filling the LLM's context window. **Disabled tool categories by default:** `broadcasts, rates, packages_upload, packages_validate, user_token, user_tokens, webhooks, status, repos_ecdsa, repos_geoip, repos_gpg, repos_rsa, repos_x509, repos_upstream, orgs_openid, orgs_saml, orgs_invites, files, badges, quota, users_profile, workspaces_policies, storage_regions, entitlements, metrics_entitlements, metrics_packages, orgs_teams, repo_retention` ### List Available Tools and Groups ```bash # Show all available tools $ cloudsmith mcp list_tools -a # Show all tool groups $ cloudsmith mcp list_groups -a ``` After reviewing the available tools and groups, you can customize which ones are exposed to your AI assistant. ## Customizing Available Tools You can customize which tools are available by editing `~/.cloudsmith/config.ini`: ```bash [default] mcp_allowed_tools=workspaces_policies_simulate_list mcp_allowed_tool_groups=metrics ``` This configuration exposes: - Individual tools listed in `mcp_allowed_tools` - All tools within the groups specified in `mcp_allowed_tool_groups` ## Authentication The MCP server uses your existing Cloudsmith CLI credentials: - **API Keys**: Automatically used if configured - **SSO**: Complete authentication via `cloudsmith auth` before configuring and launching your LLM client, as MCP clients cannot trigger the SSO flow automatically once the server is running ## Multi-Profile Support Configure separate MCP server instances for different Cloudsmith profiles using the `-P` flag with the configure command. # Integrating Microsoft Teams How to integrate Microsoft Teams with Cloudsmith This guide shows you how to post a message like an example below to a Microsoft Teams channel every time a package is successfully uploaded. ## Configure Incoming Webhooks on Microsoft Teams At the time of writing you can configure your Microsoft Teams Channel according to the official [Microsoft Teams Documentation](https://docs.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook). Copy the webhook URL, which looks like: ``` https://outlook.office.com/webhook/0f7eb16f-f6de-csm1-8272-3028fbf63fdb@1e93d1ef-4a1b-4b18-b2da-50e8cbfabd1e/IncomingWebhook/a13e5467392546e0bbad03ea34559ba7/262349195-a10e-4018-9359-ds5a02y3702b ``` ## Create Webhook on Cloudsmith 1. Open your repository in [Cloudsmith](https://app.cloudsmith.com) and browse to the 'Webhooks' page using the menu on the left 2. Choose to 'Create Webhook' 3. Add the webhook URL from above to the box labeled 'Target Endpoint URL' 4. Choose 'Handlebars Template' as the 'Payload Format' 5. Choose 'application/json' as the Content type 6. Select 'Package Synchronized' from the row of events 7. Add your handlebars code; the template below produced the message shown above 8. Under 'Webhook Event Subscriptions' select 'Send Specific Events (choose)' then check 'Package Synchronized' ```json { "text": "New package from _{{ data.uploader }}_", "attachments": [ { "fallback": "{{ data.filename }} version {{ data.version }} by {{ data.uploader }}", "text": "{{ data.summary }}", "title": "{{ data.filename }}", "title_link": "https://app.cloudsmith.com/user/login/?next=/package/{{ data.workspace }}/{{ data.repository }}/detail/{{ data.slug }}", "mrkdwn_in": ["fields"], "color": "good", "fields": [ { "title": "Repository", "value": "{{ data.repository }}", "short": true }, { "title": "Version", "value": "{{ data.version }}", "short": true }, { "title": "Description", "value": "{{ data.description }}", "short": false } ] } ] } ``` Upload a package and enjoy the awesome automated Microsoft Teams posts. **Webhook Event Types** For more information about the events triggered by the package synchronization process please see [our Webhook documentation.](/developer-tools/webhooks#webhook-event-types) # Integrating MLflow How to integrate MLflow with Cloudsmith The MLflow Cloudsmith plugin lets you use Cloudsmith as an artifact store for MLflow runs. Artifacts are stored as Cloudsmith RAW packages, with tagging that preserves MLflow experiment and run context. For full details and examples, see the [MLflow Cloudsmith Plugin repository](https://github.com/cloudsmith-io/mlflow-cloudsmith-plugin). ## Using Cloudsmith as the MLflow artifact store Install the MLflow Cloudsmith plugin via pip: ```bash pip install mlflow-cloudsmith ``` Configure MLflow to use a Cloudsmith repository as the artifact destination by setting the MLflow artifact URI and your Cloudsmith API key: ```python import os import mlflow os.environ["CLOUDSMITH_API_KEY"] = "" os.environ["MLFLOW_ARTIFACT_URI"] = "cloudsmith:///" with mlflow.start_run(): mlflow.log_param("learning_rate", 0.01) mlflow.log_metric("accuracy", 0.95) mlflow.log_artifact("model.pkl") ``` Cloudsmith will store each logged artifact as a RAW package in the specified repository, tagged for easy filtering (e.g. `mlflow`, `experiment-`, `run-`, `path-`). ## Direct repository usage You can also work with Cloudsmith artifact storage directly using the plugin's repository class: ```python from plugin.cloudsmith_repository import CloudsmithArtifactRepository repo = CloudsmithArtifactRepository("cloudsmith:///") repo.log_artifact("model.pkl", "models/production") for info in repo.list_artifacts("models"): print(info.path, info.file_size, info.is_dir) ``` ## URI format Cloudsmith URIs for MLflow artifacts use the following format: ```text cloudsmith:///[/] ``` Examples: - `cloudsmith://my-workspace/ml-artifacts` - `cloudsmith://my-workspace/ml-artifacts/experiments` # Integrating Octopus Deploy _How to integrate Octopus Deploy with Cloudsmith_ [Octopus Deploy](https://octopus.com/) is a deployment tool. It takes the packages and artifacts generated by your build server and deploys them to various targets, be it Windows, Linux, Azure, AWS, or Kubernetes, in a safe and consistent process. Cloudsmith can easily be configured to be an external feed for Octopus Deploy to consume. ## Create an Octopus Account You can install a self-hosted instance of the Octopus Server or use the hosted version Octopus Cloud. You can get started with either Octopus Server or Octopus Cloud for [free](https://octopus.com/start). Once you have registered you can follow Octopus's instructions to [get started](https://octopus.com/docs/getting-started/first-deployment) with Octopus Deploy. Our instructions to add a Cloudsmith Feed will assume you are using an Octopus Cloud account but similar instructions apply to Octopus Server. ## Create a Cloudsmith Repository After you have created your [Cloudsmith repository](/repositories/create-a-repository) you can add your details to the external feed of your Octopus Deploy build. ## Create the External Feed in Octopus Log into your Octopus control centre and navigate to the Octopus dashboard for your Organization. Navigate to your Organisations Cloud instance, by clicking the _View_ button and selecting your plan. Then navigate to your Octopus app, by selecting the link on the top right. Select the Library tab. Then, create a new Octopus Feed by navigating to **External Feeds->Add Feed**. In the _Create Feed_ page: - Select the Feed type (NuGet, Helm, Docker, Maven), - Give the feed a name and in the URL field, enter the HTTP/HTTPS URL of your Cloudsmith repository. For more information on the URL Feed follow the [section below](#urls-for-feeds) - Populate the credentials of your Cloudsmith repository if necessary. For more information on the URL Feed follow the [section below](#adding-credentials-for-private-repositories) ## URLs for Feeds This section contains information about what Cloudsmith feed URL to use for your specific package. ### NuGet Create a new Octopus Feed by navigating to **Library->External Feeds** and selecting the _NuGet_ Feed type. - Give the NuGet feed a name - Enter the HTTP/HTTPS URL of the feed for your Cloudsmith NuGet repository following the instructions in the box below: ```text NuGet V3 https://nuget.cloudsmith.io/OWNER/REPOSITORY/v3/index.json ``` ```text NuGet V2 https://nuget.cloudsmith.io/OWNER/REPOSITORY/v2 ``` Private repositories require authentication- refer to the [section below](#adding-credentials-for-private-repositories) for information on how to add your credentials. ### Docker Create a new Octopus Feed by navigating to **Library->External Feeds** and selecting the _Docker Container Registry_ Feed type. - Give the Docker feed a name - Enter the HTTP/HTTPS URL of the feed for your Cloudsmith Docker repository following the instructions in the box below: ``` https://docker.cloudsmith.io/v2/OWNER/REGISTRY/ ``` Private repositories require authentication- refer to the [section below](#adding-credentials-for-private-repositories) for information on how to add your credentials. ### Maven Create a new Octopus Feed by navigating to **Library->External Feeds** and selecting the _Maven_ Feed type. - Give the feed a name - Enter the HTTP/HTTPS URL of the feed for your Cloudsmith Maven repository following the instructions in the box below: ```text Public URL with no authentication https://dl.cloudsmith.io/public/OWNER/REPOSITORY/maven/ ``` ```text Entitlement Token Authentication https://dl.cloudsmith.xyz/TOKEN/OWNER/REPOSITORY/maven/ ``` ```text HTTP Basic Authentication https://dl.cloudsmith.io/basic/cloudsmith-test/ciara-repo1/maven/ ``` Private repositories require authentication- refer to the [section below](#adding-credentials-for-private-repositories) for information on how to add your credentials. ### Helm Create a new Octopus Feed by navigating to **Library->External Feeds** and selecting the _Helm_ Feed type. - Give the feed a name - Enter the HTTP/HTTPS URL of the feed for your Cloudsmith Helm repository following the instructions in the box below: ```shell Entitlement Token Authentication https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/helm/charts/ ``` ```text Public URL with no authentication https://dl.cloudsmith.io/public/OWNER/REPOSITORY/helm/charts/ ``` ```text https://dl.cloudsmith.io/basic/OWNER/REPOSITORY/helm/charts/ ``` Private repositories require authentication- refer to the [section below](#adding-credentials-for-private-repositories) for information on how to add your credentials. ## Adding Credentials for Private Repositories Private Cloudsmith repositories require authentication. If you used a token in the URL then you do not need to add additional credentials. **Private Repositories** Private Cloudsmith repositories require authentication. You can choose between two types of authentication, Entitlement Token Authentication or HTTP Basic Authentication. The setup method will differ depending on what authentication type you choose to use. **Warning** Entitlement Tokens, User Credentials and API-Keys should be treated as secrets, and you should ensure that you do not commit them in configurations files along with source code, or expose them in any logs. When you are adding or editing your external feed, you can add credentials for your feed by populating the _Credentials_ section. Provide one of the following three types of credentials: - Cloudsmith Username and Password - Cloudsmith API Key - An Entitlement Token These will be populated in the Credentials section of the Octopus External Feed. ```text Username + password Feed username: USERNAME Feed password: PASSWORD ``` ```text API Key Feed username: USERNAME Feed password: API-KEY ``` ```text Entitlement Token Feed username: token Feed password: TOKEN ``` For more information about credentials check [here](/formats/docker-registry#private-registries). # Integrating Puppet How to integrate Puppet with Cloudsmith Puppet is open-core software for provisioning, configuration management, and application deployment. Puppet declaratively manages the configuration of Unix-like, macOS and Microsoft Windows systems. - [Puppet](https://puppet.com/): Puppet Website - [Puppet Docs](https://puppet.com/docs/): Official Puppet documentation In the following examples: | Identifier | Description | | :--------------- | :---------------------------------------------------------------------------------------- | | OWNER | Your Cloudsmith organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository identifier (also called "slug") | | DISTRO | Your distribution (i.e el, fedora, debian etc) | | VERSION | Your version name (i.e 7, 29, hardy, buster etc) | | FINGERPRINT-LONG | The 20 Byte fingerprint of the Public GPG key for the repository | | FINGERPRINT | The 8 Byte fingerprint of the Public GPG key for the repository | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | USERNAME | Your Cloudsmith username | | PASSWORD | Your Cloudsmith password | | API-KEY | Your Cloudsmith API Key | | PACKAGE | The name of the package | ## Debian repository ### Configuration To add a Cloudsmith repository for Debian packages using Puppet, you would use the Puppet [apt module](https://forge.puppet.com/modules/puppetlabs/apt/readme) . Example Puppet Class using apt module: **Public Repository** ``` class cloudsmith_repo { include apt apt::key { 'cloudsmith': id => 'FINGERPRINT-LONG, source => 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key', } apt::source { 'cloudsmith': comment => 'A Description added to repo config in /etc/apt/sources.list.d/', location => 'https://dl.cloudsmith.io/public/OWNER/REPOSITORY/deb/DISTRO', release => 'VERSION', repos => 'main', pin => 500, include => { 'src' => true, 'deb' => true, }, } } ``` **Private Repository** ```text Entitlement Token Auth class cloudsmith_repo { include apt apt::key { 'cloudsmith': id => 'FINGERPRINT-LONG, source => 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key', } apt::source { 'cloudsmith': comment => 'A Description added to repo config in /etc/apt/sources.list.d/', location => 'https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/deb/DISTRO', release => 'VERSION', repos => 'main', pin => 500, include => { 'src' => true, 'deb' => true, }, } } ``` ```text HTTP Basic Auth (User & Pass) class cloudsmith_repo { include apt apt::key { 'cloudsmith': id => 'FINGERPRINT-LONG, source => 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key', } apt::source { 'cloudsmith': comment => 'A Description added to repo config in /etc/apt/sources.list.d/', location => 'https://USERNAME:PASSWORD@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO', release => 'VERSION', repos => 'main', pin => 500, include => { 'src' => true, 'deb' => true, }, } } ``` ```text HTTP Basic Auth (API-Key) class cloudsmith_repo { include apt apt::key { 'cloudsmith': id => 'FINGERPRINT-LONG, source => 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key', } apt::source { 'cloudsmith': comment => 'A Description added to repo config in /etc/apt/sources.list.d/', location => 'https://USERNAME:API-KEY@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO', release => 'VERSION', repos => 'main', pin => 500, include => { 'src' => true, 'deb' => true, }, } } ``` ```text HTTP Basic Auth (Token) class cloudsmith_repo { include apt apt::key { 'cloudsmith': id => 'FINGERPRINT-LONG, source => 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/cfg/gpg/gpg.FINGERPRINT.key', } apt::source { 'cloudsmith': comment => 'A Description added to repo config in /etc/apt/sources.list.d/', location => 'https://token:TOKEN@dl.cloudsmith.io/basic/OWNER/REPOSITORY/deb/DISTRO', release => 'VERSION', repos => 'main', pin => 500, include => { 'src' => true, 'deb' => true, }, } } ``` ### Package Installation After you have configured the repository, you can then install a package using: ``` package { 'PACKAGE': ensure => 'latest', } ``` # Renovate _How to integrate Mend Renovate with Cloudsmith._ --- [Renovate](https://github.com/renovatebot/renovate) is a popular open-source dependency update automation tool. It can help you by automating the process of looking for references to dependencies (both public and private) and updating them if newer versions are available. Renovate supports a wide range of package ecosystems (Docker, npm, PyPI, Maven, NuGet, etc.) and works across multiple version control systems (GitHub, GitLab, Azure DevOps, Bitbucket). **Other Package Formats** This guide shows how to configure Renovate to work with a private Cloudsmith Docker repository, using the Renovate GitHub App (hosted by Mend) as its primary example.However, the same approach applies to other package types supported by Cloudsmith, including npm, PyPI, Maven, NuGet, and more. ## Why use Renovate with Cloudsmith? - Keep your dependencies secure and up-to-date. - Automate the process of consuming new builds published to your Cloudsmith repositories. - Works with all major package types supported by Cloudsmith. ## Configuration Steps ### Prerequisites - You are using either: - The Renovate GitHub App (hosted version), OR - A self-hosted instance of Renovate (for full control of credentials). - A private Cloudsmith repository for your chosen package type. - A valid authentication method (Entitlement Token, or User/Service API Key) for your Cloudsmith repository. Cloudsmith repositories are private by default. Renovate requires authentication to list available package versions and perform version checks. **Entitlement Tokens** We recommend using a Cloudsmith [Entitlement token](/software-distribution/entitlement-tokens) instead of an API key. ### 1. Add your Cloudsmith Entitlement Token as a Secret In the Mend Developer Dashboard, navigate to Settings → Credentials → Add Secret. This step can be completed at the Repository level or the Organisation level, and define your: - **Secret Name**: use `MEND_CLOUDSMITH_TOKEN`. - **Secret Value**: Use your Cloudsmith entitlement token. Click [here](/software-distribution/entitlement-tokens-via-the-website-ui#creating-entitlement-tokens) to learn how to generate a new one. - Check the **Env var** box (if available). ### 2. Add a Host Rule The primary purpose of hostRules is to configure credentials for host authentication (in this case, your private repository). In the next step, you'll tell Renovate how to match against your Cloudsmith repository and which credentials to use. In the Mend Developer Dashboard, navigate to Settings → Host Rules → Add Host Rule and fill in the fields: | Field | value | | :------------ | :---------------------------------------------------------------- | | Description | Cloudsmith Docker Updates | | Host Type | docker (or npm, pip, maven, nuget depending on your package type) | | Host URL | `https://docker.cloudsmith.io` | | Secret Type | Password | | Host Username | `YOUR_ORG_NAME/YOUR_REPO_NAME` | | Password | `{{ MEND_CLOUDSMITH_TOKEN }}` | **renovate.json** You do not need to add a renovate.json file to your repository — the hosted App manages this for you. ### 3. Example: version upgrade for docker Once configured, Renovate will automatically: 1. Authenticate to your Cloudsmith repository using the newly created Host Rule. 2. Scan your project (Dockerfile, package.json, requirements.txt, pom.xml, etc.). 3. Check Cloudsmith for newer versions. In the image below, you can observe how Renovate detected an upgraded version of the `datadog-cloudsmith-agent` docker container, from version `2.1.0` to `2.10.0`. 4. Automatically open a pull request with the latest version available. Please, note that [Silent Mode](https://docs.renovatebot.com/configuration-options/#mode) needs to be disabled. ## Summary - Renovate can be used with any Cloudsmith-supported package type. - The Renovate GitHub App hosted via Mend works perfectly with Cloudsmith private repositories with little configuration:. # Integrating Roadie (Backstage) ## How to integrate Roadie (Backstage) with Cloudsmith [Roadie (Backstage)](https://roadie.io/) is a service catalog platform that allows users to integrate a number of different APIs into one dashboard for ease of tracking key metrics in one place. Roadie currently has many [available plugins](https://roadie.io/backstage/plugins/) and we are happy to announce that Cloudsmith is one of them. An example picture of what is possible to have displayed through this integration: # Getting Started In order to get started please visit Roadie's [GitHub Page](https://github.com/cloudsmith-io/roadie-backstage-plugins#getting-started) and follow the steps of spinning up your own open-source instance. Once setup and running, you can add Cloudsmith Plugins by inserting them into `packages/app/src/components/home/HomePage.tsx`. ## Plugins There are 4 available plugins: ### CloudsmithStatsCard Displays how many packages are in use/inactive in the provided repository [Image: Repository Stats] ### CloudsmithQuotaCard Displays the current bandwidth and storage usage and how much is available. [Image: Repository Stats] ### CloudsmithRepositoryAuditLogCard [Image: Repository Audit Logs] Displays audit logs for a specified repository. ### CloudsmithRepositorySecurityCard Displays Medium to Critical vulnerabilities found in a specified repository. [Image: Repository Security Scan Results] ## Integration In order to use the plugins import them into `HomePage.tsx`: ```typescript import { CloudsmithStatsCard, CloudsmithQuotaCard, CloudsmithRepositoryAuditLogCard, CloudsmithRepositorySecurityCard } from '@roadiehq/backstage-plugin-cloudsmith'; ``` Once imported you can add them to your grid: ```typescript ``` You can add multiple elements by copying and pasting the code above and changing `CloudsmithStatsCard`. Some elements will only require `owner` field and the syntax should highlight in red that it only takes one argument. Then you can edit the `app-config.yaml` for the backstage backend application, adding the following proxy configuration. ``` ... proxy: ... '/cloudsmith': target: 'https://api.cloudsmith.io' headers: X-Api-Key: ${CLOUDSMITH_API_KEY} ``` When you run the backstage backend, you will need to set the `CLOUDSMITH_API_KEY` environment variable. # Integrating Semaphore CI How to integrate Semaphore CI with Cloudsmith Semaphore is a hosted continuous integration and deployment service used for testing and deploying software projects hosted on GitHub and BitBucket. You can create your workflow and pipelines by writing YAML configuration files and committing them along with your source code, or using the intuitive Visual Builder that Semaphore includes. - [Semaphore CI](https://semaphoreci.com/): Semaphore Website - [Semaphore Docs](https://docs.semaphoreci.com/): Official Semaphore Documentation In the following examples: | Identifier | Description | | :----------- | :------------------------------------------------------------ | | OWNER | Your Cloudsmith account name or organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | FORMAT | The format of the package, i.e "deb", "maven", "npm" etc | | PACKAGE_FILE | The filename of the package | ## API Key Configuration You need to add your Cloudsmith [API Key](https://app.cloudsmith.com/settings/api-keys) in order to authenticate to Cloudsmith for pushing packages. We recommend storing your Cloudsmith API Key as a [secret in Semaphore](https://docs.semaphoreci.com/using-semaphore/secrets). Secrets in Semaphore are organization-level objects that can contain environment variables or files which you can then access in your build jobs. You can add a secret using the Semaphore CLI or the UI: ## Push Package Example To Push a package to Cloudsmith from a Semaphore CI Pipeline, you just need to add your Cloudsmith API Key secret and the `cloudsmith push` CLI command to one of the blocks in your pipeline For example: ```json - name: Push Package task: secrets: - name: Cloudsmith-API jobs: - name: Push commands: - cache restore package - cloudsmith push FORMAT OWNER/REPOSITORY/ PACKAGE_FILE prologue: commands: - sem-version python 3.7 - pip install cloudsmith-cli ``` The push command will vary with the package format, an example of the push command for a debian package would look like: `cloudsmith push deb my-org/my-repo/ubuntu/xenial foo-1.0.deb` . Please see the [Cloudsmith CLI](/developer-tools/cli) for full details of the push command for other formats and additional help). In the above example, we use the command executed during the block prologue to install the Cloudsmith CLI, so that the `cloudsmith push` command is available. In addition, we also use the `cache restore` command to retrieve the package file that was built in a previous block. Please see the [Semaphore Cache documentation](https://docs.semaphoreci.com/using-semaphore/optimization/cache) for more detail on storing and retrieving artifacts using the Semaphore cache Where supported, you can also use native publishing commands like `docker push` or `npm publish` instead of using the Cloudsmith CLI, provided you have installed the necessary tooling as part of your pipeline block setup. # Integrating Slack How to integrate Slack with Cloudsmith This guide shows you how to post a message like an example below to a Slack channel every time a package is successfully uploaded. [Image: Slack message from Cloudsmith after a Python package synchronizes] **Prerequisites** You need a paid Cloudsmith account, a repository where you are an admin and access to a Slack Workspace. ## Configure Incoming Webhooks on Slack 1. [Create a Slack App](https://api.slack.com/apps?new_app=1) on their site, in this example we called ours `Cloudsmith.io` 2. On the next page, near the bottom, add a short description, add an icon choose a colour, we picked a [Cloudsmith Icon](#section-cloudsmith-icon) and `#333333` and 'Save Changes' 3. Choose 'Incoming Webhooks', back towards the top, and on that page: 'Activate Incoming Webhooks' then 'Add New Webhook to Workspace'; choose a channel and then 'Authorize' 4. Test the webhook with the `curl` command 5. Copy the webhook URL, which looks like `https://hooks.slack.com/services/TXXXXXXXX/BXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXX`, to use below _After creating your app you can view its settings at https://api.slack.com/apps._ ## Create Webhook on Cloudsmith 1. Open your repository in [Cloudsmith](https://app.cloudsmith.com) and browse to the 'Webhooks' page using the menu on the left 2. Choose to 'Create Webhook' 3. Add the webhook URL from above to the box labelled 'Target Endpoint URL' 4. Choose 'Handlebars Template' as the 'Payload Format' 5. Select 'Package Synchronized' from the row of events 6. Add your handlebars code; the template below produced the message shown above 7. Under 'Webhook Event Subscriptions' select 'Send Specific Events (choose)' then check 'Package Synchronized' 8. 'Create Webhook' **Payload Format** Although Slack consumes JSON, the format of a Cloudsmith webhook payload differs from Slack's expected input and is required to be adapted. Please ensure that _Handlebars Template_ is selected as the Payload Format. ```json { "text": "New package from _{{ data.uploader }}_", "attachments": [ { "fallback": "{{ data.filename }} version {{ data.version }} by {{ data.uploader }}", "text": "{{ data.summary }}", "title": "{{ data.filename }}", "title_link": "{{ data.self_webapp_url }}", "mrkdwn_in": ["fields"], "color": "good", "fields": [ { "title": "Repository", "value": "{{ data.repository }}", "short": true }, { "title": "Version", "value": "{{ data.version }}", "short": true }, { "title": "Description", "value": "{{ data.description }}", "short": false } ] } ] } ``` Upload a package and enjoy the awesome automated Slack posts. **Webhook Event Types** For more information about the events triggered by the package synchronization process please see [here](/developer-tools/webhooks#webhook-event-types) ## References This guide is based up Slack's [own documentation](https://api.slack.com/incoming-webhooks). Slack also has documentation about [message formatting](https://api.slack.com/docs/message-formatting) and the Slack [message builder](https://api.slack.com/docs/messages/builder) shows what a message will look like. {/* ``` ## Cloudsmith Icon Using ImageMagick we created an icon to matching both [Slack's guidelines](https://api.slack.com/docs/slack-apps-guidelines#app_icon) and [Cloudsmith's branding page](https://corp.cloudsmith.io/branding) : curl -O https://corp.cloudsmith.io/assets/cloudsmith-icon-master-color.svg convert -resize 800x800 cloudsmith-icon-master-color.svg 800x800.png ``` */} # TeamCity _How to integrate TeamCity with Cloudsmith_ Integrating TeamCity with Cloudsmith enables seamless management of your software packages within your CI/CD pipelines. This integration allows you to push build artifacts directly to Cloudsmith repositories from TeamCity, leveraging TeamCity’s build pipelines for efficient artifact publishing. The Cloudsmith CLI gives you full control when connecting to any CI/CD process; allowing you to upload any of our support formats or query your repositories. Just configure your API Key, install the CLI, and you'll be all set. ## Prerequisites Before proceeding, ensure you have the following: 1. **TeamCity Server**: A functioning installation of TeamCity. 2. **Cloudsmith Account**: An active Cloudsmith account with the necessary repository access. 3. **API Key**: A Cloudsmith API key for authentication, obtainable from your Cloudsmith account under Account > API Settings. 4. **Build Agent**: A configured build agent to execute TeamCity builds. *** ## Integration Steps Go to Project Build Configuration in TeamCity and add the following build steps: ### Step 1: Install Python - Runner Type: Command Line - Custom Script: ```bash apt-get update -y apt-get install -y python3 python3-pip ``` - Execution Policy: If all previous steps finished successfully. 📘 You can skip this step if the build agent is consistent as Python installation is a one-time setup. ### Step 2: Install Cloudsmith CLI - Runner Type: Command Line - Custom Script: ```bash pip install cloudsmith-cli ``` - Execution Policy: If all previous steps finished successfully. ### Step 3: Add Cloudsmith API Key Parameter 1. Navigate to **Project Settings > Parameters**. 2. Add the following parameter: - **Name**: env.CLOUDSMITH_API_KEY - **Kind**: Environment variable(env.) - **Value Type**: Password (to ensure security). - **Value**: Your Cloudsmith API key. [Image: teamCityParameters] ### Step 4: Push Your Artifacts - Runner Type: Command Line - Custom Script: ```bash cloudsmith push raw / ``` - Replace `/` and `` with: - Replace `` with your Cloudsmith Organisation name. - **Your Cloudsmith Repository Name**: For example, `my-repository`. - **Artifact Path**: Artifact path, such as `./example-artifact.tar.gz`. - Execution Policy: If all previous steps finished successfully. ## Enable Versioned Settings in TeamCity **Optional step** Optional: execute the next step only if you want to track build configuration in your VCS. 1. In Project Settings, navigate to Versioned Settings. 2. Configure the following options: 1. **Synchronization**: Enable. 2. **Project settings VCS root**: Set as per your preference. 3. **Settings format**: Choose either from Kotlin or XML. 4. **Settings path in VCS**: .teamcity 5. **Allow editing project settings via UI**: Enable. 6. **Store passwords and API tokens outside of VCS**: Enable. 3. Apply the changes to ensure your build configuration is tracked in the version control system (VCS). 4. Allow it to complete and then hit Commit current project settings button and TeamCity will commit the build configuration to your VCS. [Image: Build config] **Here's a Kotlin Snip for You Reference** ```kotlin import jetbrains.buildServer.configs.kotlin.* import jetbrains.buildServer.configs.kotlin.buildFeatures.perfmon import jetbrains.buildServer.configs.kotlin.buildSteps.script import jetbrains.buildServer.configs.kotlin.triggers.vcs version = "2024.03" project { buildType(TestCloudsmith) } object TestCloudsmith : BuildType({ name = "TestCloudsmith" artifactRules = "./" params { password("env.CLOUDSMITH_API_KEY", "credentialsJSON:4ad9ee86-2dd4-4acf-8070-123e96c647fc") } vcs { root(DslContext.settingsRoot) } steps { script { name = "InstallPython" id = "InstallCloudsmith" scriptContent = """ apt-get update -y apt-get install -y python3 python3-pip """.trimIndent() } script { name = "InstallCloudsmith" id = "InstallCloudsmith" scriptContent = "pip3 install cloudsmith-cli" } script { name = "CheckCloudsmithAuth" id = "CheckCloudsmithAuth" scriptContent = "cloudsmith whoami" } script { name = "Cloudsmith Push" id = "Cloudsmith_Push" scriptContent = "cloudsmith push raw testenv/rawrepo raw-example.tar.gz" } } triggers { vcs { } } features { perfmon { } } }) ``` *** ## Best Practices - **Use Entitlement token Authentication**: If you only need to pull packages, use Entitlement tokens for authentication to avoid using long-lived API keys. - **Secure Secrets**: Store sensitive information like API keys/Entitlement tokens as Passwords instead of any other Value type. ## Troubleshooting - **Authentication Issues**: Verify the API key is correctly configured as a hidden parameter. - **Artifact Upload Errors**: Ensure owner/your-repository and artifact-path are properly specified in the push command. - **Build Failures**: Review the TeamCity build logs for errors in specific steps. # Integrating Terraform How to integrate Terraform with Cloudsmith Cloudsmith is a verified provider on the [Terraform Registry](https://registry.terraform.io/browse/providers). **[Terraform Provider](https://github.com/cloudsmith-io/terraform-provider-cloudsmith)**: cloudsmith-terraform-provider ## Installation To install the Cloudsmith Terraform Provider, add the following to the `required_providers` in your Terraform module: ``` required_providers { cloudsmith = { source = "cloudsmith-io/cloudsmith" version = "0.0.5" } } ``` You also need to add your Cloudsmith API Key, for use by the provider: ``` provider "cloudsmith" { api_key = "API-KEY" } ``` You then run `terraform init` and Terraform will automatically install the provider. To specify a particular provider version when installing providers, see the Terraform documentation on [provider versioning](https://developer.hashicorp.com/terraform/language/providers/configuration#version-provider-version). ## Data Sources ### namespace The namespace data source allows fetching of metadata about a given Cloudsmith namespace. The fetched data can be used to resolve permanent identifiers from a namespace's user-facing name. These identifiers can then be passed to other resources to allow more consistent identification as user-facing names can change. ``` data "cloudsmith_namespace" "my_namespace" { slug = "my-namespace" } ``` | Argument | Description | Required | | :------- | :---------------------------------------- | :------- | | slug | The slug identifies the namespace in URIs | Yes | ## Resources ### repository The repository resource allows the creation and management of package repositories within a Cloudsmith namespace. Repositories store packages and are the main entities with which Cloudsmith users interact. ``` resource "cloudsmith_repository" "my_repository" { name = "My Repository" namespace = "${data.cloudsmith_namespace.my_namespace.slug_perm}" description = "A certifiably-awesome private package repository" } ``` |Argument|Description|Required| |--|--|--| |name|A descriptive name for the repository.|Yes| |namespace|Namespace / account to which this repository belongs.|Yes| |description|A description of the repository's purpose/contents.|No| |index_files|If checked, files contained in packages will be indexed, which increase the synchronisation time required for packages. Note that it is recommended you keep this enabled unless the synchronisation time is significantly impacted.|No| |repository_type|Private repositories are visible only to users that have been granted access. Public repositories are visible to all Cloudsmith users.|No| |slug|The slug identifies the repository in URIs.|No| |storage_region|The Cloudsmith region in which package files are stored. Valid values are: `default`, `ie-dublin`, `de-frankfurt`, `ca-montreal`, `us-norcal`, `us-ohio`, `us-oregon`, `sg-singapore`, `au-sydney`|No|" |wait_for_deletion|If true, terraform will wait for a repository to be permanently deleted before finishing.|No| See the [repository documentation](/repositories) for more information on creating and managing repositories. ### repository privileges The repository privileges resource allows the management of privileges for a given Cloudsmith repository. Using this resource it is possible to assign users, teams, or service accounts to a repository, and define the appropriate permission level for each. Note that while users can be added to repositories in this manner, since Terraform does not manage those user accounts, you may encounter issues if the users change or are deleted outside of Terraform. ``` resource "cloudsmith_repository_privileges" "privs" { organization = data.cloudsmith_organization.my_organization.slug repository = cloudsmith_repository.my_repository.slug service { privilege = "Write" slug = cloudsmith_service.my_service.slug } team { privilege = "Write" slug = cloudsmith_team.my_team.slug } team { privilege = "Read" slug = cloudsmith_team.my_other_team.slug } user { privilege = "Read" slug = "some-user-slug" } } ``` When creating a repository via Terraform using a service account and subsequently adjusting the privileges on that same repository, ensure to include a block giving the provisioning account the 'Admin' privilege (as it would get by default by being the creator). This will ensure that the account can continue to provision resources (e.g. entitlements). Otherwise, the resource may overwrite that privilege and further provisioning on the repository when using that account would fail. | Argument | Description | Required | | :----------- | :------------------------------------------------------------------------------------- | :------- | | organization | Organization to which this repository belongs. | Yes | | repository | Repository to which these privileges apply. | Yes | | service | Namespace / Account to which this token belongs | No | | privilege | The service's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | slug | The slug/identifier of the service. | Yes | | team | Variable number of blocks containing teams that should have repository privileges. | No | | privilege | The team's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | slug | The slug/identifier of the team. | Yes | | user | Variable number of blocks containing users that should have repository privileges. | No | | privilege | The user's privilege level in the repository. Must be one of Admin, Write, or Read. | Yes | | slug | The slug/identifier of the user. | Yes | ### entitlement The entitlement resource allows the creation and management of Entitlement tokens for a Cloudsmith repository. Entitlement tokens grant read-only access to a repository and can be configured with a number of custom restrictions if necessary. ``` resource "cloudsmith_entitlement" "my_entitlement" { name = "Entitlement Token 1" namespace = "${cloudsmith_repository.test.namespace}" repository = "${cloudsmith_repository.test.slug_perm}" } ``` | Argument | Description | Required | | :-------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------- | | name | A descriptive name for the token | Yes | | namespace | Namespace / Account to which this token belongs | Yes | | repository | Repository to which this token belongs | Yes | | token | The literal string value of the token to be created | No | | is_active | If true, the token will be enabled and will allow downloads based on configured restrictions (if any). | No | | limit_date_range_from | The starting date/time the token is allowed to be used from. | No | | limit_date_range_to | The ending date/time the token is allowed to be used to. | No | | limit_num_clients | The maximum number of unique clients allowed for the token. | No | | limit_num_downloads | The maximum number of downloads allowed for the token. | No | | limit_package_query | The package-based search query to apply to restrict downloads to. This uses the same syntax as the standard search used for repositories, and also supports boolean logic operators such as OR/AND/NOT and parentheses for grouping. This will still allow access to non-package files, such as metadata. | No | | limit_path_query | The path-based search query to apply to restrict downloads to. This supports boolean logic operators such as OR/AND/NOT and parentheses for grouping. The path evaluated does not include the domain name, the namespace, the entitlement code used, the package format, etc. and it always starts with a forward slash. | No | Please see the [Entitlements](/software-distribution/entitlement-tokens) documentation for more details on managing entitlement tokens and restrictions. ## Example Module A complete, but minimal, example of a Terraform module that uses the Cloudsmith Provider to create a repository and an Entitlement Token (with some basic token restrictions) is: ``` terraform { required_providers { cloudsmith = { source = "cloudsmith-io/cloudsmith" version = "0.0.5" } } } provider "cloudsmith" { api_key = "abcdefghijlkl1234567890" } data "cloudsmith_namespace" "demo-organization { slug = "demo-org" } resource "cloudsmith_repository" "terraform-demo" { description = "Example repo provisioned by Terraform" name = "Terraform Demo" namespace = "${data.cloudsmith_namespace.demo.slug_perm}" slug = "terraform-demo" repository_type = "Private" } resource "cloudsmith_entitlement" "demo-entitlement" { name = "Token 1" namespace = "${cloudsmith_repository.terraform-demo.namespace}" repository = "${cloudsmith_repository.terraform-demo.slug_perm}" limit_num_downloads = "15" limit_num_clients ="1" limit_package_query = "name:,my-package" } ``` ### More example modules Further examples can be found on github in our [terraform-provider](https://github.com/cloudsmith-io/terraform-provider-cloudsmith/tree/master/examples) # Integrating Travis-CI *How to integrate Travis-CI with Cloudsmith.* --- [Travis CI](https://www.travis-ci.com/) is a cloud-based continuous integration (CI) and continuous delivery platform that enables open source and private projects on GitHub and other providers to automate the build, test, and deployment processes of your software. ## Using the Cloudsmith CLI Here you can find a template to start building your own. This Travis pipeline example uses a Debian *trusty* build environment to: - Install the [Cloudsmith CLI]() and required dependencies. - Convert an existing file `setup.py` with your Python project into a wheel (.whl) file for distribution, and store it in the `dist/` directory. - Set the python version to 3.6 and installs the Cloudsmith CLI. - And finally use the Cloudsmith CLI to upload a Python package to your `WORKSPACE/REPOSITORY` only when new changes are pushed to the `main` branch in the associated GitHub repository. ```yaml dist: trusty script: - python setup.py bdist_wheel before_deploy: - pyenv global 3.6 - pip install cloudsmith-cli deploy: provider: script skip_cleanup: true script: cloudsmith push python --republish WORKSPACE/REPOSITORY dist/cloudsmith_python_example-100-py2.py3-none-any.whl on: branch: main ``` # Integrating Zapier How to integrate Zapier with Cloudsmith Cloudsmith provides a [Zapier](https://zapier.com/) integration to manage your entitlement codes; allowing you to hook up almost any shopping cart to enable Cloudsmith as your digital asset distribution provider. ## Setup The Cloudsmith Zapier Integration uses your Cloudsmith API Key for authentication: [Image: Zapier Cloudsmith API Key setup] You can get your Cloudsmith API key from your user [settings](https://app.cloudsmith.com/settings/api-keys) page when you have logged into the Cloudsmith website, or by using the [Cloudsmith CLI](https://github.com/cloudsmith-io/cloudsmith-cli) `cloudsmith login` command. ## Actions The Cloudsmith Zapier Integration provides two actions: - Create Entitlement Token - Disable Entitlement Token Please see [Entitlements](/software-distribution/entitlement-tokens) for more information on Cloudsmith Entitlement Tokens. ## Create Entitlement Token The Create Entitlement Token action has three required fields: [Image: Zapier Create Entitlement Token Fields] **Fields** |Field Name|Description|Required| |:--|:--|:--| |Token Name|A name for the Token within the repository|Yes| |Repository Owner|The username or organisation name that owns the repository|Yes| |Repository|The repository identifier (also called the slug)|Yes| |Token String|A string for the token itself. If one is not provided, a random string will be generated.|No| |Is Active|A boolean defines if the token is enabled or disabled|No| |Valid From|The start date that the token is valid from|No| |Valid To|The end date that the token is valid until|No| |Limit Number of Downloads|The maximum number of downloads that the Token allows|No| |Limit Number of Clients|The maximum number of clients (Unique IP addresses) allowed by the token|No| |Limit Package Query|The package-based search query to apply to restrict downloads. See Searching / Filtering for further details.|No| |Limit Path Query|The path-based search query to apply to restrict downloads.The path evaluated does not include the domain name, the namespace, the entitlement code used, the package format, etc. and it always starts with a forward slash.|No| ## Disable Entitlement Token The Disable Entitlement Token action has three required fields: [Image: Zapier Delete Entitlement Token Fields] **Fields** |Field Name|Description|Required| |:--|:--|:--| |Repository Owner|The username or organisation name that owns the repository|Yes| |Repository|The repository identifier (also called the slug)|Yes| |Entitlement Identifier|The unique identifier for a token within a repository (see [Entitlements](/software-distribution/entitlement-tokens) for further information on tokens and identifiers)|Yes| # Log Exports to S3 Client and Audit log exports to S3 bucket # Prerequisites To use this feature, you'll need administrative access to an [AWS account](https://aws.amazon.com/) (in order to create an S3 bucket for receiving the log exports from Cloudsmith). **If you already have one type of export set up with us and don't require updating your role/permissions, please skip to [Existing Setup](/logs-and-observability/access-log-exports-to-s3#existing-setup).** *** # New Setup {/* First of all, please familiarise yourself with the [changelog post for the feature](https://changelog.cloudsmith.com/en/now-testing-access-log-exports-to-s). */} Next, you'll need to **follow these steps**: **1**. Create an S3 bucket for the logs, such as `cloudsmith-acmecorp-logs` where you can replace `acmecorp` with your own organization name. **2**. If using an existing bucket, pick a prefix for the Cloudsmith logs to go into, such as `cloudsmith-logs`. This is configured in the next step and configured on our side in tandem. **3**. If planning to exports multiple types of logs (e.g. Client and Audit logs), make sure to create separate folders inside of your Cloudsmith log export bucket (Note them down for later): **4**. [Contact us](https://cloudsmith.com/company/contact-us) to tell us your: **One bucket to rule them all** We recommend setting up one bucket with folders for each log type and use the same AWS account, if you require each log type to have a different AWS account, please let us know. - AWS account ID - S3 Bucket name - The IAM Role name that will be used in Step 6, such as `CloudsmithLogsWriter` - The log format that you want to export: - JSON (Stream) **(RECOMMENDED)** - JSON (Stream+TimeStamp) - CSV - Apache Style - The log type that you want to export: - Audit Logs - Client Logs - Prefix (folder names) for each type of log export - Export Options: - Entire org **(RECOMMENDED)** - Selected repositories only **5**. We'll then tell you the External ID value that will be used during the role's creation. This value will be part of the authentication we use to assume your IAM Role. The use of External ID is a [best practice recommended by AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html). **6**. Create a new IAM Role for "Another AWS account", with the same name as determined in ** step 4**, and specify `884446598447` as the Account ID, tick "Require external ID", and then specify the External ID that we provided to you. ``` { "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:PutObject", "s3:PutObjectAcl" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::cloudsmith-acmecorp-logs/cloudsmith-logs/*" ] } ] } ``` **Bucket Name and Prefix** Make sure to replace `cloudsmith-acmecorp-logs` with your own bucket name, and replace `cloudsmith-logs` with your own prefix. If you don't need a prefix, delete the `cloudsmith-logs/` string. The IAM Role should have the following Role Trust Relationship Policy: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::884446598447:root" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "your-external-id" } } } ] } ``` **External ID** Make sure the value for sts:ExternalId matches what Cloudsmith provided you. *** # Existing Setup **We will use your existing AWS account for new log types** If you already have an export set up with us, we will use the same account that you have initially provided us with. If you wish to use a new account for a new type, please go through the _**[New Setup](/logs-and-observability/access-log-exports-to-s3#new-setup)**_ and provide relevant information. If you already have S3 exports setup with us, we will use the same AWS account that you have provided us with. The only change that may be required is a creation of folders inside your Cloudsmith log bucket to separate the different log types. For example if your current solution has no folders inside of the Cloudsmith export bucket and all exports go to `/`, we recommend creating folders: - client-logs - audit-logs _e.g. Client logs will export to `//`_ **Please let us know if you plan to separate the logs into different folders in order for us to reflect the changes in both the new export requested and the old one.** # Summary of Information Required You should have provided us with the following information: - ID of your AWS account (e.g. `995557609558`, typically 12 digits). - Name of your S3 bucket (e.g. `cloudsmith-acmecorp-logs`). - Name of your S3 prefix (folder name) (e.g. `cloudsmith-logs`), if any. - The type of logs you would like to export (e.g. Client and/or Audit logs). - Name of your IAM Role that we'll use for exporting logs (e.g. `CloudsmithLogsWriter`). - The format of the logs you'd like to export (e.g. Apache-style, CSV, JSON Stream, JSON Stream + Timestamp, etc.) **Which format should I choose?** It depends. Each format is "streaming," meaning there's one line per entry for each download rather than a large object. In terms of detail, the Apache (or Nginx) format has the least to conform with those formats, CSV has the next level of detail but is limited due to columns, and JSON has the most level of detail as a structured format. **We recommend choosing JSON Stream + Timestamp**, in which a JSON blob representing the download is prefixed with a timestamp, making this nice to parse and import into your favorite Business Intelligence (BI) or Security Information Event Monitoring (SIEM) tooling. We should have provided you with the following information: - An ExternalId value used for assuming your IAM Role, in order to export the logs. **What can I include/exclude?** It is possible to include or exclude the following: - EULA acceptance data. - location-based (Geo) data. - IP data. - namespace identifiers/paths. - repository identifiers/paths. # Data Provided ## Client Logs Some of the data provided includes: - **Edge Location**: Nearest CDN Edge Node Location (e.g., "LHR61-C2") - **Edge Response (`edge_response`)**: Identifies whether a package was served from an existing cache or required a cold load. Possible values (and casing) are `Hit` and `Miss`. - **EULA ID**: Unique identifier for EULA accepted (if any) - **EULA Number**: Revision of EULA accepted (if any) - **IP Address**: IP of the client - **Host**: Hostname for the request (e.g. your download domain) - **Method**: HTTP Method (e.g. "GET") - **Geo/IP Fields**: E.g., "City," "Continent", e.g. enriched based on IP (we pay for this) - **Package Identifier**: Unique identifier for downloaded package (if any) - **Package Name**: Name for the downloaded package (if any) - **Package Version**: Version for downloaded package (if any) - **Protocol**: HTTP Protocol Used (e.g. "https/1.3") - **Referer**: HTTP Referrer - **Request ID**: Globally unique identifier for the request - **Status**: HTTP Status Code (e.g. "200") - **Time Taken**: Time taken to service request, in seconds - **Token ID**: Unique identifier for entitlement token used (if any) - **Token Name**: Name for entitlement token used (if any) - **URI**: Uniform Resource Identifier, i.e. path, for the request - **User Agent**: HTTP User Agent sent by the client - **User ID**: Unique identifier for the client user (if any) - **User Name**: Name for the client user (if any) ## Audit Logs - **Actor**: The user or system that performed the action. - **Actor IP Address**: The IP address of the actor. - **Actor Kind**: The type of actor, such as "user" or "system". - **Actor Location**: The location of the actor, as determined by their IP address. - **Actor Slug Perm**: A unique identifier for the actor, used for permissions. - **Actor URL**: The URL of the actor's profile. - **Context**: The context in which the event occurred, such as "web" or "api". - **Event**: The type of event that occurred. - **Event At**: The date and time the event occurred. - **Object**: The object that was affected by the event. - **Object Kind**: The type of object, such as "user" or "file". - **Object Slug Perm**: A unique identifier for the object, used for permissions. - **Target**: The target of the event, such as the user who was affected. - **Target Kind**: The type of target, such as "user" or "file". - **Target Slug Perm**: A unique identifier for the target, used for permissions. - **UUID**: A unique identifier for the event. # Format Examples ## JSON (Stream) [recommended] One JSON record per line, without a header line: ``` {"bytes": 2150, "datetime": "2024-05-30T15:14:27+00:00", "edge": "us-east-1", "edge_response": "Hit", "eula": null, "format": "Terraform", "host": "dl.cloudsmith.io", "ip_address": "3.234.139.83", "location": {"city": "Ashburn", "continent": "North America", "country": "United States", "country_code": "US", "latitude": "39.046900", "longitude": "-77.490300"}, "method": "GET", "namespace": "cloudsmith", "package": {"identifier": "wKSkBFHpsNWB", "name": "cloudsmith-terraform-example", "tags": {"info": ["local"]}, "version": "1.0.1717082040651684"}, "recorded": "2024-05-30T15:42:56+00:00", "referer": "", "repository": "testing-private", "request_id": "8v19hTgCw3rVWw9gl1QiPuSGf3JOpKMktSNSk0vn-rTSqcmCx7bYqw==", "status": 200, "token": null, "uri": "/terraform/terraform-local-cloudsmith-terraform-example-1.0.1717082040651684.tar.gz", "user": {"identifier": "svHuImXP38Ev", "username": "end-to-end-tests"}, "user_agent": {"browser": "Go-http-client 2.0", "device": "Other", "os": "Other", "raw": "Go-http-client/2.0"} ``` ## JSON (Stream+Timestamp) One JSON record per line, prefixed by a timestamp, separated by a single whitespace character\*, without a header line: ``` 2024-05-30T15:14:27+00:00 {"bytes": 2150, "datetime": "2024-05-30T15:14:27+00:00", "edge": "us-east-1", "edge_response": "Hit", "eula": null, "format": "Terraform", "host": "dl.cloudsmith.io", "ip_address": "3.234.139.83", "location": {"city": "Ashburn", "continent": "North America", "country": "United States", "country_code": "US", "latitude": "39.046900", "longitude": "-77.490300"}, "method": "GET", "namespace": "cloudsmith", "package": {"identifier": "wKSkBFHpsNWB", "name": "cloudsmith-terraform-example", "tags": {"info": ["local"]}, "version": "1.0.1717082040651684"}, "recorded": "2024-05-30T15:42:56+00:00", "referer": "", "repository": "testing-private", "request_id": "8v19hTgCw3rVWw9gl1QiPuSGf3JOpKMktSNSk0vn-rTSqcmCx7bYqw==", "status": 200, "token": null, "uri": "/terraform/terraform-local-cloudsmith-terraform-example-1.0.1717082040651684.tar.gz", "user": {"identifier": "svHuImXP38Ev", "username": "end-to-end-tests"}, "user_agent": {"browser": "Go-http-client 2.0", "device": "Other", "os": "Other", "raw": "Go-http-client/2.0"} ``` **Future Compatibility** To ensure compatibility, please parse the "whitespace character" as a space OR tab (for future compatibility). ## CSV Starts with a header line, followed by one CSV record per line: ```csv datetime,repository,status,method,uri,host,ip_address,bytes,city,country,edge,eula,format,package,recorded,referer,request_id,token,user,user_agent 2021-04-01T19:40:55+00:00,cloudsmith/testing-private,200,GET,/rpm/fedora/29/x86_64/cloudsmith-redhat-example-1.0.16173057145-1.x86_64.rpm,dl.cloudsmith.io,82.1.4.8,10421,Newmarket,GB,LHR61-C2,,RedHat,vmN1OlMDi3Iy,2021-04-01T19:46:23.817658+00:00,,dZ3DmJ8yHU36FyH9uHoyWH4LocmlkzdKXSEuOBmnvLhA4ZF7exa5Qw==,wKSkBFHpsNWB,eP1B3YCtTlJX,libdnf ``` ## Apache Style ``` 3.222.115.18 - t-6GXPs3OxkOio [30/May/2024:15:15:16 +0000] "GET /deb/ubuntu/dists/bionic/main/source/by-hash/SHA256/c6f31d4574a468d87347ea68b1420d9d5cbc7f1704df40b4ec45c3a7b1cb11e4 " 200 2247 - "Debian APT-HTTP/1.3 (1.6.17)" eula:none ``` # Analyzing Client Logs with Athena Analysis on package downloads via client logs can provide valuable insights into the usage patterns of your users. In this guide, we'll explore how to export client logs to S3 and use AWS Athena to extract valuable insights about package downloads, as well as preparing the data for consumption by other applications. ## Why Client Logs are Important Client logs are a record of every package downloaded by your users, along with information about the download such as the date and time, the user agent, and the IP address. By analyzing these logs, you can gain valuable insights into how your users are interacting with your product. For example, you can identify the most popular packages, track download trends by country, and monitor usage by package versions. This information can be used to optimize your product development, improve your marketing efforts, and identify potential security threats. ## Exporting Client Logs to S3 Head over to our [help documentation on access log exports to S3](/logs-and-observability/access-log-exports-to-s3) Once configured to export client logs to S3, you will start receiving new log files at the specified interval. These log files will contain detailed information about each download event that takes place in your repositories. ## Analysing Client Logs with AWS Athena Athena is an interactive query service that makes it easy to analyze data in S3 using standard SQL. Here's how to use Athena to analyze your client logs: Create an Athena table that matches the schema of your client logs. This can be done using a CREATE TABLE statement that specifies the location of the log files in S3 and the table properties. The exact SERDE settings and properties will depend on the format your logs are exported in, the below shows an example for CSV-formatted logs. ```sql CREATE EXTERNAL TABLE IF NOT EXISTS my_repository_downloads ( `datetime` STRING, `repository` STRING, `status` INT, `method` STRING, `uri` STRING, `host` STRING, `ip_address` STRING, `bytes` INT, `city` STRING, `country` STRING, `edge` STRING, `format` STRING, `package` STRING, `package_tags` STRING, `recorded` STRING, `referer` STRING, `request_id` STRING, `token` STRING, `user` STRING, `user_agent` STRING, `eula` STRING ) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES ( "separatorChar" = ",", "quoteChar" = "\"", "escapeChar" = "\\" ) LOCATION 's3://bucketname/repositoryname/' TBLPROPERTIES ("skip.header.line.count"="1", "serialization.encoding"="UTF-8") ``` Once the table is created, you can start querying the data using standard SQL statements. For example, you can write a query to identify your most active tokens: ```sql SELECT count(*) as num_downloads, token FROM my_repository_downloads GROUP BY token ORDER BY count(*) desc LIMIT 10 ``` ## Examples of Insights Here are some examples of the types of insights that can be gained from analyzing client logs: **Most popular packages**: By counting the number of downloads for each package, you can identify the most popular packages and prioritize their development and marketing. **Download trends by country**: By filtering the data by country, you can identify where your product is most popular and adjust your marketing efforts accordingly. **Usage by package versions**: By filtering the data by package version, you can identify which versions are most widely used and prioritize bug fixes and feature improvements. ## Aggregation and export While it's great to have all of your client logs in one place on S3, sometimes you want to be able to quickly answer high-level questions about usage. This is where rolled-up views come in. By aggregating the data in different ways, you can create summary tables that can be queried or exported into other systems more quickly and easily than the raw logs. Here's an example of how you could use Athena to create a summary table that shows the total number of downloads per IP Address. ```sql CREATE TABLE package_downloads_per_ip_address WITH ( format = 'PARQUET', external_location = 's3://my-bucket/package_downloads_per_ip_address/' ) AS SELECT ip_address, COUNT(*) AS downloads FROM my_repository_downloads GROUP BY ip_address ``` In this example, we're creating a new table called package_downloads_per_ip_address. The table is stored in Parquet format and partitioned by package to make querying more efficient. The SELECT statement does the actual aggregation using the GROUP BY clause to group downloads by IP Address. Once you have this rolled-up view, you can ETL it into your application database to make these insights available to end users. You can also create additional views that roll up the data in different ways, depending on the questions you want to answer. ## Conclusion Whether you use Cloudsmith for internal development or for distributing software to customers, understanding how your users interact with your product is essential. By taking advantage of the rich data provided by client logs, you can gain a deeper understanding of your users and optimize your product and workflows accordingly. We hope this guide has been helpful in showing you how to generate insights and track usage with client logs exported to S3. If you have any questions or feedback, please let us know in the comments below. # Audit Logs Audit logs provide a log of events across your organization, such as [creating](/repositories/create-a-repository) or [deleting](/repositories/repository-settings#dangerous-actions) a repository, or modifications to repository settings/configuration. [Image: Audit Logs] Cloudsmith retains audit logs for 1 year. Your plan level determines how far back you can review and search audit logs. | **Plan** | **Audit log review period** | | --------------------------------- | ----------------------------------------- | | Pro | Last 30 days | | Ultra | Last 60 days | | Enterprise | Custom, up to 1 year To update your audit log review period, please reach out to your customer success manager. | ## Key Concepts Clicking on a row in your audit log will expand to show more details. [Image: Expanded log entry] Each entry in the log represents an event or a state change and consists of four main components. - **Actor**: The object that performed the Action, such as a User, Service Account or System - **Verb**: The verb (phrase) identifying "what_happened", such as `login`, `retention_settings_changed` or `token_created` - **Action Object**: The object which was created, deleted or updated by the action. - **Target**: (Optional) The object within which the Action was performed, such as a repository or account ## Searching / Filtering You can Search and Filter the Audit Log using the search box at the top. You can also use boolean logic (e.g. AND/OR/NOT) for complex search queries. ### Search Terms |Search By|Search Terms Example| |:--|:--| |Actor|`actor:some-user`| |Actor Kind|`actor_kind:user` (user) `actor_kind:service_account` (service account) `actor_kind:system`|" |Event Time|`event_at:>"1 day ago"` `event_at:<"June 21, 2022 EST"`| |Event Kind|`event_kind:action` (action) `event_kind:create` (create) `event_kind:read` (read) `event_kind:update` (update) `event_kind:delete` (delete) | |Event (Fuzzy)|`event:api_key` (api key events) `event:entitlement` (entitlement events) `event:login `(login events) `event:package` (package events) `event:retention` (retention events) `event:service_account` (service account events)| #### Field type modifiers (depending on the type, you can influence behaviour) - For all queries, you can use: `~foo` for negation - For string queries, you can use: `^foo` to anchor to start of term `foo$` to anchor to end of term `foo*bar` for fuzzy matching - For number/date queries, you can use: `>foo` for values greater than `>=foo` for values greater / equal ` # Client Logs Cloudsmith's Client Logs provide a single pane of glass to visualize, filter, and export information about package usage within your entire workspace. It keeps the same _artifact-agnostic_ support to unify information about all of your repositories, packages, and users in one place. [Image: Client Logs Overview] **Disambiguation** If you want to learn how to review an audit of actions performed by users of your workspace, visit the [*Audit Logs*](/logs-and-observability/audit-logs) section. If you need information about usage to understand its impact in billing, click in the *Usage* tab in the top right corner of your Workspace. ## Use cases No matter if your goal is gaining visibility, performing an exploratory analysis, or reporting to stakeholders, Client Logs provides all the information that you need. Here you can find a few example use-cases: - Analyze which Maven artifacts were downloaded last month - Understand all the different artifacts used in a staging pipeline - Understand usage of different repository types; private repos, OSS repos, public broadcasts, private broadcasts etc. ## Requirements To access Client Logs, you should be a *Manager* or an *Owner* of your Workspace. **About error logs** For customers using custom domains, client logs will show both successful and unsuccessful requests. For customers who aren't using custom domains, client logs only contain successful requests. ## Navigating the Client Logs overview You can access Client Logs directly from your Workspace Overview page or from any Repository main page. To access your Client Logs, browse to any of your workspaces and click the *Logs* tab in the top right corner. Executing the same action from a Repository Overview page takes you to the Client logs, but automatically adding a filter to include only download events associated with this repo. The Client Logs area displays the following information: [Image: Menus] | Section | Description | | :--------------------- | :---------- | | Filters | A Filter menu in the left to scope your search by any Target or Actor parameters. Learn more about fields available in [Fields and filters](#filters). | | Time-graph | An interactive time-frame graph in the top of the screen to visualize how events distribute across the time-frame selected. Use this visualization to zoom into your data. | | Log details | A list of events result from applying the filters to the search. Here you can visualize every single package downloaded from your workspace. You can review a detailed sample entry [here](/logs-and-observability/client-logs#analyzing-a-specific-entry). | | Actions | Two action buttons in the top right corner to fetch new data or download a report of events with your selected scope. | ### Filters Client Logs allow you to filter events by *time* or any of the download *fields available*. There are two main categories: - **Target**: Information about the artifacts or repositories. - **Actor**: Identify who's downloading the package. | Filter | Description | Filter | Description | | :--------------------- | :---------- | :--------------------- | :---------- | | **Target** || **Actor** | | Package Format | _Maven_, _Python_, _Docker_... you name it. Cloudsmith supports 30 different [formats](/formats). | IP Address | IP address of the download request | User Type | User or Service. | | Repository Type | Public, Private, or OSS. | Users | Name, determined by the owner of the _API Token_ used. | | Repository | the name of your repo. | Entitlement Token | Name of the service or non-human agent that downloaded the package. | **About search limits** Some of these fields (for example, IP Address, Users and Entitlement Token) could potentially return many thousands of results in a dropdown. A 1k limit is applied, displaying the first 1k results ordered by descending most results. #### Using the time filters The time graph shows how your package downloads distribute across time. It helps you to visualize in a quick view when there's more or less demand. [Image: Filtering events in time] Each block in the graph represents the atomic time interval for the current view, for example one week. Hover your mouse over each of the block to preview the number of items downloaded in that period. Use the left menu to select your desired time scope: _last 30 minutes_, _last hour_, _last day_, _last year_, or a _custom time range_ for more specific periods. Alternatively, you can use the time graph to browse for a specific time range. Click and drag your mouse in the graph to zoom in your data. to select the specific period you want to visualize. This action displays the total number of items downloaded during the specified period. Then, to apply the selected filter to the list of events, just click **Set as time range** and the web app returns a new filtered list of results. Once applied the desired scope, you can preview the list of downloads matching the conditions. ## Visualizing events When you perform a single download, the client may also need to fetch metadata about the package, indexes, and keys. As a result, a single package download usually results in more than one log. The number of requests that you see for a single download varies based on the package format. For each of entries, you have an overview of the next information available: | Column | Description | | :---------------- | :-------------------------------------------------------------------------------------------------------------------- | | Event Type | Cache Miss, Hit, etc. | | Timestamp | The date and time of the request. | | Status Code | The HTTP status code for the request. | | Target name | The name of the asset requested | | Target EULA | If requires acceptance of a End-User License Agreement (_EULA_). | | File-size | Size of the asset requested. | | Latency | Response time. | | IP Address | The IPv4 or IPv6 address for the request. | | Authentication | The type of authentication provided, if its a private repository. | ## Analyzing a specific entry Below you can find a preview of one of the event entries. [Image: Filtering events in time] This log entry captures an event where the package `bartosz-testing` was requested. It is divided into several logical sections: **request**, **response**, **target**, and **actor**. ### Request details This section describes the specifics of the client's request. | Field | Example Value | Description | | :-------------- | :--------------------------------------------- | :-------------------------------------------------------------------------- | | **Identifier** | `BXAwZg98KnQXZOlE8Io48iWr-RoxWcTFQ_9FCV72pIE...` | A unique identifier for this specific request within the system. | | **Timestamp** | `2025-05-22 08:15:25 UTC` | The precise date and time the server received the request. | | **Method** | `GET` | The HTTP method used for the request (for example: `GET` for retrieval, `POST` for submission). | | **Host** | `dl.cloudsmith.xyz` | The hostname or domain to which the request was directed. | | **Path** | `/npm/debug/4.3.4/debug-4.3.4.tgz` | The specific path to the requested resource on the host. | | **URI** | `/9tQrQ5beoGpaQ6nGh/bartosz-testing/npm/debug/4.3...` | The complete Uniform Resource Identifier of the requested resource. | | **Query string**| `-` | Any parameters appended to the url after a `?` (empty in this example). | | **User agent** | `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) Apple...` | A string identifying the client's web browser, operating system, and other details. | | **Referer** | `https://app.cloudsmith.com/` | The url of the page that referred the client to the requested resource. | ### Response details This section provides information about the server's reply to the request. | Field | Example Value | Description | | :-------------- | :------------ | :------------------------------------------------------------- | | **Response** | `200 1,484ms` | The HTTP status code and the time (in milliseconds) it took for the server to generate and send the response. | | **Edge location** | `eu-west-2` | The geographical location of the server (edge node) that processed the request (for example: an AWS region). | | **Cache** | `Cache Miss` | Indicates whether the requested item was served from a cache (`Cache Hit`) or had to be fetched from the origin (`Cache Miss`). | | **Size** | `14.0kB` | The size of the response body, typically the size of the downloaded file. | ### Target details This section focuses on the specific package or artifact that was the subject of the request. | Field | Example Value | Description | | :----------- | :---------------------- | :---------------------------------------------------------- | | **Repository** | `bartosz-testing` | The name of the repository where the package is stored. | | | `PRIVATE` | Indicates the access level of the repository (for example: `PRIVATE`, `PUBLIC`). | | **Package** | `debug 4.3.4` | The name and version of the requested package. | | **File** | `debug/-/debug-4.3.4.tgz` | The specific file within the package that was accessed. | | **EULA** | `-` | End-User License Agreement information (if applicable). | ### Actor details This section provides information about the entity (user or token) that initiated the event. | Field | Example Value | Description | | :---------------------- | :------------------ | :--------------------------------------------------------- | | **Token** | `Default` | The authentication token used to authorize the request. | | **User** | `-` | The specific user account associated with the token (if applicable). | | **Authentication method** | `Token` | The method of authentication used for the request. | | **IP Address** | `88.7.241.39` | The public IP address of the actor. | | **Location** | `Zaragoza Spain` | The geographical location (city and country) derived from the actor's IP address. | **Download Logs for Docker** Due to the nature of Docker and how layers may be shared across images, download counts for Docker are collected differently from other formats. The download count displayed for a Docker package is based on pulls of the "configuration" layer for an image, so pulls for other data-based layers aren't counted. However, this is typically enough to gauge usage for an image. ## Reporting Once you have found what you were looking for, click in the top right corner download button and then confirm the **Download** to get a report of all the download actions in CSV format. **Log entries limit** There is a limit of 500 log entries for download operations. ## FAQs and troubleshooting ### Why is the downloaded package name not populated? There are two possible reasons: - There wasn't a package download, it was a metadata file download. - Deleted package since upload. ### Why is a repository/user/token field not populated? Logs only contain anonymous identifiers for certain objects like packages, users, or repositories. So, if the field is empty, it's likely the repository/user/token has been deleted since the download occurred. # Log Exports to Azure ## Prerequisites To use this feature, you'll need administrative access to an [Azure Tenant](https://portal.azure.com/) (in order to create a Blob Container for receiving the log exports from Cloudsmith). **If you already have one type of export set up with us and don't require updating your role/permissions, please skip to [Existing Setup](/logs-and-observability/exports-to-azure#existing-setup).** ## New Setup Next, you'll need to **follow these steps**: **1**. In the Storage Account you would like to use, create a Blob Container for the logs, such as `cloudsmith-acmecorp-logs` where you can replace `acmecorp` with your own organization name. **2**. If using an existing container, pick a folder name for the Cloudsmith logs to go into, such as `cloudsmith-logs`. This is configured in the next step and configured on our side in tandem. **3**. If planning to exports multiple types of logs (e.g. Client and Audit logs), let us know and we will make sure to use separate folders names inside of your Cloudsmith log export container (Note them down for later): **4**. Create a Managed Identity and assign a Role to it that only has write access to the Blob Container you created in the previous step. Make note of the Managed Identity Client ID as we will need it for our next step. **5**. [Contact us](https://cloudsmith.com/company/contact-us) to tell us your: **One container to rule them all** We recommend setting up one container with folders for each log type and use the same Azure Tenant, if you require each log type to have a different Azure Tenant, please let us know. - Azure Tenant ID - Azure Managed Entity Client ID - Azure Storage Account Name - Azure Blob Storage Container Name - Azure Blob Storage Folder Name - The log format that you want to export: - JSON (Stream) **(RECOMMENDED)** - JSON (Stream+TimeStamp) - CSV - Apache Style - The log type that you want to export: - Audit Logs - Client Logs - Prefix (folder names) for each type of log export - Export Options: - Entire org **(RECOMMENDED)** - Selected repositories only **6**. We will use an OIDC Token to authenticate against your Azure Managed Entity. For this, you will need to create a Federated Credential in the Managed Entity you previously created. - Go to Managed Identities > Federated Credentials, and create a new one. - We will send you all the details you need to fill in. # Existing Setup **We will use your existing Azure Tenant for new log types** If you already have an export set up with us, we will use the same account that you have initially provided us with. If you wish to use a new account for a new type, please go through the _**[New Setup](/logs-and-observability/exports-to-azure#new-setup)**_ and provide relevant information. If you already have Blob Container exports setup with us, we will use the same Azure account that you have provided us with. The only change that may be required is a change of folders inside your Cloudsmith log container to separate the different log types. For example if your current solution has no folders inside of the Cloudsmith export container and all exports go to `/`, we recommend creating folders: - client-logs - audit-logs _e.g. Client logs will export to `//`_ **Please let us know if you plan to separate the logs into different folders in order for us to reflect the changes in both the new export requested and the old one.** # Summary of Information Required You should have provided us with the following information: - Azure Tenant ID - Azure Managed Entity Client ID - Azure Storage Account Name - Azure Blob Storage Container Name (e.g. `cloudsmith-acmecorp-logs`). - Azure Blob Storage Folder Name (e.g. `cloudsmith-logs`), if any. - The type of logs you would like to export (e.g. Client and/or Audit logs). - The format of the logs you'd like to export (e.g. Apache-style, CSV, JSON Stream, JSON Stream + Timestamp, etc.) **Which format should I choose?** It depends. Each format is "streaming," meaning there's one line per entry for each download rather than a large object. In terms of detail, the Apache (or Nginx) format has the least to conform with those formats, CSV has the next level of detail but is limited due to columns, and JSON has the most level of detail as a structured format. **We recommend choosing JSON Stream + Timestamp**, in which a JSON blob representing the download is prefixed with a timestamp, making this nice to parse and import into your favorite Business Intelligence (BI) or Security Information Event Monitoring (SIEM) tooling. We should have provided you with the following information to configure a Federated Credential: - Token Issuer URL - Identity ID - Token Audience **What can I include/exclude?** It is possible to include or exclude the following: - EULA acceptance data. - location-based (Geo) data. - IP data. - namespace identifiers/paths. - repository identifiers/paths. # Data Provided ## Client Logs Some of the data provided includes: - **Edge Location**: Nearest CDN Edge Node Location (e.g., "LHR61-C2") - **Edge Response (`edge_response`)**: Identifies whether a package was served from an existing cache or required a cold load. Possible values (and casing) are `Hit` and `Miss`. - **EULA ID**: Unique identifier for EULA accepted (if any) - **EULA Number**: Revision of EULA accepted (if any) - **IP Address**: IP of the client - **Host**: Hostname for the request (e.g. your download domain) - **Method**: HTTP Method (e.g. "GET") - **Geo/IP Fields**: E.g., "City," "Continent", e.g. enriched based on IP (we pay for this) - **Package Identifier**: Unique identifier for downloaded package (if any) - **Package Name**: Name for the downloaded package (if any) - **Package Version**: Version for downloaded package (if any) - **Protocol**: HTTP Protocol Used (e.g. "https/1.3") - **Referer**: HTTP Referrer - **Request ID**: Globally unique identifier for the request - **Status**: HTTP Status Code (e.g. "200") - **Time Taken**: Time taken to service request, in seconds - **Token ID**: Unique identifier for entitlement token used (if any) - **Token Name**: Name for entitlement token used (if any) - **URI**: Uniform Resource Identifier, i.e. path, for the request - **User Agent: ** HTTP User Agent sent by the client - **User ID: ** Unique identifier for the client user (if any) - **User Name**: Name for the client user (if any) ## Audit Logs - **Actor**: The user or system that performed the action. - **Actor IP Address**: The IP address of the actor. - **Actor Kind**: The type of actor, such as "user" or "system". - **Actor Location**: The location of the actor, as determined by their IP address. - **Actor Slug Perm**: A unique identifier for the actor, used for permissions. - **Actor URL**: The URL of the actor's profile. - **Context**: The context in which the event occurred, such as "web" or "api". - **Event**: The type of event that occurred. - **Event At**: The date and time the event occurred. - **Object**: The object that was affected by the event. - **Object Kind**: The type of object, such as "user" or "file". - **Object Slug Perm**: A unique identifier for the object, used for permissions. - **Target**: The target of the event, such as the user who was affected. - **Target Kind**: The type of target, such as "user" or "file". - **Target Slug Perm**: A unique identifier for the target, used for permissions. - **UUID**: A unique identifier for the event. # Format Examples ## JSON (Stream) [recommended] One JSON record per line, without a header line: ``` {"bytes": 2150, "datetime": "2024-05-30T15:14:27+00:00", "edge": "us-east-1", "edge_response": "Hit", "eula": null, "format": "Terraform", "host": "dl.cloudsmith.io", "ip_address": "3.234.139.83", "location": {"city": "Ashburn", "continent": "North America", "country": "United States", "country_code": "US", "latitude": "39.046900", "longitude": "-77.490300"}, "method": "GET", "namespace": "cloudsmith", "package": {"identifier": "wKSkBFHpsNWB", "name": "cloudsmith-terraform-example", "tags": {"info": ["local"]}, "version": "1.0.1717082040651684"}, "recorded": "2024-05-30T15:42:56+00:00", "referer": "", "repository": "testing-private", "request_id": "8v19hTgCw3rVWw9gl1QiPuSGf3JOpKMktSNSk0vn-rTSqcmCx7bYqw==", "status": 200, "token": null, "uri": "/terraform/terraform-local-cloudsmith-terraform-example-1.0.1717082040651684.tar.gz", "user": {"identifier": "svHuImXP38Ev", "username": "end-to-end-tests"}, "user_agent": {"browser": "Go-http-client 2.0", "device": "Other", "os": "Other", "raw": "Go-http-client/2.0"} ``` ## JSON (Stream+Timestamp) One JSON record per line, prefixed by a timestamp, separated by a single whitespace character\*, without a header line: ``` 2024-05-30T15:14:27+00:00 {"bytes": 2150, "datetime": "2024-05-30T15:14:27+00:00", "edge": "us-east-1", "edge_response": "Hit", "eula": null, "format": "Terraform", "host": "dl.cloudsmith.io", "ip_address": "3.234.139.83", "location": {"city": "Ashburn", "continent": "North America", "country": "United States", "country_code": "US", "latitude": "39.046900", "longitude": "-77.490300"}, "method": "GET", "namespace": "cloudsmith", "package": {"identifier": "wKSkBFHpsNWB", "name": "cloudsmith-terraform-example", "tags": {"info": ["local"]}, "version": "1.0.1717082040651684"}, "recorded": "2024-05-30T15:42:56+00:00", "referer": "", "repository": "testing-private", "request_id": "8v19hTgCw3rVWw9gl1QiPuSGf3JOpKMktSNSk0vn-rTSqcmCx7bYqw==", "status": 200, "token": null, "uri": "/terraform/terraform-local-cloudsmith-terraform-example-1.0.1717082040651684.tar.gz", "user": {"identifier": "svHuImXP38Ev", "username": "end-to-end-tests"}, "user_agent": {"browser": "Go-http-client 2.0", "device": "Other", "os": "Other", "raw": "Go-http-client/2.0"} ``` **Future Compatibility** To ensure compatibility, please parse the "whitespace character" as a space OR tab (for future compatibility). ## CSV Starts with a header line, followed by one CSV record per line: ```csv datetime,repository,status,method,uri,host,ip_address,bytes,city,country,edge,eula,format,package,recorded,referer,request_id,token,user,user_agent 2021-04-01T19:40:55+00:00,cloudsmith/testing-private,200,GET,/rpm/fedora/29/x86_64/cloudsmith-redhat-example-1.0.16173057145-1.x86_64.rpm,dl.cloudsmith.io,82.1.4.8,10421,Newmarket,GB,LHR61-C2,,RedHat,vmN1OlMDi3Iy,2021-04-01T19:46:23.817658+00:00,,dZ3DmJ8yHU36FyH9uHoyWH4LocmlkzdKXSEuOBmnvLhA4ZF7exa5Qw==,wKSkBFHpsNWB,eP1B3YCtTlJX,libdnf ``` ## Apache Style ``` 3.222.115.18 - t-6GXPs3OxkOio [30/May/2024:15:15:16 +0000] "GET /deb/ubuntu/dists/bionic/main/source/by-hash/SHA256/c6f31d4574a468d87347ea68b1420d9d5cbc7f1704df40b4ec45c3a7b1cb11e4 " 200 2247 - "Debian APT-HTTP/1.3 (1.6.17)" eula:none ``` # Logs and Observability Cloudsmith's observability features provide the tools to understand how your artifacts are being accessed, used, and managed. This section covers the key data sources that enable deep visibility into your artifact management lifecycle. - [Client Logs](/logs-and-observability/client-logs): these logs are the foundational data layer, capturing the granular, real-time interactions with your repository. They record every request, download, and deployment, providing a detailed view from the perspective of the developer machines and CI/CD pipelines that consume your artifacts. They are invaluable to understand package consumption patterns. - [Audit Logs](/logs-and-observability/audit-logs): Provides a critical security and compliance perspective of your Workspace. This feature tracks significant events related to governance of your Workspaces and repositories, such as user authentications, permission changes, or modifications to repository settings. This creates an immutable record of "_who did what, and when_" which is fundamental for security forensics and meeting regulatory requirements. - [Usage](/logs-and-observability/usage): analyzes and aggregates client logs information to provide insights and a general overview of your usage data. Together, these logging mechanisms offer a comprehensive view of your artifact ecosystem. While this section also touches on advanced capabilities like exporting this data for long-term retention or specialized analysis, the core principle is to provide you with the foundational visibility needed to secure and manage your software supply chain effectively. # Usage To access this feature, click on **Usage** on the top right corner of your Workspace/Repository Overview page. The Client Statistics page shows usage information of all access to your Workspace or Repositories. It includes the next areas: - Overview of usage - Delivery Summary - Package Delivery - Artifact data {/* TODO update with new gif, legacy charts were recently removed */} [Image: Statistics Page] ## Overview of usage The total amount accumulated during the **Current Billing Cycle** for: - Package Delivery. - Artifact data. - Packages. - Container images. - Open Source delivery. When applicable, it also includes a comparative (percentage variation) with the previous comparable cycle. [Image: Statistics Page] ## Delivery Summary Displays a list of all of the package bandwidth for the selected time scope. The results can be grouped by package or repository, and display the trend compared with the previous comparable time scope. All of the data available in this chart is sourced from the [Client Logs](./client-logs) feature. Click on any of the entries in the list to view its own data in the Client Logs. [Image: Statistics Page] ## Package Delivery The "Package Delivery" chart presents data related to package consumption, enabling monitoring and analysis of download trends over defined periods. It displays metrics such as total downloads and bytes transferred, contributing to an understanding of artifact distribution and utilization. [Image: Statistics Page] This section can be displayed as a chart or a table. In both cases, it displays the same information: package download activity. Users can select various aggregate types, including "Bytes Downloaded Sum" for data transfer volume, or other metrics like download counts. Basically, aggregate allows you to decide _how you want to quantify or summarize the package download activity_. Here you can find a detailed list of options available: - **Bytes Downloaded Average**. - **Bytes Downloaded Sum**. - **Unique IP Address Count**. - **Request Count**. - **Time Taken** (Avg, Max, P50, P75, P90, P95, P99, Sum). ### Time Period Selection - **Period**: Pre-configured options are available for historical data analysis (e.g., "Previous 12 months"). - **Date Selection**: A configurable date range picker (e.g., "10 Jun 2024 - 10 Jun 2025") allows for specific data window definition. - **Interval**: Data can be viewed by different time intervals, such as "Month" or other available options. In order words, this is the _bucket size for which the "Aggregate" calculation is performed_. ### Filtering Options These filters allow you yo refine displayed data based on specific criteria: - **Package format**: Filters by package type (e.g., npm, Maven, Docker). - **Repository type**: Filters by repository classification (e.g., private, public). - **Repository**: Filters data for a specific repository. - **IP address**: Filters usage originating from specific IP addresses. - **User type**: Differentiates between user classifications. - **User**: Filters by individual user download activity. - **Entitlement token**: Filters usage associated with specific access tokens. - Other filters available for **plot Configuration**: - **Group by**: Allows data aggregation based on selected criteria (e.g., package format, repository). - **Plot Type**: Options include "Interval" (data per period) and "Accumulated" (cumulative data). - **Scale**: Selection between "Linear" and "Log" scales for data display. - **Reset Filters**: A "Reset Filters" function is available to restore default filter settings. ### **Purpose** This chart serves to provide data for: * Monitoring bandwidth consumption, and impact on billing. * Identifying package download frequencies. * Tracking user-specific download activities, although **Client Logs** might be better suited for this goal. * Evaluating the impact of different repositories or package formats. * Reviewing access patterns via entitlement tokens. ## Artifact Data Total amount of Artifact data stored within the Workspace/Repository. [Image: Statistics Page] This will increase with time as more artifacts are stored within your Workspace. [Retention Rules](/artifact-management/retention-rules) can help to automate storage management. ## Usage via the CLI You can query the number of active (has been downloaded) and inactive (has not been downloaded) packages via the Cloudsmith CLI with the `cloudsmith metrics packages` command: ```shell cloudsmith metrics packages OWNER/REPOSITORY ``` Example: ```shell cloudsmith metrics packages demo/examples-repo ``` [Image: cloudsmith metrics packages] You can also query package activity during a specific time range using the `--start` and `--finish` parameters and ISO 8601 formatted datetimes. Example: ```shell cloudsmith metrics packages demo/examples-repo --start=2019-01-01T00:00:00Z --finish=2019-12-31T00:00:00Z ``` ### Specific Packages You can query one or more specific packages by adding the `--packages` flag with package identifiers as a comma-separated list (See [Package Identification](/artifact-management/identifying-a-package) for more information). Example: ```shell cloudsmith metrics packages demo/examples-repo --packages=ZGCV58VqT8Sl ``` # Export from JFrog Artifactory Migrating from JFrog Artifactory requires you to migrate hosted repositories only, since any proxy repositories configured in Artifactory can just be set up with the same configuration in Cloudsmith Repository Manager, and all data will be retrieved from the upstream repositories again. All the upstream repositories supported by Cloudsmith are listed [here](/formats). Hosted repositories will have to be migrated from Artifactory to Cloudsmith. To migrate from JFrog Artifactory cloud or a JFrog Artifactory cluster you will be required to have access to Artifactory. ## Exporting via Artifactory's UI You can use the import/export feature of Artifactory and migrate one hosted repository after another. Please consult the Artifactory [documentation](https://www.jfrog.com/confluence/display/JFROG/Import+and+Export) for step-by-step instructions on how to export a repository. ## Exporting via the CLI Exporting via Jfrogs CLI can provide more control over how and what you export. Below are some instructions on how to conduct your export. ### Install jfrog-cli All the information about installing jfrog-cli can be found [here](https://jfrog.com/getcli/). It can be installed on Linux, Mac, or Windows. ### Configure jfrog-cli Log into jfrog-cli with your Artifactory credentials ```shell jfrog config add Server ID: server_name JFrog platform URL: https://server_name.jfrog.io/ JFrog access token (Leave blank for username and password/API key): JFrog username: artifactory_user JFrog password or API key: Is the Artifactory reverse proxy configured to accept a client certificate? (y/n) ``` ### Download your packages from Artifactory The `jfrog rt dl` command can be used to download all packages from Artifactory. To get all artifacts from all repos in Artifactory do the following: ```shell jfrog rt dl "*/*" ``` The following examples use the `--flat` option to dump all the packages into the same folder. > NOTE > Be careful that the use of the _--flat_ option doesn't cause issues due to packages with the same name in different repositories. To get all packages from a particular repository called "REPO" ```shell jfrog rt dl REPO --flat ``` The command below shows you how to just get a particular package type from all repositories, using .tgz as an example ```shell jfrog rt dl "*/*.tgz" --flat ``` --- title: Export from Nexus Sonatype --- # Export from Nexus Sonatype Migrating from Nexus requires you to migrate hosted repositories only, since any proxy repositories configured in Nexus can just be set up with the same configuration in Cloudsmith Repository Manager, and all data will be retrieved from the upstream repositories again. All the upstream repositories supported by Cloudsmith are listed [here](/repositories/upstreams). Hosted repositories will have to be migrated from Nexus to Cloudsmith. To migrate from Nexus you will be required to have access to Nexus and all of the repositories that you wish to migrate. As Nexus doesn't provide an official way to download all of the packages, a [community written script](https://stackoverflow.com/questions/70610642/download-entire-repository-from-nexus-3-37-1) can assist in the process. This script will produce a LogFile and a file for the specified repository with all of the download links, the second part of the script will then pull the information with all of the download links and download them to your directory. > 🚧 > We do not maintain this script and it should be used as a guidance only. ```shell sourceServer= sourceRepo= sourceUser= sourcePassword= filters=("\\.zip$") # Grab only the packages specified in the filter or leave the variable blank to grab everything logfile=$sourceRepo-backup.log outputFile=$sourceRepo-artifacts.txt # ======== GET DOWNLOAD URLs ========= url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo contToken="initial" while [ ! -z "$contToken" ]; do if [ "$contToken" != "initial" ]; then url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo fi echo Processing repository token: $contToken | tee -a $logfile response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"` artifacts=( $(echo $response | sed -n 's|.*"downloadUrl" : "\([^"]*\)".*|\1|p') ) printf "%s\n" "${artifacts[@]}" > artifacts.temp if [ ! -z "$filters" ]; then for filter in "${filters[@]}"; do cat artifacts.temp | grep "$filter" >> $outputFile done else cat artifacts.temp >> $outputFile fi contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') ) done # ======== DOWNLOAD EVERYTHING ========= echo Downloading artifacts... IFS=$'\n' read -d '' -r -a urls < $outputFile for url in "${urls[@]}"; do url="$(echo -e "${url}" | sed -e 's/^[[:space:]]*//')" path=${url#https://*/*/*/} dir="\""$sourceRepo"/"${path%/*}"\"" curFolder=$(pwd) mkdir -p $dir cd $dir url="$(echo -e "${url}" | sed -e 's/\s/%20/g')" curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O >> /dev/null 2>&1 responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2` if [ "$responseCode" == "200" ]; then echo Successfully downloaded artifact: $url else echo ERROR: Failed to download artifact: $url with error code: $responseCode fi rm response.header > /dev/null 2>&1 cd $curFolder done ``` # Exporting NuGet You can download the packages using the native NuGet/Chocolately/Powershell tools or using your package repository-specific tooling. Using package repository-specific tooling may mean that you don't need to disable your upstreams. ## NuGet packages bulk Export using Powershell Before continuing, you must **disable your upstreams**- otherwise, this powershell script will download all the packages from your upstreams as well as the packages stored directly in your package repository. ```powershell $credential = New-Object PSCredential('token',$(ConvertTo-SecureString '' -AsPlainText -Force)) $SOURCE_REPOSITORY = $NUGET_V2_SOURCE_REPOSITORY_FEED_URL = $DESTINATION_DIR = Register-PackageSource -Name $SOURCE_REPOSITORY -Location $NUGET_V2_SOURCE_REPOSITORY_FEED_URL -Trusted -Credential $credential -ProviderName NuGet Find-Package -AllVersions -Source $SOURCE_REPOSITORY -ProviderName NuGet -Credential $credential | ForEach-Object { Save-Package -Name $_.Name -RequiredVersion $_.Version -Path $DESTINATION_DIR -Source $SOURCE_REPOSITORY -Credential $credential -ProviderName NuGet } ``` Some issues: - The above may fail saving packages if it can't find it's dependent package(s). In this case you may want to use the powershell command _Install-Package_ to download the packages. ```powershell $credential = New-Object PSCredential('token',$(ConvertTo-SecureString '' -AsPlainText -Force)) $SOURCE_REPOSITORY = $NUGET_V2_SOURCE_REPOSITORY_FEED_URL = $DESTINATION_DIR = Register-PSRepository -Name $SOURCE_REPOSITORY -SourceLocation $NUGET_V2_SOURCE_REPOSITORY_FEED_URL -InstallationPolicy 'trusted' -Credential $credential Find-Module -AllVersions -Source $SOURCE_REPOSITORY -Credential $credential | ForEach-Object { Save-Module -Name $_.Name -RequiredVersion $_.Version -Path $DESTINATION_DIR -Source $SOURCE_REPOSITORY -Credential $credential } ``` ## Export Chocolately packages You can use the Chocolately CLI to download packages, [here](https://docs.chocolatey.org/en-us/choco/commands/download) # Import Debian Bulk import your Debian packages to Cloudsmith using a script. There are a few prerequisites that need to be set up prior to importing the packages in order for the script to work, and this can be changed to automate according to your setup. **Setup**: 1. [Install the Cloudsmith CLI ](/developer-tools/cli#installation) 2. Create a folder to place the script and content into 3. Create a folder with exactly the same name as your repository name in Cloudsmith 4. Inside the created (repo) folder, create a distro folder you wish to upload, for a list of available distros, you can run `cloudsmith list distros deb` and use the right side of the table as name for the folder. **PLEASE NOTE** that since the distro/release includes a slash, please create a folder replacing it with a dot (`.`). For example, if your distro is `debian/any-version`, the folder should be named `debian.any-version`. In full context: `//`. **Example folder structure**: ```text folder_structure deb_import/ ├─ config.ini ├─ credentials.ini ├─ deb_import.sh ├─ repo_name_1/ │ ├─ any-version.any-release/ │ │ ├─ some_package.deb ├─ repo_name_2/ │ ├─ debian.bullseye/ │ │ ├─ some_package.deb ``` 5. Inside the folder insert all the `.deb` files that will belong to that repository and distro/release. Finally, you can run the script below to start uploading files to the selected repository by running `sh deb_import.sh`. ```shell deb_import.sh # Check if cloudsmith creds exist, if not run cloudsmith login if [ ! -f config.ini ]; then echo "config.ini not found, running cloudsmith login" cloudsmith login fi # set org name and repo read -p "Enter the repo name: " repo_name read -p "Enter the organization name: " org_name # get all folder names in repo_name and save to variable as release names release_names=$(ls $repo_name) # loop through release names and upload to cloudsmith for release in $release_names; do # create a new variable for release name and replace dot with slash release_name=$(echo $release | sed 's/\./\//g') # get all .deb files in release and upload to cloudsmith for deb in $(ls $repo_name/$release/*.deb); do cloudsmith push deb $org_name/$repo_name/$release_name $deb --no-wait-for-sync done done ``` # Import Docker ## Bulk Import of Docker images Since Docker can be used in different ways including remote repositories and different naming conventions this documentation will include a guide as to how you can script around bulk-importing your images to a Cloudsmith repository. ### Process #### Get image names and tags - Get an output of image name+version, this can be achieved by using: - `docker images --format "{{json . }}"` - This will provide a JSON list of all the images and their attributes. We are primarily looking for 2 values, `Repository` and `Tag`. Export the above to a file or assign the output to a dictionary in your script. - If your organization is using remote repositories you can access this information by utilizing Docker's v2 API, more information can be found [here](https://www.baeldung.com/ops/docker-registry-api-list-images-tags#:~:text=3.%20Docker%20Registry%20API%20V2). #### Re-tag the images with Cloudsmith URL **Custom Domain** If you are an Ultra or Enterprise customer, you are entitled to a [custom domain](/workspaces/custom-domains) for your package types including Docker. Before proceeding, if you would like to use a custom domain we highly recommend requesting it first as you will have to re-point the images to your new domain. **Be careful** Before you proceed: 1. Make sure that you are logged into the Cloudsmith repository you wish to push the images to. More information can be found in our [docs](/formats/docker-registry#publish-via-docker). 2. The format that we will be tagging the images is as follows: `docker.cloudsmith.io/v2/OWNER/REGISTRY/` 3. If you are using a custom domain, the URL will change to `/v2/OWNER/REGISTRY/` Once all of the images are saved either to variable or file, they now need to be parsed into `docker tag` command to include the new name for the docker image. 1. Configure your script to insert the following: `docker tag old_image:version docker.cloudsmith.io/v2/OWNER/REGISTRY/image_name:version` 2. Then we can use `docker push :` ### Example If your images are named the following: `docker.other-registry.com/v2/orgname/repository/redhat:latest` Cloudsmith equivalent should look like this: `docker.cloudsmith.io/v2/YourOrgName/YourRepository/redhat:latest` Finally the push command: `docker push docker.cloudsmith.io/v2/YourOrgName/YourRepository/redhat:latest` # Import Files from a Folder Once you have exported all your npm packages you can upload them to Cloudsmith. First make sure you [install the Cloudsmith CLI ](/developer-tools/cli#installation) and export your token. A folder of packages in the correct format can be published to Cloudsmith using the script below. We support over 28+ format types. The supported formats can be found [here](/formats). Create the bash script below and call it `migrate_npm_to_cs.sh` and give it execute privileges. Access the folder with your packages and run the next script: ```shell #!/bin/bash FILES="." FILES=$(find . -type f) for f in $FILES do echo "Processing $f file..." cloudsmith push "$1" "$2" "$f" done ``` Execute the `migrate_npm_to_cs.sh` file passing the [format type](/formats) and the path of `/`. ```shell ./migrate_npm_to_cs.sh python cloudsmith_org/cloudsmith_repo ``` # Import Maven Cloudsmith provides both upstream proxying and caching for Maven packages. Together they mean simpler, more reliable integration of third-party packages into the development process. There is more information about Cloudsmith's Maven repositories [here](/formats/maven-repository) ## Prerequisites - Maven artifacts exported to a folder. - Create a repository on Cloudsmith using the instructions found [here](/repositories/create-a-repository) ## Update settings.xml Maven is configured using a settings.xml file located under your Maven home directory (typically, this will be /user.home/.m2/settings.xml). Follow the instructions [here](/formats/maven-repository#upload-via-maven) to update your settings.xml file for Maven. ## Update pom.xml A POM, the Project Object Model, is the XML file that describes all the aspects of your project that relate to building and packaging the source code into a package file. You will need to update the distribution and dependency values in the POM for each Maven artifact ### Retrieve your Maven endpoint for Cloudsmith The endpoint for the native Maven API is: ``` https://maven.cloudsmith.io/OWNER/REPOSITORY/ ``` ### Update your distributionManagement To enable the retrieval of Cloudsmith hosted packages via Maven, you will need to update the distributionManagement section of your pom.xml file. The distribution repositories define where to push your artifacts. In this case it will be a single repository, but you can configure alternatives. Edit the url tags of distributionManagement to point to your Cloudsmith Maven endpoint like the following to your project `pom.xml` file: ```xml NAME https://maven.cloudsmith.io/OWNER/REPOSITORY/ NAME https://maven.cloudsmith.io/OWNER/REPOSITORY/ ``` ### Update your dependencies The dependencies section of the project pom.xml file may need to be updated for dependencies that were migrated to Cloudsmith. You may not need to update your dependencies if they have the same groupId, artifactId, and version. The following XML is an example of a dependency in a pom.xml: ```xml GROUP_ID ARTIFACT_ID PACKAGE_VERSION ``` ### Update your pluginRepository A repository holds two main kinds of items, called artifacts. The first and most common kind are dependencies, which other projects need to work. The second kind are Maven plugins. Because plugins are a special type of artifact, they can have their own separate repositories. The setup for these `pluginRepositories` is very similar to the setup for regular repositories: ```xml NAME https://maven.cloudsmith.io/WORKSPACE/REPOSITORY/ true true ``` ## Deploy to Cloudsmith Once you have your pom.xml and settings.xml file all updated with the Cloudsmith information, you just need to package your Maven package and deploy it to Cloudsmith. ```shell mvn package mvn deploy ``` # Import npm Once you have exported all your npm packages you can upload them to Cloudsmith. First make sure you [install the Cloudsmith CLI ](/developer-tools/cli#installation) and export your token. A folder of npm packages, in the .tgz format, can be published to Cloudsmith using the script below. Create the bash script below and call it `migrate_npm_to_cs.sh` and give it execute privileges ```shell #!/bin/bash FILES="." for f in "$2/"*.tgz do echo "Processing $f file..." cloudsmith push npm "$1" $f done ``` Execute the `migrate_npm_to_cs.sh` file passing the path of `/` and `` ```shell ./migrate_npm_to_cs.sh cloudsmith_org/cloudsmith_repo ``` # Import NuGet ## Bulk Import of NuGet Packages using the Cloudsmith CLI Once you have exported all your NuGet packages (.nupkg files) you can upload them to Cloudsmith. First make sure you [install the Cloudsmith CLI ](/developer-tools/cli#installation). A folder of NuGet packages, in the .nupkg format, can be published to Cloudsmith using the scripts below in either powershell or bash ```powershell # Script to upload all package in a directory into a Cloudsmith repo, using the Cloudsmith CLI $SourceFolder = '/your/full/path/to/folder/containing/nupkgs/here' $TargetRepo = 'YOUR_CLOUDSMITH_ORG/YOUR_CLOUDSMITH_REPO' Get-ChildItem -Path $SourceFolder -Filter '*.nupkg'| ForEach-Object { Write-Host -ForegroundColor Green "Pushing package "$_.Name"" cloudsmith push nuget $TargetRepo $_.FullName } ``` ```shell #!/bin/bash #Script to upload all package in a directory into a Cloudsmith repo, using the Cloudsmith CLI # How to use: # Save this bash script as 'migrate_nuget_to_cs.sh' and give it execute privileges # this script takes in 3 parameters from the command line: # $1: '/' # $2: '' # $3: '' # To execute the script from the command line and pass in the 3 parameters: # ./migrate_nuget_to_cs.sh / FILES="." for f in "$2/"*.nupkg do echo "Processing $f file..." cloudsmith push nuget "$1" $f -k $3 done ``` # Importing Python Bulk import your packages from the remote repository to Cloudsmith using a simple Python script and upstream. ## Process **Remote Repository** Make sure that the remote repository that you are using doesn't have an upstream configured to a large public distribution (e.g. `https://pypi.org/`) as this will start proxying for all of the packages inside there too. 1. Setup an upstream for your repository: - Create a new repository that you wish to bulk import your packages to. - Once that's created click `Upstream Proxying`. - Create a new Python upstream by pointing your URL at the remote repository (e.g. Artifactory) and fill in any credentials that may be needed. - Create the upstream and wait for the indexing to complete. 2. Copy over the following script which will: - Grab the index of all packages in the setup upstream - Download them into a `Downloads` folder where the script is running from. - Upload them to your chosen repository. The script below includes the `main.py` and `requirements.txt` files: **Installing the CLI** Please make sure that you have Cloudsmith CLI downloaded. Installation steps can be found [here](/developer-tools/cli#installation). ```python main.py import requests from bs4 import BeautifulSoup import urllib.request from time import sleep import os from os.path import exists org_name = input("Your workspace name (Same as on Cloudsmith): ") repository = input("Repository that you wish to upload your Python packages to: ") print(f"Please grab `index-url` from: https://app.cloudsmith.io/WORKSPACE/REPOSITORY/setup/#formats-python") url = input("Paste in your `index-url`: ") reqs = requests.get(url) soup = BeautifulSoup(reqs.text, 'html.parser') print("Checking for packages...") # Grab all package names and download links urls = [] for link in soup.find_all('a'): urls.append(link.get('href')) urls2 = [] packages = [] # Project Root ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) download_dir = ROOT_DIR + "/Downloads" # Create Downloads folder if not os.path.exists(download_dir): os.mkdir(download_dir) # Change to downloads directory os.chdir(download_dir) # Download all of the packages from the URL print(f"Found {len(urls)} packages. Downloading...") for package_name in urls: url2 = url+package_name reqs2 = requests.get(url2) soup2 = BeautifulSoup(reqs2.text, 'html.parser') for package in soup2.find_all('a'): package_name = package.get('href')[6:] packages.append(package_name.split("#")[0]) download_url = url[:-7] + package_name urllib.request.urlretrieve(download_url, package_name.split("#")[0]) print("Packages downloaded. Please login to your Cloudsmith account to upload the packages.") # Login to Cloudsmith if config files are not found if not exists("config.ini") and not exists("credentials.ini"): os.system("cloudsmith login") # Upload all of the packages to Cloudsmith print(f"Uploading to {org_name}/{repository}") sleep(5) # Use Cloudsmith CLI to push the packages for package in packages: os.system(f"cloudsmith push python {org_name}/{repository} {package} --no-wait-for-sync") ``` The `main.py` python script above requires several Python packages to function correctly. The `requirements.txt` file below lists all of these required dependencies and their specific versions. Before running the main script, create a file named `requirements.txt` in the same directory and paste the following content into it. You can then install all the necessary packages at once by running this command in your terminal: ```python pip install -r requirements.txt ``` ```text requirements.txt autopep8==1.7.0 beautifulsoup4==4.11.1 blessed==1.19.1 boto3==1.24.59 botocore==1.27.59 bs4==0.0.1 certifi==2022.6.15 charset-normalizer==2.1.1 cloudsmith-api==1.142.3 defusedxml==0.7.1 gitdb==4.0.9 GitPython==3.1.27 gpg==1.18.0 idna==3.3 inquirer==2.10.0 jira==3.4.0 jmespath==1.0.1 oauthlib==3.2.0 packaging==21.3 pycodestyle==2.9.1 pyparsing==3.0.9 python-dateutil==2.8.2 python-editor==1.0.4 readchar==4.0.2 requests==2.28.1 requests-oauthlib==1.3.1 requests-toolbelt==0.9.1 s3transfer==0.6.0 sh==1.14.3 six==1.16.0 smmap==5.0.0 soupsieve==2.3.2.post1 toml==0.10.2 typing_extensions==4.3.0 urllib3==1.26.12 wcwidth==0.2.5 ``` # Import RPM Bulk import your RPM packages to Cloudsmith using a script. There are a few prerequisites that need to be set up prior to importing the packages in order for the script to work, and this can be changed to automate according to your setup. **Setup**: 1. [Install the Cloudsmith CLI ](/developer-tools/cli#installation) 2. Create a folder to place the script and content into 3. Create a folder with exactly the same name as your repository name in Cloudsmith 4. Inside the created (repo) folder, create a distro folder you wish to upload, for a list of available distros, you can run `cloudsmith list distros rpm` and use the right side of the table as name for the folder. **PLEASE NOTE** that since the distro/release includes a slash, please create a folder replacing it with a dot (`.`). For example, if your distro is `el/any-version`, the folder should be named `el.any-version`. In full context: `//`. **Example folder structure**: ```text rpm_import/ ├─ config.ini ├─ credentials.ini ├─ rpm_import.sh ├─ repo_name_1/ │ ├─ any-distro.any-version/ │ │ ├─ some_package.rpm ├─ repo_name_2/ │ ├─ el.7/ │ │ ├─ some_package.rpm ``` 5. Inside the folder insert all the `.rpm` files that will belong to that repository and distro/release. Finally, you can run the script below to start uploading files to the selected repository by running `sh rpm_import.sh`. ```shell rpm_import.sh # Check if cloudsmith creds exist, if not run cloudsmith login if [ ! -f config.ini ]; then echo "config.ini not found, running cloudsmith login" cloudsmith login fi # set org name and repo read -p "Enter the repo name: " repo_name read -p "Enter the organization name: " org_name # get all folder names in repo_name and save to variable as release names release_names=$(ls $repo_name) # loop through release names and upload to cloudsmith for release in $release_names; do # create a new variable for release name and replace dot with slash release_name=$(echo $release | sed 's/\./\//g') # get all .rpm files in release and upload to cloudsmith for rpm in $(ls $repo_name/$release/*.rpm); do cloudsmith push rpm $org_name/$repo_name/$release_name $rpm --no-wait-for-sync done done ``` # Migrating to Cloudsmith This help doc will walk you through the steps involved in migrating packages, artifacts, binaries, images, or zip files into Cloudsmith. If you have any trouble migrating, contact us, and we will be happy to assist in the process. Migrating from another package repository requires the following: - Export the packages out of your old repositories like JFrog's Artifactory or Sonatype Nexus. - Importing to Cloudsmith - Optional setup requirements ## Exporting from another repository If you have been running another repository manager, such as Artifactory, Nexus, MyGet or ProGet and you want to migrate your package hosted on this repository to Cloudsmith, the first step is exporting the packages out of your old repository. Depending on your package repository and/or package format, you will have to use different approaches to exporting your packages. We have the following documentation on how to: - [Export any package from JFrog Artifactory](/migrating-to-cloudsmith/export-from-jfrog-artifactory) - [Export from Nexus Sonatype](/migrating-to-cloudsmith/export-from-nexus-sonatype) - [Export NuGet packages from any package repository](/migrating-to-cloudsmith/exporting-nuget) > 📘 Export details not here? Contact us and we'll help. ## Import into Cloudsmith After the export, you are ready to import the packages into your new Cloudsmith repository . We will assume you have exported all your packages/artifact-management/binaries into a folder or a shared location (e.g. S3 bucket). Importing your packages into Cloudsmith should be a stress-free process; you will need to: - Create an [organization](/workspaces) and at least one [repository](/repositories) on your Cloudsmith account. - Install the [Cloudsmith CLI](/developer-tools/cli) and export your API token. - Do a bulk import of your packages per package format using the Cloudsmith CLI. For a build import from a folder, follow the instructions [here](/migrating-to-cloudsmith/import-files-from-a-folder). - We have specific instructions for certain package types : - [Import Maven](/migrating-to-cloudsmith/import-maven) - [Import npm](/migrating-to-cloudsmith/import-maven) - [Import NuGet](/migrating-to-cloudsmith/import-maven) - [Import Docker](/migrating-to-cloudsmith/import-docker) - [Import Python](/migrating-to-cloudsmith/import-python) > 📘 Import details not here? Contact us and we'll help. ## Other settings to consider If you are migrating packages and setting up Cloudsmith for the first time, you may need to consider other tasks when getting started, including: - Update your build tools and configurations to push/pull to the new Cloudsmith endpoints. You may have a configuration file for every package format in your code base, you will find information on how to set up Cloudsmith for each format [here](/formats). - Setup your [Single Sign-On](/authentication/single-sign-on) and [Group Mapping](/authentication/single-sign-on#saml-group-sync) instructions if this is your preferred way to log in to your users. - Setup your [upstreams](/repositories/upstreams), which will set up Cloudsmith to fetch and cache your 3rd party dependencies hosted on public repositories like NuGet Gallery, RubyGems.org or Maven Central. - Setup [signing keys](/supply-chain-security/signing-keys). - Setup your [service accounts](/accounts-and-teams/service-accounts). - Setup your [logs export to s3](/logs-and-observability/access-log-exports-to-s3). - If you are a distributor, you may want to set up [Entitlement tokens](/software-distribution/entitlement-tokens) to distribute to 3rd parties - Setup [Custom Domains](/workspaces/custom-domains), which allow you to present your brand and domain when using native tooling or pulling packages from your Cloudsmith Workspace. # API Key Policy ## API Key Policy Overview The API key policy allows workspaces to customize the maximum age of API keys across all workspace accounts, by specifying the duration after which keys should be rotated. Frequent API Key rotation ensures that API keys older than the designated age are invalidated for authentication, with the added flexibility to enforce a refresh for keys that exceed the specified duration. **Early Access** The API Key Policy feature is in early access; if you would like to try this feature, please [Contact Us](https://cloudsmith.com/company/contact-us). ## View API Key Policy To view your workspace's API Key Policy, go to your Workspace settings > Manage policies and select Authentication Policies from the left-hand menu. The API Key Policy is disabled by default. [Image: API Key Policy] ## Enable API Key Policy To enable the API Key Policy, set the "Maximum permitted age (hours)" field and click "Update policy". [Image: Enable API Key Policy] ||| |:---|:---| |**Maximum permitted age (hours)**|The maximum permitted age of an account's API key.Note: the minimum value this can be set to is “24”.| |**Days**|Read-only field, displaying the value of "Maximum permitted age (hours)" field in days.For example, if the "Maximum permitted age (hours)" field is set to “720” hours, this field will display “30” days.| |**Enforce automatic API key refresh**|Enable to automatically refresh API keys that violate the policy. Take extreme caution when enabling this option, as this cannot be undone.See section [ Enforce Automatic API Key Refresh](/policy-management/api-key-policy#enforce-automatic-api-key-refresh) for further details.| Once the policy is enabled, permission will be revoked for any API key older than the specified maximum age. This applies to all accounts associated with the workspace, including members, managers, owners, service accounts, and collaborators. If a user attempts to utilize an API key that violates the policy, they will receive a "permission denied" error message To regain access, the user should update their [API Key](/accounts-and-teams/api-key) by clicking "Refresh" within the API settings page. [Image: Refresh API Key] ### Enforce Automatic API key Refresh **API key refresh** Exercise extreme care when utilizing this option, as refreshing an API key is an irreversible action. To automatically force refresh an API key that violates the policy, enable "Enforce automatic API key refresh". This force refreshes API keys of all users (member, manager, owners and service accounts) that violate the policy, except for collaborators. [Image: Enable API Key Policy with enforced automatic API key refreshing] The process of force refreshing all API keys may take up to 60 minutes to complete. To enforce an immediate refresh of API keys that violate the policy, enable both the "Enforce automatic API key refresh" and "Refresh immediately" options. [Image: Enable immediate for refresh of API keys] ## Disable API Key Policy To disable the API key policy, clear the value within the "Maximum permitted age (hours)" field and click "Update policy". [Image: Disable API Key Policy] The UI will then update to confirm the Policy has been disabled. [Image: Disabled Confirmation] Once disabled, restrictions to API keys that previously violated the policy are lifted and these API keys will become operational again. ## Members of Multiple Workspaces If a user is a member of multiple workspaces the user's permissions will only be revoked for the workspace where the API key policy is enforced and their API key violates the policy. Their permissions in all other workspaces are unaffected. However, if the user is a member of a workspace which also has "Enforce automatic API key refresh" enabled, the user's API key will be refreshed across all their workspaces. ## Notifications When an API key is approaching its expiration age, users will receive a notification at least 24 hours in advance. This notification will be sent via email and will also appear as a notification when they log into the application. Additionally, users will receive an email and UI notification when their API key has expired. If the API policy is enabled and "Enforce automatic API key refresh" is set, users will also receive an email notification when their API key has been refreshed. This email will include details about the user who initiated the refresh process. For service accounts, the creator of the service account will receive an email notification informing them of the upcoming expiration of the service's API key. The email also provides a URL that the user can use to reset the API key. Additionally, when the creator logs into the application's user interface, they receive a warning message. If a service account has been created by another service account, no email notification will be sent for an upcoming API key expiration. In this scenario, with the API Key Policy enabled, you can use the [Get a list of all services within a workspace](/api/services/list) REST API call to retrieve your services, and find each respective key expiry in the `key_expires_at` field. ## Audit Logs All actions and notifications related to the API Key Policy are also logged within the Audit logs. In scenarios where a user is a member of multiple workspaces, it is important to note that the user who enforces the API key refresh will only be visible as the initiator within the workspace where they hold the ownership role. In other workspaces where the user is a member, the API key refresh action will still be recorded in the audit logs. However, the user who enforced the refresh will be obscured, and the action will be listed as a "refreshed API key" rather than an "enforced API key refresh." # Block Until Scan **Block Until Scan** is a security feature designed to enhance the integrity and security of software packages served by Cloudsmith, guaranteeing that all relevant security and compliance policy checks (licenses, vulnerabilities, package deny policies) are fully completed *before* a package is made available for download. --- By enabling Block Until Scan, all package requests originating from upstreams configured in "Cache and Proxy" mode are subjected to a mandatory scanning and policy evaluation process. This ensures that only packages meeting your organization's security policies can be accessed by users. Without Block Until Scan, packages could be served to clients *before* all necessary policy checks were completed (unless the package was already synced in the repository, and policy checks had been completed prior to that). This allowed for the initial download of packages that may subsequently fail policy checks. ## How to Use It 📘 The Block Until Scan feature is in **Early Access (EA)**. To enable Block Until Scan for your workspace, please [contact Cloudsmith Support](https://cloudsmith.com/company/contact-us). **Workspace-Wide Setting** Please note that Block Until Scan is a global setting that affects **all repositories** within a given workspace. Its activation will impact all applicable package flows. ### Requirements - **Active Security Policies**: At least one active classic policy with a _quarantine_ action enabled is required. Without such a policy, the system would have no criteria to evaluate incoming packages. - **Upstream Configuration ('Proxy and Caching' mode)**: the repository must have an upstream configured with ['Proxy and Caching'](/repositories/upstreams#supported-formats) mode enabled. Block Until Scan is explicitly applied to packages undergoing the caching and evaluation pipeline within Cloudsmith. ### Steps to Validate Block Until Scan To verify the functionality of Block Until Scan, follow these steps: #### 1. Configure an Upstream [Configure a repository](/repositories/upstreams) with an upstream in 'Proxy and Caching' mode. For example, enable a Python upstream with PyPI. 📘 Note that incoming packages from sources different to an upstream are also subjected to Block Until Scan. Downloads won't be allowed until all security checks have been completed. #### 2. Create a Vulnerability policy Browse to your Workspace Settings and [create a new vulnerability policy](/policy-management/vulnerability-policy). Define your policy with: - A name. - A [package Search Query](/artifact-management/search-filter-sort-packages) to scope the filter to `format:python AND requests`. We're targeting only packages named `requests` from Python, following the example in the next step. Adjust the policy so that it applies only to your desired scope if you use a different package for this example. - A severity threshold set to `High`. - A Quarantine action enabled. Then, click in **+ Create Policy**. #### 3. Pull a new package from your repository Execute a command to pull a new package. For example, `requests==0.2.2`. This package contains a known vulnerability with a high severity CVE: `2018-18074`. If this package already exists in your repo, find a different one that hasn't been synced and cached before: ```bash pip download requests==1.2.0 --index-url https://dl.cloudsmith.io/public/demo-docs/awesome-repo/python/simple/ ``` 📘 **Performance Impact** When a package is not yet cached within Cloudsmith, every package and its dependencies must undergo a series of synchronous operations before being served: * **Parsing**: The package metadata must be parsed. * **Scanning**: The package is scanned for malware, vulnerabilities, and license compliance. * **Policy Evaluation**: The package is evaluated against all active policies. #### 4. Observe Behavior When Block Until Scan is enabled, the initial download request for an uncached package will be temporarily blocked until the policy evaluation has completed: - **Successful Scan**: If the package passes all policy checks, the download will proceed, and the package will be served normally. - **Policy Matched**: If the package is matched against a policy, the package is processed exactly as the matched policy specifies: - If the policy's action is **quarantine**, the package is quarantined and any download attempt returns **403** Forbidden. - If the policy's action is **tag, allow**, or any other non-blocking action, the package is served normally (with the tag or other metadata applied). If the package fails a policy check (e.g., due to a vulnerability or license issue), or if the scanning process encounters an unrecoverable error, the download will be blocked with a `403 Forbidden` error code. **First download without Block Until Scan** Without Block Until Scan, the same package would be delivered to the user immediately, before the full evaluation phase of the synchronization process had taken place. Cloudsmith would then asynchronously cache and synchronize the package after the first download. At this point, the system would apply the policy to the package and could then quarantine it. While future downloads of the package would then be blocked, this asynchronous processing means the initial downloads of the package could not be prevented if required. ## Summary Block Until Scan helps your organization ensure that all software packages and their dependencies downloaded through Cloudsmith meet your security and compliance rules. While this scanning happens before you can download packages, the advantages are significant: - **Enforce Security Policies across all your organization**: It blocks any package downloads until security checks has been completed. Only approved software components are used in your workloads, guaranteeing compliance with organizational standards and practices. - **Reduced Risk**: checking packages before running them reduces your exposure to non-compliant software. In short, Block Until Scan is a crucial tool for securing and reducing risk in your software supply chain. # Deny Policy Package deny policy rules let you control which packages can be downloaded from your organization's repositories. Deny Policies allow users to specify criteria, in the form of a `package_query_string`, to identify packages that should be blocked from download. By defining these rules, organizations can enforce stricter security measures and maintain tighter control over their software artifacts. ## Supported Package Formats This feature applies to all formats. **Docker SBOM** Please, note that deny policies don't apply to components inside of a Docker image. While deny policies can be applied to Docker images by their **package name** (pull string) or **image digest**, they won't apply to any internal components. For example, a deny policy with the package query: `format:python`, won't block Docker images containing Python artifacts within the Software Bill of Materials (SBOM). ## Scope Deny Policies are applied at the organization level. ## View Deny Policies To view the license policies in your organization, go to your organization's “Settings”->"Manage policies"->"Package Deny Policies" page. [Image: Package Deny Policies] ## Create Deny Policies To create a new Deny Policy, go to your organization's "Settings" page and select "Package Deny Policies" from the left side menu. Then click the "Create new policy" button. You are then presented with the “Create Package Deny Policy” form: The following fields and options are available: ||| |:---|:---| |**Policy Name**|A display name for the Deny Policy. This name will be visible to users who view a package that violates this policy, as a reason for the violation.| |**Description**|A description of the Deny Policy. This description will be visible to users who view a package that violates this policy, as a reason for the violation.| |**Package Query**|A search query that you can use to target this policy at a specific repository, package format or package name. See [Searching / Filtering](/artifact-management/search-filter-sort-packages) for more details on the available fields you can filter on.The query string box (limited to 256 chars in length) to specify a search-query string to match packages against. Matched packages will then be blocked across the org.TIP: To ensure your search query will block the packages you want to block from download, test your search query first, using the "Search all packages..." box at the top of the Repositories page. Make sure the results of your query are the packages that you intend to block.| |**Enabled**|You can disable a rule temporarily, without deleting the rule.| ## Propagating a Newly Created Deny Policy When a new Deny Policy is created, Cloudsmith runs the policy against all packages in all repositories in your organization. This can take some time, depending on how many artifacts are stored in your repositories. You can check the status of a newly created Deny Policy via our [Get a package deny policy](/api/deny-policy/list) REST API call, which will return a `status` of "Pending" while the new policy is being propagated; the `status` will change to "Complete" when the policy is fully implemented across your organization. (You'll need to get the slug_perm identifier for your newly created Deny Policy first - use the [Get a list of all package deny policies](/api/deny-policy/list) REST API call for this.) Even after a new Deny Policy is fully propagated, a user who has recently downloaded a package may still be able to download it again for a short period of time, due to CDN caching. You can still confirm that your Deny Policy has been successfully applied to a package by confirming the policy violation identifier appears on that package (see below). ## What Gets Blocked Download requests are blocked for a specific package (or version of a package, if your search string specifies one). This includes the first download of the package, even if the package is proxied through from an upstream. ## Policy Violation Identifiers When local packages are blocked by a deny rule, it's available in the application as a red-flag, and the download link on the right-hand-side is faded-out and will not click. {/* When you click into the package itself, you'll be able to see which deny rule is blocking the package. */} You can use the `deny_policy_violated:true` search facet to see all packages that have been denied by Deny Policies. ## Audit Logs There are also audit logs for whenever users configure (create, update or delete) Deny Policies: ## Unblock a package that violates a policy To remove a policy violation, you have several options (depending on your user permissions): - Delete the Deny Policy. - Upgrade the package to a version outside of the Deny Policy. - Edit the Deny Policy to allow this package using the query string language syntax. - Temporarily disable the Deny Policy. # Geo/IP Rules If you need to add physical location security to your package management, then Geo/IP Restriction is what you need. You can restrict or grant access to your packages based on geographical location, IP ranges or specific IP addresses. You could, for example, use these restrictions to only allow access from your organization VPN IP address(es) or from specific organization locations/sites. ## Enabling and Configuring Geo / IP Restrictions These restrictions only apply to downloads-based traffic and do not apply to web-based traffic. - The deny lists are applied before the allow lists and therefore take precedence. - Geo-based restrictions are also applied before IP-based restrictions. - The first rule matching the incoming traffic is applied. To enable IP restrictions for uploads please see [API-Key](/accounts-and-teams/api-key) To enable Geo / IP restrictions, select "Geo/IP Rules" from the left menu on a repository detail page, then click the green "Enable Geo /IP" button on the right side. [Image: Geo / IP Restrictions configuration page] ## Geo-based Restrictions Use the "Deny" or "Allow" fields under "Geo-based Restrictions" to specify the geographic regions/countries that you wish to block or allow respectively. When you start to type the name of a region or country in these fields you will be offered a matching selection from the drop-down list. You can add multiple regions/countries to block or allow: If the region/country can't be resolved from an IP address (e.g for an internal or private IP), then setting this will have no effect. *** ## IP-based Restrictions Use the "Deny" or "Allow" fields under "IP-based restrictions" to specify the IP addresses / CIDRs that you wish to block or allow. You can enter multiple IP addresses / CIDRs in each field: [Image: IP-based restrictions - IP address / CIDR entry] # Policy Management By leveraging Cloudsmith's Policy Management, organizations can streamline their security operations, maintain compliance with software licensing requirements, and proactively mitigate potential threats, thereby safeguarding their software supply chain and maintaining the integrity of their software artifacts. Cloudsmith's Policy Management capabilities cover authentication policies, license policies, vulnerability policies, and deny list policies. Cloudsmith's Policy Management feature offers organizations a powerful tool for ensuring the security, compliance, and efficiency of their software artifact management processes. # License Compliance License Compliance provides you with an overall view of license policy violations for your Workspace or Repository as a whole. [Image: License Compliance Overview] If you're interested in restricting the use of packages with certain licenses, please review our [License Policy documentation](/policy-management/license-policy) to learn how to use policies to automatically quarantine packages and ensure organizational compliance. {/* The License Compliance page shows a breakdown of the license associated with each package stored in the repository. It provides basic statistics on overall license usage and coverage for the repository. **Total Packages Chart** The total packages chart shows the number of packages of each format in the repository **Licensed Packages Chart** The licensed packages chart shows the number of each license type in the repository **Licensed / Unlicensed Chart** The licensed / unlicensed chart shows the total number of licensed versus unlicensed packages in the repository */} **Dependencies** Cloudsmith provides License data for the packages it knows about or has access to. A package might have a dependency on another package with a different license which is unknown and therefore not reported. {/* ## Edit a License The license defined within a package's metadata is automatically matched as accurately as possible to the SPDX License List. The SPDX License List is a list of commonly found licenses and exceptions used in free and open source and other collaborative software or documentation. The purpose of the SPDX License List is to enable easy and efficient identification of such licenses and exceptions in an SPDX document, in source files or elsewhere. The SPDX License List includes a standardized short identifier, full name, vetted license text including matching guidelines markup as appropriate, and a canonical permanent URL for each license and exception. When a license is matched automatically, a description is provided of the license as defined within the package's metadata, a confidence percentage of how accurate the match is, and the new license that has been applied. You will see a description like the following on the edit page for any license that has been automatically applied: _The Apache 2.0 license provided within this package's metadata is a 97% match to a Apache License 2.0 License SPDX license and was automatically added to this package_ Anytime the match is not accurate to a high percentage (less than 75%), the parsed license is greater than 60 characters, or the license is not supplied, we leave the license empty for you to decide how you want to resolve it. Click the blue "Edit" button on the package that you wish to change / edit the license for. You are then presented with the Edit License form: [Image: Edit License form] You can then select a license from the drop-down menu. In addition, you can enter a URL for the license, and any notes that you require. You can also add a license override, where you can mark a package as "ignored" or "purchased". Click the green "Edit" button to apply your changes and return to the license overview page. Please Note: Currently editing the License changes the Cloudsmith held metadata. It does not write the new License to the package itself. */} ## Disclaimer Cloudsmith will not be held accountable for any package or dependency used that has an undesirable license or affects your IP rights in any way. # License Policy License Policies allow you to define which package licenses are allowed within your organization. You can specify one or more licenses to exclude and an action to perform when a specified license is detected. This is useful when you need to flag or optionally automatically quarantine a package based on the associated license. ## Supported Package formats Cloudsmith will automatically match licenses for the following package formats: |Format|
    License Support
    | |:---|:---| |[Alpine](/formats/alpine-repository)| ✅ | |[Cargo](/formats/cargo-registry)| ✅ | |[Cocoapods](/formats/cocoapods-repository)| ✅ | |[Composer](/formats/composer-repository)| ✅ | |[Conan](/formats/conan-repository)| ✅ | |[Conda](/formats/conda-repository)| ✅ | |[CRAN](/formats/cran-repository)| ✅ | |[Dart](/formats/dart-repository)| ✅ | |[Debian](/formats/debian-repository)| ❌ | |[Docker](/formats/docker-registry)| ❌ | |[Go](/formats/go-registry)| ✅ | |[Helm](/formats/helm-chart-repository)| ✅ | |[Hex](/formats/hex-repository)| ✅ | |[LuaRocks](/formats/luarocks-repository)| ✅ | |[Maven](/formats/maven-repository)| ✅ | |[npm](/formats/npm-registry)| ✅ | |[NuGet](/formats/nuget-feed)| ✅ | |[Python](/formats/python-repository)| ✅ | |[RPM](/formats/redhat-repository)| ✅ | |[Ruby](/formats/ruby-repository)| ✅ | |[Terraform](/formats/terraform-modules-repository)| ❌ | |[Vagrant](/formats/vagrant-repository)| ❌ | ## View License Policies To manage the license policies in your workspace, go to your workspace's Settings page and select Manage Policies >> License Policies from the left-hand menu. Deactivating a policy is not supported at this time, resulting in any policy created being active until deleted. ## Create License Policy To add a new license to the deny list, click “Create License Policy”. [Image: Create License Policy Button] You are then presented with the “Create a License Policy” form: Here you can define the following: ||| |:---|:---| |Name|A display name for the License Policy.| |Description|A description of the License Policy.| |Allow unknown or absent licenses|Set to “Yes” to ignore packages with missing or unknown licenses. Set to “No” to flag and quarantine packages with a missing or unknown license.Note: SPDX License List is used as the license source. If a package has a license that is not listed in the [SPDX License List](https://spdx.org/licenses/), this will be treated as an unknown license.| |Quarantine on violation|If set to "Yes", any package that violates this license policy is flagged and automatically quarantined. If set to "No", packages that violate this license policy are flagged, but not automatically quarantined.| |Select licenses to deny|Select the licenses to be denied as part of this policy. The licenses listed in this table are sourced from the [SPDX License List](https://spdx.org/licenses/).| {/* [block:parameters] { "data": { "h-0": "", "h-1": "", "0-0": "Name", "0-1": "A display name for the License Policy.", "1-0": "Description", "1-1": "A description of the License Policy.", "2-0": "Allow unknown or absent licenses", "2-1": "Set to “Yes” to ignore packages with missing or unknown licenses. Set to “No” to flag and quarantine packages with a missing or unknown license. \n \nNote: SPDX License List is used as the license source. If a package has a license that is not listed in the [SPDX License List](https://spdx.org/licenses/), this will be treated as an unknown license.", "3-0": "Quarantine on violation", "3-1": "If set to \"Yes\", any package that violates this license policy is flagged and automatically quarantined. \n \nIf set to \"No\", packages that violate this license policy are flagged, but not automatically quarantined.", "4-0": "Select licenses to deny", "4-1": "Select the licenses to be denied as part of this policy. The licenses listed in this table are sourced from the [SPDX License List](https://spdx.org/licenses/)." }, "cols": 2, "rows": 5, "align": [ "left", "left" ] } [/block] */} Once saved, the policy is enabled across your workspace and the policy compliance check will be performed automatically during the package synchronisation process. This process occurs when a package is uploaded, moved, copied, or cached (such as in upstream caching). You can also manually trigger a synchronisation of a package using the resync functionality. See [Package Resynchronization](/artifact-management/resync-a-package) for instructions on how to do this. ## Policy Violations License policy violations can be viewed on the workspace and repository compliance dashboards. By selecting License Policy Violations from the overview, the table will filter to show all license policy violations across the workspace or repository. [Image: violated policy warning] You can also search for policy violations using `policy_violated:true`. [Image: violated policy search] You can view this violation list at any time using our package search [(link)](/artifact-management/search-filter-sort-packages#searching-packages-via-the-website-ui) feature and setting the filter to `policy_violated:true` for all policy violations, or `license_policy_violated:true` to only return license policy violations. ## Policy Violation Identifiers Within a repository, packages with policy violations are identified with the Policy Violation and Vulnerabilities icons. [Image: Policy Violation Indicators] |||| |:---|:---|:---| ||Policy Violation|This package is in violation of policy. Click on the package to view more details.| ||Quarantined|This package has been quarantined, downloads will be blocked until the package is released from quarantine.Reason given: the package was quarantined as a result of a policy violation| Clicking on the package name displays the Package overview, which provides details on the policy violations for the package. [Image: package page warning] By selecting License Policy Violation from the summary, you can view a list of all license policies the package violates and view and modify the policies from there. ## Package Logs Logs of policy violations and quarantining actions are also displayed within the Package Logs page. Hover over the “policy violation” text to display additional details on the policy and license violation. ## Restore a package that violates a policy It's important to note that packages cannot be restored from quarantine if they are still in violation of an existing policy which has **Quarantine on violation** set to "Yes". To remove a package from quarantine, you have four options: - Change the package license to an allowed license via the [license compliance](/policy-management/license-compliance#edit-a-license) section of the package's repository. - Apply a manual license override of "ignored" or "purchased" via the [license compliance](/policy-management/license-compliance#edit-a-license) section of the package's repository. - Edit the license policy to remove the license from the deny list. - Edit the license policy to set **Quarantine on violation** to "No". If you change the package license, the package will be resynchronized and automatically undergo the policy check again. If the new license is not on the deny list, the package will no longer be marked as a policy violation and can be manually removed from quarantine. If you update the policy to remove the license from the deny list or set **Quarantine on violation** set to "No", the package can be manually removed from quarantine. See [Package Quarantine](/artifact-management/package-quarantine#release-from-quarantine) for instructions on restoring a package from quarantine. # Vulnerability Policy Vulnerability Policies enable you to set the acceptable severity threshold for vulnerabilities within your organization. You can create several policies at different levels and determine the actions to take when a package returns a vulnerability level that meets or exceeds a specified threshold. This feature is useful when you want to either flag or automatically quarantine a package based on the level of vulnerability detected. ## Supported Package formats Vulnerability scanning and vulnerability policies are supported for the following package formats: |Format|Vulnerability Scanning Support| |:---|:---| |[Alpine](/formats/alpine-repository)|❌| |[Cargo](/formats/cargo-registry)|❌| |[Conan](/formats/conan-repository)|✅| |[Cocoapods](/formats/cocoapods-repository)|❌| |[Composer](/formats/composer-repository)|✅| |[Conan](/formats/conan-repository)|❌| |[Conda](/formats/conda-repository)|❌| |[CRAN](/formats/cran-repository)|❌| |[Dart](/formats/dart-repository)|❌| |[Debian](/formats/debian-repository)|❌| |[Docker](/formats/docker-registry)|✅| |[Go](/formats/go-registry)|✅| |[Helm](/formats/helm-chart-repository)|❌| |[Hex](/formats/hex-repository)|✅| |[LuaRocks](/formats/luarocks-repository)|❌| |[Maven](/formats/maven-repository)|✅| |[npm](/formats/npm-registry)|✅| |[NuGet](/formats/nuget-feed)|✅| |[Python](/formats/python-repository)|✅| |[RPM](/formats/redhat-repository)|❌| |[Ruby](/formats/ruby-repository)|✅| |[Swift](/formats/swift-registry)|✅| |[Terraform](/formats/terraform-modules-repository)|❌| |[Vagrant](/formats/vagrant-repository)|❌| ## View Vulnerability Policies To manage vulnerability policies in your organization, go to your organization's "Settings" page and select "Vulnerability Policy" from the left-hand menu. Deactivating a policy is not supported, so any policy created will remain active until deleted. [Image: View Vulnerability Policies] ## Create Vulnerability Policy To create a new vulnerability policy, click "Create Vulnerability Policy." [Image: Create Vulnerability Policy Button] You are then presented with the “Create Vulnerability Policy” form: [Image: Create Vulnerability Policy Form] Here you can define the following: |Field|Description| |:---|:---| |**Policy Name**|A display name for the Vulnerability Policy.| |**Description**|A description of the Vulnerability Policy.| |**Package Query**|A search query that you can use to target this policy at a specific repository, package format or package name. See [Searching / Filtering](/artifact-management/search-filter-sort-packages) for more details on the available fields you can filter on| |**Select Vulnerability Threshold**|The vulnerability threshold for the policy.Following a package security scan, if a vulnerability matching or exceeding this value is returned, the package will be in violation of the policy. Vulnerability thresholds are based on the following CVSS v3.1 severity ratings: - Critical (9.0 - 10.0) - High (7.0 - 8.9) - Medium (4.0 - 6.9) - Low (0.1 - 3.9)| |**Quarantine packages which violate policy**|If set to "Yes", any package that has a vulnerability equal to or higher than the threshold will be flagged and automatically quarantined.If set to "No", packages that violate the policy will only be flagged, but they will not be automatically quarantined.| |**Allow Unknown severity level**|Vulnerabilities may have no CVSS score or severity rating, in which case the severity level is categorized as "Unknown".This setting determines whether or not the Vulnerability Policy should be applied to packages which have vulnerabilities in that category.If set to "Yes", a package with vulnerabilities of an unknown severity level will not be considered to be in violation of the policy. If set to "No", a package with vulnerabilities of an unknown severity level will be considered to be in violation of the policy and the configured policy actions will be taken.| Once saved, the policy is enabled across your organization, and the vulnerability policy check is performed on a package every time it undergoes a security scan. Packages are automatically security scanned on upload and subsequent scans can be triggered manually via the Web UI, and the Cloudsmith API. See [\[Security Scanning\]](/policy-management/vulnerability-policy) for further details on our security scanning feature. ## Policy Violation Notification When a package violates a vulnerability policy a warning is displayed within the UI: [Image: Vulnerability Violation Warning] Clicking the link within this warning, displays a custom search page, displaying all packages with policy violations: [Image: Policy Violation List] You can view this violation list at any time using our package search [(link)](/artifact-management/search-filter-sort-packages#searching-packages-via-the-website-ui) feature and setting the filter to `policy_violated:true` for all policy violations, or `vulnerability_policy_violated:true` to only display vulnerability policy violations. ## Policy Violation Identifiers Within a repository, packages with policy violations are identified with the Policy Violation and Quarantined icons. [Image: Policy Violation Indicators] |||| |:---|:---|:---| ||Policy Violation|This package is in violation of a vulnerability policy. Click on the package to view more details.| ||Quarantined|This package has been quarantined, downloads will be blocked until the package is released from quarantine.Reason given: the package was quarantined as a result of a policy violation| ||Security Vulnerability Detected|The security scan has detected vulnerabilities within this package.| Clicking on the package name displays the Package view page, which provides details on the policy that has been violated. ## Package Logs Logs of policy violations and quarantining actions are also displayed within the Package Logs page: Hover over the “policy violation” text to display additional details on the policy and vulnerability violation. ## Restore a package that violates a policy To remove a policy violation, you have three options: - Upgrade the package to a version that has resolved the security vulnerability. - Edit the vulnerability policy to lower the threshold below the violated level. A [manual rescan](/supply-chain-security/vulnerability-scanning#additional-security-scans-via-the-cloudsmith-web-ui) of the package is required to remove the policy violation flag and the package must also be manually [released from quarantine](/artifact-management/package-quarantine#release-from-quarantine). - Use the restore functionality to remove the package from quarantine. Keep in mind, however, that subsequent vulnerability scans may return the package to quarantine. See [Release from quarantine](/artifact-management/package-quarantine#release-from-quarantine) for instructions on restoring a package from quarantine. # Create a Repository You can create a new repository in three ways: - [Via the Cloudsmith CLI](#via-the-cloudsmith-cli) - [Via the Cloudsmith web app](#via-the-cloudsmith-web-app) - [Via the Cloudsmith API](#via-the-cloudsmith-api) ## Via the Cloudsmith CLI In the following examples: |Identifier|Description| |:---|:---| |OWNER|A Cloudsmith organization / workspace / namespace account name| |REPO-NAME|A name for your repository. Maximum 50 characters.| |REPO-DESC|Description for your Repository| |REPO-TYPE|The type of repository - "Public" or "Private"| |REPO-IDENTIFIER|(Optional) A unique identifier for the repository, also referred to as a 'slug'. This will form part of the URI for your repository. It can only contain lowercase alphanumeric characters, hyphens, and underscores and must be a minimum of 2 characters.| If you do not specify a repository identifier, we will generate one for you. We will try to base this on the repository name where possible but may also append some characters to it. If you require a specific repository identifier, we recommend that you do specify one. Create a `REPO-CONFIG.json` file with the following: ```json { "name": "REPO-NAME", "description": "REPO-DESC", "repository_type_str": "REPO-TYPE", "slug": "REPO-IDENTIFIER" } ``` You can create then a repository via the [Cloudsmith CLI](/developer-tools/cli) using the following command: ``` cloudsmith repos create OWNER REPO-CONFIG.json ``` ### Example `example-repo-config.json` file: ```json { "name": "Example Repository", "description": "Example packages repository", "repository_type_str": "Private", "slug": "example-repo1" } ``` Create repository command: ```shell cloudsmith repos create demo example-repo-config.json ``` [Image: cloudsmith repos create command] ## Via the Cloudsmith web app You can create a repository by navigating to the Repositories overview and clicking the "New repository" button. That will take you to the "Create a new Repository" form: Here, you simply name your repository and click "Create Repository," and you will create a new private repository. **Multi-Format Repositories** All Cloudsmith repositories are multi-format. This means you can store artifacts of different formats in the same logical grouping. A Maven package can sit beside a Debian package, a Ruby Gem or a Python package. You can manage your Repository by navigating to your new repository and selecting Settings. From there, you can manage: - General settings - Access control - Retention rules - EULA enforcement - GEO / IP rules - Key management - Webhooks **Changing Repository Settings** The good thing about Cloudsmith is that you can change everything later. The name, slug, description, repository type can all be changed if necessary. Although once shared, if you change the settings or naming you may need to communicate this to all interested parties. ### Private Repositories All paid accounts come with unlimited Private repositories. Private repositories are private. If required, external access to any files in a private repository is controlled by our Entitlement system. See "[How To: Sharing a Private Package](/artifact-management/sharing-a-private-package)" for more details. ### Broadcasts You can create a branded public repository using our new distribution feature, Broadcasts. Enabling broadcasting during repository creation creates a public page where users can browse, search, and download repository packages without requiring authentication. See [Broadcasts](/software-distribution/broadcasts) for more information. ## Via the Cloudsmith API For details on how to create a repository via the Cloudsmith API, please see the "Create a Repository" section of the [API Reference](/api/repos/create). It is not possible to create Open Source repositories via the Cloudsmith API. # Custom Storage Regions Custom storage regions allow you to control where in the world your artifacts are geographically stored. For regulatory and compliance reasons you may wish to store your artifacts in a specific country or region. In addition, storing your artifacts closer to where your services and teams operate can also provide significant performance benefits (lower latency) in many cases. You can choose a storage region when creating a new repository or you can transfer an existing repository to a different storage region. ## Choosing a Custom Storage Region for a new Repository During the creation of a new package repository, you have the option to specify a custom storage region (other than the default region of Dublin, Ireland) for your new repository via the "Create Package Repository" form. The following storage regions are supported: - Dublin, Ireland - Frankfurt, Germany - London, United Kingdom - Montreal, Canada - Ohio, United States - Oregon, United States - Singapore - Sydney, Australia - Tokyo, Japan Selecting a storage region enables you to use one of our supported locations to store your packages at an edge location that is geographically closer to your region of operations. With some consideration regarding your intended usage, selecting your nearest storage region can significantly reduce latency. [Image: Create Package Repository Form] Use the "Storage Region" drop-down menu to select your custom storage region, and then click the green "Create" button to create a new repository in your chosen region. ## Transferring an existing Repository to a new storage region To transfer an existing repository to a new storage region, click the repository "Settings" button on the left-hand menu, scroll to the bottom and click the orange "Transfer" button. By default, your storage region will be based in Dublin, Ireland if this is the first time you are specifying a storage region. [Image: Transfer Repository Button] You will then be presented with the Transfer Repository Form: [Image: Transfer Repository Form] In order to prevent accidental transfers, you need to confirm the identifier / slug for the repository by typing it into the "Confirm Current Repository Slug/Identifier" field. Then choose your new storage region from the drop-down menu, complete the Captcha and click the orange "Transfer" button to start the transfer. Whilst the transfer is processing, synchronisation for any new uploads will be paused and will not complete until the transfer is complete. However, downloads will continue to work as normal throughout the transfer process. The time taken to transfer to a new region is directly related to the number of packages stored within a repository. # Repositories A Repository, or repo, is a storage location for software packages and containers. Cloudsmith provides polyglot repositories that can contain packages of all support types. You can create an unlimited number of repositories; providing you with the flexibility to suit any use-case. At Cloudsmith we use "repository" as a general term. We also refer to "registry" and "feed" in line with the format's specific nomenclature. ## Repository Controls Cloudsmith provides comprehensive controls to allow you to secure, monitor and control the packages inside each repository. Use the top menu items to access the controls available: [Image: Repository Settings] |Control|Description| |:---|:---| |[Packages](/formats)|A list of all uploaded packages and artifacts.| |[Package Groups](/artifact-management/package-groups)|A list of grouped packages by name (ignoring version/distro info)| |[Audit Logs](/logs-and-observability/audit-logs) | A log of repository events| |[Client Logs](/logs-and-observability/client-logs)| View access logs for this repository| |[Client Statistics](/logs-and-observability/usage)|See all the statistics gathered about your packages - downloads, bandwidth usage, etc.| |[License Compliance](/policy-management/license-compliance)|View and edit all licenses for packages in the repository| |[Security Scanning](/policy-management/vulnerability-policy)|Security Scanning for artifacts and Docker images| |[Signing Keys](/supply-chain-security/signing-keys)|GPG and RSA signing keys| |[Main Settings](/repositories/repository-settings)|Change the Name, Permissions, Visibility and if needed, delete the repository.| |[Access Controls](/repositories/repository-privileges#access-control)|Manage teams and user access to this repository| |[Custom Domains](/workspaces/custom-domains)|Setup domain configuration to support your named endpoint. [Contact us](https://cloudsmith.com/company/contact-us) for help.| |[Entitlement Tokens](/software-distribution/entitlement-tokens)|Manage entitlement tokens for external, read-only access to private repositories.| |[EULA Enforcement](/software-distribution/eula) | Manage End-User License Agreement enforcement| |[Geo/IP Rules](/policy-management/geoip-rules) | Manage download permissions by IP and/or Geographic area| |[Retention Rules](/artifact-management/retention-rules) | Manage the size of your repositories with retention settings| |[Upstream Proxying](/repositories/upstreams)|Configure upstream sources for this repository| |[Connected Repositories](/repositories/connected-repositories)|Link multiple repositories together for unified package resolution| |[Webhooks](/developer-tools/webhooks)|Create and configure webhooks to allow external tools/systems to see your Cloudsmith events| {/* |[Package Logs](/logs-and-observability/package-logs)|View logs of package events for this repository| */} {/* */} # Per-repository privileges There are three different repository settings defining user's access to artifacts and repository settings: - [Repository Privileges](#repository-privileges): Defines the minimum level of privilege required for a user to perform specific actions. This setting is override by _User/Self_ privileges when the user is the ownser of a package. - [Self Privileges](#self-privileges): Defines which actions users can always do with their own packages, no matter what are the privileges defined at other privilege levels (workspace, repository, team). - [Access control](#access-control): Defines the default level of privilege assigned to all workspace members (no collaborators) for this repository. Before we dig deeper into each of them, here we can find the different levels of access that you can assign to users, services and members of teams in a repository: | Privilege | Description | | :-------- | :----------------- | | **Admin** | Can manage entitlements, privileges, and settings, in addition to those permissions granted by Write and Read access. | | **Write** | Can upload packages and edit existing packages, in addition to those permissions granted by Read access. | | **Read** | Can view and download packages. | **Fine-tuning Privileges** Note that this privileges can be fine tuned per action (see [Repository privileges](#repository-privileges)). **Public repositories** Public repositories are open by definition, hence there's no level of privileges or belonging to the workspace required to have read access to its assets. ## Repository privileges With this options, you can define the **minimun level of privileges granted** required to perform certain actions within a repository: | Action | Minimum Privilege | | :--- | :--- | | Copy packages | Admin, Write, Read | | Move packages | Admin, Write, Read | | Delete packages | Admin, Write | | Resync packages | Admin, Write | | Scan packages | Admin, Write, Read | | Replace packages | Admin, Write | | View statistics packages| Admin, Write, Read | | Manage entitlements | Admin, Write, Read | | See/Use entitlements | Admin, Write, Read | ## Self privileges With this option, we can define which of the following actions can always be performed by users with their own packages, no matter which other permissions are stablised per workspace, repository, team, or user: - Scan - Copy - Move - Delete - Resync Additionally, we can enable/disable user entitlements for a private repository. This setting allows users to use and manage their own entitlement token for the repository. ## Access Control Access Controls allows you to configure the default level of privilege assigned to all workspace members (no collaborators) for this repository: - Admin - Write - Read - None [Image: Access Controls] ### Privileges for Specific Users, Services and Teams Additionally, you can increase the level of repository privileges for specific: - Users - Services - Teams ## Effective privilege The effective privilege for an account (User or Service) is the greatest privilege granted to them via: - Assignment to them directly, via [Privileges for Specific Users/Services](#privileges-for-specific-users,-services-and-teams). - Derived from their team membership via [Privileges for Specific Teams](#privileges-for-specific-users,-services-and-teams). - By default on the repository, via [Default Privileges](#access-control). - By default on the workspace, via the org-wide "Default Object Privileges" (see [Workspace Settings](/workspaces/privileges#default-object-privileges). When granting a Team or User access to a repository, you can select from the following privilege levels. ## External User Access To allow external (non-Cloudsmith) users access to a private repository, please see our [Entitlement Tokens](/software-distribution/entitlement-tokens) documentation. # Repository Settings ## Repository Name and Description You can change the name or description of a repository at any time. Please be aware that the name is just a descriptive name, it is not the repository slug / identifier. Changing the name of the repository will not result in breaking any configurations used by client programs or other users. [Image: Repository Name and Description] To change the repository slug / identifier, see the section on [Potentially Unsafe Actions](/repositories/repository-settings#potentially-unsafe-actions). When changing or modifying any repository settings please be sure to click the blue "Update Settings" button to apply your changes. ## Repository Privileges [Image: Repository Privileges] Repository privileges allow you to configure what permissions are required to perform actions on the repository. You assign a permission level to a Team or User when you grant them access to the repository (see [Access Controls](/repositories/repository-privileges#access-control) for information on granting access to a repository), and then you can set precisely what actions that permission level can perform using the repository privileges For Example: If you wanted a Team or User to only be able to set or use Entitlement Tokens, then you could set the permission level required for **Set/Use Entitlements** to **Read**, and set all other action to **Write** or above. Then in the repository [Access Controls](/repositories/repository-privileges#access-control), you would set the Team or User to only have **Read** permissions on the repository. As a result, they would be able to set or use Entitlement Tokens, but would not have permission for any other repository actions. *** ## User / Self Privileges [Image: Repository User / Self Privileges] **User Entitlements Enabled** If checked, users can use and manage their own user-specific entitlement token for the repository (if the repository is a private repository). If not checked, user-specific entitlement tokens are disabled for all users. **Copy / Move / Delete / Resync** If checked, users can copy/move/delete and resync packages that they have uploaded. This assumes that they still have **write** privileges for the repository. These override the repository privilege level setting for the repository. ## Miscellaneous Settings [Image: Miscellaneous Repository Settings] **Use/Configure NoArch Packages** Enables noarch packages (if supported by the package type) in installations/configurations. A noarch package is one that is not tied to a specific system architecture (like i686) **Use/Configure Source Packages** If checked, source packages (if supported) are enabled in installations/configurations. A source package is one that contains source code rather than built binaries. **Index Package File Contents** Enables the indexing of files within a package. This will increase the synchronization time for a package but it is recommended to keep this enabled unless synchronization time is significantly impacted. **Replace Packages By Default** Enables uploaded packages to overwrite/replace any others with the same attributes (e.g version) by default. This only applies if the user has the required privilege for republishing AND has the required privilege to delete packages that they do not own. **Contextual Authentication Realm** If checked, missing credentials for this repository where basic authentication is required shall present an enriched value in the 'WWW-Authenticate' header containing the namespace and repository. This can be useful for tools such as SBT, where the authentication realm is used to distinguish and disambiguate credentials. **Strict Npm Validation** Enables strict validation of npm packages to ensure that they match the specification. If you have packages that are old or slightly off-spec, you can disable this, but we can't guarantee that the packages will work with the npm CLI or other tooling. **Serve index for raw packages** Enables the generation of HTML and JSON indexes that list all available raw packages in the repository **Display generated GPG signatures for the raw package index** If checked, the HTML and JSON indexes will display raw package GPG signatures alongside the index packages. **Docker Auth Refresh Enabled** Enables the issuing of refresh tokens in addition to access tokens for Docker authentication. This allows unlimited extension of the lifetime of access tokens. **Use Debian Labels** Enables a 'Label' field in Debian-based repositories. It will contain a string that identifies the entitlement token used to authenticate the repository in the form of `source=t-`; or `source=none` if no token was used. You can use this to help with pinning. **Scan for Vulnerabilities** If checked, vulnerability scanning will be enabled for all supported packages within this repository. **Use crates.io as default Cargo upstream** Enables the assumption that Cargo crates which do not set an explicit value for 'registry' will be available from crates.io. If not enabled, dependencies with unspecified 'registry' values will be assumed to be available in the registry being uploaded to. Disable this if you want to ensure that dependencies are only ever installed from Cloudsmith unless explicitly specified as belonging to another registry. **Apply Latest Tag for Pre-Release Versions** If checked, packages pushed with a pre-release component on that version will be marked with the 'latest' tag. Please note, if unchecked, a repository containing ONLY pre-release versions will have no version marked latest which may cause incompatibility with native tools. ## Custom GPG Signing Key You can specify a custom GPG key for signing repository metadata and packages. We will automatically derive the public key and fingerprint for any custom GPG key specified. If your custom GPG key is encrypted, please also provide the passphrase for it. ## Custom RSA Signing Key You can specify a custom RSA key for signing repository metadata and packages. We will automatically derive the public key and fingerprint for any custom RSA key specified. The private key must be in PKCS8 format. If your custom RSA key is encrypted, please also provide the passphrase for it. Adding a custom GPG or RSA key will take effect immediately, but will not affect any existing packages that were signed with the previous key. Packages signed with the previous key will still be available and the previous key can still be fetched. ## Potentially Unsafe Actions [Image: Potentially Unsafe Actions] ## Dangerous Actions **Dangerous Actions** The following actions are marked as dangerous. The following actions are marked as dangerous as initiating can affect a user's or customer's ability to access a repository. For example, a Repository Transfer will require the repository users to update their configuration and may require coordination with your user base. Alternatively, a Repository Delete is irreversible will prevent exiting users from accessing it. In both situations, it's worth considering your users' impact and what communication is required before using these actions. [Image: Dangerous Actions] ### Transferring a repository A repository transfer is initiated from within the repository settings of the repository you wish to transfer. The action to perform the transfer is listed under "Actions (Danger)". When you initiate a repository transfer, the owner of the destination account/namespace will be notified that there is a transfer pending approval, and they then need to confirm that the transfer is accepted on the receiving repository. Once the destination account/namespace has accepted the transfer, the repository will then be transferred to the destination account/namespace immediately. The repository storage size will immediately count towards the storage limits for the destination account/namespace. Any Entitlement Tokens associated with the repository will also be transferred. Collaborators for the repository will be reset and any existing clients of the repository will need to update their URIs to point to the new location. # Bug Bounty Program ## Overview Cloudsmith aims to keep its Service safe, and data security is of utmost priority. Suppose you are a security researcher and have discovered a security vulnerability in the Service. In that case, we appreciate your help in disclosing it to us privately and letting us fix it before publishing technical details. Cloudsmith will engage with security researchers when vulnerabilities are reported to us, as described here. We will validate, respond to, and fix vulnerabilities to support our commitment to security and privacy. We won’t take legal action against, suspend, or terminate access to the Service of those who responsibly discover and report security vulnerabilities. Cloudsmith reserves all of its legal rights in the event of any noncompliance. We are pleased to offer thanks and/or a bounty reward for vulnerability information that helps us protect our customers thanks to the security researchers who choose to participate in our bug bounty program. This will range from the public thanks to a variable monetary amount or loot (depending on our budget and the impact). ## Rules & Process If you identify a verified security vulnerability in compliance with this Security Disclosure Policy (i.e. Bug Bounty Program), you can submit this to our bug bounty program with the following process: 1. You should determine if an exploit is _viable_ (i.e., practically possible). 2. You should also check to ensure it's not explicitly out-of-scope (see below). 3. Please feel free to report the exploit to us using the relevant information (see below). 4. We will let you know we received the disclosure and triage impact and then schedule a review. 5. The urgency for review will be determined based on the type and impact of the exploit. 6. It may take up to one month to triage, depending on how busy the team is. 7. Timeframes are typically hours for critical/high, days for medium and (up to) weeks for low. 8. Please do NOT send continual "any update?" messages; we get reminded automatically. 9. Any disrespectful communications to our staff including but not limited to frequent checkups on the status of a report will make the report ineligible for any award. 10. We will let you know if the exploit is viable and whether we accept it as a bug bounty (or not). 11. If viable and fixed, we will publicly thank you on our [Hall of Fame](/resources/exploits-hall-of-fame). 12. You may also be issued with an award in addition to being placed in the Hall of Fame. 1. N.B. if the same exploit is reported by multiple reporters prior to remediation, then the first valid report that documents the exploit will receive the credit. **❗️ Unauthorized Cross-Account Access** Please note that in _absolutely no event_ are you permitted to access, download, or modify data residing in any other Account or one that is not registered to you unless permission is _expressly provided_ by the Account Owner. **🚧 Monetary Award Tax** If a monetary award is provided as a bounty, you are responsible for paying any taxes associated with the reward. We are obliged by law to report any such awards to the tax authorities, so please make sure that you handle this appropriately. ## In Scope The bug bounty scope applies to: - The [Cloudsmith Classic App](https://cloudsmith.io) (the "old" app); - The [Cloudsmith Web App](https://app.cloudsmith.com) (the "new" app); - The [Cloudsmith Core API](/api); - The [Cloudsmith Corporate Site](https://cloudsmith.com). The vulnerability classifications that are considered in-scope are: - Access Control Bypass; - API Misuse Issues; - Authentication (Broken or Bypassed); - Business Logic Issues; - Cross-Site Scripting (XSS); - Cross-Site Request Forgery (CSRF); - Complexity Bomb; - Decompression Bomb; - Directory Traversal; - Improper TLS protection; - Open URL Redirection; - Privilege Escalation; - Provisioning Errors; - Remote Code Execution (RCE); - Sensitive/Private Data Leaks; - Session Fixation; - Session Management (Broken or Bypassed); - Subdomain/Domain Takeover; - SQL Injection. If a vulnerability isn't explicitly listed here as either in-scope or explicitly mentioned in the next section as out-of-scope, then it might be applicable. Still, we reserve the right to determine this in communication with the security researcher. ## Out Of Scope Application bugs that are **explicitly NOT considered to be in-scope**: - Any standard (i.e., non-security) application or API bugs (e.g., display issues, ordering issues, search issues, input/output issues, etc.), unless they involve privilege escalation or cross-account access. - Any UI-based injection bugs in the [Classic App](https://cloudsmith.io) (e.g., XSS or CORS issues); however, these are now applicable within the new [Web App](https://app.cloudsmith.com), which was released for preview in November 2024. Actions/areas that are **explicitly NOT considered to be in-scope**: - Executing or attempting to execute any _Denial of Service_ (DoS) attack; or - Knowingly posting, transmitting, uploading, linking to, sending, or storing any Malicious Software; or - Attempting to social engineer support or other staff; or - Testing in a manner that would result in the sending of unsolicited or unauthorized junk mail, spam, pyramid schemes, or other forms of duplicative or unsolicited messages; or - Testing in a manner that would degrade the operation of the Service; or - Testing or otherwise accessing or using the Service from any jurisdiction that is Prohibited Jurisdiction; or - Testing third-party applications, websites, or services that integrate with or link to the Service. Vulnerabilities that are **explicitly NOT considered to be in-scope**: - Clickjacking, CSRF, and content spoofing issues without demonstrable security impact. - Publicly known vulnerable libraries without a working Proof of Concept. - Self-XSS or having a user paste JavaScript into the browser console. - Issues relating to non-Cloudsmith products. - Open ports scanning, banner grabbing, and software version disclosure issues. - Issues that affect only outdated user agents or unsupported platforms. Non-qualifying best practices that are **explicitly NOT considered to be in-scope**: - Missing cookie flags on non-authentication cookies. - Missing best practices in DNS configuration (e.g., DKIM/DMARC/SPF/TXT). - Missing best practices in SSL/TLS configuration. - Missing best practices in Content Security Policy (CSP) or lack of other security-related headers. - Leakage of sensitive tokens (e.g., reset password token) to trusted third parties on a secure connection (HTTPS). Non-qualifying business practices that are **explicitly NOT considered to be in-scope**: - Institution access code enumeration or demonstrating access codes leaked in internet forums. - Credential re-usage from public dumps. - UUID enumeration of any kind. - Ability to determine if a username or email has a Cloudsmith account. - Signing up with multiple accounts to abuse referral code usage. - Password length, complexity, and re-use requirements. - Email verification feature. Websites/services that are **explicitly NOT considered to be in-scope**: - Any "third-party" website that we use to host a service but do not control uses _their_ bug bounty program, e.g.: - [Status Website (https://status.cloudsmith.com)](https://status.cloudsmith.com) - this is hosted by [Incident.io](https://incident.io). - [Support Website (https://support.cloudsmith.com)](https://support.cloudsmith.com) - this is hosted by [Zendesk](https://www.zendesk.com). - [Amazon Web Services (https://aws.amazon.com)](https://aws.amazon.com/) - this is our infrastructure provider. - [GRC / Trust Centre Website (https://trust.cloudsmith.com)](https://trust.cloudsmith.com/) - this is hosted by [Vanta](https://www.vanta.com/). - Cloudsmith non-production systems; only production systems are in scope for the bug bounty program. - Any third-party widgets, such as: - Chat widget (powered by [Intercom](https://intercom.com), [ZenDesk](https://zendesk.com) or [Qualified](https://qualified.com)). - Status widget (powered by [Incident.io](https://incident.io)). - Any websites under the following domains: - \*.googleapis.com - \*.googletagmanager.com - \*.google-analytics.com - \*.doubleclick.net - \*.hubspot.com - \*.bing.com - \*.simpleanalytics.com - \*.getbeamer.com - \*.zendesk.com ## Severity When we issue bug bounties, we assign them a severity level. This represents the level at which we assess the impact of the disclosed security issue to be, and it influences the bug bounty award. Please remember, any decisions for assessed severity and impact are final, and we reserve the right to determine them. ### Severity Level: CRITICAL Vulnerabilities that score in the critical range usually have most of the following characteristics: - Exploitation of the vulnerability likely results in root-level compromise of servers or infrastructure devices. - Exploitation is usually straightforward, in the sense that the attacker does not need any special authentication credentials or knowledge about individual victims, and does not need to persuade a target user, for example via social engineering, into performing any special functions. ### Severity Level: HIGH Vulnerabilities that score in the high range usually have some of the following characteristics: - The vulnerability is difficult to exploit. - Exploitation could result in elevated privileges. - Exploitation could result in significant data loss or downtime. ### Severity Level: MEDIUM Vulnerabilities that score in the medium range usually have some of the following characteristics: - Vulnerabilities that require the attacker to manipulate individual victims via social engineering tactics. - Denial of service vulnerabilities that are difficult to set up. - Vulnerabilities where exploitation provides only very limited access. - Vulnerabilities that require user privileges for successful exploitation. ### Severity Level: LOW - Vulnerabilities in the low range typically have little impact on an organization's business. ## Timeliness We endeavor to try and fix security bugs within the following timelines: | Severity | Typical Mean Time To Fix (MTTF) | | :------- | :-------------------------------- | | CRITICAL | Within 2 weeks of being verified | | HIGH | Within 4 weeks of being verified | | MEDIUM | Within 6 weeks of being verified | | LOW | Within 26 weeks of being verified | ## Reporting To start the reporting process, please share the details of any suspected vulnerabilities with the Cloudsmith Security Team by sending us the details to [bug-bounties+nospam@cloudsmith.io](mailto:bug-bounties+nospam@cloudsmith.io) (remove the `+nospam` to reach us). Please do not publicly disclose these details outside this process without explicit permission. In reporting any suspected vulnerabilities, please include the following information: - **Vulnerable URL**: The URL where the vulnerability occurs; - **Vulnerable Parameter**: If applicable, the parameter where the vulnerability occurs; - **Vulnerability Type**: The type of vulnerability; - **Steps to Reproduce**: Step-by-step information on how to reproduce the issue; - **Screenshots**: A demonstration of the attack to aid description; and - **Attack Scenario**: An example attack scenario may help demonstrate the risk and resolve your issue faster. ## Reserved Rights Cloudsmith reserves all rights to decide on scope, impact, and reward. If another researcher has previously reported an exploit, only the first disclosure will be considered. However, you'll always have our appreciation, and we'll let you know if this happens. This often occurs due to the number of submissions that we get. If we cannot replicate the issue, regardless of the timeframe between the report and our triage, we cannot award any bounties. We'll try our best to replicate this, but please know that we don't receive reports, so we fix them without acknowledgment. Finally, regardless of the reason, we do not tolerate abuse or threats to our staff or company. We reserve the right to terminate and block all communication with such people. Everyone should acknowledge and respect that there is a Human being on both sides of every communication. ## Acknowledgement Thank you for helping us make the package management world a bit safer. # Resources A collection of useful links and resources, including information about our [Bug Bounty Program](/resources/bug-bounty-program). | Resource Name | URL | Description | | :--- | :--- | :--- | | **Blog** | [cloudsmith.com/blog](https://cloudsmith.com/blog) | Marketing website describing Cloudsmith's key use cases, benefits, pricing options. | | **Bug Bounty Program** | [Bug Bounty Program](/resources/bug-bounty-program) | Our bug bounty program. | | **Changelog** | [cloudsmith.com/changelog](https://cloudsmith.com/changelog) | A public record of all new platform features, updates, and bug fixes. | | **CircleCI Orb** | [circleci.com/orbs/cloudsmith](https://circleci.com/developer/orbs/orb/cloudsmith/cloudsmith) | A reusable integration for automating package uploads and management within CircleCI pipelines. | | **CLI (GitHub Repo)** | [github.com/cloudsmith-cli](https://github.com/cloudsmith-io/cloudsmith-cli) | Source code and installation for the official Python-based Command Line Interface. | | **Customer Stories** | [cloudsmith.com/customers](https://cloudsmith.com/customers) | Case studies from organizations like Font Awesome, Carta, and Thrivent. | | **G2 Reviews** | [g2.com/products/cloudsmith](https://www.g2.com/products/cloudsmith/reviews) | Reviews written by Cloudsmith customers. | | **GitHub Organization** | [github.com/cloudsmith-io](https://github.com/cloudsmith-io) | Central hub for Cloudsmith’s open-source projects, including providers and examples. | | **LinkedIn** | [linkedin.com/company/cloudsmith](https://www.linkedin.com/company/cloudsmith) | Official company updates, hiring news, and professional networking. | | **Pricing** | [cloudsmith.com/pricing](https://cloudsmith.com/pricing) | Breakdown of pricing plans. | | **Status Page** | [status.cloudsmith.com](https://status.cloudsmith.com) | Live monitoring of service availability and incident history. | | **Terraform Provider** | [registry.terraform.io](https://registry.terraform.io/providers/cloudsmith-io/cloudsmith/latest) | Official provider to manage Cloudsmith resources as Infrastructure as Code. | | **Trust Center** | [trust.cloudsmith.com](https://trust.cloudsmith.com) | Compliance documentation, including ISO 27001 certifications and security audit reports. | | **VS Code Extension** | [marketplace.visualstudio.com/...](https://marketplace.visualstudio.com/items?itemName=Cloudsmith.cloudsmith-vsc) | Cloudsmith extension for the popular VSCode IDE | | **YouTube Channel** | [youtube.com](https://www.youtube.com/channel/UCO8NZYdzB3RgO4HXkHG7QiQ) | Video tutorials, webinars, and recorded talks on artifact management best practices. | # Open-Source Hosting Policy Whether you use Cloudsmith for private or public distribution, free or paid, you always have the option of creating open-source repositories. An open-source repository works a little bit different from others, with the following differences: - Usage for open-source repositories is tracked _separately_ from your public/private repositories. - You don't need to sign up for a specific open-source plan; you create an open-source repository. - You get _at least_ 50GB of artifact data + 200GB of package delivery for _free_, across all open-source repositories. - You get some features on the OSS repository that are normally reserved for paid _Pro_ plan users, such as geo/IP restrictions, access logs, and statistics. - You can request more features, artifact data, or package delivery in return for sponsorship (more details later). Sounds good. Right? So, how do you qualify for open-source repositories? Read on. ## Rules To qualify for an open-source repository, we have a few simple rules for you: 1. The primary project you're distributing for must be [free and open-source by definition](https://en.wikipedia.org/wiki/The_Open_Source_Definition). 2. You must have significant control and/or responsibility for the project; i.e. you're a maintainer. 3. The project must not be a minor fork; i.e. it is either the primary project or a **major** fork of one. 4. The main artifacts being served by the repository are for the project and not dependencies only. 5. You must adequately and fairly attribute Cloudsmith as the source of free hosting (see below). 6. A paid plan or significant sponsorship may be more appropriate if you're a commercial, for-profit entity. 7. A Cloudsmith org can contain multiple OSS projects, but a project must belong to one org only. If you need clarity on any of these, please feel free to [contact us](https://cloudsmith.com/company/contact-us) to ask. We'll be delighted to help. We don't require advance permission to create open-source repositories. (We might check them later, to confirm they qualify.) In other words, feel free to create the open-source repositories you need today, and as long as you're following the rules above, you'll be able to use them immediately. If your repository doesn't fit within our rules, this is usually something simple that can be fixed, such as using the wrong license. For more severe cases, we may have to suspend the open-source repository (e.g., doesn't have attribution, or is for a fork, etc.) In the case of suspension, we'll contact you and let you know the steps forward. We take suspension as seriously as we would do with any other user, paid or not. If this occurs, please know that we do so thoughtfully and have likely tried to contact you. **👍 Are you a forked project?** Although we require distributors to be the primary project, or a major fork of one, there's an exception to the rule: If you get the blessing from the primary maintainers to be the distributor, then you can bypass the rule. Typically though, we advise that forks either try to integrate their changes into the main project instead. ## Attribution Want to meet the attribution rule? That's easy, just include the next snippet: ``` Static Badge Package repository hosting is graciously provided by [Cloudsmith](https://cloudsmith.com). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence. ``` ... to render a badge such as the following: Static Badge
    Package repository hosting is graciously provided by [Cloudsmith](https://cloudsmith.com). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence. *** Feel free to shake it up a little. As long as it contains a link back to [cloudsmith.com](https://cloudsmith.com) or [https://app.cloudsmith.io](https://app.cloudsmith.com), and addresses that Cloudsmith provided free hosting, you're good to go. You can see other examples of badges on [Shields](https://shields.io). **Please Spread The Word** We'd also _really_ _REALLY_ appreciate it if you tweeted to your followers that you're now using Cloudsmith for package management, and tag us in. For example: **"AcmeCorp is now using @cloudsmith for open-source package management and distribution; check it out."**. Replacing _AcmeCorp_ with your own company or project name. You could also add a link to your new repository; be proud and show it off. ## Commercial Entities Suppose you're a commercial, for-profit entity, especially one with significant funding or that's post-revenue. In that case, it may be more appropriate to proceed with a paid plan or sponsorship (as below). You can still avail yourself of the base open-source usage we offer for everyone, but beyond that, we'd recommend a commercial relationship where we can work together. If you're for-profit, it's in your best interest to keep vendors, such as ourselves, going; in return, we can provide the best possible solution and full commercial support. ## Sponsorship If you need to vary the rules a little (e.g., a repository full of dependencies), need more ([Ultra or Enterprise plans](https://cloudsmith.com/pricing)) features, or need more GBs of artifact data / package delivery? We can do that for you with sponsorship that's a pinch more formal (but only a pinch). Our primary rule for sponsorship is that you're a "significant" open-source project. By significant, there isn't a hard and fast rule, but typically it means you have some level of notability. This typically means a community of users in the thousands or beyond, a website with several hundred thousand genuine hits per month, or over a thousand followers on sites like Twitter. If you feel like you fit, just [let us know, and we can work it out with you](https://cloudsmith.com/company/contact-us), a sponsorship usually takes the form of providing more significant linkbacks on your site, blog, and/or social media. Essentially it is helping us with marketing in return for free hosting and support. Nothing too demanding; just quid pro quo. Please help us; help you. Some examples of open-source projects we've sponsored formally: - [HoneyDB](https://honeydb.io) - [Shields](http://shields.io) - [Pony Language](https://www.ponylang.io) - [CloudPosse](https://github.com/cloudposse/packages) --- title: "Troubleshooting" --- # Troubleshooting Please see the [individual format pages](/formats) for format-specific troubleshooting guides and frequently asked questions. If there is anything you need further help or assistance with, or you have an issue that isn't answered here then please just [contact us](https://cloudsmith.com/company/contact-us) and we will be more than happy to help. ## Package fails to synchronize Synchronization is where we extract the metadata and files within a package, process them and make the package available for download. There are a few reasons why a package may fail to complete this process: - Incomplete or malformed package metadata - Corruption or errors in creating the package - A temporary problem with back-end package processing In some cases, especially with a temporary problem with package processing, synchronizing the package may resolve the issue. Please see [Package Resynchronization](/artifact-management/resync-a-package) for instructions. In the case of a corrupt package or missing/incomplete metadata, resynchronizing the package will not resolve the issue. You would need to fix the problem with the package itself and then re-upload the package again. ## Error: Could not import the GPG key for this repository You likely have not installed the GPG key for this repository. You can download the repository GPG key from Cloudsmith and install it. There is contextual documentation for this in your repository for whichever format you are working in. ## Still Need Help? Contact us [here](https://cloudsmith.com/company/contact-us). We're always happy to help. # Entitlement Tokens via the API ## Managing Entitlement Tokens via the Cloudsmith API You can create, delete, update and configure Entitlement Tokens programmatically using the Cloudsmith API. Please see the Cloudsmith API [reference documentation](/api/entitlements/list) for examples, including an interactive environment where you can test your API calls. # Entitlement Tokens via the CLI In the following examples: |Identifier|Description| |:---|:---| |OWNER|Your Cloudsmith account name or organization name (namespace)| |REPOSITORY|Your Cloudsmith Repository name (also called "slug")| |IDENTIFIER|The unique identifier for an Entitlement Token| |TOKEN-NAME|The name of an Entitlement Token| |TOKEN-STRING|The token itself - It must only contain alphanumerics, dashes, dots or underscores and it must begin with an alphanumeric.| ## Listing Entitlement Tokens To view all entitlement tokens for a repository, you use the `cloudsmith ents list` command: ```shell cloudsmith ents list OWNER/REPOSITORY ``` Example: ```shell cloudsmith ents list demo/examples-repo ``` This will show all the tokens associated with a repository and their creation / updated date and identifier. The token identifier is unique to each individual entitlement token and is quite important, as you use it with other entitlement commands (such as delete) [Image: ents list command] Note that by default, the token itself is not shown. To see the token, you must add the `--show-tokens` option to the command. For example: ```shell cloudsmith ents list demo/examples-repo --show-tokens ``` [Image: Entitlements list] ## Creating Entitlement Tokens To create an entitlement token via the Cloudsmith CLI, you use the `cloudsmith ents` command: ```shell cloudsmith ents create OWNER/REPOSITORY --name=TOKEN-NAME ``` Example: ```shell cloudsmith ents create demo/examples-repo --name=TestToken ``` The entitlement token identifier is shown in the output of the `cloudsmith create` command: [Image: Entitlements create command] Additionally, you can manually specify the token itself (rather than have Cloudsmith create it for you) using the `--token=TOKEN-STRING` option: ```shell cloudsmith ents create demo/examples-repo --name=TestToken --token=abcdef123456 ``` [Image: ents-create-string-show] If you manually specify a token, it must be between 8 - 48 characters in length. It must only contain alphanumerics, dashes, dots or underscores and it must begin with an alphanumeric. ### Batch Create Example If you wanted to create a batch of Entitlement Tokens at once, you could script it with a loop using the Cloudsmith CLI: ```shell bash for I in $(seq 1 10); do cloudsmith ents create OWNER/REPOSITORY --name "Customer $I" done ``` You could then retrieve a list of all entitlements, plus their tokens with: ```shell cloudsmith ents ls OWNER/REPOSITORY --show-tokens -F json | jq -r '.data[] | "\(.name) = \(.token)"' ``` ## Deleting Entitlement Tokens Before you can delete an entitlement token, you need to get the identifier for the entitlement token. This can be obtained from the `cloudsmith ents list` command or from the `cloudsmith ents create` command. To delete an entitlement token, you use the `cloudsmith ents delete` command: ```shell cloudsmith ents delete OWNER/REPOSITORY/IDENTIFIER ``` Example: ```shell cloudsmith ents delete demo/example-repo/Ym4xadpEU4YS ``` [Image: Cloudsmith Entitlements delete] A deleted token will still be able to be used for static assets (that are cached at the Package Delivery Network) for approximately 10 minutes until the PDN has to re-authenticate once its cache expires ## Refreshing Entitlement Tokens Refreshing an entitlement token generates a new token, but retains any permissions granted by the token. This has the effect of invalidating the current token. You can refresh an entitlement token using the `cloudsmith ents refresh` command: ```shell cloudsmith ents refresh OWNER/REPOSITORY/IDENTIFIER ``` Example: ```shell cloudsmith ents refresh demo/examples-repo/RJol6tyAIssl ``` [Image: Entitlements refresh command] The example above shows the **TestToken** before and after using the `cloudsmith ents refresh` command. ## Updating Entitlement Tokens To modify a token name or the token itself, you use the `cloudsmith ents update` command: ```shell cloudsmith ents update OWNER/REPOSITORY/IDENTIFIER --name=TOKEN-NAME --token=TOKEN-STRING ``` Examples: ```shell cloudsmith ents update demo/examples-repo/RJol6tyAIssl --name=NewTestToken ``` [Image: Entitlements update] ```shell cloudsmith ents update demo/examples-repo/RJol6tyAIssl --token=abcedf123456 ``` [Image: Entitlements update token] A refreshed token will still be able to be used for static assets (that are cached at the Package Delivery Network) for approximately 10 minutes until the PDN has to re-authenticate once its cache expires ## Synchronising Tokens You can synchronise all of the entitlement tokens from one Repository to another Repository using the `cloudsmith ents sync` command: ```shell cloudsmith ents sync OWNER/TARGET-REPO SOURCE-REPO ``` Example: ```shell cloudsmith ents sync demo/demo-repo examples-repo ``` This example synchronises all the entitlement tokens from the **examples-repo** repository to the **demo-repo** repository: [Image: Cloudsmith Entitlements sync command] If we now list the entitlement tokens in each of these repositories, we can see that they contain the same entitlement tokens: [Image: Cloudsmith Entitlements list demo repo] The token identifiers are still unique to each token, the token names and any associated permissions that the tokens grant are synchronised. ## Token Restrictions You can add usage restrictions to a token with the `cloudsmith ents restrict` command: ```shell cloudsmith ents restrict OWNER/REPOSITORY/IDENTIFIER ``` The available flags that you can add to set restrictions are: |Restrict Command Flag|Description| |:---|:---| |--limit-bandwidth|The maximum download bandwidth allowed for the token. Please note that since downloads are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point.| |--limit-bandwidth-unit|The unit of bandwidth to limit by. Note that 1GB = 1000000000 Bytes.| |--limit-num-clients|The maximum number of unique clients allowed for the token. Please note that since clients are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point.| |--limit-num-downloads|The maximum number of downloads allowed for the token. Please note that since downloads are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point.| |--limit-package-query|The package-based search query to apply to restrict downloads to. This uses the same syntax as the standard search used for repositories (see [Searching / Filtering](/artifact-management/search-filter-sort-packages) for more details). This will still allow access to non-package files, such as metadata. For package formats that support dynamic metadata indexes, the contents of the metadata will also be filtered.| |--limit-path-query|The path-based search query to apply to restrict downloads to. This supports boolean logic operators such as OR/AND/NOT and parentheses for grouping. The path evaluated does not include the domain name, the namespace, the entitlement code used, the package format, etc. and it always starts with a forward slash.| |--limit-date-range-from|The starting date/time (UTC) that the token is allowed to be used from.| |--limit-date-range-to|The ending date/time (UTC) that the token is allowed to be used until.| |--refresh-token|The reset period used will be used to automatically trigger a reset of the token limits. Available options are daily, weekly, fortnightly, monthly, bi-monthly, quarterly and annual| For Example: ```shell cloudsmith entitlements restrict demo/examples-repo/GYwg00eEElKs \ --limit-bandwidth=1 \ --limit-bandwidth-unit=gigabyte \ --limit-num-clients=10 \ --limit-num-downloads=1000 \ --limit-package-query="package-darwin-amd64" \ --limit-path-query=tag:latest \ --limit-date-range-from=2020-01-01T00:00:00Z \ --limit-date-range-to=2077-01-01T00:00:00Z \ --refresh-token=daily ``` ## Token Usage Metrics You can check the bandwidth usage and token metrics for a repository with the `cloudsmith metrics` command: ### Repository Usage You can check the total bandwidth usage for a repository with the `cloudsmith metrics tokens` command: ```shell cloudsmith metrics tokens OWNER/REPOSITORY ``` Example: ```shell cloudsmith metrics tokens demo/examples-repo ``` [Image: Metrics Tokens] ### Specific Token Usage To check the bandwidth used by a specific token you add the `--tokens` parameter and the token identifier, or a comma-separated list of token identifiers, to the `cloudsmith metrics tokens` command: ```shell cloudsmith metrics tokens OWNER/REPOSITORY --tokens=IDENTIFIER ``` Example: ``` cloudsmith metrics tokens demo/examples-repo --tokens=YcLoVS7BsLHf ``` [Image: Metrics per token] In addition, you can add the `--start` and `--finish` parameters, to display usage for a specific time period: Example: ```shell cloudsmith metrics tokens demo/examples-repo --tokens=YcLoVS7BsLHf \ --start=2020-04-01T00:00:00Z --finish=2020-09-07T00:00:00Z ``` [Image: Metrics Token Data] # Entitlements via the UI ## Viewing Entitlement Tokens To view Entitlement Tokens for a repository, click the "Settings" tab, then click "Entitlement Tokens": [Image: Viewing Entitlement Tokens] ## Searching Entitlement Tokens You can search / filter tokens using the "Filter Tokens field: [Image: Entitlement Token Search] When filtering/searching Entitlement Tokens, you can use the following search criteria with boolean logic (e.g. AND/OR/NOT) for complex search queries: | Search Query | Description | | :----------------------------- | :--------------------------------------------------------------------------------------------------------------------------- | | Name | The Token Name. For Example: `name:some-name` | | Identifier | The Token Identifier. For Example: `identifier:abced12345` | | Active | If the Token is Active or Disabled, either `true`or `false` For Example: `active:false` | | No. of clients (since reset) | The Number of Client IP addresses that have used the token since the usage counter was last reset. For Example: `clients:>1` | | No. of downloads (since reset) | The Number of package downloads attributed to the Token since the usage counter was last reset. For Example: `downloads:>1` | | Bandwidth limit | The bandwidth limit for a Token. For Example: `bandwidth_limit:100` | | Bandwidth limit unit | The units for a bandwidth limit. For Example `bandwidth_limit_unit:GB` | | Bandwidth usage (since reset) | The bandwidth used by a Token since the usage counter was last reset. For Example: `bandwidth_usage:>10` | | Token Type | The Token Type (Standard Token or User Token). For Example `token_type:standard` | | User | User Name, for user-based tokens. For Example: `user:foo` | | Created date | The Token creation date. For Example: `created:>10/10/2024` | | Last usage reset date | The date the usage counter for the token was last reset. For Example `last_reset:>10/10/2024` | | Limit from date | The date that the token will be valid from. For Example:`limit_date_from:10/10/2024` | | Limit to date | The date that the token will be valid until. For Example: `limit_date_to:10/10/2024` | ## Creating Entitlement Tokens You can create and configure Entitlement Tokens via the Cloudsmith web app by clicking the "Settings" tab in a repository, then click "Entitlement token", the click the blue "Create New Token" button: [Image: Create Token Button] You will then be presented with a form where you can name the token and configure permissions/restrictions that the token grants on the repository: [Image: Create Token Form] ### Visibility Restrictions You can add visibility restrictions to a token which will control what packages the token has access to: | Restriction | Description | | :--------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Restrict by Search** | The package-based search query to apply to restrict downloads to. This uses the same syntax as the standard search used for repositories (see [Searching / Filtering](/artifact-management/search-filter-sort-packages) for more details). This will still allow access to non-package files, such as metadata. For package formats that support dynamic metadata indexes, the contents of the metadata will also be filtered. | ### Usage Limits You can add usage limits restrictions to a token which will control how the token can be used. The configurable restrictions are: |Limit|Description| |:---|:---| |**Statistics Reset Interval**|A token reset refreshes the maximum downloads, clients/IPs, and bandwidth restrictions to zero and maintains the existing limits. The reset period used will be used to automatically trigger a reset of the token limits during the configured period.| |**Valid From (UTC)**|The starting date/time the token is allowed to be used from.| |**Expires at (UTC)**|The ending date/time the token is allowed to be used until.| |**Maximum Downloads**|The maximum number of downloads allowed for the token. Please note that since downloads are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point."| |**Maximum Clients/IPs**|The maximum number of unique clients allowed for the token. Please note that since clients are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point.| |**Maximum Bandwidth**|The maximum download bandwidth usage allowed for the token. Please note that since downloads are calculated asynchronously (after the download happens), the limit may not be imposed immediately but at a later point.| |**Unit of Bandwidth**|The selected unit of bandwidth to apply to the Maximum Bandwidth restriction. **Please Note** 1GB = 1000000000 (1000^3) Bytes, not 1073741824 (1024^3) Bytes.| Please see [Sharing a Private Package](/artifact-management/sharing-a-private-package) for an example of configuring an Entitlement Token using the Cloudsmith web app. ### Additional Metadata In addition, you can optionally add metadata to the token that is specific to your use case. This could be used to store information such as licensing information, but the format and contents are defined by you. You add this metadata as JSON into the "Token Metadata (JSON)" field: ### EULA Acceptance EULA for Entitlement Tokens is in Early Access. If you'd like to be included in early access to this feature please [contact us](https://cloudsmith.com/company/contact-us). You can specify that a EULA must be accepted before an Entitlement is enabled: If checked, then a client will be compelled to go to the token-based URL for EULA acceptance, before they are able to use the token to download files. Note that this also requires EULA enforcement to be enabled on the repository. ## Editing Entitlement Tokens Editing an Entitlement Token allows you to change the token name, modify any permissions/restrictions associated with the token or change the token's metadata. You edit an Entitlement Token by clicking the dots to the right of a token and then clicking "Edit Token": [Image: Edit Token Button] User Entitlement Tokens cannot be edited. You are then presented with the Edit Entitlement Token form where you can make any changes and click the "Save" button to apply them: [Image: Edit Token Form] ## Setting an Entitlement Token When you create an Entitlement Token, we generate a random string for the token secret itself and it is this secret that will be used in configuration files that use the token. The token is not displayed in the Cloudsmith web app but can be copied using the copy button beside the secret: [Image: Token Field] Setting an Entitlement Token secret allows you to use your own custom string for an Entitlement Token secret. Please note, setting a custom string for a token will not change the token name or any permissions/restrictions associated with the token, but it will have the effect of invalidating any users/clients using the current token. You set an Entitlement Token secret by clicking the dots to the right of a token and then clicking "Set Token": [Image: Set Token Button] You will then be presented with the Set Token form, where you have to confirm the repository slug/identifier (to prevent the accidental setting of a token) and enter the new string for the token: [Image: Set Token Form] If you specify a custom string for a token, it must be between 8 - 48 characters in length. It must only contain alphanumerics, dashes, dots or underscores and it must begin with an alphanumeric. ## Resetting Entitlement Token Statistics Resetting Entitlement Token Statistics will reset the download and client counts to zero. You can reset the statistics associated with an Entitlement Token by clicking the dots to the right of a token and then clicking "Reset Token Statistics": [Image: Reset Token Statistics Button] You will then be presented with the Reset Token Statistics form where you must confirm the current repository slug/identifier (to prevent accidental resets of statistics) and then click the "Confirm" button: [Image: Reset Token Statistics Form] ## Refreshing Entitlement Tokens Refreshing will generate a new Entitlement Token secret and this will invalidate the current token in use by existing users/clients. Refreshing an Entitlement Token will not change the token name, or any restrictions/permissions associated with the token, it just generates a new token secret itself. As long as the user who created this token has privileges for this repository, they can recreate/retrieve the token at anytime To refresh an Entitlement Token, click the dots to the right of a token and then click "Refresh Token": [Image: Refresh Token Button] You will then be presented with a form that will ask you to enter the repository slug/identifier (this is to prevent accidental token refreshes), and then click "Confirm": [Image: Refresh Token Confirmation Form] A refreshed token will still be able to be used for static assets (that are cached at the Package Delivery Network) for approximately 10 minutes until the PDN has to re-authenticate once its cache expires. ## Synchronising Entitlement Tokens Synchronising Entitlement Tokens replaces all the tokens associated with a repository with those from another repository. This will invalidate any current tokens in use by existing users/clients. You can synchronise Entitlement Tokens between repositories by clicking the dots to the right of the "Create New Token" button and then clicking "Sync Tokens": [Image: Sync Tokens Button] You will then be presented with the Sync Tokens form, which requires you to confirm the current repository slug/identifier (to prevent the accidental synchronisation of tokens) and also choose the source repository from a drop-down list: [Image: Sync Tokens Confirmation Form] When you have confirmed the current repository slug/identifier and selected the source repository for the tokens you wish to synchronise, click "Sync Tokens" to synchronise the tokens. ## Deleting Entitlement Tokens You can delete an Entitlement Token via the Cloudsmith web app. This is a soft-delete, in that the token will no longer be available for use but the history of the token will be retained for logging/auditing purposes. To delete an Entitlement Token click the dots to the right of the token and then click "Delete token": [Image: Delete Token Button] You will be presented with a form that will ask you to enter the current repository slug/identifier (this is to prevent accidental deletion of a token), and then click "Delete": [Image: Delete Token Confirmation Form] A deleted token will still be able to be used for static assets (that are cached at the Package Delivery Network) for approximately 10 minutes until the PDN has to re-authenticate once its cache expires. Deleted Entitlement Tokens cannot be re-enabled. # Entitlement Tokens Entitlement tokens are read-only tokens that your users (or customers) can use for downloading files - or more precisely, for getting read-only access to repository contents. You can use them to distribute files to users who don't have logins to Cloudsmith, or for "licensing" software by associating an entitlement token per customer. If you'd like to track downloads for specific customers, the best practice is to create an entitlement token per customer, then use that token when you're providing download links for them. It also means you can revoke their token if they've lapsed their subscription with you, or change it for other reasons. We also offer a "basic auth" method of authenticating with the website, which means it uses your username/password to authenticate instead of embedding a token into the URL. You could use this to create a separate "bot" user whose sole purpose is to authenticate for downloads via server installations. **For Vendors** If you're selling software and distributing it via Cloudsmith, you'll likely have a license that is associated per Customer and which dictates their terms of usage. Associating the license with an entitlement allows you to control and track downloads of the software specifically for that license. For example, you could only allow the Customer to download specific packages, between August 1st 2024 and August 1st 2024, up to a maximum of 10 downloads from one location (i.e. a single client). Each entitlement can have different restrictions, and you can use the freeform (JSON-based) metadata to add your own information into the entitlement. Entitlement tokens are only available on private repositories, as by definition, public repositories are open and provide read-only access to anyone. Entitlement Tokens also cannot be used to authenticate to the Cloudsmith API. Entitlement Tokens are used to authenticate for package downloads only. ## Standard Entitlement Tokens You can create Standard Entitlement Tokens via the [Cloudsmith web app](/software-distribution/entitlement-tokens-via-the-website-ui), via the [Cloudsmith CLI](/software-distribution/entitlement-tokens-via-the-cloudsmith-cli) or via the [Cloudsmith API](/software-distribution/entitlement-tokens-via-the-cloudsmith-api) ## User Entitlement Tokens User Entitlement Tokens are very similar to Standard Entitlement Tokens. They are created automatically when a user with permissions views a repository. They are associated with the user and the user's permissions. Only the user themselves and administrators of the repository can see the user-based tokens. If the user is removed from the repository, teams or the organisation, then the token will no longer work for authentication. ## Multi-Repository Tokens We don't support creating entitlement tokens that are valid across multiple repositories. However, there are two ways you can achieve something similar: - You can [sync entitlement tokens](/software-distribution/entitlement-tokens-via-the-cloudsmith-cli#synchronising-tokens) across multiple repositories. - You can create a token in several repositories with a custom token string, thereby giving you a token where the string will be valid in more than one repository. ## Permissions **How can I define a user with minimum rights to only create entitlements on a certain repository?** The permissions aren't granular enough to allow for entitlement token creation only (which requires at least "Write" access). However, a user (or a user in a team) with "Write" access can only modify (move/delete/etc) their own packages. So if you create a user/team for updating entitlement tokens, although they will have write access to the repository, to create tokens, they will not be able to modify the existing packages that are there. In other words, separate the "uploader" account (this could also just be your normal staff) from the account that will create entitlement tokens. ## Counting Token Usage Some Cloudsmith pricing plans include a specific number of entitlement tokens. Only active tokens count towards this limit; disabled and deleted tokens are not counted. # EULA Enforcement Cloudsmith provides the ability for all Raw format files, if enabled, to require End-User License Agreement (EULA) when a user attempts to download it. ## Create a EULA To create a EULA, navigate to the repository and select "Settings" -> "EULA enforcement." Click the blue "Create Revision" button: You will then be presented with the Create EULA Revision form, where you can add the content/terms that you wish to display to the user before they can download the file: You then click the blue "Create Revision" button to create the EULA. You can repeat this process if you need to create subsequent revisions of the EULA. To require a user to accept the EULA before downloading Raw files in the repository, EULA enforcement must be set to "Enabled." ## How do users view and accept a EULA? ### View and accept via WebsiteUI The first time a user attempts to download a raw package using a download link, they will instead see the EULA: Once the user has clicked the "Yes, I Agree + Download File" button, the download will start. ### Accept via URL / Command Line If a user wishes to accept the EULA without visiting the HTML page, `?accept_eula=1` can be suffixed onto the URL link for the raw package (which would otherwise display the EULA) to accept it. The number appended to the `accept_eula` parameter specifies the revision of the EULA that is being accepted: ```shell curl https://dl.cloudsmith.io/TOKEN/OWNER/REPOSITORY/raw/files/FILENAME.zip?accept_eula=1 ``` Where: | Identifier | Description | | :--------- | :---------------------------------------------------------------------------------------- | | TOKEN | Your Cloudsmith Entitlement Token (see [Entitlements](/software-distribution/entitlement-tokens) for more details) | | OWNER | Your Cloudsmith account name or organisation name (namespace) | | REPOSITORY | Your Cloudsmith Repository name (also called "slug") | | FILENAME | The name of the raw file | For Example: ```shell curl https://dl.cloudsmith.io/uy7de6tvI8O9/cloudsmith/demo/raw/files/test.zip?accept_eula=3 ``` ## EULA for Entitlement Tokens EULA for Entitlement Tokens is in Early Access. If you'd like to be included in early access to this feature please [contact us](https://cloudsmith.com/company/contact-us). You can specify that a EULA must be accepted before an [Entitlement Token](/software-distribution/entitlement-tokens) can be used. This will allow you to enforce EULA acceptance for any package format, as Entitlement Tokens are used to provide controlled, read-only access to any artifact in a repository. To require EULA acceptance for an Entitlement Token, you need to select the option when editing an Entitlement Token: If checked, then a client will be compelled to go to the token-based URL for EULA acceptance, before they are able to use the token to download files. Note that this also requires EULA enforcement to be enabled on the repository. ## How do I track what my customers have downloaded? You can see them in "Download Logs" within a repository. These are processed asynchronously so they don't appear immediately after a download happens, but within a short-time (usually within 5 minutes). If EULA enforcement is enabled, then each Raw package file has gone through the EULA acceptance before download. In other words, it's not possible to download without accepting the latest revision of the EULA. Hovering over the EULA icon provides detail on which revision was accepted for it, and clicking it brings you to the EULA overview. It will show the name of the entitlement token you've created for that specific customer (or group of customers). E.g. "Microsoft (Token)" if a customer at Microsoft had downloaded it. In summary, we show which customer downloaded which file, when, and having accepted what EULA revision to do so. ## Can you change a EULA? Once a EULA has been created, you (a person with privileges) has exactly one hour to make modifications, then it gets locked. Afterwards you'll no longer be able to edit the EULA revision again. It's only possible to edit the most recent EULA revision within this one hour window. Any previous EULA revisions are never editable. ## How do you prevent EULA entities from being lost or deleted? Any EULAs are covered by the same strong guarantees for data sanctity as the rest of the system; as described in our Security Policy. You cannot directly delete a EULA revision. ## How long are EULA entities stored in database? Permanently until the repository or account is deleted; this can only be done by _Admin_ of a repository, or an _Owner_ of the account. See RBAC question later. ## What happens if I unsubscribe? As per the Privacy Policy we'll keep your data in-tact for a period of time, "Cloudsmith will retain Personally Identifiable Information on your behalf whilst either a) valid grounds for processing exist; or b) a maximum of seven (7) years following termination of your account.". Usually we don't delete customer information early unless specifically requested by you (as the data owner), or for some legal reason. ## Will we still be able to have access to the data? If you want to export your data out of the system, we are happy to help with that. We don't believe in vendor lock-in or restricting the portability of customers. :) ## How do you ensure that only authorised persons can manage EULA? Only users (or users in a team) with _Admin_ access on a repository, or _Owner_ access on the account, can modify EULA revisions. As mentioned previously, they can't edit or delete earlier EULA revisions. What they can do: Add a new EULA revision, enable/disable EULA enforcement for that repository. ## Is it possible to restrict access to EULA to certain roles? As above, EULA are restricted to specific roles already, but it's possible to define your own roles (as such) by creating Teams with the appropriate privileges. For example, you could create a team of users that specifically has _Admin_ access to the repositories (to manage the repository), and then another one that has _Write_ access (for adding new packages). Our recommendation would be to ensure that only appropriate employees have the _Admin_ access, and to greatly restrict who has _Owner_ access on the account itself. # Software Distribution Whether you're shipping packages to customers, partners, or internal teams, Cloudsmith ensures every artifact is delivered reliably and at speed, anywhere in the world. But distribution isn’t just about moving files quickly. It’s about making sure your software reaches the right people, under the right conditions. Cloudsmith gives you both: - A global package delivery network with intelligent edge caching for fast, reliable performance everywhere - A powerful set of features for secure, branded and controlled software distribution to customers ## Cloudsmith for distribution - [Broadcasts](/software-distribution/broadcasts) - [Entitlement Tokens](/software-distribution/entitlement-tokens) - [EULA Enforcement](/software-distribution/eula) - [Custom Domains](/workspaces/custom-domains) - Connect to payment providers via our [Zapier Connector](/integrations/integrating-with-zapier) ## Link Software Licenses to Entitlement Tokens If you're selling software and distributing it via Cloudsmith, you'll likely have a license associated per customer, which dictates their terms of usage. Associating the license with an entitlement token allows you to control and track software downloads specifically for that license. For example, you could only allow your customer to download specific packages, between November 5, 2024, and November 5, 2024, up to a maximum of 10 downloads from one location (i.e. a single client). Each entitlement can have different restrictions, and you can use the freeform (JSON-based) metadata to add your own information to the entitlement. If you'd like to track downloads for specific customers, then the best practice is to create an entitlement token per customer and then use it when providing download links. It also means you can revoke their token if they've lapsed their subscription with you or change it for other reasons. # Continuous Security Continuous Security is a feature that provides an hourly feed of vulnerability and malicious package data from multiple sources (see [refresh intervals](#data-sources) for each source). Whenever vulnerabilities or malicious packages are published or modified, we match them to artifacts in your Cloudsmith repositories. This enables faster identification of affected artifacts compared to scheduled or on-demand [vulnerability scanning](./vulnerability-scanning). **Early Access** Continuous Security is in Early Access (EA) as part of the Cloudsmith's Enterprise Policy Manager feature. ## Vulnerability Scanning vs. Continuous Security ### Vulnerability Scanning Cloudsmith [scans](./vulnerability-scanning) artifacts as they are introduced into a repository for the first time. This scan analyzes the components of a package and then checks for vulnerabilities associated with them, which are listed in a vulnerability report. Note that when new vulnerabilities are disclosed, any existing report for impacted artifacts won’t flag them; a re-scan will be required to get the most up-to-date information. ### Continuous Security **EPM Required** Continuous Security is only available for Cloudsmith workspaces where Enterprise Policy Manager (EPM) has been enabled. Continuous Security matches the software artifacts in your Repository with vulnerability data from several sources. Unlike Vulnerability Scanning, Continuous Security responds automatically in real time as vulnerabilities are reported or modified in those sources, with no manual effort required. Whenever a threat that affects an artifact in your workspace is detected, Continuous Security triggers EPM policy evaluation, with the vulnerability data included in the policy input. For more information about using EPM to address vulnerabilities identified by Continuous Security, see [Getting Started with Enterprise Policy Manager](./epm/getting-started) and [Rego Recipes](./epm/rego). Details of the specific vulnerability data available via Continuous Security are available in our API documentation under the vulnerabilities object in the [PolicyInputV0 schema](https://api.cloudsmith.io/v2/swagger/). #### Data Sources Continuous Security uses the following data sources: | Source | Refresh Interval | | :-------------------------------------------------------------------------------------- | :--------------: | | Common Vulnerabilities and Exposures (CVE) databases aggregated in [Aqua Trivy DB](https://trivy.dev/v0.39/docs/vulnerability/detection/data-source/) | 6 hours | | [Exploit Prediction Scoring System (EPSS)](https://www.first.org/epss/) | 24 hours | | Vulnerabilities and Malicious Packages from the [Open Source Vulnerability DB (OSV)](https://osv.dev/list) | 1 hour | #### Supported Formats Continuous Security is available for [all package formats](./vulnerability-scanning) supported by Cloudsmith’s Vulnerability Scanning feature. # Supply Chain Security Modern software delivery pipelines face growing complexities and a rising wave of sophisticated supply chain attacks, requiring artifact management to go beyond traditional storage. According to the [Cloudsmith 2025 Artifact Management Report](https://cloudsmith.com/campaigns/2025-artifact-management-report), this is top of mind for engineering teams, with 56% of respondents citing improved security as the leading benefit of their artifact management tools. Our platform is designed to meet this challenge head-on. We provide a comprehensive suite of features to help you secure your software supply chain by enabling you to define your policies and responses to policy violations as code; identifying, mitigating, and blocking risks from vulnerabilities and non-compliant licenses in your dependencies. In this section you can learn how to: - Ensure with [Block Until Scan](/policy-management/block-until-scan) that no package can be downloaded until a security scan is successfully completed, helping you enforce security mandates across the whole organization. - Reduce risk associated with [Software Vulnerabilities](/supply-chain-security/vulnerability-scanning), [Malware](/supply-chain-security/malware-detection/malware-scanning), and [Malicious Packages](/supply-chain-security/malware-detection/malicious-packages), by proactively quarantining packages and preventing them from entering your software supply chain. - **License Compliance**: Automatically scan for and identify the licenses of your dependencies, ensuring compliance with your organization's policies. - **Digital Signatures**: Verify the integrity and authenticity of your software artifacts using GPG, PGP, and other signing standards. Adding to this, Enterprise Policy Manager (EPM) and Continuous Security: - [Policy-as-Code](/supply-chain-security/epm/) (Early Access): The next evolution in policy management, using Open Policy Agent (OPA). This new feature allows you to define all your security and license compliance rules in rego. While our existing policy features remain fully supported, Policy-as-Code is intended to become the primary method for managing supply chain security rules at scale. - [Continuous Security](/supply-chain-security/continuous-security) (Early Access): Continuously monitor your dependencies to detect in real time and automatically act upon vulnerable and malicious packages. # Package signing Cloudsmith uses GPG or RSA signatures (where applicable) in addition to package checksums to detect tampering. We calculate a signature for every file that is uploaded, but only some of the package formats make it available or use it. Only some of the formats also offer metadata signing. For increased trust, you can also provide your own [GPG key](/repositories/repository-settings#custom-gpg-signing-key) or [RSA key](/repositories/repository-settings#custom-rsa-signing-key) for signing. ## Key Support by Package Format |Package Format|Key Type|Key Use| |:--|:--|:--| |[Alpine](/formats/alpine-repository)| RSA | Index | |[Cargo](/formats/cargo-registry)||Not Supported by Format| |[CocoaPods](/formats/cocoapods-repository)||Not Supported by Format| |[Composer](/formats/composer-repository)| GPG || |[Conan](/formats/conan-repository)||Not Supported by Format| |[CRAN](/formats/cran-repository)||| |[Dart](/formats/dart-repository)||Not Supported by Format| |[Debian](/formats/debian-repository)| GPG |Index| |[Docker](/formats/docker-registry)| RSA | Index | |[Generic](/formats/generic-repository)| Not Supported by Format || |[Go](/formats/go-registry)||| |[Gradle](/formats/gradle-repository)| GPG | Index Package | |[Helm Charts](/formats/helm-chart-repository)| GPG || |[LuaRocks](/formats/luarocks-repository)||| |[Maven](/formats/maven-repository)| GPG | Index Package | |[npm](/formats/npm-registry)| GPG || |[NuGet](/formats/nuget-feed/)||| |[Python](/formats/python-repository)| GPG || |[Raw](/formats/raw-repository)| GPG || |[RPM](/formats/redhat-repository)| GPG | Index Package | |[sbt](/formats/sbt-repository)| GPG | Index Package | |[Terraform Modules](/formats/terraform-modules-repository)||Not Supported by Format| |[Unity](/formats/unity-registry)| GPG || |[Vagrant](/formats/vagrant-repository)| GPG || ## Docker and Cosign When a Docker image is uploaded to a repository, Cloudsmith automatically generates a Cosign signature using the repository’s ECDSA private key. Customers can download the corresponding ECDSA public key to verify the specific image, as below: ``` docker push docker.cloudsmith.io///alpine: cosign verify --private-infrastructure=true --key public-ecdsa-key.key docker.cloudsmith.io///alpine: ``` See [Docker Images: Signing and Verifying SBOMs with Cosign](/artifact-management/sbom#docker-images-signing-and-verifying-sboms-with-cosign) for more. **Docker image verification** Cloudsmith does not log to Rekor when generating signatures on image upload. When verifying these using `cosign verify` pass `--private-infrastructure=true` to prevent cosign querying the Rekor log. As Docker images within Cloudsmith are predominantly private, when a cosign signature is automatically generated on image upload, Cloudsmith does not add any data to the Rekor log. The Rekor log contains a record of image names and corresponding public keys, and is used to enable software maintainers to record signed metadata, and for verifiers to monitor and query the log for an appropriate identity. When using `cosign verify` to verify cosign signatures generated by Cloudsmith, pass `--private-infrastructure=true` to ensure cosign does not query the Rekor log. If this parameter is not passed, the following warning will be displayed: ``` WARNING: "docker.cloudsmith.io///alpine: appears to be a private repository, please confirm uploading to the transparency log at "https://rekor.sigstore.dev" ``` Please note customers who create their own signatures using the `cosign sign` command, will be asked if they wish to upload a transparency log to Rekor. **Supported Keys** If a key is used which is not supported by Cosign, it will not be possible to generate the associated OCI image signature. Cosign supports the following ECDSA curves: * NIST P-224 (secp224r1) * NIST P-256 (secp256r1, prime256v1) * NIST P-384 (secp384r1) * NIST P-521 (secp521r1) If a key is used which is not supported by Cosign, Cloudsmith will not be able to generate the associated OCI image signature. # Upstream Trust Upstream trust is a supply chain security feature that prevents namesquatting attacks where bad actors hijack your internal package name in public repositories. By designating upstream sources as trusted or untrusted, you control which sources are permitted to serve versions of packages that exist in your private repository or other trusted sources. This is particularly important for organizations that publish private packages alongside public open-source dependencies. Without upstream trust, a malicious actor could publish a package with the same name as your private package to a public registry, potentially tricking your build systems into pulling the attacker's version instead of your own. **Early Access** Upstream Trust is in Early Access (EA) and is currently supported for NPM, Python, and Maven. ## How it works Every upstream source configured for a repository has a trust status that can be set to either Trusted or Untrusted. Your local Cloudsmith repository is always implicitly trusted. When a package is requested from a repository with upstreams configured, Cloudsmith evaluates trust as follows: 1. Cloudsmith checks whether the requested package name (including any format-specific qualifiers or scopes, such as @cloudsmith/cloudsmith-web for npm) exists in any trusted source — either the local repository or any upstream marked as trusted. 2. If the package name is found in a trusted source, any upstream marked as untrusted is blocked from serving versions of that package. Only versions available from trusted sources will resolve. 3. If the package name is not found in any trusted source, versions from all sources (including untrusted upstreams) will resolve as normal. This means your private packages are protected from namesquatting, while open-source packages that don't collide with your private package names continue to resolve without interruption. **Proxied and cached packages** If a package has already been proxied or cached from a particular upstream, that upstream is permitted to continue serving versions of that package — even if it is marked as untrusted. The block only applies to other untrusted upstreams that have not previously served the package. This ensures that your existing open-source dependencies (such as `requests` or `numpy` proxied and cached from PyPI) continue to resolve without disruption. ## Examples The following examples illustrate how upstream trust affects package resolution for a package named cloudsmith_web across different source configurations. ### Untrusted upstream with multiple trusted sources | Source | Trust Status | Available Versions | | :-------------------------------------------------------------------------------------- | :--------------: | :--------------: | | PyPI | Untrusted | 2.1.1, 2.1.2, 2.1.3 | | Private repository | Trusted | 2.1.1, 2.1.2 | | Local repository | Trusted | 2.1.1 | Resolved versions: 2.1.1 (local), 2.1.2 (private) The package cloudsmith_web exists in trusted sources (private and local), so PyPI is blocked from serving any versions. Only versions from trusted sources resolve. ### Untrusted upstream with local trusted source only | Source | Trust Status | Available Versions | | :-------------------------------------------------------------------------------------- | :--------------: | :--------------: | | PyPI | Untrusted | 2.1.1, 2.1.2, 2.1.3 | | Local repository | Trusted | 2.1.1 | Resolved versions: 2.1.1 (local) The package exists in the local repository (trusted), so all versions from the untrusted PyPI upstream are blocked. ### Untrusted upstream with no trusted sources containing the package | Source | Trust Status | Available Versions | | :-------------------------------------------------------------------------------------- | :--------------: | :--------------: | | PyPI | Untrusted | 2.1.1, 2.1.2, 2.1.3 | Resolved versions: 2.1.1, 2.1.2, 2.1.3 (PyPI) No trusted source contains a package with this name, so all versions from PyPI resolve normally. This is the typical flow for open-source packages that don't collide with any of your private package names. When a package was originally cached from an untrusted source and a newer version is now available from the same source, you will be allowed to pull that new version. ### Untrusted upstream with trusted upstream (no local package) | Source | Trust Status | Available Versions | | :-------------------------------------------------------------------------------------- | :--------------: | :--------------: | | PyPI | Untrusted | 2.1.1, 2.1.2, 2.1.3 | | Private repository | Trusted | 2.1.1, 2.1.2 | Resolved versions: 2.1.1, 2.1.2 (private) The package exists in a trusted upstream (private repository), so PyPI is blocked. Only the versions available from the private repository resolve. ### All trusted sources | Source | Trust Status | Available Versions | | :-------------------------------------------------------------------------------------- | :--------------: | :--------------: | | PyPI | Trusted | 2.1.1, 2.1.2, 2.1.3 | | Local repository | Trusted | 2.1.1 | Resolved versions: 2.1.1 (local), 2.1.2, 2.1.3 (PyPI) When all sources are trusted, versions from every source resolve. No blocking is applied. # Configuring upstream trust Trust status can be configured when creating a new upstream or by editing an existing upstream. ## Setting trust on a new upstream 1. Navigate to your repository and select Upstreams. 2. Click Add Upstream and select the format for your upstream source. 3. Complete the upstream configuration fields as usual. 4. Under Trust Status, select either Trusted or Untrusted. 5. Click Create Upstream. ## Updating trust on an existing upstream 1. Navigate to your repository and select Upstreams. 2. Click the upstream you want to modify. 3. Toggle the Trust Status between Trusted and Untrusted. 4. Click Save. **Default trust status** All upstreams are set to Trusted by default. To take advantage of namesquatting protection, you should mark public registries (such as PyPI, npmjs, or Maven Central) as Untrusted if you also publish private packages with the same format. # Package identity and scoping Upstream trust matches packages by their full identifier, which includes any format-specific qualifiers or scopes. For example: ### Example identifiers | Format | Example Identifier| | :-------------------------------------------------------------------------------------- | :--------------: | | Python (PyPI) | cloudsmith-web | | npm | @cloudsmith/cloudsmith-web | | Maven | io.cloudsmith:cloudsmith-web | | NuGet | Cloudsmith.Web | A match occurs when the full package identifier in an untrusted upstream is identical to a package identifier in a trusted source. There is no partial or fuzzy matching. # Recommended configuration For most organizations that publish private packages, we recommend the following approach: - Mark public registries (PyPI, npmjs, Maven Central, NuGet Gallery, etc.) as Untrusted. - Mark internal upstreams (private Artifactory instances, other Cloudsmith repositories, etc.) as Trusted. - Artifacts sourced locally (rather than from upstream sources) are always trusted. This ensures that your private packages are only sourced from locations you control, while open-source dependencies that don't share names with your private packages continue to resolve from public registries as expected. **Important** Upstream trust protects against namesquatting by controlling which sources can serve packages. It does not replace other supply chain security practices such as vulnerability scanning, package signing, or policy enforcement. For a comprehensive security posture, use upstream trust alongside Cloudsmith's other security features. # Vulnerability scanning Cloudsmith Security Scanning will automatically scan supported package types for CVEs upon upload of a package. You can also trigger subsequent scans manually via the Web UI, and via the Cloudsmith API. You can use the results of a Cloudsmith Security Scan to drive other actions such as to [quarantine a package](/artifact-management/package-quarantine), or as part of a package promotion workflow. Scan results are available via the [Web UI](#scan-results-via-the-cloudsmith-web-ui), the [Cloudsmith API](/api/vulnerabilities/read) or even as a [Webhook](/developer-tools/webhooks). ## Supported Formats Cloudsmith's Security Scanning feature is available for the following package formats: - [Cargo](/formats/cargo-registry) - [Composer](/formats/composer-repository) - [Conan](/formats/conan-repository) - [Dart](/formats/dart-repository) - [Docker](/formats/docker-registry) - [GoLang](/formats/go-registry) - [Hex](/formats/hex-repository) - [Maven](/formats/maven-repository) - [npm](/formats/npm-registry) - [NuGet](/formats/nuget-feed) - [Python](/formats/python-repository) - [Ruby](/formats/ruby-repository) - [Swift](/formats/swift-registry) ## Data Sources Vulnerability scanning scans artifacts using the Trivy scanner, which uses TrivyDB as its data source. Below you can find a list of supported data sources: | Language / Framework | Source | | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | C, C++ | 1. [GitLab Advisories Community](https://gitlab.com/gitlab-org/advisories-community) | | Dart | 1. [GitHub Advisory Database](https://github.com/advisories?query=ecosystem%3Apub) | | Go | 1. [GitLab Security Advisories](https://gitlab.com/gitlab-org/advisories-community) 2. [Go Vulnerability Database](https://pkg.go.dev/vuln/) | | Hex | 1. [GitHub Advisory Database](https://github.com/advisories?query=ecosystem%3Aerlang) | | Java | 1. [GitHub Maven Security Advisories](https://github.com/advisories?query=ecosystem%3Amaven) 2. [GitLab Security Advisories](https://gitlab.com/gitlab-org/advisories-community) | | .NET | 1. [GitHub .NET Security Advisories](https://github.com/advisories?query=ecosystem%3Anuget) | | Node.js | 1. [GitHub NodeJS Security Advisories](https://github.com/advisories?query=ecosystem%3Anpm) 2. [NodeJS Ecosystem Security Working Group](https://github.com/nodejs/security-wg) | | PHP | 1. [GitHub PHP Security Advisories](https://github.com/advisories?query=ecosystem%3Acomposer) 2. [Friends of PHP Security Advisories](https://github.com/FriendsOfPHP/security-advisories) | | Python | 1. [GitHub Python Security Advisories](https://github.com/advisories?query=ecosystem%3Apip) 2. [Safety DB](https://github.com/pyupio/safety-db) | | Ruby | 1. [GitHub Ruby Security Advisories](https://github.com/advisories?query=ecosystem%3Arubygems) 2. [Ruby Advisory Database](https://github.com/rubysec/ruby-advisory-db) | | Rust | 1. [RustSec Advisory Database](https://github.com/RustSec/advisory-db) | | Swift | 1. [GitHub Advisory Database](https://github.com/advisories?query=ecosystem%3Aswift) | ### NVD | Name | Source | | :------------------------------ | :------------------------------ | | National Vulnerability Database | 1. [NVD](https://nvd.nist.gov/) | If an advisory does not provide severity information, it falls back to NVD's CVSS scoring. ## Security Scan Results The results of a security scan are available from the Cloudsmith Web UI, the Cloudsmith API and also via a Webhook. ### Scan results via the Cloudsmith Web UI You can find an overview of all the packages that have been scanned, or are awaiting an additional scan, via the "Compliance / Security" page in any repository: The Security Scanning page shows a list of packages scanned, with information about: - Maximum severity of vulnerabilities. - Affected package. - Number of vulnerabilities found. - Time since last scan. For more information and details, you can view the individual vulnerabilities found on the "Security" tab on any package detail page (alternatively, from the overview page click in the **Vulnerabilities Found** card): ### Scan results via the Cloudsmith API You can use the Cloudsmith [Vulnerabilities API endpoints](/api/vulnerabilities/namespace/list) to return the scan results for an entire organization account, a specific repository, an individual package or just a single scan id. ### Scan results via Webhook. Please see our [Webhooks](/developer-tools/webhooks) documentation for details of how to create a webhook and the full range of package events that are supported. ## Performing additional vulnerability scans You can perform additional vulnerability scans after the scan that is performed when the package is uploaded using the Cloudsmith UI or API. ### Additional vulnerability scans via the Cloudsmith Web UI You can request an additional scan for an individual package from the package details page: ### Additional vulnerability scans via the Cloudsmith API You can request an additional vulnerability scan for a package using the [packages_scan API endpoint](/api/packages/scan). ### Early Access: Recurring vulnerability scans You can now also set up vulnerability scans to occur on a recurring basis. This feature is in Early Access. To set up recurring vulnerability scans for your workspace, contact us to be added to Early Access. # Custom domains Your company brand and trust are important to you and your customers, and with custom domains you can present your own company as the endpoint for distribution, APIs, and configuration (e.g. retrieving GPG keys). This is a great idea if you are: - A vendor that is selling and distributing software to your customers - A security-conscious DevOps team that is looking to control the source endpoint for artifacts **Disambiguation** If you are looking for Broadcast URL customization, please refer to the [Broadcasts](/software-distribution/broadcasts) documentation or [contact us](https://cloudsmith.com/company/contact-us). Custom domains are available with our Ultra and Enterprise plans. We strongly recommend - and may require - custom domains for any customer with significant Cloudsmith package delivery traffic. ## Configuration Options ### Account-wide vs per-repository An **account-wide** (also called organizational-level) custom domain works across all repositories in your Cloudsmith organization. When using an account-wide custom domain, the organization/account identifier (slug) will be removed from the URI for a repository. A **per-repository** custom domain works for a single repository in your Cloudsmith organization. When using a per-repository custom domain, the organization/account identifier (slug) and the repository identifier are both removed from the URI for the repository. ### Package support (Download vs Native API) For API-based (Native) package formats, such as NuGet, Docker, Cargo, npm, Conan, Conda, Go and Terraform, you can have a specific custom domain, such as _docker.yourdomain.com_, _npm.yourdomain.com_ or _cargo.yourdomain.com_. For other package formats, and downloads in general, you can have a custom domain such as _dl.yourdomain.com_. It is possible to have multiple download domains for a package type, however, all of them will be accessible through each other, for example, if we set up a general `Downloads` domain for raw packages and a `Debian` domain (which is also Download), we will be able to hit the `Downloads` domain to get Debian packages. Please refer to the table below to see which package will utilize Download/API Native domains. | Package Type | Download | Native Uploads | Native Downloads | |:--- |:--- |:--- |:--- | | Alpine | ✅ | ❌ | ❌ | | Cargo (Rust) | ✅ | ❌ | ✅ | | Chocolatey | - | ✅ | ✅ | | Cocoapods | ✅ | ❌ | ❌ | | Composer | ✅ | ❌ | ✅ | | Conan | - | ✅ | ✅ | | Conda | ✅ | ❌ | ✅ | | CRAN | ✅ | ❌ | ❌ | | Dart | - | ✅ | ✅ | | Debian | ✅ | ❌ | ❌ | | Docker | - | ✅ | ✅ | | Go | ✅ | ❌ | ✅ | | Gradle | ✅ | ✅ | ❌ | | Helm | ✅ | ❌ | ❌ | | Hex | - | ✅ | ✅ | | LuaRocks | ✅ | ❌ | ❌ | | Maven | ✅ | ✅ | ❌ | | npm | - | ✅ | ✅ | | NuGet | - | ✅ | ✅ | | PowerShell | - | ✅ | ✅ | | Python | ✅ | ✅ | ❌ | | Raw | ✅ | - | - | | RPM | ✅ | ❌ | ❌ | | Ruby | ✅ | ✅ | ❌ | | sbt | ✅ | ✅ | ❌ | | Swift | ✅ | ✅ | ❌ | | Terraform | ✅ | ❌ | ✅ | | Unity | - | ✅ | ✅ | | Vagrant | ✅ | ❌ | ❌ | ### Top Level Redirects You can request that the top-level of your domain (e.g. npm.example.com) redirect to a location of your choice. For example, to the UI for your repositories if public, or to your own customer support page. If you'd like to configure this, please include it in your request. ## Setup To configure a custom domain within Cloudsmith, we require you to provide: - A list of domains that you wish to have for each package type (refer to the table above). We recommend having only 1 domain for Download types, but this can be set up to include multiple if needed. - For each domain, do you wish to have an account-wide or per-repository domain - Finally, would you like to have a top level redirect for your domain Example request: ``` Domain: `docker.mycompany.com` (Docker) Account wide No redirect ``` Once created within Cloudsmith, you will need to create two DNS CNAME entries for each domain you wish to configure within your DNS provider. ### Step 1 - The process begins when you [contact us](https://cloudsmith.com/company/contact-us) via support with a list of custom domains you would like to configure. We will review and start processing your request, ensuring your plan is eligible to make use of the custom domain feature. ### Step 2 - After confirming the domains you need, a member of the team will review and configure them within Cloudsmith. Once set, the DNS CNAME entries will be visible in your account (in the "Custom Domains" section of any repository) ### Step 3 - You then add the DNS CNAME entries to your domain DNS records with your DNS provider. The first CNAME is for authorizing the domain, and the second is to make it visible and accessible to your users. - Once you have configured the CNAME's, you will then be able to make use of your custom domains. - Please just [contact us](https://cloudsmith.com/company/contact-us) if you would like to set this up or require any further information. # Workspaces Creating a Workspace in Cloudsmith allows you to configure access for teams, individuals and services that map to your company's organizational structure. Building security and resilience in managing teams and workflows is essential in today's ecosystem. 📘 We refer to your top-level account as **Workspaces**. Depending on the context, you might think of a Workspace as an **Organization**, a **Namespace**, or an **Owner**. These terms all refer to the same concept of a shared, top-level container for your work, but preferably **Workspace** will be used. ## Account Workspaces view Click in the Cloudsmith Logo in the top left corner to see all the Workspaces that you can access with your account. Alternatively browse to: [https://app.cloudsmith.com/workspaces](https://app.cloudsmith.com/workspaces). ### Joining a workspace Here you can see the workspaces you've joined and also accept new invitations: E-mail notifications will be sent after a new invitation. From this page you can also Accept or Reject new invitations. To learn how to invite users to a workspace, review the [Users](/accounts-and-teams/users) documentation. ## Primary Workspace You can access multiple workspaces from a single account. This provides both isolation and flexibility for convenience of your Cloudsmith users. To find out what's your primary workspace, browse to your [workspace settings](https://app.cloudsmith.com/settings/workspaces). **SAML and SCIM** A primary workspace defines the landing page when you login into Cloudsmith. This also have implications in terms of login with [SAML](/authentication/single-sign-on) and [SCIM](/authentication/scim) configuration, please refer to their documentation pages to learn more about how primary workspaces influence your login experience. ### Selecting or changing your Primary Workspace From your Account Settings, click in Workspaces (or alternatively browse to [https://app.cloudsmith.com/settings/workspaces](https://app.cloudsmith.com/settings/workspaces)). Here, you'll be presented with a list of all the Workspaces available for you. The star icon identifies your primary workspace. ## Workspace Overview Page Browse to the [Workspace Overview Page](/workspaces/overview) documentation page to learn more about it. ## Workspace Settings Browse to the [Workspace Settings](/workspaces/workspace-settings) documentation page to learn about the different controls available. ## Multiple Email Management It's often the case that one ends up juggling multiple email accounts and using them simultaneously to access different services. This is often the case if you belong to multiple workspaces, and each provides you with an email address that is specific to it. To help manage and facilitate this, you can now add multiple email addresses to your Cloudsmith User Account. Once an email is verified, you can log in with it. You can also specify your primary email address, which we'll email when contacting you. In addition to that, we're proud to say that we're fully compliant with the latest privacy laws in the EU (the GDPR, which you've likely heard about more often than you'd like to). As such, you can now manage your email communication preferences and choose whether you want to opt-in or opt-out for emails. # Workspace Overview The workspace overview page provides a summary of your package **usage** and **delivery**, with all the information and controls required to understand and manage all your repositories activity. It is organized into several key sections: - **Header Navigation**: access your Workspace Overview, Repositories, Packages, Accounts, Teams, Policies, and Settings. It also includes access to Logs, Usage and Security. - **Workspace Analytics Overview**: provides a high level view of usage and delivery analytics. - **Recent activity**: stay on top of the latest activities within your workspace. ### Header Navigation A primary navigation bar at the top allows you to switch between different sections of the workspace: - **Overview**: The current view, summarizing all activity. - **Repositories**: View and manage all your [repositories](/repositories). - **Packages**: Browse and [manage all packages](/artifact-management/) within your workspace. Use the [search engine](/artifact-management/search-filter-sort-packages) to narrow the view to your desired packages. - **Accounts**: Manage [user and service accounts](/accounts-and-teams/) and [permissions](/workspaces/privileges). - **Teams**: Organize users into [teams](/accounts-and-teams/teams). - **Policies**: Define and manage [EPM policies](/supply-chain-security/epm) for your workspace. - **Settings**: Configure your [Workspace Settings](./workspace-settings). Secondary navigation includes: - **Logs**: View [client and audit logs](/logs-and-observability) for your workspace. - **Usage**: access [analytics](/logs-and-observability/usage) and understand in more detail your package usage and delivery. - **Security**: Get an overview of [security findings](/supply-chain-security) and [license violations](/policy-management/license-compliance). ### Workspace Analytics overview Under the top bar, 4 cards summarize information about: - **Quarantined Packages**: The total number of packages that have been quarantined due to policy violations. - **Vulnerable Packages**: The total count of packages with known vulnerabilities. - **Vulnerability Policy Violations**: The number of instances where packages do not comply with Workspace vulnerability policies. - **License Policy Violations**: The number of packages that violate Workspace license policies. The **Workspace** card provides a snapshot of the entities in the workspace: - **Repositories** - **Packages** - **Container Images** - **Accounts** And also, the main area in your workspace overview page shows two different cards to help you understand in a quick glance the **package usage and delivery** of your users. This card details the data consumption for the workspace: - **Artifact Data**: The total amount of storage used during the current month. - **Package Delivery**: The total amount of bandwidth used during the current month. This section also displays a graph visualizing the accumulated package delivery over the last month. It also provides a breakdown of usage and delivery by repository. To get a detailed view of package usage and delivery, browse to the Workspace Usage section. ### Recent Activity Two lists at the bottom of the page highlight the latest activities: #### Recent Packages This list shows the most recently added or updated packages in the workspace. Each entry includes: - Package name and version. - Repository it belongs to. - Tags. - File size. - Time of the last update and associated user. #### Recent Repositories This list shows the repositories that have been recently created or have had recent activity. ### Cloudsmith Service Status & News A sidebar on the right provides information directly from Cloudsmith: - **Cloudsmith Service Status**: A link to check the operational status of all Cloudsmith services. - **New at Cloudsmith**: A feed of the latest updates, features, and announcements from Cloudsmith. # Personalization Cloudsmith allows you to personalize certain parts of the web app to suit your workflow. Your preferences are saved automatically and persist across sessions. Personalization settings are user-specific. Changes you make to table layouts and column visibility only affect your own view and do not impact other members of your workspace. ## Table customization You can customize the tables throughout Cloudsmith to control which columns are visible, how they are ordered, and how much vertical space rows use. Table customization is available on the following tables: ### Customize columns To open the column customization menu, click the **customize columns** icon (grid icon) in the top-right corner of any supported table. You can also open it by clicking the three-dot menu on any column header and selecting **customize columns**. [Image: Select which columns are visible in the packages table] ### Choosing visible columns In the **customize columns** panel, you will see a list of all available columns under the **Visible** section, each with a toggle to show or hide it. - Toggle a column **on** to make it visible in the table. - Toggle a column **off** to hide it from view. The primary column (for example, **Name and version** in the packages table) cannot be hidden. Its toggle will appear disabled. ### Reordering columns Within the **customize columns** panel, columns can be reordered using the drag handle to the left of each column name. Drag a column up or down to change the order in which it appears in the table. You can also use the **search** field at the top of the panel to quickly find a specific column by name. ### Hiding a column directly You can hide a column by clicking the three-dot menu on any column header and selecting **hide column**. This is equivalent to toggling the column off in the **customize columns** panel. ### Saving and resetting your preferences Once you have made your changes, click **Save** to apply them. To restore the default column configuration, click **Reset**. ### Comfortable and compact view You can switch between two row density options to control how much vertical padding is displayed in table rows: - **Comfortable** — increases vertical padding for easier readability. - **Compact** — reduces vertical padding to display more rows on screen at once. This setting can be found in the **Customize columns** panel or in the table view options. # Workspace Privileges Once your Cloudsmith Workspace [is set up](/workspaces/workspace-create), you can implement a least privilege approach to secure your software supply chain. This is achieved by defining roles and permissions that control access to your artifacts and workspace settings. This page describes these roles and details how they can be modified and tailored for your specific use cases. ## Workspace User Roles Cloudsmith provides the next levels of user privileges, based on four roles: | Role | Permissions | | :--- | :--- | | **Owners** | Can modify all workspace settings, manage other owners or delete the workspace. They are the "root" of the workspace, the only user that can manage all other users (including owners) or delete the workspace, and have full access over all artifacts within the Workspace repositories. | | **Manager** | Can manage workspace settings and all non-owner users and teams. By default, they have no access to artifacts within the Workspace repositories, but it can be assigned to them (see [Default Object Privileges](#default-object-privileges)). | | **Member** | Can see other members and visible teams. They inherit privileges from workspace and team membership. | | **Collaborator** | Can see other team members and inherit privileges from their teams, but they can't access Workspace or Repository settings. | **Account and Service Accounts** Please note that this roles only apply to Account Member Accounts. For Service Accounts, only **Manager** and **Member** roles can be assigned. ### User Role Assignment You can assign any of the roles described above when you [invite users](/accounts-and-teams/user-accounts) to a Workspace: Once assigned, roles can also be modified from the **Accounts** tab in your Workspace. ## Workspace User Team Roles Additionally, users can play different roles in teams: | Role | Permissions | | :--- | :--- | | **Manager** | Can manage team settings and members. Inherits all privileges assigned to the team. | | **Member** | Inherits all privileges assigned to the team. | ### User Team Role Assignment Team roles can be edited for each of the users belonging from the **Teams** tab in your Workspace. When a user is added to a team, you can select its role in the team. ## Global Privileges By default, accounts with the **Member** role assigned have no access over certain Workspace settings. This behaviour can be overridden in the Workspace **Global Privileges** settings. To access this setting, browse to your _Workspace Settings_ and, in the left menu, click _Privileges_. 📘 Please note that this setting applies to **workspace members** only. **Owners and managers** can always perform these actions by default, and **Collaborators** can't be granted this level of privileges. The next extra privileges can be granted to workspace members: - Create new teams. - Invite new users with member or collaborator access role (no manager or owner roles allowed, as this could be use to escalate privileges). - See other member's unredacted email addresses. - Create new repositories. ## Default Object Privileges You can set the default repository privileges for workspace members and managers. **Member and Manager Privileges only** Note that these settings apply only to **Members** and **Managers**. **Owners** always have full administrative access to all repositories. **Collaborators** do not inherit these workspace-level privileges, but you can grant them permissions on a per-repository basis. By default, users are granted no specific repository privileges. You can choose from the following options: | Privilege | Description | | :--- | :--- | | **Admin** | Allows users to manage repository settings, user permissions, and entitlements. Includes Write and Read privileges too. | | **Write** | Allows users to upload, edit, and delete packages within the repository. Includes Read privileges too. | | **Read** | Allows users to view and download packages. This permission is for authenticated users working directly in the system. In contrast, an entitlement token is typically used by automated systems (like a CI/CD pipeline) for programmatic, read-only access without a user account. | | **None** | (Default) No specific repository permissions are granted. | To configure more specific access rules for individual users, teams, or services, see the [Repository Access Control](/repositories/repository-privileges) documentation. # Create a Workspace Creating a _Workspace_ in Cloudsmith allows you to configure access for teams, individuals and machines that map to your company's organizational structure. Building security and resilience in managing teams and workflows is essential in today's ecosystem. This is a quick start guide to the powerful permission system within Cloudsmith and how you can get started. To create a new Workspace: 1. Navigate to "Workspaces" in the global menu 2. Click "Workspaces" 3. Click "Create Workspace" [Image: Create a new Workspace] You are now presented with the "Create a new workspace" form. You are required to enter a name for your Workspace (don't worry, we will check your workspace name is unique for you before creating it). # Workspace Settings To configure the settings for the Workspace, just click on the Settings menu item: ## Workspace Settings The menu on the left is where you can modify/configure Workspace settings, and has the following sections: - Workspace - Billing - Privileges - Accounts and Teams - Authentication - Custom domains - Usage limits - Manage policies ### Workspace The Workspace profile settings are where you configure the Workspace name, avatar and contact email addresses (including billing email address if different from the primary email address): From within the Workspace profile settings, you have the ability to rename your organization's slug/identifier, and if you really need too, you can delete an organization (caution: this is a permanent action and cannot be undone). If you rename the Workspace, then the URI that is used to connect to any repositories will change. This change will affect any users that use the repositories as the URI would no longer be valid. ### Billing The Workspace Billing settings are where you add or change your payment source, view your invoices and select / modify your current plan. ### Privileges The Workspace Privileges settings are where you set global privileges and default object privileges. For a high-level description, visit the [Workspace privileges](/workspaces/privileges) documentation pages. ### Accounts and Teams Workspace Accounts and Teams settings is used to control the default visibility of teams: - **Hidden**: Non-Members are not able to view the team. - **Visible**: Non-Members are able to view the team. ### Authentication Workspace Authentication settings is where you can configure and modify the authentication settings for your users. For a high-level description, visit the [Authentication](/authentication) documentation pages. ### Custom Domains Custom domains let you utilize branded domains for any endpoint. Custom Domains settings are where you can view what custom domains have been configured for your Workspace account. ### Usage Limits The Usage Limits settings are where you can configure additional usage of artifact data and package delivery for the Workspace. Setting a limit of 0GB will disable any overage ### Manage policies Cloudsmith's Policy Management lets your protect your workspace by defining policies that suit your business. Manage Policies settings is where you can configure and manage your workspace policies. Policy types include: - License policies - Vulnerability policies - Authentication policies - Package deny policies Please see [Policy Management](/policy-management) documentation for more details on each policy type. # Connected Repositories **Early Access** Connected Repositories are in Early Access (EA) and are currently supported for Maven, Docker, Python, NPM, Go, Cargo (Rust), NuGet and Helm. Please contact us if you are interested in trying them. Connected Repositories allow you to natively link multiple repositories together within a single repository, making packages from all connected repositories available through the repository that the connections are created on. ## How It Works When a repository has connections, package resolution happens within the context of a single request. There is no redirection between repositories, and packages pushed to any connected repository are immediately visible to consumers of the primary repository. - Packages are available instantaneously from all connected repositories as soon as they are pushed or updated - no indexing required. This happens because cache invalidation is triggered within the primary repository as soon as a package is pushed or updated in any connected repository. - Package resolution happens inside a single request with no redirection, significantly improving performance. Native package management tools receive a flattened blended view of packages from all configured repositories. ## Priority Each connected repository is assigned an integer priority, which determines the order in which repositories are evaluated during package resolution. This is important when the requested package exists in multiple connected repositories. Resolution order follows `1..n`, where `1` is the highest priority. When multiple connected repositories share the same priority value, the configuration creation date is used as a tiebreaker. ## Upstream Inheritance Upstreams configured on connected repositories are automatically available through the primary repository. For example, if Repository A is connected to Repository B, and Repository B has upstreams pointing to PyPI and PyTorch, consumers of Repository A can resolve packages from both upstreams. Packages fetched from inherited upstreams are cached in the repository where the upstream is configured (i.e., Repository B in this example). When blending upstreams across connected repositories, relative priorities are dynamically calculated based on the priority of each connection. For example, if Repository A has two connections each with priority `1` on their respective repositories, the upstream from the higher-priority connection will be assigned relative priority `1`, and the upstream from the lower-priority connection will be assigned relative priority `2`. Packages are always resolved in the following order: 1. Packages within the primary repository. 2. Packages within any of the connected repositories, evaluated in priority order. 3. Upstream packages, evaluated in relative priority order (beginning with upstreams defined on the primary repository). ## Example Repository Configuration [Image: Example Connected Repository Configuration] ## Requirements To create a connected repository configuration, you must have: - **Administrator** privileges on the repository you are creating the connection on (the "primary" repository). - At least **Read** privileges on the repository you are connecting to. This mirrors the permission model used for upstreams. ## Supported Formats | Format | Connected Repositories Support | | :----- | :----------------------------- | | Maven | ✅ | | Docker | ✅ | | Python | ✅ | | NPM | ✅ | | Go | ✅ | | NuGet | ✅ | | Cargo (Rust)| ✅ | | Helm | ✅ | Support for additional formats is planned. ## Configuring Connected Repositories Connected repositories can be configured via the UI or API. ### Using the UI Navigate to the Sources tab on the repository you want to configure as the primary repository where you will find the **Connected Repositories** settings. [Image: Connected Repositories Configuration] Select the target repository and assign it a priority to create the connection. [Image: Connect To A Repository] ### Using the API Connected Repositories [API endpoints](/api/repos/connected/create) are available to manage connections programmatically. Connections can also be configured using the [Cloudsmith Terraform Provider](https://registry.terraform.io/providers/cloudsmith-io/cloudsmith/latest). In summary, the following operations are available: | Operation | Description | | :-------- | :---------- | | Create | Create a new connected repository configuration | | List | List all connected repository configurations for a repository | | Update | Update the priority or target of an existing configuration | | Delete | Remove a connected repository configuration | ## Behavior and Considerations ### Access Control Connecting Repository A to Repository B gives consumers of Repository A implicit access to view and download packages within Repository B via native tooling. ### Audit Logs Audit log entries are created when connected repository configurations are created, updated, or deleted. ### Download Attribution When a package download occurs through the primary repository for a package that belongs to a connected repository, the client log entry is produced for the primary repository. The individual package download count is incremented on the package in the repository where it resides. Client logs do not currently indicate that a package was resolved through a connected repository. Work is planned to add connected repository attribution to download logs. ### Package Signatures Connected repositories do not cache packages. Package signatures are sourced from the repository where the package belongs and are signed using that repository's key. Policies continue to apply to packages within the repository they belong to. ### Nested Connections We do not support the ability to nest connected repositories (i.e. Repository A is connected to Repository B, which is connected to Repository C). All connections must be directly configured on the primary repository. This limitation is purposeful as we believe it is a more intuitive model and nested configurations can almost always be flattened into a non-nested configuration with the same effective resolution order. ## Current Limitations The following is a non-exhaustive list of known limitations during Early Access: - **OSS repositories**: Connected repository configurations cannot be created on OSS repositories, or targeting OSS repositories. - **Cross-workspace**: Connected repositories can only be configured between repositories within the same workspace. - **Download log attribution**: Client logs do not yet attribute package downloads to connected repositories. - **Repository restrictions**: Repository level restrictions such as Geo/IP Rules and EULA enforcement must be defined on the source repository. These restrictions are not inherited from connected repositories. - **Repository package list and search results**: In the Cloudsmith UI and API, packages from configured connected repositories do not appear in the package list or search results for the primary repository, even though they are still resolvable through the primary repository using native package management tools. # Upstream Proxying and Caching Upstream proxying and caching allows you to upload and use the packages you own, while Cloudsmith fetches and caches other packages (such as dependencies). This enables you to use Cloudsmith as a first-class cache and a central source of truth for packages, to protect you from outages of external services (which is especially important when running behind your firewall). ## Upstream Concepts Cloudsmith upstream support centers around several key concepts: - _Proxying_: The act of transparently allowing access to a package that exists on an upstream repository. Package managers see the remote package as one which belongs to the Cloudsmith repository. - _Caching_: An extension to the proxying functionality, where requested packages from an upstream are fetched and permanently stored in your Cloudsmith repository. This helps to ensure package dependencies are always available and helps to protect from upstream outages or security breaches. - _Indexing_: In order to be aware of the packages available from an upstream, Cloudsmith builds an index. This process occurs when an upstream is first added to your Cloudsmith repository and is scheduled for a resync on a regular basis. ### Indexing Index availability is a critical factor for upstream handling on Cloudsmith, helping to ensure deterministic performance for upstream requests and a deeper insight into the availability of packages. The indexing process can differ, depending on the package format and upstream itself. Where possible, Cloudsmith will determine the availability of all packages on an upstream repository _ahead-of-time_, which generally means that an upstream repository is unavailable when first added, until this indexing process has occurred. For package formats that do not maintain a centralized mechanism for retrieval of all packages, Cloudsmith employs a _just-in-time_ indexing mechanism. In this approach, awareness of packages is made the first time a package is successfully cached from an upstream repository. Going forwards, Cloudsmith maintains a list of all versions available on the source upstream for the package and ensures this is kept in sync. When neither indexing mechanism is available for an upstream, Cloudsmith falls back to a _real-time_ unindexed approach. When requests are made for upstream packages, Cloudsmith determines availability across each upstream in your repository, in real-time. This is the least performant approach. We strive to ensure that at least _just-in-time_ indexing is available for each package format and upstream source, although this is not always possible. ### Priority When defining upstreams for a repository, a _priority_ can be specified. The priority of an upstream is used to determine the order in which upstream requests are resolved. Cloudsmith evaluates upstreams by the order of `1..n`. A good approach when determining what priority to apply to upstreams is to ensure that the lowest value is specified for the upstream which is most likely to contain upstream packages you request and that there are no multiple upstreams with the same priority value. This helps to improve performance in the event that an upstream source does not support any of our indexing mechanisms. ## Supported Formats | Format |
    Configurable Proxy
    |
    Caching
    |
    Indexing
    |
    Indexing Type
    |
    API Reference
    | | :----------------------------------------------- | :---------------------------------------------- | :----------------------------------- | :------------------------------------ | :----------------------------------------- | :-------------------------------------------- | | [Alpine](/formats/alpine-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/apk/create) | | [Cargo (Rust)](/formats/cargo-registry) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/cargo/create) | | [Conda](/formats/conda-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/conda/create) | | [Composer](/formats/composer-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/composer/create) | | [CRAN](/formats/cran-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/cran/create) | | [Dart](/formats/dart-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/dart/create) | | [Debian](/formats/debian-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/deb/create) | | [Docker](/formats/docker-registry) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/docker/create) | | [Generic](/formats/generic-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/generic/create) | | [Golang](/formats/go-registry) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/go/create) | | [Gradle](/formats/gradle-repository) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/maven/create) | | [Helm](/formats/helm-chart-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/helm/create) | | [Hex](/formats/hex-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/hex/create) | | [Hugging Face](/formats/hugging-face-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/huggingface/create) | | [Maven](/formats/maven-repository) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/maven/create) | | [npm](/formats/npm-registry) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/npm/create) | | [NuGet](/formats/nuget-feed) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/nuget/create) | | [Python](/formats/python-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/python/create) | | [RedHat](/formats/redhat-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/rpm/create) | | [Ruby](/formats/ruby-repository) | ✅ | ✅ | ✅ |
    Ahead-of-Time
    | [API](/api/repos/upstream/ruby/create) | | [sbt](/formats/sbt-repository) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/maven/create) | | [Swift](/formats/swift-registry) | ✅ | ✅ | ✅ |
    Just-in-Time
    | [API](/api/repos/upstream/swift/create) | You can also create and manage upstreams via the [Cloudsmith API](/api). ## Create an Upstream Proxy ### Using the Quick Configure Wizard Cloudsmith helps you proxying and caching canonical registries in a few clicks. Click the "+ Configure new upstream" button to open the configuration modal. From the navigation, select 'Pre-configured upstreams'. [Image: Create Upstream Quick] Select the Upstream(s) you need and they will be automatically configured for you. You can then pull packages from them, proxying and caching all content within Cloudsmith and applying all the controls you require. ### Using the CLI You can create upstreams without leaving your terminal. For example, you can create a debian upstream with the Cloudsmith CLI. 1. Create a `upstream_config.json` file with the next fields: ```json { "name": "Debian Upstream Demo Docs", "upstream_url": "http://archive.ubuntu.com/ubuntu", "mode": "Cache and Proxy", "distro_versions": ["debian/trixie"], "component": "main", "auth_mode": "None", "priority": 1 } ``` 2. Use the Cloudsmith CLI to create the Debian upstream in your workspace `WORKSPACE` and repository `REPOSITORY`: ```bash cloudsmith upstream deb create WORKSPACE/REPOSITORY ./upstream_config.json ``` This command will return the details about your new configured upstream. Please, refer to the [Config Reference](#supported-formats) column in the supported formats table to review fields available for each of the supported formats. ### Using the manual configuration Click the "+ Configure new upstream" button to open the configuration modal. From the navigation, select 'Configure your own' and then select the format you want to create an upstream for: [Image: Create Upstream button] #### Create a Alpine Upstream [Image: Create Alpine Upstream] | Form Field | Description | | :------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. For Alpine Linux, use the mirror root (e.g., `https://dl-cdn.alpinelinux.org/alpine/`). For Wolfi, use `https://packages.wolfi.dev/os/`. The distribution type (Alpine or Wolfi) is automatically detected from the domain — you do not need to configure it separately. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommend leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible. | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | **Distribution Type Detection** Cloudsmith automatically detects whether an upstream is Alpine Linux or Wolfi based on the URL domain. URLs containing `alpinelinux.org` are treated as Alpine Linux; URLs containing `wolfi.dev` are treated as Wolfi. For unrecognized domains, Alpine Linux is assumed by default. **Alpine and Wolfi are incompatible** Alpine Linux and Wolfi both use the APK package format but are distinct, incompatible distributions. Do not mix Alpine and Wolfi packages in the same Cloudsmith repository — create separate repositories for each distribution. #### Create a Cargo (Rust) Upstream [Image: Create Cargo (Rust) Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Conda Upstream [Image: Create Conda Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. In the example above, the upstream used is Anaconda: `https://repo.anaconda.com/pkgs/main`. Remember to specify the channel when setting your upstream. For other upstreams like Condaforge/Bioconda, the URL would look like: `https://conda.anaconda.org/conda-forge/` or `https://conda.anaconda.org/bioconda/`. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a NPM Upstream [Image: Create NPM Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Ruby Upstream [Image: Create Ruby Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Python Upstream [Image: Create Python Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Maven Upstream [Image: Create Maven Upstream Form] | Form Field | Description | | :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | (Default) Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Cache Only | Requests made for packages that aren't yet in this repository will self-redirect until available. This mode ensures that packages served are guaranteed to be signed with the associated repository signing key | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | GPG key | The source of a package signing key. When a signing key is provided, the Cloudsmith setup script will ensure this signing key is deployed to allow packages available on this upstream to be installed | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | Package caching is only supported for Maven packages that have a `.pom` file present on the upstream source. #### Create a Composer Upstream [Image: Create Composer Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a CRAN Upstream [Image: Create CRAN Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Dart Upstream [Image: Create Dart Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Debian Upstream [Image: Create Debian Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | (Default) Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Distribution Version | The distribution version that packages from the upstream will be associated with. | | Upstream Component | The component to fetch from the upstream. | | Upstream Distribution | (optional) The distribution to fetch from the upstream. Useful for repositories that have custom naming schemes. If left blank, the Distribution Version will be used. | | Source Packages | If selected, source packages will be available from the upstream. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Docker Upstream [Image: Create Docker Upstream Form] | Form Field | Description | | :------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible.**Note**: Docker Hub requires that requests to their service via an upstream proxy be authenticated. As such, when you configure an upstream to Docker Hub, you will be required to provide credentials for authentication. | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Generic Upstream [Image: Create Generic Upstream Form] | Form Field | Description | | :------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Upstream Prefix | Path prefix to prevent file collisions when using multiple upstreams. All files from this upstream will be grouped under this path (e.g., `node_distributions/`). | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible. | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | **HTML-based upstreams** Generic upstreams work with simple HTML-based sources. The upstream must consist of simple, unstyled HTML pages with no JavaScript. Each page should contain only links to package files or subdirectories. Examples include Gradle distributions, Apache releases, and Node distributions. **Authentication requirements** Some upstream sources require authentication to access packages or interact with their APIs. For example, GitHub package registries require token-based authentication for API access. Review the documentation for your upstream source to determine if credentials are needed and configure them accordingly in the upstream settings. **Enterprise Repository Support** Enterprise repository managers like Artifactory and Sonatype Nexus require additional configuration and will not work out of the box. These sources also have practical size limits—repositories with millions of packages may not be processable. [Contact support](https://cloudsmith.com/company/contact-us) to discuss setup for these upstream sources. #### Create a Go Upstream [Image: Create Go Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Helm Upstream [Image: Create Helm Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Hex Upstream Hex upstreams are in Early-Access [Image: Create Hex Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Weighting | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | (Default) Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a NuGet Upstream [Image: Create NuGet Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | (Default) Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a RedHat (RPM) Upstream [Image: Create RedHat Upstream Form] | Form Field | Description | | :------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Weighting | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | (Default) Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Distribution | The distribution version to index from the upstream, such as el/8r or fedora/32. | | Source Packages | If selected, source packages will be available from the upstream. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | GPG Key | The source of a package signing key. When a signing key is provided, the Cloudsmith setup script will ensure this signing key is deployed to allow packages available on this upstream to be installed. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | #### Create a Swift Upstream [Image: Create Swift Upstream Form] | Form Field | Description | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | A descriptive name for this upstream source. A shortened version of this name will be used for tagging cached packages retrieved from this upstream. | | Priority | The weighting of the Upstream source. Upstream sources are selected for resolving requests by sequential order (1..n), followed by creation date. | | Upstream URL | The URL for this upstream source. This must be a fully qualified URL including any path elements required to reach the root of the repository. | | Proxy Only | Proxy requests through to upstream sources in order to match assets that are not present in this repository. | | Cache and Proxy | Proxy the initial request for an asset through to the upstream source and then store (cache) resolved assets in this repository for future requests. | | Verify SSL Certificates | If enabled, SSL certificates are verified when requests are made to this upstream. We recommended leaving this enabled for all public sources to help mitigate Man-In-The-Middle (MITM) attacks. | | Authentication (optional) | Optional credentials that can be provided if the upstream is not publicly accessible | | Headers (optional) | Optional Key-Value headers that can be passed to upstreams with each request. | ## Edit an Upstream Proxy Once you have a list of configured Upstreams, use the action buttons in the right to Delete, Edit, or Disable/Enable an Upstream. Click the "Edit Upstream" button to edit an upstream source. # Customizing a Broadcast The default Broadcast **color scheme** can be customized to match your organization's brand colors. To customize the appearance of your Broadcasts, click the **Theme Tab** from the Broadcasts Overview page. Customizations are applied at the workspace level, meaning the theme will be consistent across all of your broadcasts. In the **Theme** tab, you can modify: - **Logo**: Add your company's logo. - **Colors**: Adjust the background, text, and accent colors. Additionally, you can configure visible metadata for each of the three different user-facing broadcast pages: - **Broadcast overview**: Customize the main broadcast page by showing or hiding the total package and download counts. - **Broadcast page**: View key repository information for your broadcast, including its description, package count, total downloads, and license. - **Package details**: Publish package information, including size, downloads, status, checksums, GPG signature, GPG fingerprint, upload date, uploading user, and the README file. # Broadcasts _Share branded repositories via a public facing URL._ Broadcasts help you distribute your software packages to your end-users through a customizable, secure, and user-friendly interface. It allows you to create a seamless experience for your customers, making it feel as though they are interacting directly with your own infrastructure, rather than a third-party service. Users can browse and discover software artifacts from this public Broadcast page, and download artifacts via the browser, or pull them using native tooling such as their IDE or CI/CD build systems. Cloudsmith Broadcasts offers an enterprise-level range of controls: - **Branded Experience**: Customize the user interface with your own logo and color scheme, ensuring your users have a consistent experience with your brand. - **Metadata**: You decide which metadata is visible to your users. You can hide or show details like package tags, download counts, and more. - **Centralized Analytics**: Get an overview of how your distributed packages are performing with aggregated download statistics for all your broadcasts. **Public repositories** With Broadcasts, you no longer create a "public repository" directly. Instead, you create a standard repository and then enable it to broadcast its contents to the world. If you previously used public or open-source repositories, they have been automatically converted into broadcasts. To ensure a smooth transition from our legacy systems, we have implemented automatic redirects from old public repository URLs to their new broadcast equivalents. **Private Repositories** You can continue distributing artifacts within private repositories using entitlement tokens. ## Broadcasts user-facing pages - **Broadcasts Overview**: displays all repositories broadcasting artifacts inside of your organization. A [custom domain can be configured](#broadcast-custom-domains) for it. - **Broadcast**: displays all the artifacts available within a repository broadcast. Each of the repositories in a Workspace have their own Broadcast user facing page. - **Package**: displays different levels of information for your packages: - **Overview**: this page include instructions for the installation or configuration required for the format of the package, including package metadata and the README. - **Files**: all package files are accessible from this section, including associated metadata as tags, filenames, downloads, size, or checksums. - **Versions**: access information about all available versions of the package. **Customization** Visit the [Broadcast Customization](/software-distribution/broadcasts/broadcasts-customization) documentation page to learn how to configure the feel and look and metadata available in these user facing pages. ## Managing Broadcasts You can manage all aspects of your broadcasts by navigating to the Broadcasts Overview section of the Cloudsmith web app. To get there, click on the Cloudsmith logo in the top-left corner and switch from "Artifact Management" to **Broadcasts**. Alternatively, browse to: [https://app.cloudsmith.com/distribution](https://app.cloudsmith.com/distribution). This section contains three main areas: - **Overview**: Displays aggregated analytics for all of your broadcasting repositories, including total downloads, a list of your most popular broadcasts, and your most popular packages over the last 30 days. - Analytics: - Popular Broadcasts - Popular Packages - Workspace total public and OSS broadcasts count. - **Broadcasts**: Shows a complete list of all your repositories that are being broadcasted. Clicking on a specific repository will show you analytics scoped just to that broadcast. - **Theme**: Allows you to [customize the look and feel of your broadcast](/software-distribution/broadcasts/broadcasts-customization) pages. ### Creating a New Broadcast 1. When you create a new repository, or in the repository main page, you will see a toggle to **Enable broadcast**. 2. Once enabled, you can choose the broadcast type: - **Public**: Makes the repository and its packages freely available for the public to view and pull through the public-facing broadcasts page. Ideal for public SDKs and assets you want to be widely accessible. - **Open Source**: Similar to public, allows you to publicly distribute your open-source projects. Open-source broadcasts are tracked separately from your public/private broadcasts. You get at least 50GB of artifact data + 200GB of package delivery for free, across all open-source broadcasts. - **Private**: A private broadcast allows you to limit access to your distributed artifacts through authentication. Your end-users can access a private broadcast using an entitlement token. Ideal for internal SDKs and premium artifact distribution. Private broadcasts is in early access. When a broadcast has been enabled on your repository, you’ll see a status indicator on the packages tab. You can view the public Broadcast page via your workspace's broadcast URL: ```text https://broadcasts.cloudsmith.com/WORKSPACE/REPOSITORY ``` ### Disabling a Broadcast To disable a Broadcast, press status indicator, and choose "Stop broadcasting" from the drop down menu. When a Broadcast is disabled, the URL associated with it and any package within your repository will not be accessible. ### Broadcast Analytics **Plans** Broadcast analytics are only available for Ultra and Enterprise customers. The Broadcast Overview displays analytics for all Broadcasts in your Workspace. Broadcast analytics allow you to keep track of the following: - The number of downloads across all of your Broadcasts in the last 30 days - The most popular/downloaded broadcasts. - The most popular/downloaded packages. #### Analytics for a specific Broadcast To view analytics specific to a given Broadcast, select that Broadcast. The Overview page for the Broadcast will include total downloads over the last 30 days and the most popular packages downloaded. ### Broadcast custom domains **Plans** Broadcast custom domains are only available for Ultra and Enterprise customers. Broadcasts can be displayed on a custom domain. To do so, please contact Cloudsmith via support. You will need to provide your chosen custom domain, and Cloudsmith will provide you a `CNAME` to add to this domain. Cloudsmith will then let you know when your domain is published and available to use. To learn more about it, visit the [Custom domains]() documentation. **Disambiguation** It is important to understand the two types of custom addresses associated with broadcasts and your repositories, as they serve different purposes. 1. **Broadcast URL**: this is the web address your end-users will visit to browse and discover your packages: - **Default URL Structure**: `broadcast.cloudsmith.com/YOUR_WORKSPACE_NAME/YOUR_REPO_NAME/`. - **Custom URL**: You can have a fully branded URL for your broadcast portal (e.g., `packages.yourcompany.com`). This provides a completely seamless experience for your users. To set this up, you will need to contact our support team. 2. **Workspace Custom Domains**: this is the address used in the command line or build scripts to pull packages with native tooling (e.g., Docker, npm, Maven): - Cloudsmith has long supported custom domains for package downloads (e.g., using `npm.cloudsmith.com` instead of the default cloudsmith domain). - The installation instructions provided on your broadcast pages will automatically use any custom domain you have configured for your workspace. This ensures that your users see your branded domain in the setup instructions they copy. - To learn how to configure it, visit the [Workspace Settings](/workspaces/custom-domains) documentation. # Private Broadcasts Private broadcasts are a secure, access-controlled distribution portal, built on the same infrastructure as public and open-source broadcasts. They allow you to deliver packages to selected partners and internal users through your own broadcast site. To access your broadcasts, visit: ```text https://broadcasts.cloudsmith.com/WORKSPACE ``` ## How private broadcasts differ from public broadcasts | Feature | Public broadcast | Private broadcast | | :----------------------- | :------------------------------------ | :---------------------------------------------- | | **Visibility** | Publicly accessible | Restricted access | | **Access control** | None, open access | Requires authentication via entitlement tokens | | **Branding & analytics** | Supported | Supported | | **Use cases** | Public SDKs, open-source distribution | Internal SDKs and premium artifact distribution | ## Enabling a private broadcast ### New repositories To create a private broadcast when creating a new repository, click the "Enable Broadcast" toggle, then select "Private Broadcast" from the list below. [Image: Create new repository modal] ### Existing repositories If you want to enable a private broadcast on an existing Cloudsmtih repository, go to your repository’s general settings. Scroll down to the broadcasts section and click the "start broadcasting" button. Then select "private broadcast" from the modal. [Image: Settings: Start Broadcasting] When a private broadcast has been enabled on your repository, you’ll see a status indicator on the packages tab. [Image: Broadcast status] ## Accessing a private broadcast Private broadcasts are secured using entitlement tokens. To gain access to the restricted content, follow these steps: 1. **Navigate to the Broadcasts Page:** Go to your main broadcasts page (e.g., `https://broadcasts.cloudsmith.com/WORKSPACE/`), which initially displays all public and open-source broadcasts. 2. **Sign In:** Click the "Sign in" button located in the top right corner of the page. 3. **Authenticate:** A login screen will appear. Authenticate your access by entering the secret key from your unique entitlement token. [Image: Broadcast sign in screen] Once authenticated, you will be able to view and download packages from your authorized private broadcasts. ### Entitlement tokens Managing entitlement tokens is the fundamental access control mechanism for your private broadcasts. These tokens ensure secure, authenticated access for partners, customers, or internal users to view and download restricted artifacts distributed through your broadcast site. Creating a unique entitlement token for each customer or partner is the recommended practice. This allows administrators to: * Monitor individual download activity. * Easily revoke or modify access rights to the broadcast if the customer's subscription or relationship changes. Each entitlement token intended for use with your private broadcasts must have the "Access private broadcasts" setting explicitly enabled. This setting grants the token the necessary permission to authenticate against your private broadcast portal, allowing the user to view and download restricted packages. If this setting is not enabled, the token cannot be used to sign in to the broadcast site. [Image: Entitlement token modal] For comprehensive instructions on how to create and manage these tokens, please consult the official [entitlement tokens documentation](software-distribution/entitlement-tokens). ### Access links An "access link" provides an alternate, direct-link method for users to authenticate and access a private broadcast without manually entering an entitlement token secret key. Customers can embed these direct links into their own applications. This is ideal for a seamless user experience, as end-users can securely access the private repository's artifacts directly without needing to save or manually enter their entitlement token secret key. Here is how you can set up an access link: 1. **Generate a Temporary Token:** Use the [broadcast-token endpoint](/api/broadcasts/create/broadcast/token) to generate a temporary access token. The API requires the secret key from an existing entitlement token as input. 2. **Token Expiration (optional):** The generated access token is temporary, expiring by default after one hour. You can configure this duration via the API when you make the generation request. 3. **Construct the Access Link:** Embed the temporary access token into a URL to create the direct link your users will follow to access the private broadcast. Example format: `https://broadcasts.cloudsmith.com/WORKSPACE/REPO?token={token}` Access links support [Cloudsmith search syntax](/artifact-management/search-filter-sort-packages), allowing you to pre-filter the packages displayed to users. For example, to show only Docker packages, add the query parameter to the access link URL: `https://broadcasts.cloudsmith.com/WORKSPACE/REPO?token={token}&query=format%3Adocker` In this case, `format:docker` is URL-encoded as `format%3Adocker`. # Cooldown Policy **Coming soon** This feature is coming soon. To learn more and register your interest, please [contact us](https://cloudsmith.com/company/contact-us). A cooldown policy hides newly published versions of packages from your repository index until they reach a minimum age based on the package version’s publish date. Packages that do not meet the age requirement are hidden from the index, preventing package managers from accessing them. This protects your supply chain from recently published packages that may carry malware or have not yet undergone sufficient community scrutiny, while automatically resolving to the latest compliant version when one is available. A cooldown policy applies at the Cloudsmith index level and is supported for npm and Python packages that originate from a public upstream source. It can apply to packages proxied directly from an upstream source as well as packages that are cached in your Cloudsmith workspace. **Automatic reversal** If a cooldown policy is updated, disabled, or deleted, packages will be reevaluated. This is different from other policies that only apply to cached packages and do not have automatic reversal. ## How it works Only one cooldown policy can be created per workspace. A cooldown policy is evaluated directly on the package index, which is made up of two types of packages: - Upstream cached packages already in your Cloudsmith workspace. - Packages that originate from an upstream source but are not yet cached in your Cloudsmith workspace. By default, a cooldown policy applies to upstream cached packages already in your Cloudsmith workspace and packages coming directly from an upstream source. When a cooldown policy is enabled, packages that do not meet the configured age requirement are hidden from the index. - Packages that are not yet cached in Cloudsmith and need to be requested directly from a public upstream source are hidden from the index. They will not be downloaded or cached in Cloudsmith. - Packages already cached in Cloudsmith are quarantined and hidden from the index until the cooldown period elapses. Once the cooldown period expires, the package is unquarantined and becomes available for download and visible in the index, provided it does not violate any other active policies. You can exclude upstream packages that are already cached in your Cloudsmith workspace from the policy scope by unchecking the **Apply to packages already in use** option in the **Edit policy** view. Decision logs for a cooldown policy are only generated for upstream packages that are already in your Cloudsmith workspace. They are accessible via the Cloudsmith web app and the Cloudsmith API. Cloudsmith determines a package's age by using metadata retrieved from the upstream source. Where package age metadata is not populated, the cooldown policy will fail open, meaning the package will not be hidden from the index. ### Supported package formats | Format | Description | | :------------ | :---------------------------------------------------------------------------------------------- | | Python | Applies to packages where the `upload-time` field has been populated by the package maintainer. | | npm | Applies to packages where the `time` field has been populated by the package maintainer. | ### Policy configuration You can configure the following fields in your cooldown policy Rego: | Field | Description | | :---------------------------------- | :---------------------------------------------------------------------------------------------- | | **Policy name** | A display name for the cooldown policy. The default policy name can be customized as needed. For package managers where enhanced 403 error messaging is supported, the policy name is returned with any 403 error. | | **Policy description** | A description of the cooldown policy. The default policy description can be customized as needed. For package managers where enhanced 403 error messaging is supported, the policy name is returned with any 403 error.| | **Enabled** | Enable or disable the cooldown policy. | | **Repositories** | The repositories that the cooldown policy applies to. By default, the cooldown policy applies to all repositories in a workspace unless configured otherwise. | | **Formats** | The package formats the policy applies to. Currently supported: `npm`, `python`. | | **Cooldown period** | The minimum age, in days, a package must reach before it appears in the index. Packages published more recently than this value are hidden from the index. | | **Apply to packages already in use**| Checked by default. When unchecked, packages already in Cloudsmith will not be quarantined by the cooldown policy or hidden from the repository index. | A cooldown policy is always set to **Terminal**, which prevents the evaluation of any further policies if this policy matches. This setting cannot be edited. Precedence for a cooldown policy is always set to `0`. Policies within an organization are ordered based on the precedence integer defined in each policy. The policy with the lowest number is evaluated first, then each remaining policy is evaluated in order of ascending precedence. This setting cannot be edited. ## The developer experience What developers experience depends on how their dependencies are specified: - **Compliant version available**: The package manager resolves to the latest compliant version. Non-compliant versions are skipped automatically. - **No compliant version available**: Cloudsmith returns a 404 Not Found or 403 Forbidden error - this is common when pinned versions or a lockfile is used to resolve dependencies. Behavior may vary in different package managers. To resolve this, choose a version that satisfies the cooldown requirement. | Error | Cause | | :---------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------ | | 404 Not Found | The specific pinned version you requested does not exist, or the version (or version within your declared range) is not old enough to be allowed. | | 403 Forbidden | The package has been quarantined, or you do not have the requisite permissions to access the package. | | 200 OK | The requested version is accessible and has been downloaded. | ### Impact on existing packages If caching is enabled on upstreams, packages previously pulled through Cloudsmith are already stored in your workspace. Excluding those from the policy means your existing builds are unlikely to break, because only new package versions entering the workspace are subject to the cooldown period. If a cooldown policy is applied to packages that are already in your workspace, or if upstreams are proxy-only, builds are more likely to break because compliant versions may be quarantined or unreachable. ### Enhanced 403 error messaging For supported package managers, Cloudsmith returns a customizable error message in the terminal or build log when a package is hidden from the index. The message is returned as part of the standard 403 response body and surfaced directly by the package manager, with no client-side configuration required. The message includes: - **Policy name**: The name of the cooldown policy that matched the package. - **Policy description**: The policy description field, which you can use to include internal guidance — for example, who to contact, a link to the exemption process, or a link to internal documentation. - **Policy ID**: The unique identifier for the policy action, for reference and troubleshooting. #### Supported package managers: - uv (Python) - npm ## Creating and managing a cooldown policy via the Cloudsmith web app (recommended method) To create and manage a cooldown policy, you must have access to Policy Manager in your workspace. ### Create a cooldown policy via the Cloudsmith web app (recommended method) Only one cooldown policy can be created per workspace. 1. From your Cloudsmith dashboard, go to the **Policies** tab and click **Create a new policy**. 2. Select **Start from a template**, and then select the **Cooldown period** template. The **Cooldown period** policy template opens. 3. Click **Use template** to create the cooldown policy. When created from a template, your cooldown policy is disabled by default. 4. (Optional) Click the edit icon next to the policy name to customize the policy name and description. 5. Customize the policy options. For guidance on which policy options are configurable, see [Policy configuration](#policy-configuration). 6. Click **Save policy** to save and apply your changes. After the policy is created, it might take a few minutes to take effect on packages in your workspace. [Image: Cooldown policy view] ### Edit a cooldown policy via the Cloudsmith web app (recommended method) To edit an existing cooldown policy, go to **Policies** and select the cooldown policy from the policy list. The policy opens in read-only mode. Click **Edit policy** to make changes. The following fields can be updated after the policy is created: - Policy name - Policy description - Enabled - Repositories - Formats - Cooldown period - Apply to packages already in use For more information about configurable policy fields, see [Policy configuration](#policy-configuration). ### Enable a cooldown policy via the Cloudsmith web app (recommended method) You can enable a cooldown policy from the **Policies** tab of the Cloudsmith dashboard: 1. From the Cloudsmith dashboard, go to the **Policies** tab. 2. In the policy list, find the cooldown policy that you want to enable. Hovering over the toggle next to the policy displays **Click to enable policy**. 3. Click the toggle to enable the policy. ### Disable a cooldown policy via the Cloudsmith web app (recommended method) You can disable a cooldown policy from the **Policies** tab of the Cloudsmith dashboard: 1. From the Cloudsmith dashboard, go to the **Policies** tab. 2. In the policy list, find the cooldown policy that you want to disable. Hovering over the toggle next to the policy displays **Click to disable policy**. 3. Click the toggle to disable the policy. When a policy is disabled, it might take a few minutes to take effect on packages that are already in your workspace. ### Delete a cooldown policy via the Cloudsmith web app (recommended method) You can delete a cooldown policy from the **Policies** tab of the Cloudsmith dashboard: 1. From the Cloudsmith dashboard, go to the **Policies** tab. 2. In the policy list, find the cooldown policy that you want to delete. 3. Click the options menu next to the policy. 4. Click **Delete policy**. 5. In the **Delete policy: DANGER** view that opens, click **Delete** to confirm that you want to delete the policy. When a policy is deleted, it might take a few minutes to take effect on packages that are already in your workspace. ## Create and manage a cooldown policy via the Cloudsmith API To create and manage a cooldown policy, you must have access to Policy Manager in your workspace. A cooldown policy can also be created and managed via the Cloudsmith API by using the following endpoints: - [Create a policy](https://docs.cloudsmith.com/api/workspaces/policies/create) - [Update a policy](https://docs.cloudsmith.com/api/workspaces/policies/update) - [Partially update a policy](https://docs.cloudsmith.com/api/workspaces/policies/partial-update) - [List policies](https://docs.cloudsmith.com/api/workspaces/policies/list) - [Retrieve a policy](https://docs.cloudsmith.com/api/workspaces/policies/retrieve) As with the web app, only the fields listed in [Policy configuration](#policy-configuration) can be edited via the API. Example policy: ```rego package cloudsmith import rego.v1 default match := false within_past_days := 14 supported_formats := {"python", "npm"} included_repositories := {} excluded_repositories := {} include_local_packages := true match if count(reason) != 0 _should_evaluate if { not input.v0.package.is_local } _should_evaluate if { include_local_packages == true } _repo_allowed if { count(included_repositories) == 0 not input.v0.repository.slug in excluded_repositories } _repo_allowed if { input.v0.repository.slug in included_repositories not input.v0.repository.slug in excluded_repositories } reason contains msg if { _should_evaluate _repo_allowed pkg := input.v0.package within_past_days_date := time.add_date(time.now_ns(), 0, 0, 0 - within_past_days) publish_date := time.parse_rfc3339_ns(pkg.upstream_metadata.published_at) publish_date >= within_past_days_date pkg.format in supported_formats msg := sprintf("Package %v within cooldown", [pkg.upstream_metadata.published_at]) } ``` ## Exemptions When creating a cooldown policy, you can exclude specific repositories, formats, and local packages from the policy scope by using the configuration fields described in [Policy configuration](#policy-configuration). For the initial early access release, it is not possible to add exemptions for specific package versions directly. We are treating this as a priority area ahead of general availability, and are gathering user feedback during early access to ensure that we take the right approach. In the meantime, if a specific package version needs to bypass the cooldown policy urgently, we recommend that your security or platform team downloads the required package directly and uploads it into Cloudsmith manually. This bypasses the cooldown requirement and makes the package available to developers immediately. ## Viewing packages that violate your cooldown policy Upstream packages that are already cached in your Cloudsmith workspace and violate your cooldown policy are displayed as **HIDDEN** in the Cloudsmith web app and via the Cloudsmith API. These packages cannot be downloaded, unhidden, or unquarantined. To find packages currently hidden due to a cooldown policy, search your workspace using the search term `status:hidden`: [Image: Hidden package search results] Hovering over the **HIDDEN** status in the web app will display why the package is hidden: [Image: Hidden status] When a cooldown policy is deleted or disabled, packages previously displayed as **HIDDEN** revert to their prior status. After a cooldown policy is created, it may take several minutes for the non-compliant packages in your Cloudsmith workspace to appear as **HIDDEN**. # Getting Started with Enterprise Policy Manager _An introduction to creating Enterprise Policy Manager policies and actions._ --- This guide offers an introduction to creating Enterprise Policy Manager policies and actions. Using this guide, you will: - Create policy matching logic using the Rego language to match packages above a specified vulnerability threshold. - Create a policy via the API that employs this matching logic. - Create actions to quarantine and tag matched packages, and assign these actions to the created policy. - Use the policy simulator to simulate this policy running without impacting any live packages or data. **Policy Creation** You must have administrator permissions within your Workspace to create or update a policy. ## Step 1: Creating policy matching logic with rego Let's create a policy in Rego. This policy will: - Check if any vulnerabilities in the package exceed a CVSS (Common Vulnerability Scoring System) value of 4. - Check if the vulnerability detected was published 10 or more days ago. - Check if there is a patch available for the vulnerability. - Ignore a specified list of CVEs. If all these criteria are met on policy evaluation, and the CVE has not been included on the specific ignore list, the package will be matched (the exported `match` variable will be `true`), and any actions associated with the policy will be run against the package. This policy matching logic is shown below: ``` package cloudsmith default match := false # Define minimum CVSS score threshold max_cvss_score := 4 # Define time-based policy threshold (Vulnerabilities older than 10 days) older_than_days := -10 # Define CVEs to ignore ignored_cves := {"CVE-2023-45853", "CVE-2024-12345"} match if { some target in input.v0.security_scan some vulnerability in target.Vulnerabilities not ignored_cve(vulnerability) vulnerability.FixedVersion vulnerability.Status == "fixed" some _, val in vulnerability.CVSS val.V3Score >= max_cvss_score t := time.add_date(time.now_ns(), 0, 0, older_than_days) published_date := time.parse_rfc3339_ns(vulnerability.PublishedDate) published_date <= t } ignored_cve(vulnerability) if { vulnerability.VulnerabilityID in ignored_cves } ``` ## Step 2: Create a policy using the API **Placeholder values** The example API requests in this guide below make use of placeholder variables for consistency and brevity. It is advised to export the following variables such that they can be used in any example requests: ```shell export CLOUDSMITH_API_KEY= export CLOUDSMITH_WORKSPACE= ``` Save the Rego policy created in Step 1 to a file named `policy.rego`. Then use the script below to create a JSON request body which includes the Rego content in the `rego` field: ```shell escaped_policy=$(jq -Rs . < policy.rego) cat < payload.json { "name": "cvss_gt_4", "description": "Policy to quarantine and tag CVSS > 4", "rego": $escaped_policy, "enabled": false, "is_terminal": false, "precedence": 1 } EOF ``` **Policy testing** When creating a policy via the API, setting the `enabled` field to `false` prevents the policy from being triggered, but still allows it to be tested via the Simulation API. Policies are created using the [workspaces_policies_create](https://api.cloudsmith.io/v2/redoc/#tag/workspaces/operation/workspaces_policies_create) REST API method. The following curl command makes a request to this method, providing in the request the `payload.json` payload created above: ```shell curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d @payload.json ``` A successful request will return an HTTP `201` response, indicating the policy was created. ### Retrieving a policy's unique identifier When a policy is created via the API, the unique identifier for the policy will be provided in the `slug_perm` field in the response body. This identifier will be required, for example, if you need to update the policy or add actions to it. This identifier can be retrieved by directly extracting it from the policy creation response (as in the example below): ```shell POLICY_SLUG=$(curl ... | jq -r '.slug_perm') ``` Or alternatively, the policy identifier can be retrieved via the [workspaces_policies_actions_list](https://api.cloudsmith.io/v2/redoc/#tag/workspaces/operation/workspaces_policies_actions_list) REST API method. ## Step 3: Adding actions to a policy After a policy is created, actions can be assigned to it via the [workspaces_policies_actions_create](https://api.cloudsmith.io/v2/redoc/#tag/workspaces/operation/workspaces_policies_actions_create) REST API method. In this example, two actions will need to be added to the policy: 1. An action to quarantine a package matched by the matching logic. 2. An action to tag a packages matched by the matching logic. ### Adding an action to quarantine a package To create an action to quarantine a package, use the curl command below. This example request specifies the required `action_type`, sets the quarantined package state via the `package_state` field, and provides an action `precedence` value of `1`: A successful request will return a HTTP `201` response, with the unique identifier of the action returned in the `slug_perm` field. ```shell curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/{POLICY_SLUG}/actions/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{ "action_type": "SetPackageState", "precedence": 1, "package_state": "QUARANTINED" }' ``` ### Adding an action to tag a package Similarly, the following curl command can be used to create an action to tag a matched package. This request specifies the required `action_type`, the relevant tag in the `tags` array, and a `precedence` value of `32767`. ```shell curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/{POLICY_SLUG}/actions/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{ "action_type": "AddPackageTags", "precedence": 32767, "tags": ["policy-violated"] }' ``` ## Step 4: Simulating the policy The policy referenced in Step 2 was not enabled when it was created. It is possible to test a policy, even if not enabled, using the simulator [workspaces_policies_simulate_list](https://api.cloudsmith.io/v2/redoc/#tag/workspaces/operation/workspaces_policies_simulate_list) REST API method. The response contains a list of tested packages, whether or not there was a `match` for each page, any reason messages, and the actions that would be taken if the policy is enabled. ## Step 5: Enabling or disabling a policy Once you confirm the policy works as expected, you can enable it via a PATCH request. Enabling the policy means it will run matching logic against packages and apply any associated actions (in this example, quarantining and tagging) to packages that are matched. ```shell curl -X PATCH "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/{POLICY_SLUG}/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{"enabled": true}' ``` Similarly, if a policy needs to be disabled, a PATCH request can be sent for the relevant policy specifying an `enabled` value of `false`. ## Step 6: Pushing a package to trigger the policy Once enabled, if you wish to test the policy against an actual package, you can trigger this policy by: 1. Uploading a vulnerable package to the required repository. 2. Wait for the vulnerability scan to complete. The policy will run at the end of package synchronization. 3. Confirm the package's status and tags in Cloudsmith. ## Checking Decision Logs **Decision Logs** Decision log entries are not added when a policy is simulated via the simulator. When a package is scanned or triggers the policy, EPM creates a decision log entry. Logs are accessed via the decision logs API, which provides two endpoints: a list endpoint returning lightweight summaries, and a retrieve endpoint for fetching the full decision log detail. **List decision log summaries** (the `created_after` parameter is required): ```shell curl -X GET \ "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/decision-logs-v1/?created_after=2026-06-01T00:00:00Z" \ -H "Accept: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" | jq . ``` Each summary in the list includes: `id`, `correlation_id`, `started_at`, `ended_at`, `match`, `package_name`, `package_format`, `policy_name`, and an `actions` summary dict. **Retrieve the full decision log** using the `id` from the list response: ```shell curl -X GET \ "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/decision-logs-v1/$DECISION_LOG_ID/" \ -H "Accept: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" | jq . ``` The full log additionally provides: - `policy_input`: The exact data used to evaluate the policy. - `policy_output`: The results (match or not, partial rule states). For more information on building common matching criteria in Rego, please see the [Enterprise Policy Manager Rego Recipes guide](rego). ## Enterprise Policy Manager **Early Access** Enterprise Policy Manager is in early access; if you would like to try this feature, please [Contact Us](https://cloudsmith.com/company/contact-us). Enterprise Policy Manager, or EPM, provides a way for Workspaces (previously known as Organizations) to define policies that can match and act on packages within Cloudsmith. EPM is implemented on top of [Open Policy Agent](https://www.openpolicyagent.org/) (OPA), a general-purpose and widely adopted policy evaluation engine. OPA provides a high-level declarative language called [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) to define policies as code. ### OPA and EPM EPM allows you to create an OPA policy that triggers when certain events occur within Cloudsmith, ensuring your policies are consistently enforced. When a policy is triggered, Cloudsmith provides the policy evaluation engine with package metadata (input data). Through the rego-based policies, a set of actions associated with the policy (called policy actions) act on the package if the policy is matched. At a high level, the EPM policy evaluation workflow is as follows: 1. An event within Cloudsmith triggers a policy evaluation (for example, completing a package security scan). 2. Package metadata and your rego-based policy are provided to the policy evaluation engine. 3. The policy evaluation engine determines if the policy "matches" the package, based on the logic/criteria provided in your policy. 4. If the policy matches the package, the action(s) associated with the policy are then applied to the package. An example policy, explained in more detail below, quarantines a package following a security scan if vulnerabilities of a certain severity are found within the package. ## Policies Overview - Key Concepts ### Policy Triggers Cloudsmith policies are continuously evaluated, so we detect changes in your security posture in near real time. Certain events will explicitly trigger a policy re-evaluation: - When a package is added to a repository, it goes through a [synchronization process](/artifact-management/resync-a-package#what-is-package-synchronization) that includes evaluating existing policies that may affect the package. Vulnerability Scanning should be enabled for the Workspace. - When a package is resynchronized, provided it wasn't scanned within the 30 minutes before that (failed scans) or 24 hours before (successful scans). - When a package is copied from one repository to another. - When a vulnerability scan is triggered via the UI or API (Note that manual scan requests will only result in a re-scan 30 minutes after the previous scan). - When a [recurring vulnerability scan](/supply-chain-security/vulnerability-scanning#early-access-recurring-security-scans) is triggered. - When Cloudsmith receives updated package data, such as CVSS or EPSS updates ([Continuous Security](../continuous-security)). ### Policies Policies are written in [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/), a declarative language used to define OPA policies. Policies evaluate packages against a set of criteria, such as package metadata, workspace, repository, and security information. ### Policy Matching Policy matching occurs when the policy evaluation engine evaluates a policy against a package and determines that the package matches the criteria outlined in the policy. ### Policy Actions Actions can be assigned to policies to act on packages following policy evaluation. If a package has been matched in the matching step above, the policy actions associated with the policy will then be applied to the package. Multiple actions can be associated with a given policy, and the following actions are supported: - `set_state`. This allows you to control the state of the package. For example, whether to delete or quarantine the package. See the [PackageStateEnum](https://api.cloudsmith.io/v2/swagger/#/:~:text=MovePackageActionTyped-,PackageStateEnum,string,-AVAILABLE%20%2D%20The%20package) for more information. - `add_package_tags`. Add a set of tags to the package. - Note on quarantining action: If you set the action to quarantine, any request for the package returns a **403 Forbidden** error code. For more detail on creating Enterprise Policy Manager policies and actions, see the [Getting Started with Enterprise Policy Manager guide](/supply-chain-security/epm/getting-started). ### Decision Logs The details of every policy evaluation and its outcome are captured in a decision log. Decision logs record the result of policy evaluations, including why the decision was made and what actions were taken as a result. The [Getting Started](/supply-chain-security/epm/getting-started) with Enterprise Policy Manager guide provides more details on creating Enterprise Policy Manager policies and actions. # Enterprise Policy Manager: Rego Recipes ## Recipe 1: Simple Tag Check **Use case**: Match any package with a certain tag. For example: `ready-for-production`. For this rule, the value of `match` will be `true` when any `tag.name` contains the value provided in `required_tag`. ```rego package cloudsmith required_tag := "ready-for-production" default match := false match if { has_required_tag } has_required_tag if { some i input.v0["package"].tags[i].name == required_tag } ``` ## Recipe 2: Time-Based CVSS Policy **Use case**: Evaluate vulnerabilities older than 30 days, check CVSS threshold ≥ 7, filter a specific repo, ignoring certain CVEs. What It Does: - Scopes to the `testing-policy` repository. - Ignores certain CVEs, requires CVSS ≥ 7. - Only triggers if vulnerability is older than 30 days. ```rego package cloudsmith max_cvss_score := 7 older_than_days := -30 target_repository := "testing-policy" ignored_cves := {"CVE-2023-45853", "CVE-2024-12345"} default match := false match if { in_target_repository count(reason) != 0 } in_target_repository if { input.v0.repository.name == target_repository } reason contains msg if { some scan in input.v0.security_scan some vulnerability in scan.Vulnerabilities not ignored_cve(vulnerability) vulnerability.FixedVersion vulnerability.Status == "fixed" some val in vulnerability.CVSS val.V3Score >= max_cvss_score t := time.add_date(time.now_ns(), 0, 0, older_than_days) published_date := time.parse_rfc3339_ns(vulnerability.PublishedDate) published_date <= t msg := sprintf( "CVSS Score: %v | Package: %v | Vulnerability: %v | Reason: %v", [val.V3Score, input.v0["package"].name, vulnerability.VulnerabilityID, vulnerability.Description], ) } ignored_cve(vulnerability) if { vulnerability.VulnerabilityID in ignored_cves } ``` ## Recipe 3: CVSS Score + Fix Version + CVE Exclusion + Repo **Use case**: Another approach for ignoring certain CVEs, focusing on one repository, with high/critical CVSS threshold. What It Does: - Matches packages in repository testing-policy if at least one vulnerability is “fixed,” CVSS > 7, and not in `ignored_cves`. ```rego package cloudsmith max_cvss_score := 7 target_repository := "testing-policy" ignored_cves := {"CVE-2023-45853"} default match := false match if { input.v0.repository.name == target_repository some scan in input.v0.security_scan some vulnerability in scan.Vulnerabilities vulnerability.FixedVersion vulnerability.Status == "fixed" not ignored_cve(vulnerability) exceeded_max_cvss(vulnerability) } exceeded_max_cvss(vulnerability) if { some val in vulnerability.CVSS val.V3Score > max_cvss_score } ignored_cve(vulnerability) if { vulnerability.VulnerabilityID in ignored_cves } ``` ## Recipe 4: CVSS Score + Tag + Time-Based **Use case**: Combine tag requirements with older vulnerabilities that surpass a threshold. What It Does: - Requires package to have a tag containing "internal-only" - Only triggers if a vulnerability is older than 21 days, fixed, and has a CVSS ≥ 7. ```rego package cloudsmith customer_face_tag := "internal-only" max_cvss_score := 7 default match := false match if { has_given_tag count(reason) != 0 } has_given_tag if { some tag in input.v0["package"].tags contains(tag.name, customer_face_tag) } reason contains msg if { t := time.add_date(time.now_ns(), 0, 0, -21) some scan in input.v0.security_scan some vulnerability in scan.Vulnerabilities published_date := time.parse_rfc3339_ns(vulnerability.PublishedDate) published_date <= t vulnerability.FixedVersion vulnerability.Status == "fixed" some val in vulnerability.CVSS val.V3Score >= max_cvss_score msg := sprintf( "CVSS Score: '%v' for Package: '%v' has VulnerabilityID: '%v' with Reason: '%v'", [val.V3Score, input.v0["package"].name, vulnerability.VulnerabilityID, vulnerability.Description], ) } ``` ## Recipe 5: Compare software package license to a list of copyleft licenses **Use case**: Check whether a software package's license is a copyleft license. What It Does: - Defines a set of known copyleft SPDX identifiers. - Triggers if the package's license SPDX identifier matches any of those copyleft licenses. ```rego package cloudsmith default match := false # GNU General Public License (GPL) variants gpl_licenses := { "GPL-1.0-only", "GPL-1.0-or-later", "GPL-2.0", "GPL-2.0-only", "GPL-2.0-or-later", "GPL-3.0", "GPL-3.0-only", "GPL-3.0-or-later", } # GNU Lesser General Public License (LGPL) variants lgpl_licenses := { "LGPL-2.0", "LGPL-2.0-only", "LGPL-2.0-or-later", "LGPL-2.1", "LGPL-2.1-only", "LGPL-2.1-or-later", "LGPL-3.0", "LGPL-3.0-only", "LGPL-3.0-or-later", } # GNU Affero General Public License (AGPL) variants agpl_licenses := { "AGPL-1.0", "AGPL-1.0-only", "AGPL-1.0-or-later", "AGPL-3.0", "AGPL-3.0-only", "AGPL-3.0-or-later", } # Mozilla Public License (MPL) variants mpl_licenses := { "MPL-1.0", "MPL-1.1", "MPL-2.0", } # Common Development and Distribution License (CDDL) variants cddl_licenses := { "CDDL-1.0", "CDDL-1.1", } # Eclipse Public License (EPL) variants epl_licenses := { "EPL-1.0", "EPL-2.0", } # Open Software License (OSL) variants osl_licenses := { "OSL-1.0", "OSL-2.0", "OSL-3.0", } # GNU Free Documentation License (GFDL) variants gfdl_licenses := { "GFDL-1.1-only", "GFDL-1.1-or-later", "GFDL-1.2-only", "GFDL-1.2-or-later", "GFDL-1.3-only", "GFDL-1.3-or-later", } # Creative Commons Share Alike (CC-BY-SA) variants cc_by_sa_licenses := { "CC-BY-SA-1.0", "CC-BY-SA-2.0", "CC-BY-SA-2.5", "CC-BY-SA-3.0", "CC-BY-SA-4.0", } # Other copyleft licenses other_copyleft_licenses := { "QPL-1.0", "Sleepycat", "SSPL-1.0", "copyleft-next-0.3.0", } # Combined copyleft license set copyleft := gpl_licenses | lgpl_licenses | agpl_licenses | mpl_licenses | cddl_licenses | epl_licenses | osl_licenses | gfdl_licenses | cc_by_sa_licenses | other_copyleft_licenses # Main policy rule match if { input.v0.package.license.oss_license.spdx_identifier in copyleft } ``` ## Recipe 6: Detect malicious packages **Use case**: Check if a package was part of a supply chain attack. ```rego package cloudsmith default match := false match if count(malicious_packages) > 0 malicious_packages := [vulnerability.id | some vulnerability in input.v0.osv startswith(vulnerability.id, "MAL-") ] ``` ## Recipe 7: Detect non-compliant licenses used in a Docker image **Use case**: Check whether any components or transitive dependencies of a Docker image have a copyleft license. What It Does: - Defines a set of known copyleft SPDX identifiers. - Triggers if any component of the Docker image has a copyleft license. ```rego package cloudsmith import rego.v1 default match := false # Match components with copyleft SPDX licenses # Comprehensive list of copyleft licenses from https://spdx.org/licenses/ # GNU General Public License (GPL) variants gpl_licenses := { "GPL-1.0-only", "GPL-1.0-or-later", "GPL-2.0", "GPL-2.0-only", "GPL-2.0-or-later", "GPL-3.0", "GPL-3.0-only", "GPL-3.0-or-later", } # GNU Lesser General Public License (LGPL) variants lgpl_licenses := { "LGPL-2.0", "LGPL-2.0-only", "LGPL-2.0-or-later", "LGPL-2.1", "LGPL-2.1-only", "LGPL-2.1-or-later", "LGPL-3.0", "LGPL-3.0-only", "LGPL-3.0-or-later", } # GNU Affero General Public License (AGPL) variants agpl_licenses := { "AGPL-1.0", "AGPL-1.0-only", "AGPL-1.0-or-later", "AGPL-3.0", "AGPL-3.0-only", "AGPL-3.0-or-later", } # Mozilla Public License (MPL) variants mpl_licenses := { "MPL-1.0", "MPL-1.1", "MPL-2.0", } # Common Development and Distribution License (CDDL) variants cddl_licenses := { "CDDL-1.0", "CDDL-1.1", } # Eclipse Public License (EPL) variants epl_licenses := { "EPL-1.0", "EPL-2.0", } # Open Software License (OSL) variants osl_licenses := { "OSL-1.0", "OSL-2.0", "OSL-3.0", } # GNU Free Documentation License (GFDL) variants gfdl_licenses := { "GFDL-1.1-only", "GFDL-1.1-or-later", "GFDL-1.2-only", "GFDL-1.2-or-later", "GFDL-1.3-only", "GFDL-1.3-or-later", } # Creative Commons Share Alike (CC-BY-SA) variants cc_by_sa_licenses := { "CC-BY-SA-1.0", "CC-BY-SA-2.0", "CC-BY-SA-2.5", "CC-BY-SA-3.0", "CC-BY-SA-4.0", } # Other copyleft licenses other_copyleft_licenses := { "QPL-1.0", "Sleepycat", "SSPL-1.0", "copyleft-next-0.3.0", } # Combined copyleft license set copyleft := gpl_licenses | lgpl_licenses | agpl_licenses | mpl_licenses | cddl_licenses | epl_licenses | osl_licenses | gfdl_licenses | cc_by_sa_licenses | other_copyleft_licenses match if { input.v0.sbom != null component := input.v0.sbom.components[_] component.licenses != null license_entry := component.licenses[_] license_entry.license != null license_id := license_entry.license.id license_id in copyleft } ``` ## Recipe 8: Compare package versions **Use case**: Match any package with a specific version range. What It Does: - Evaluates the version of a package against a specified package version. - Triggers if the version of the current package being evaluated is older than the specified package version. ```rego package cloudsmith default match := false match if count(reason) > 0 reason contains msg if { pkg := input.v0.package pkg.name == "h11" # Parse versions using Cloudsmith's semantic version comparator semver.compare(pkg.version, "0.16.0") == -1 msg := sprintf("Package 'h11' version '%s' is older than 0.16.0", [pkg.version]) } ``` ## Recipe 9: Package Publish Date **Use case**: Match new packages for a specific period (e.g. two weeks) after release. Enforcing a time lag before consuming a new package or package version is an effective safeguard to protect against zero-day attacks. Package publish date is currently available for npm, Python, NuGet, Docker, Ruby, Go, Rust, Conda, and Maven. **How we source publish date metadata:** To ensure the accuracy of your policies, we pull metadata from the most reliable sources for each ecosystem: **npm, Python, NuGet, Go, Rust, Ruby & Conda**: Sourced directly from the official registry metadata APIs. **Docker**: Retrieved via the Docker Registry Tags API. **Maven**: Since Maven repositories don't always provide a native "publish date," this is a best-effort calculation based on the Last-Modified HTTP header of the package's POM file on the upstream server. What It Does: - Evaluates the publish date of your npm package. - Triggers if your npm package was published within the past 14 days. ```rego package cloudsmith default match := false # A package is matched if its upstream publish date is within the past N days. within_past_days := 14 supported_formats := {"npm"} match if count(reason) != 0 reason contains msg if { pkg := input.v0.package within_past_days_date := time.add_date(time.now_ns(), 0, 0, 0 - within_past_days) publish_date := time.parse_rfc3339_ns(pkg.upstream_metadata.published_at) # Match if the publish date comes after the date of the set number of days ago. publish_date >= within_past_days_date pkg.format in supported_formats msg := sprintf("Package upstream publish date is %v (falls within the past %v days)", [pkg.upstream_metadata.published_at, within_past_days]) } ``` ## Tag Addition and Removal _An example workflow for adding and removing tags with Enterprise Policy Manager._ --- This example workflow will guide you through the process of adding and removing tags from packages using Enterprise Policy Manager policies and actions. Using this guide, you will: - Create two policies and two associated actions. Our policies will check the maximum severity and exploitability of any vulnerabilities associated with a package - Our first policy will check if a package goes above a specified severity/exploitability threshold and tag it as risky - Our second policy will check if a package goes back below the specified severity/exploitability threshold and remove the tag - We'll create these policies through the API and use the simulator to test - Then we'll discuss how to verify them using a real world package **Policy Creation** You must have administrator permissions within your Workspace to create or update a policy. ## Step 1: Creating policy matching logic with rego we'll create a policy to match any package named my-pkg which exceeds the vulnerability risk threshold we feel comfortable with. Let's create that, as follows, in Rego. Create a file named `exceeds-risk-policy.rego` and add the following content: ```rego package cloudsmith cvss_score := 7 epss_score := 0.5 package := "my-pkg" default match := false match if { count(reason) != 0 } reason contains msg if { input.v0["package"]["name"] == package some vulnerability in input.v0.vulnerabilities vulnerability.epss.score > epss_score some source, cvss_info in vulnerability.cvss cvss_info.V3Score > cvss_score msg := sprintf( " Package tagged as risky: %v ", [input.v0["package"].name], ) } ``` ## Step 2: Create our first policy using the API **Placeholder values** The example API requests in this guide below make use of placeholder variables for consistency and brevity. It is advised to export the following variables such that they can be used in any example requests: ```shell export CLOUDSMITH_API_KEY= export CLOUDSMITH_WORKSPACE= ``` We'll use [jq](https://stedolan.github.io/jq/) to help us create the JSON payload for our API request, and for managing JSON from the command line for the rest of these examples. Jq is a small, industry standard tool. It's not essential but it does solve a lot of headaches. Run the following command to convert our Rego policy into a JSON file containing the payload data we'll send to our API to create the policy. ```shell escaped_policy=$(jq -Rs . < exceeds-risk-policy.rego) cat < exceeds-risk-payload.json { "name": "tag_risk_package", "description": "Policy to tag packages with high severity/EPSS scores as risky", "rego": $escaped_policy, "enabled": false, "is_terminal": true, "precedence": 5 } EOF ``` **Policy testing** When creating a policy via the API, setting the `enabled` field to `false` prevents the policy from being triggered, but still allows it to be tested via the Simulation API. Policies are created using the [workspaces_policies_create](https://api.cloudsmith.io/v2/redoc/#tag/workspaces/operation/workspaces_policies_create) REST API method. The following curl command makes a request to this method, providing in the request the `payload.json` payload created above. The API will return a JSON response containing (amongst other metadata) the slug identifier for the policy which we'll use later. We capture that from the curl output. ```shell RESPONSE=$(curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d @exceeds-risk-payload.json ) POLICY_SLUG=$(echo $RESPONSE | jq -r '.slug_perm') ``` A successful request will return an HTTP `201` response, indicating the policy was created. ### Adding an action to tag a package to our policy Similarly, the following curl command can be used to create an action to tag a matched package. This request specifies the required `action_type`, the relevant tag in the `tags` array, and a `precedence` value of `1`. ```shell curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/${POLICY_SLUG}/actions/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{ "action_type": "AddPackageTags", "precedence": 1, "tags": ["risky"] }' ``` ## Step 3. Testing our policy with the simulator We can test our policy using the Enterprise Policy Manager policy simulator by passing the `POLICY_SLUG` to the simulator API endpoint as follows: ```shell RESPONSE=$(curl -L "https://api.cloudsmith.io/v2/workspaces/${CLOUDSMITH_WORKSPACE}/policies/${POLICY_SLUG}/simulate" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" ) # View our simulator results echo $RESPONSE | jq -r '.results[] | .policy_output, .actions[]' ``` We can inspect the output of the simulator to see if our policy matched any packages, and what actions it would have taken. Note that it includes the `cvss_score` and `epss_score` values as we checked against those in our policy: ```json { "policy_output": { "match": true, "reason": [ " Package tagged as risky: my-pkg" ], "cvss_score": 7, "epss_score": 0.5, "target_package_name": "my-pkg" }, "actions": { "precedence": 1, "slug_perm": "", "created_at": "2025-09-10T15:35:59.970073Z", "updated_at": "2025-09-10T15:35:59.970087Z", "effect": "Added package tags '['risky']'.", "tags": [ "risky" ], "action_type": "AddPackageTags" } } ``` ## Step 4. Calling our policy on a real world package If you wanted to call this policy on a real world package, you could do so by first enabling the policy as follows: ```shell curl -X PATCH "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/${POLICY_SLUG}/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{"enabled": true}' ``` And then... 1. Uploading a vulnerable package to the required repository. 2. Wait for the vulnerability scan to complete. The policy will run at the end of package synchronization. 3. Confirm the package's status and tags in Cloudsmith. ## Step 5. Create our second policy to remove tags Now we'll create a second policy which removes the "risky" tag if the vulnerability risk associated with the package is no longer deemed a concern. As above, create a file named `lower-risk-policy.rego` and add the following content: ```rego package cloudsmith cvss_score := 7 epss_score := 0.5 default match := false match if { count(reason) != 0 } reason contains msg if { input.v0["package"]["name"] == "my-pkg" some vulnerability in input.v0.vulnerabilities vulnerability.epss.score < epss_score some source, cvss_info in vulnerability.cvss cvss_info.V3Score < cvss_score msg := sprintf( " Package untagged as not risky: %v ", [input.v0["package"].name], ) } ``` Then create our correctly formatted JSON payload. Note that we've provided a higher precedence value than our first policy. this ensures that our removal policy will take precedence once it's created. ```shell escaped_policy=$(jq -Rs . < lower-risk-policy.rego) cat < lower-risk-payload.json { "name": "untag_low_risk_packages", "description": "Policy to untag packages if they drop below a certain vulnerability risk threshold", "rego": $escaped_policy, "enabled": false, "is_terminal": true, "precedence": 1 } EOF ``` And then create our second policy using the API: ```shell RESPONSE=$(curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d @lower-risk-payload.json ) POLICY_SLUG=$(echo $RESPONSE | jq -r '.slug_perm') ``` And lastly our action to remove the tag: ```shell curl -X POST "https://api.cloudsmith.io/v2/workspaces/$CLOUDSMITH_WORKSPACE/policies/${POLICY_SLUG}/actions/" \ -H "Content-Type: application/json" \ -H "X-Api-Key: $CLOUDSMITH_API_KEY" \ -d '{ "action_type": "RemovePackageTags", "precedence": 1, "tags": ["risky"] }' ``` At this point you could test this against the simulator as described above if you wish. ## Step 6. How our new policy will interact with packages At this point, you could enable this second policy as described above and if the EPSS and CVSS scores for your package drop it will remove the tag. ### A note on precedence If you look at the policies and actions we created above, you'll see that they come with a `precedence` value. This determines which policy or action runs first, if more than one should apply. In the case of actions, it is only relevant when a policy has more than one action. # Malware detection Malware (*short for malicious software*) refers to any software or code intentionally designed to harm, disrupt, steal data, or compromise a system. It includes: - Viruses - Trojans - Ransomware - Spyware - Backdoors - Keyloggers Malware can be delivered through many vectors—infected email attachments, malicious websites, compromised USB drives, or even software packages. ## Why shift malware detection left into artifact management systems Integrating malware detection into an artifact management system helps stop threats at the source, reducing response time, and maintaining a clean, trusted repository of software components—before they ever touch your codebase or production systems. Defense in depth is a strategy that employs multiple layers of protection to safeguard an organization's assets. Cloudsmith provides several barriers to protect against risky artifacts, including: - [Malware scanning with clamAV](./malware-detection/malware-scanning/) - [Malicious packages detection with OSV.dev](./malware-detection/malicious-packages) # Malicious Packages Malicious packages are artifacts that have been **intentionally crafted by an attacker to cause harm**. When developers unwittingly include these packages in their applications, they open the door to data theft, ransomware, and other damaging attacks for anyone using the affected artifacts. Malicious packages are a subset of malware (it's malicious software) and a delivery vehicle for malware. An example of a malicious package is the PyPI package `django-log-tracker`, which saw its first release in two years in 2024. This release propagated information stealer malware. The threat of malicious packages has grown in recent years, even as public registries introduce stronger security controls. **Software Vulnerabilities (CVE) vs. Malicious Packages** A **software vulnerability**, often identified by a Common Vulnerabilities and Exposures (CVE) ID, is an _unintentional_ flaw in the code. It's a bug that, if exploited by an attacker, could lead to a security breach. These vulnerabilities are typically the result of errors in design or implementation. For example, the **Log4Shell** vulnerability (**CVE-2021-44228**) was a flaw in the popular Log4j logging library for Java that allowed attackers to gain remote code execution over compromised systems. The library itself was not created with malicious intent. ## How malicious packages are introduced in dependencies Malicious actors attempt to breach the software supply chain by disguising harmful packages as legitimate, tricking developers into including them in their projects. Common techniques include: - **Typosquatting**: A new package is published with a name that is a common misspelling of a popular one. For example, publishing `reqeusts` to trick developers who meant to install the legitimate `requests` library in Python. This type of attack has been recently extended to a complete new level: [registry Typosquatting](https://cloudsmith.com/blog/typosquatting-the-ghcr-registry). - **Dependency Confusion**: This technique targets corporate build systems. An attacker discovers the name of a private, internal package (e.g., `acme-auth`) and publishes a malicious package with the same name to a public registry, often with a higher version number, tricking automated tools into downloading it. - **Account Takeover**: Attackers compromise the account of a legitimate package maintainer, then publish a new, malicious version of the trusted package. For instance, if the maintainer of `super-utils` gets their account stolen, an attacker could release version `1.2.4` containing code to steal credit card information. - **Protestware/Malicious Maintainers**: Sometimes, the threat comes from the package maintainers themselves. As an act of protest or malice, they might add harmful code to their own popular library. For example, the maintainer of `node-ipc` added code that would delete files on systems with specific IP addresses as a form of protest. ## OSV and OpenSSF Malicious Packages Cloudsmith uses the **OpenSSF Malicious Packages project** - distributed through [OSV.dev](https://osv.dev/) - as a trusted source for detecting malicious packages in open-source software. The OpenSSF Malicious Packages project is a community-driven effort to collect and document confirmed malicious packages found in the wild across multiple package registries. OSV.dev, initiated by Google, provides a unified, open, and distributed database for vulnerability information, making it easy for developers and automated security tools to consume. By cataloging malicious packages within OSV.dev, the OpenSSF project centralizes intelligence that helps developers and organizations quickly identify threats and strengthen their software supply chains. ## Malicious package detection in Cloudsmith **Plans** Malicious package detection via OSV.dev is available for Ultra and Enterprise customers. The malicious packages dataset is integrated into [Enterprise Policy Manager](/supply-chain-security/epm) and [Continuous Security](/supply-chain-security/continuous-security) (both in Early Access). This allows you to define policies that automatically take action when a malicious package is detected. Here is an EPM policy that matches any package which has been flagged as malicious by Continuous Security, quarantines it, and adds a `MALICIOUS_PACKAGE` tag: ### Example Rego policy To target malicious packages in policy-as-code, check whether the`vulnerability.id` field starts with the prefix `MAL-` - all malicious packages are identified with this string. Here’s an example policy: ```rego package cloudsmith default match := false match if count(malicious_packages) > 0 malicious_packages := [vulnerability.id | some vulnerability in input.v0.osv startswith(vulnerability.id, "MAL-") ] ``` ### Supported Formats Below you can find the package [formats supported](https://github.com/ossf/malicious-packages/tree/main/osv/malicious) by the OSSF Malicious Packages project: | Registry / Manager | Language / Ecosystem | | ------------------ | -------------------- | | `crates-io` | Rust | | `docker` | Docker | | `go` | Go | | `hex` | Hex | | `maven` | Java | | `npm` | JavaScript / Node.js | | `nuget` | .NET | | `pypi` | Python | | `rubygems` | Ruby | # Malware scanning In artifact management, malware scanning is the process of inspecting software packages, container images, and dependencies for malicious code before they are stored in a repository or used in a build. This is a critical step in securing the software supply chain. 📘 Malware scanning via ClamAV is included in all Cloudsmith plans. ## How it works Cloudsmith uses a signature-based detection engine that compares your artifacts against a library of known malware “fingerprints”. Each piece of malware has unique characteristics, or "signatures", that can be used to identify it. A signature might be: - A specific byte sequence from the malware's code - A cryptographic hash of a malicious file - Another distinct pattern unique to the malware Before any artifact is synchronized, its files are scanned. If the scanner detects a match in the malware signature database: - The artifact is flagged as malicious - Synchronization fails, and the package status is set to `FAILED` - The package is blocked from being downloaded, copied, moved, or otherwise used For example, trying to upload a malicious file from a terminal will result in the following output: Even if you retry synchronization, it will fail until the issue is resolved, effectively eliminating the risk. **Sample malware: eicar** To test the scanner without using real malware, you can use the harmless [sample malware file](https://www.eicar.org/download/eicar-com-2/?wpdmdl=8842&refresh=68497c10a78741749646352). This file contains a known signature in the malware database, and you can use it to test any malware scanner. First, upload the file using the Cloudsmith CLI with: ```bash cloudsmith push raw WORKSPACE/REPOSITORY ./eicar.com.txt ``` Now, we can get more information and retrieve package metadata with the API. First, update the `envvars` with your own data, and then get the package identifier (`slug_perm`): ```bash API_TOKEN= WORKSPACE= REPO= curl -s -X GET \ 'https://api.cloudsmith.io/v1/packages/WORKSPACE/REPO/' \ -H 'accept: application/json' \ -H 'X-Api-Key: API_TOKEN' \ | jq '.[] | "\(.name) \(.slug_perm)"' | grep eicar ``` Using the `package id` (in this case `IfwsHhsYbTaB` ), we can retrieve the status reason and more information from its metadata fields: ```bash curl -s -X 'GET' \ 'https://api.cloudsmith.io/v1/packages/WORKSPACE/REPOSITORY/IfwsHhsYbTaB/' \ -H 'accept: application/json' \ -H 'X-Api-Key: API_TOKEN' \ | jq '{ status: .status_reason, malware: .extended_metadata.Malware }' ``` ``` { "status": "FAILED", "malware": "Potentially malicious content detected: Win.Test.EICAR_HDB-1 - See the package Metadata section for more information. Otherwise, please contact us." } ``` Here, the initial scan detected malicious content and automatically blocked synchronization. ## Supported Formats | Package Format | Supported Archive Types | |---|---| | Python | zip (.whl), tar.gz | | Java/Maven | zip (.jar) | | NuGet | zip (.nupkg) | | npm | tar.gz (.tgz) | | Helm | tar.gz (.tgz) | | Conan | zip, tar.gz | | Raw files | Scanned based on [file type](https://docs.clamav.net/appendix/FileTypes.html) | ## Conclusions Malware scanning is a foundational security layer in Cloudsmith. Every artifact is scanned before it's made available—helping protect your infrastructure and end users from compromised code. For additional protections against other subtypes of malware like malicious packages, see [Malicious Packages](./malicious-packages). --- # Guides # Cloudsmith Quickstart Guide In this guide we'll quickly step through just some of the initial steps we recommend when starting with Cloudsmith. --- ### Step 1: Sign up First things first. Sign up for Cloudsmith, and create your first workspace. Sign up to Cloudsmith ### Step 2: Your workspace Take a quick tour around your new workspace. Your Cloudsmith workspace ### Step 3: Pulling a package In this step we'll pull a package from an upstream registry through Cloudsmith. Pulling your first package ### Step 4: Adding a user Cloudsmith works best with teams, so we'll add our first additional user. Adding users ### Step 5: Securing your workspace In the final step of this guide, we'll show you just some of the ways you can make your workspace more secure. Securing your workspace # Guides Welcome to the guides section! Here, we move beyond the basics and show you how to apply Cloudsmith's features to solve real-world problems. Each guide is a practical, step-by-step walkthrough of a specific use case, designed to help you get the most value from our product. Whether you're just getting started or are looking to become a power user, these guides are for you. We recommend starting here to build a strong foundation: **[Getting Started with Cloudsmith](/guides/getting-started)**: First steps with Cloudsmith **[Managing permissions in Cloudsmith: A comprehensive guide](/guides/managing-permissions)**: Learn how Cloudsmith organizes roles and permissions. ## Is there something missing? If you have a specific scenario in mind that isn't covered here, feel free to: - Dive deep into our [API Reference](/api) for technical details. - [Contact us](https://cloudsmith.com/company/contact-us) for personalized help. # Managing permissions in Cloudsmith: A comprehensive guide Secure and efficient management of access is vital for organizations that rely on cloud-based tools. As a cloud-native artifact management platform, Cloudsmith provides robust and flexible permission models to help enterprises maintain security without sacrificing productivity. In this guide, we explore how Cloudsmith organizes [roles and permissions](/accounts-and-teams/about-privileges) to facilitate secure collaboration and streamlined workflows. At the end of this guide, you'll understand how the different layers of permissions can be combined to define a user's ultimate permissions. ## Understanding the Hierarchy At the core of Cloudsmith's access control is the [**Workspace**](/workspaces): a top-level entity that groups things like users, services, teams, policies and, of course, repositories. ### Workspace-Level Privileges These are the foundational settings that apply across your entire workspace. They are configured in **Workspace Settings > Privileges**. #### Workspace User Roles Users in your Cloudsmith workspace can have one of the following roles: | Role | Permissions | | :------------ | :------------------------------------------------------------------------------------------------------------------ | | **Owners** | The "Superusers" of the workspace. They have access to all settings and repositories implicitly. They can manage or delete users, including other Owners. This is the only role that has such broad privileges. | | **Managers** | They have access to workspace settings, can manage the workspace itself and can join teams, but have no repository access by default: managers must be members of teams that have access to a repository (or have explicit access granted as an individual user). | | **Members** | The default role for new users. They can not manage any workspace settings but can see other members and visible teams, and they inherit privileges from the teams they belong to. | | **Collaborators** | A limited user who can only see other team members and inherit privileges from their teams. Cannot access general Workspace or Repository settings. This role can be useful for things like granting a third party, such as a consultant, controlled access to your Cloudsmith workspace. | **Entitlement Tokens** The permissions described above apply to authenticated Cloudsmith users within your workspace. For providing artifact read-only access to external users or services, you should use **Entitlement Tokens**. These tokens provide access to a repository without requiring a Cloudsmith user account. For more information, please see the [**Entitlement Tokens Documentation**](/software-distribution/entitlement-tokens). #### Global and Default Object Privileges Furthermore, you can modify these roles to have additional privileges through Global Privileges and Default Object Privileges. ##### Global Privileges You can modify the Member role through global privileges so that Members can create teams, invite new users, see other users' email addresses and create repositories. Essentially, a subset of the privileges that the Manager role has. ##### Default Object Privileges You can set the default privileges (`Admin`, `Write`, `Read`, or `None`) for **Members** and **Managers** on package repositories. For example, setting this to “Read” will mean that all Managers and Members will have read privileges on all package repositories. The default for this setting is “None” - meaning that all package repository privileges must be explicitly granted to Managers and Members. For a complete breakdown of these settings, please see the [**Workspace Privileges**](/workspaces/privileges) documentation. **Exception: Collaborator role** Note that this setting does not apply to the **Collaborator** role, which must always be explicitly granted package repository privileges - either as an individual or inherited from a team membership. ### Team-based Permissions [Teams](/accounts-and-teams/teams) are a powerful way to manage permissions in Cloudsmith, especially for workspaces with numerous users. Teams allow you to group users together for role-based access at scale, avoiding the complexity of managing repository privileges for individual accounts. If you are using [Single Sign On (SSO)](/authentication/single-sign-on), it is also possible to automate team membership based on groups from your Identity Provider using SAML Group Sync. #### Team Roles When adding users or services to a team, you choose one of the following roles within the team: - **Manager**: can manage the team settings (visible or invisible team) and can add/remove members. - **Member**: only inherits privileges from the team. ### Repository Access Control Cloudsmith supports detailed [privileges for repositories](/repositories/repository-privileges), allowing organizations to define who can access what and to what extent. | Setting | Description | | :---------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Default Privileges** | Sets the default privilege (`Admin`, `Write`, `Read`, `None`) for all workspace **Members** on *this specific repository*, overriding the workspace-wide default. | | **User/Team/Service Privileges** | You can grant a higher level of privilege to specific users, services, or teams, beyond the default. For example, you could give the "CI/CD" team `Write` access while the default for everyone else is `Read`. | | **Repository Privileges** | Fine-tune the *minimum privilege level* required to perform specific actions like `Copy`, `Delete`, or `Resync` packages within this repository. | | **Self Privileges** | Define actions that users can *always* perform on their own packages, regardless of other permissions. Default and User/Team/Service privileges are found within a repository, under “Settings > Access control”. The possible privileges for a package repository are: - **Read**: permits viewing repository contents and downloading artifacts. Ideal for developers who just need to be consumers of the repository. - **Write**: extends privileges to include adding or updating packages, often assigned to services. - **Admin**: grants complete control, including modifying repository settings and managing privileges. In addition, it is possible to fine-tune exactly what these privilege levels can do in a repository, or actions that users can perform on their own packages. To do so, browse to “Repository Settings > General". For example, it is possible to configure a package repository so that accounts with “Write” privileges can copy or move packages, but deleting packages requires “Admin” privileges. This granular approach helps to align with modern DevOps practices, enabling seamless collaboration while maintaining strict security protocols. ## Putting It All Together: Effective Privilege The final or "effective" privilege for any user is the **greatest privilege** they are granted through any of the mechanisms explained above. The system checks for permissions on each of the privilege layers: 1. Privileges assigned directly to the **User** on a repository. 2. Privileges inherited from any **Team** the user is a member of. 3. The **Default Privilege** set on the repository. 4. The workspace-wide **Default Object Privilege**. For example, if the workspace default is `Read`, but a user is on a team that has `Write` access to a repository, that user will have `Write` access. ## Permissions Best Practices 1. Use Teams for Simplified Management: - Create teams aligned with functional groups (e.g., developers, QA, DevOps). - Use SAML Group Sync to automate team memberships based on groups from your IdP. - Assign repository-specific privileges for teams. 2. Leverage Services for Automation: - Create dedicated service accounts for CI/CD tools and restrict their access to only necessary repositories. - Use OIDC to allow services to authenticate to Cloudsmith with short-lived tokens, negating the need to store long-lived service API Keys in external tools. 3. Audit and Review Access Regularly: - Periodically review permissions to ensure compliance with security policies. - Utilize Cloudsmith's [Audit Logs]() to track changes in access control settings. 4. Adopt the Principle of Least Privilege: - Grant users, services and teams the minimum access they need to perform their tasks. For example, “Read” Privileges on a repository for Member accounts, but services have “Write” privileges to publish packages from CI/CD pipelines. This reduces the risk of accidental or malicious actions. ### Why Flexible Access Control Matters Effective access control is a cornerstone of operational security. Cloudsmith's approach ensures that organizations can: - Minimize risks of unauthorized access or data breaches. - Adapt permissions to evolving team structures or projects. - Maintain compliance with industry standards and best practices. By enabling precise control at both organizational and repository levels, Cloudsmith simplifies complex workflows while bolstering security—a must-have for businesses in DevOps, FinTech, and other industries. Whether you're setting up your first repository or managing access for a large team, Cloudsmith provides the tools to do so securely and efficiently. With this guide, you're equipped to harness Cloudsmith's capabilities and ensure that your software supply chain is secure, scalable, and streamlined. --- # API Reference # Error Handling Expected errors that occur when interacting with the API result in a `4xx` error code, which varies depending on the condition. All error responses contain at least a `detail` attribute explaining why the error occurred. The following is a list of some of the most common errors that are encountered and why: ## Not Authenticated A `401 Unauthorized` status code is sent when the resource endpoint expects the user to be authenticated but the user isn't. For example, if sending a request as an anonymous user to an authenticated endpoint the following error is sent as a response: ```bash HTTP/1.0 401 Unauthorized Allow: POST, OPTIONS Content-Length: 58 Server: Cloudsmith MCP { "detail": "Authentication credentials were not provided." } ``` ## No Permissions A `403 Forbidden` status code is sent when the resource endpoint expects the user to have specific permissions, which will be detailed in the resource endpoint documentation. For example, if sending a request as an authenticated user to a resource endpoint in which we don't have permissions (such as one belonging to another user), the following error is sent as a response: ```bash HTTP/1.0 403 Forbidden Allow: POST, OPTIONS Content-Length: 63 Server: Cloudsmith MCP { "detail": "You do not have permission to perform this action." } ``` > 🚧 Variable Messages > > The `detail` field in response can be customised by the resource endpoint, so it might be different than the text shown above. ## Invalid JSON Object Error A `400 Bad Request` status code is sent when the body of the document is expected to be JSON encoded, but the actual content is otherwise. For example, if the body was sent as `foo` then the following error is sent as a response: ```bash HTTP/1.0 400 Bad Request Allow: POST, OPTIONS Content-Length: 68 Content-Type: application/json Server: Cloudsmith MCP { "detail": "JSON parse error - No JSON object could be decoded" } ``` ## Malformed JSON Error A `400 Bad Request` status code is sent when the body of the document is cannot be parsed as syntactically correct JSON. For example, if the body was sent as `{` then the following error is sent as a response: ```bash HTTP/1.0 400 Bad Request Content-Length: 79 Content-Type: application/json Server: Cloudsmith MCP { "detail": "JSON parse error - Expecting object: line 1 column 1 (char 0)" } ``` ## Field Validation Error A `422 Unprocess Entity` status code is sent when the resource endpoint receives a JSON body that is syntactically but not semantically correct. These error responses contain a `fields` attribute that lists the specific fields affected by the error condition. For example, if the resource endpoint expected a filename attribute in the body to be populated but isn't then the following error is sent as a response: ```bash HTTP/1.0 422 Unprocessable Entity Content-Length: 132 Content-Type: application/json Server: Cloudsmith MCP { "fields": { "filename": [ "This field may not be null." ] }, "code": "invalid", "detail": "Invalid input." } ``` The Cloudsmith REST (REpresentational State Transfer) API (Application Programming Interface) provides everything you love about Cloudsmith but in a programmatic machine-accessible RESTful format. We believe in providing a rich API to enable exciting and powerful integrations, hopefully in ways that we couldn't imagine - just like your favourite brick-like toys. ## Versioning The API is versioned to help reduce future compatibility issues if we need to change the API. ## Authentication Most resources provided by the API require some form of authentication, which identifies the client to the API in the context of a particular user. Other resources are accessible anonymously, so they don't need authentication (although they may provide expanded detail for authenticated users). You can use the following methods to authenticate: ### API Key Authentication The most straightforward way to authenticate against the API is by using your personal API key. You can [find your API Key](/accounts-and-teams/api-key) within your User Settings or you can request (or reset) it via the Users Token API Endpoint. Cloudsmith Entitlement Tokens cannot be used to authenticate to the Cloudsmith API. Entitlement Tokens are used to authenticate for package downloads only. You can specify your API Key via the Authorization header when making requests (replacing `key` with your actual API Key): ```bash curl -H "Authorization: token $key" https://api.cloudsmith.io/user/self/ ``` Specifying an invalid API Key will result in an 401 Unauthorized status code, and the body will specify that an invalid token was received. Let's see it in action: ```bash curl -i -H "Authorization: token foobar" -X OPTIONS https://api.cloudsmith.io/user/self/ HTTP/2 401 date: Thu, 09 Mar 2023 16:55:35 GMT server: Cloudsmith MCP {"detail": "Invalid token."} ``` **Using Service Accounts** If your API key is given to someone else they will be able to access the API in its entirety as you. This should be viewed as a security risk and every effort should be taken to protect your API Key from disclosure. If you need to add read-only access, we suggest creating a lesser privileged [service account](/accounts-and-teams/service-accounts) and using that instead for scripts/automation. ## Pagination API requests that return more than one item may be paginated, which simply means the total number of items is split into logical pages, in the same way that a book is split into pages. Each page will have a certain number of items in it from one up to the page limit - empty datasets are never paginated. When pagination is enabled, the following query string parameters are supported: | Parameter | Description | | :--- | :--- | | `page` | The current page of the pagination dataset to view. The page number is 1-based, so omitting or specifying a non-positive number will return the first page. | | `page_size` | The page size to divide the dataset into. The default amount (if not specified) is 30 items per page and the maximum configurable is 500 items per page. | When pagination occurs, the following headers will be represent in API responses: | Header | Description | | :--- | :--- | | Link (based on H+C6988) | Hypermedia links for the previous page (if any) and for the next page (if any). Some of which may require expansion as URI templates. | | X-Pagination-Count | The total number of items in the dataset. | | X-Pagination-Page | The number of the current page. | | X-Pagination-PageTotal | The total number of pages in the dataset. | | X-Pagination-PageSize | The size of each page in the dataset. | The `Link` header can contain the following `rel` values: | Name | Description | | :--- | :--- | | `first` | The link relation to the first page of results. This will only be present if there is more than one page. | | `prev` | The link relation to the previous page of results. This will only be present if the client has requested a page greater than 1. | | `next` | The link relation to the next page of results. This will only be present if the client has requested a page less than the last/final page. | | `last` | The link relation to the last/final page of results. This will only be present if there is more than one page. | Let's see it in action ```bash curl -i -H "Authorization: token $key" 'https://api.cloudsmith.io/package/example/repo/packages/?page=2&page_size=1' ``` ``` HTTP/1.0 200 OK Allow: GET, OPTIONS Content-Type: application/json Link: ; rel="first", ; rel="prev", ; rel="next", ; rel="last" X-Pagination-Count: 3 X-Pagination-Page: 2 X-Pagination-PageTotal: 3 X-Pagination-PageSize: 1 Server: Cloudsmith MCP Date: Sun, 29 Jan 2017 18:40:55 GMT [snip] ``` **Page Sizes / Remainders** If the page size is 100 and the dataset size is 400, then there will be four pages available for retrieval. If the dataset isn't cleanly divided by the page size, the remainder will be on the final page. For example, if the page size is 100 and the dataset size is 350 then the last (4th) page will have 50 items on it. # Rate Limits and Scaling with Cloudsmith Cloudsmith applies rate limits to ensure reliability, security and consistent performance for all users. These limits help protect against abusive or accidental overuse. Most users will never encounter rate limits, and if your workloads require more capacity, we’ll adjust them. [Contact us](https://cloudsmith.com/company/contact-us) and we’ll work with you to configure limits to match your requirements. ## Cloudsmith Web App and API Cloudsmith applies default request limits based on how you access the platform. Authenticated usage is the preferred way to use Cloudsmith. It provides higher throughput, better security, and a more reliable experience for production workloads. The table below shows the baseline defaults: | Description | Rate Limit | | :---- | :---- | | Non-authenticated/anonymous users (by IP address) via API | 1,800 req/hour (0.5 req/sec) | | Authenticated users on a Core plan via API | 5,400 req/hour (1.5 req/sec) | | Authenticated users on a Pro or Velocity plans via API | 10,800 req/hour (3 req/sec) | | Authenticated users on an Ultra or Enterprise plans via API | Custom rate limits apply | | Authenticated [app.cloudsmith.com](http://app.cloudsmith.com) usage | 10,800 req/hour (3 req/sec) | | Package downloads via default domain | 120,000 req/hour (33.33 req/sec) | | Package downloads via [custom domain](https://help.cloudsmith.io/docs/custom-domains) | Custom rate limits apply | ## Package Metadata APIs Cloudsmith provides native endpoints for different package formats (for example, npm, Docker, Maven, NuGet, and more). These endpoints have a default throttle of 1,512,000 requests per hour (per region, per user, and per organisation). This can be customised dynamically, applying to all package formats. **Need higher limits?** Cloudsmith regularly customizes limits to support customer use cases. If your workloads need more capacity, contact our team and we'll be happy to adjust limits to fit your needs. ## Monitoring your usage Each API response includes headers to help you monitor and manage usage. These can be used in automated workflows to prevent disruptions and optimize performance: | Header | Meaning | Example | | :---- | :---- | :---- | | `X-RateLimit-Limit` | The maximum number of requests that the client is permitted to send per hour. | `600` | | `X-RateLimit-Remaining` | The number of requests that are remaining in the current rate limit window. | `588` | | `X-RateLimit-Reset` | The [UTC epoch timestamp](https://www.epochconverter.com/) at which the current rate limit window will reset. | `1485706850` | | `X-RateLimit-Interval` | The time in seconds that client is suggested to wait until the next request in order to avoid consuming too much within the rate limit window. | `0.98256663893` | Let's see it in action: ```shell curl -i http://api.cloudsmith.io/user/self/ HTTP/1.0 200 OK X-RateLimit-Interval: 60.0 X-RateLimit-Limit: 600 X-RateLimit-Remaining: 599 X-RateLimit-Reset: 1485712175 Date: Tue, 5 Aug 2025 12:34:56 GMT ``` If the client has exceeded the rate limit in a particular rate limit window a **429 Too Many Requests** status code will be sent instead of acting upon the request. The body response will be **JSON encoded** and include a detail message: ```shell curl -i http://api.cloudsmith.io/user/self/ HTTP/1.0 429 Too Many Requests Allow: GET, OPTIONS Content-Type: application/json Retry-After: 3304 Vary: Cookie x-content-type-options: nosniff X-Frame-Options: SAMEORIGIN X-RateLimit-Interval: 3303.55762601 X-RateLimit-Limit: 1 X-RateLimit-Remaining: 0 X-RateLimit-Reset: 1485712175 Date: Sun, 29 Jan 2017 16:54:30 GMT { "detail": "Request was throttled. Expected available in 3304.0 seconds." } ``` ## Scaling with Cloudsmith Our goal is to give every team a reliable, predictable platform that can meet its needs, without surprises. Rate limits protect Cloudsmith and our users. If your workflows require more capacity, we’ll work with you to ensure you always have the limits you need.