Phase 3: Artifact migration
Configuring legacy platforms as upstreams
Configuring your existing platform as a Cloudsmith upstream lets Cloudsmith fetch artifacts from it on demand, rather than requiring every artifact to be migrated before teams can start working in Cloudsmith. Requests for packages not present in a Cloudsmith repository are served from the upstream source, and can be cached in Cloudsmith for subsequent requests.
This underpins the just-in-time migration strategy, in which artifacts migrate only when they are requested. This page is written so it can be sent to your network and security teams before the migration starts, because their approval is usually the longest lead time in the plan.
What this involves
An upstream is configured on a Cloudsmith repository with the URL of your existing repository, an optional credential, and a mode:
- Cache and Proxy fetches each artifact once from your platform and stores it in Cloudsmith. Use this during a migration. Every package a team requests is migrated as a side effect, and the dependency on your platform shrinks with use.
- Proxy Only passes every request through and stores nothing. It leaves every request dependent on your existing platform remaining available, so it is the wrong mode for a migration.
Upstream support is defined by package format rather than by source platform. Any repository that serves a supported format over HTTPS can act as one. Check each format you intend to proxy against the supported formats table before your plan depends on it.
Network access requirements
Your existing repository must be reachable from Cloudsmith over the internet. If it is not, just-in-time and registry proxy migration are unavailable to you, and you should plan on exporting and importing instead. The same connectivity is needed if the Cloudsmith team runs a bulk migration from Artifactory on your behalf.
What your security team needs to know
- Direction of traffic. Cloudsmith makes outbound requests to your repository. Nothing is required inbound to Cloudsmith from your network.
- Source addresses. Requests come from Cloudsmith's NAT gateway IP ranges and, for cached content, from AWS CloudFront. The current ranges, and a command to fetch the CloudFront ranges, are in Configuring your firewall to accept Cloudsmith and AWS CDN IP ranges.
- Protocol and port. HTTPS on port 443, using the package format's own native interface, the same one your developers' tools use today. Certificate verification is on by default.
- Identifiable requests. Requests carry the user agent
cloudsmith/x.y.z (+https://docs.cloudsmith.com/proxy-agent), so they can be identified in your logs. See Upstream proxy agent. - Credentials. Where your repository is not publicly accessible, a credential is supplied in the upstream configuration and used only for requests to that upstream. Create a dedicated read-only account on your platform for it, scoped to the repositories being migrated, so it can be revoked when the migration ends.
- Request pattern. Where the format allows, Cloudsmith indexes what the upstream offers ahead of time, then fetches individual packages as they are requested. For other formats it learns about packages on first request. See Indexing. Rate limits are respected, with back-off on errors.
- Reversibility. An upstream can be disabled or deleted at any time, and the connection is only needed for the duration of the migration.
Who should be involved
| Role | Why | When |
|---|---|---|
| Network or security team | Approves outbound connectivity from Cloudsmith to your platform and the read-only credential. Often the longest lead time. | Before you choose a strategy. |
| Owner of the existing platform | Creates the read-only account, confirms which repositories are hosted and which are proxies, and monitors load during the migration. | Before configuring upstreams. |
| Platform or DevOps team | Configures the Cloudsmith repositories and upstreams, and switches team tooling over. | Throughout. |
| Cloudsmith team | Validates the upstream configuration, raises limits for the migration window, and monitors the platform. | Before the first upstream goes live. |
Questions to resolve
Will your security team approve the connection?
This is the most common reason the just-in-time and registry proxy strategies become unavailable. Requesting an exception for connectivity to Cloudsmith is an internal process, and lead times of several weeks are common even where the team running the migration is fully engaged.
- Raise the request before you choose a migration strategy. It is the longest lead time in artifact migration.
- Send this page to your network and security teams. Explain that the connection is temporary and exists to keep artifacts available while you migrate.
- Set a date by which you need a decision, and plan an alternative in case it is declined.
What if the connection is not approved?
Some security teams will not grant an exception for external connectivity, and some firewalls block the connection regardless of platform.
- Publish new artifacts to Cloudsmith while your existing platform continues to serve current consumers, then migrate what you need from an export. See Running a bulk import.
- Reduce what you migrate rather than working around the restriction. Fewer artifacts means less that has to cross the boundary at all.
Can your existing platform be configured as an upstream?
Two things determine whether it will work:
- Whether the format is supported. See the supported formats table.
- Whether Cloudsmith can authenticate to it in the way your platform expects. The upstream configuration supports a username and password, a token, or custom request headers. Some hosted registries use authentication schemes these do not cover.
Confirm the authentication method your source requires, and test one repository before scoping the rest around this approach. Where a source cannot be used as an upstream, publish new artifacts to Cloudsmith directly and migrate existing ones from an export.
Configuring the upstream
- Create the Cloudsmith repository that will front the migrated content.
- Add an upstream for each format the source repository serves, pointing at your platform's native endpoint for that repository, with the read-only credential and the mode set to Cache and Proxy. See Create an upstream proxy.
- Install one package through Cloudsmith from a developer machine and confirm it arrives in the repository.
- Switch one team's tooling to the Cloudsmith repository and watch what gets cached.
- When usage settles, bulk import anything that has not been requested but must be kept, then disable the upstream and decommission the source.
Using your legacy platform as a proxy to Cloudsmith
The reverse arrangement is also supported: publish to Cloudsmith only, and configure your legacy platform with Cloudsmith as a remote repository so existing consumers continue to pull through it.
- Artifactory: configure Cloudsmith as a remote repository so existing CD pipelines keep working during the transition, then switch your CD pipelines to Cloudsmith once validated.
- Nexus: configure a proxy repository pointed at Cloudsmith. Only hosted repositories need active migration; proxy repositories can be reconfigured in Cloudsmith to pull from the same upstream sources.
Next steps
Return to Migrating your artifacts for the strategies this configuration supports, or see Upstreams for the full set of upstream options.
