Configuring upstreams
Adding and configuring upstreams for a registry
Upstreams
Linked registries - source and destination for packages
Upstreams are linked registries, used both as a source and destination for packages in your private registries. Examples include the public npm registry, NuGet Gallery or the Maven Central Repository.
Upstreams are used when a Bytesafe registry is unable to fulfill a request for a specific package version directly or to push package versions from the registry to other destination registries.
| Ecosystem | Upstream types | Comment |
|---|---|---|
| npm | Public npm registry | registry.npmjs.org |
| nuget | NuGet Gallery | nuget.org |
| maven | Central Maven Registry | repo1.maven.org |
| pypi | Python Package Index | pypi.org |
| npm, maven, nuget, pypi | Local registry | Other registry in the workspace |
| npm, maven, nuget, pypi | External registry | URL to any compatible registry |
| npm, maven, nuget, pypi | Registry WITHOUT upstreams | Package versions can NOT be pulled into the registry or pushed to target |
| npm | Git repository | Url ending in .git or using GitHub integration |
How upstreams work
Using upstreams users are able to fetch external packages and store them together with private packages in a registry. Enabling secure management of package dependencies within Bytesafe.
If a registry is configured without any upstreams:
- Developers can publish package versions to the private registry
- Package versions can NOT be published from the private registry to public npm or any other registry
- Installing packages from the registry requires the package version exists in the registry, else the installation fails
Adding an upstream to a registry enables a lot of different functions:
- Packages not available in the registry can be pulled from the upstream and made available for the developer
- Packages can be published or pushed from the registry to the upstream
- Packages previously not available in the registry will be stored in the registry, caching and proxying the upstream
How npm install works with upstreams
Using npm install to add a package to your project is actually a three-step process.
- A list of all available versions for the package is downloaded and the result is combined by
npm - A specific version is chosen from the list
- Lastly the actual requested content is downloaded
For a registry without upstreams this request by the npm client will only be processed by the single registry.
When adding one or more upstreams on the other hand, the request for all available package versions will be passed along to all upstream registries
and npm will receive information that includes the versions available in the upstream registries.
A version is lastly selected from this extended list of available package versions. If the requested version is not available in Bytesafe, it will be pulled from upstreams as needed.
If the package being installed is an internal package, only upstreams containing versions of the package flagged as internal will be considered (all other upstream sources will be hidden).
Upstream use cases and examples
How to best configure upstreams to suit an organization is not always straight forward.
Visit the Use cases section of the documentation for examples configurations as well as insight into ways upstreams can be used to automate work- and package flows.
Example: Use case where Bytesafe registry is used as a proxy registry and the public npm registry is configured as an upstream.
flowchart TD
A(fab:fa-npm registry.npmjs.org) <--> | pull / push | B{Bytesafe registry} <--> | install / publish | C[fa:fa-user Project workspace]
- Users in the project workspace interact with Bytesafe registry (e.g. npm install / publish).
- Bytesafe handles the interaction with registry.npmjs.org.
- Requests for package versions that are not available in Bytesafe registry, will be requested from registry.npmjs.org and the package version downloaded will be cached in Bytesafe.