Upstreams

An upstream is one registry a firewall resolves packages from. Upstreams live in the upstreams array of the firewall config. For behavior and patterns, see Upstreams and caching and Configure upstreams.

Shape

{
  "id": "e2de947a",
  "endpoint": "https://npm.acme.internal",
  "credentials": {
    "username": "",
    "password": "",
    "token": "<encrypted>"
  },
  "internal": true,
  "cached": true
}

Fields

Credentials

The auth scheme is inferred from which fields are set; there is no separate auth-type setting.

• On fetches, token is sent as Authorization: Bearer. Without a token, username/password are sent as HTTP basic auth. If both are set, the token wins. One compat case: Azure Artifacts endpoints (pkgs.dev.azure.com) receive a token as basic auth in addition to the bearer header.
• On publishes forwarded to the publish target, npm follows the fetch behavior above; Maven and NuGet send token as a bearer header (NuGet additionally as X-NuGet-ApiKey) and fall back to username/password as basic auth; PyPI sends a token as __token__ basic auth (the twine convention), or username/password as basic auth.
• Encrypted at rest. API responses and exports never contain the plaintext; they carry a placeholder value that preserves an existing stored credential on import.
• Config import rejects plaintext credential values. Leave placeholders untouched in exported files. A placeholder for a new firewall or upstream resolves to empty because that target has no stored credential; set it separately after import.

Artifact host credentials

Some registries serve metadata and package files from different hosts. Packagist is the current case: its metadata can point a download to a separate artifact host, and most Packagist dist URLs use api.github.com.

Credentials set on the upstream are only ever sent to the upstream's own host. A different host is only given credentials if it has its own entry:

{
  "id": "packagist",
  "endpoint": "https://repo.packagist.org",
  "credentials": { "username": "", "password": "", "token": "" },
  "internal": false,
  "cached": false,
  "artifactCredentials": [
    {
      "host": "api.github.com",
      "credentials": { "username": "", "password": "", "token": "<encrypted>" }
    }
  ]
}

• host is a hostname, optionally with a port (registry.internal:8443), never a URL. It is matched case-insensitively against the download URL's host; when both a port-qualified and a bare entry exist, the port-qualified one wins.
• If no entry matches, the firewall fetches from that host anonymously. Configuration still succeeds, but the host can later refuse a download after its anonymous rate limit is exhausted.
• Set them with PUT /config/<firewall-id>/upstreams/<upstream-id>/artifact-credentials/<host>, which takes a plaintext body and merges per field, exactly like the upstream's own credentials endpoint. Currently there is no bsfw command for it.
• Encrypted at rest, and redacted in API responses, exports, and webhook events, like every other credential.

Composer is the only ecosystem whose downloads use these today. For Composer upstreams, the dashboard provides a field for api.github.com; use the API for another artifact host. See the Composer artifact host token.

Cache behavior

• cached: true requires internal: true. An external upstream with caching enabled is rejected with 400: upstream <id>: caching is only allowed on internal upstreams. The rule holds for the dashboard, the API, and bsfw firewall import alike.
• cached: false disables caching for the upstream, with one override: Maven Central endpoints (https://repo1.maven.org/maven2, https://repo.maven.apache.org/maven2) are always cached. Every other Maven upstream, including a repository manager, follows its own flag.
• Maven -SNAPSHOT artifacts are never cached, even from a cached upstream, so a rebuilt snapshot is always fetched fresh.
• Cache keys include the firewall config version, so a config change is never answered from cache entries built under the previous version. The change itself propagates across the cluster within a few seconds.
• Cache keys include the upstream's credentials, and version listings from an upstream with any credentials set are cached per namespace, for a shorter interval than listings from an upstream without credentials.

Failure behavior

An upstream that cannot be reached, or rejects the firewall's credentials, is skipped during resolution. The firewall answers with what the remaining upstreams returned, so a request another upstream can serve is unaffected, and any version held only by the skipped upstream drops out of the version list.

Upstreams are not copies of each other, so that only helps when another upstream has the package. When the skipped upstream was the only one holding it, or the firewall has a single upstream, nothing is left to serve and the request fails:

• npm, PyPI, NuGet, Go, Composer, and Conda answer as if the package does not exist.
• Maven reports the failure instead, so the client sees a retryable status: an upstream 429 is relayed as 429, anything else becomes 502. This is deliberate, so a failing upstream is not mistaken for a coordinate that was never published.

A failure after resolution, while the artifact itself is being fetched, ends in a server error rather than a not-found. Only an upstream 404 or bytes that have aged out of the cache produce a not-found at that point.

In every case the client error names the package, never the upstream behind it. The skip is recorded in the firewall log as an upstream error entry, naming the upstream and what it returned. An upstream answering that it has no such package is a normal part of resolution and records nothing.

When the upstream is cached and the firewall already holds a copy of what was requested, it can answer from that copy instead of skipping the upstream. That covers a short outage for content already fetched, and nothing else: a DNS failure and an upstream 404 are not covered.

Related

• Concept: Upstreams and caching
• How-to: Configure upstreams, Prevent dependency confusion
• Reference: Firewall config