PolyhydraGames.CDN 2.0.0.16

Polyhydra CDN & Retro Media Engine

The primary backend service for the Polyhydra CDN ecosystem, consolidating media management, automated indexing, and retro-gaming metadata enrichment.

Tags

• api
• api-cdn
• dotnet
• cdn
• docs
• testing

Related Repos

• retro-library — trusted DAT/hash ROM and BIOS identity, durable inventory, recovery review, curation, and frontend export manifests. Media projection coordination is tracked by retro-library#6 and Api.CDN#67.
• Media-Metadata
• Api.VideoGames
• Api.RetroAchievements
• Api.Streaming
• core.ffmpeg

Project Structure

• WebAPI (src/WebAPI): Consolidated service for media clips, CDN indexing, and the RetroAchievements proxy.
• RetroMedia Library (src/Api.RetroMedia): Core business logic, parsers, and platform rules.
• UI & Tools: WPF and Core conversion utilities.

Api.RetroMedia interprets EmulationStation gamelist.xml and scraper media layouts. Api.CDN indexes, stores, and publishes CDN assets. Neither component replaces retro-library as the authority for DAT/hash ROM and BIOS identity.

Gitops image digest bump

After publish-image pushes an image on main, its bump-gitops job opens a PR against lancer1977/homelab-gitops (base dev, branch bot/cdn-digest-<short-digest>) that pins the api-cdn image in systems/r620/stacks/cdn/compose.yml to the new sha256 digest (never a mutable tag). Re-runs are idempotent: an existing open PR or an already-pinned digest is a no-op. A human merges the PR and deploy.yml applies it.

The bump-gitops job calls a reusable workflow, lancer1977/homelab-gitops/.github/workflows/bump-image-digest.yml@main (tracked in lancer1977/homelab-gitops#1517). It authenticates as a GitHub App, not a personal access token. It needs two repository Actions values on lancer1977/Api.CDN: the variable GITOPS_BUMP_APP_ID and the secret GITOPS_BUMP_APP_PRIVATE_KEY. Without the private key the job logs a warning and skips; image publishing is unaffected.

GITOPS_BUMP_TOKEN is no longer used. Remove it from lancer1977/Api.CDN once the App path is verified.

Setting up the GitHub App for the gitops bump

The App's private key is a credential. Paste it only into the GitHub UI, never into a file, issue, PR, chat, or command line.

1. Create a GitHub App named polyhydra-gitops-bump at GitHub Settings > Developer settings > GitHub Apps.
2. Under Repository permissions, set Contents to Read and write, Pull requests to Read and write, and Metadata to Read. Grant no other permissions.
3. Leave the webhook disabled.
4. Install the App only on lancer1977/homelab-gitops.
5. Generate a private key from the App's settings page and keep the downloaded file out of any repository.
6. In lancer1977/Api.CDN, open Settings > Secrets and variables > Actions. Add the variable GITOPS_BUMP_APP_ID with the App ID, and add the secret GITOPS_BUMP_APP_PRIVATE_KEY with the full contents of the private key file.
7. In lancer1977/homelab-gitops, open Settings > Actions > General > Access, and select "Accessible from repositories owned by the user lancer1977". This lets the reusable workflow be called from this repo.
8. Verify by re-running the Publish image workflow on main. The bump-gitops job should open a PR against homelab-gitops dev and should not print the skip warning about the missing app key.
9. Rotate by generating a new private key for the App, replacing the GITOPS_BUMP_APP_PRIVATE_KEY secret, and then deleting the old key on the App's settings page.

Documentation

Comprehensive feature and infrastructure documentation can be found in the docs/features directory:

• Docs README
• System Overview
• Folder Structure & Mapping
• WebAPI Features
• RetroMedia Library
• RetroAchievements Integration
• Infrastructure & Deployment
• Roadmap

Getting Started

Local Development

The project uses the modern .slnx solution format.

# Open the solution in VS 2022 or Rider
PolyhydraGames.Api.CDN.slnx

Running the API

cd src/WebAPI
dotnet run

Swagger UI will be available at: http://localhost:5000/swagger

Testing

The project uses NUnit for unit testing. Some tests require a mock CDN directory structure.

# Run the CI-aligned local validation path
scripts/validate.sh

# Run fixture-backed CDN media analysis and artifact checks
scripts/validate-retro-media.sh

# Build a review-only manifest for a local CDN tree
python3 scripts/build-cdn-manifest.py --root /path/to/cdn --output artifacts/cdn/cdn-manifest.json

# Run the WebAPI locally and probe /api/v1/health
scripts/smoke-webapi.sh

# Build a source-SHA-labelled local container and write inspection evidence
SOURCE_SHA="$(git rev-parse HEAD)" scripts/build-local-image.sh

Local CDN fallback for development and validation

When the expected CDN mount paths (for example /mnt/syn2/cdn/roms) are not available on a local workstation, use deterministic fixture or temporary-root validation instead of guessing at live CDN state. Local validation should point the Cdn configuration at a known fixture tree or a disposable temp directory that contains the required gamelist.xml and media samples for the test being exercised.

Required Cdn values for local runs are:

• RootPath: root of the local or fixture CDN tree.
• RetroPath: retro-media subpath when the code path expects one; keep it aligned with the fixture layout.
• ArchiveRoot: writable archive/temp location for archive-related tests or local runs.
• BaseUrl: base URL used when generated links are asserted or smoke-tested.

RetroAchievements (RaUser/RaKey) credentials are only required for live RA/online smoke paths. Do not add or invent secrets for deterministic local tests; remote CDN and RA checks should remain smoke-only and should be documented as such in the validation handoff.

The GitHub contents-client release coverage uses a local response fixture. A live GitHub contents probe is intentionally explicit and excluded from the release gate. To run it against a known public repository, supply all three values and target the integration fixture:

GITHUB_CDN_PROBE_OWNER=<owner> \
GITHUB_CDN_PROBE_REPOSITORY=<repository> \
GITHUB_CDN_PROBE_FOLDER=<folder> \
dotnet test src/Test/PolyhydraGames.Api.CDN.Test.csproj --configuration Release \
  --filter FullyQualifiedName~GithubCdnIntegrationTests

If the intended mount path, runtime host, or credential boundary differs from the existing docs, block and add an explicit note rather than guessing. The setup/readiness rule is: deterministic fixture/temp-root tests prove local behavior; live CDN/RA checks only prove connectivity and current external availability.

Documentation

XML documentation is enabled for the core library. Build the project to generate the .xml documentation file.

For public CDN inventory, classification, and controlled cleanup, see Public CDN hygiene and catalog.

Configuration

Configure the Cdn section in appsettings.json or via environment variables. Environment variables use double underscores, for example Cdn__RootPath.

"Cdn": {
  "RootPath": "/path/to/cdn",
  "SourcePath": "/path/to/roms",
  "RetroPath": "images/retro_v2",
  "ArchiveRoot": "/var/lib/polyhydra/cdn-archive",
  "ScanIntervalMinutes": 60,
  "BaseUrl": "https://your-cdn-url.com"
}

appsettings.json is a checked-in, non-secret baseline. Supply live RetroAchievements and Seq credentials only at runtime; do not add RaKey, RaUser, or SEQ_API_KEY to a tracked JSON file.

Common environment variables for local runs:

• Cdn__RootPath
• Cdn__SourcePath
• Cdn__RetroPath
• Cdn__ArchiveRoot
• Cdn__ScanIntervalMinutes
• Cdn__BaseUrl
• Cdn__RaUser
• Cdn__RaKey
• SEQ_API_KEY when Seq ingestion is enabled
• DataProtection__KeyPath pointing at a persistent, externally managed directory
• DOTNET_NUGET_SIGNATURE_VERIFICATION=false when restoring on Linux hosts that hit the current ReactiveUI/Splat signature-revocation issue
• UPLOAD_STORAGE_PROVIDER=local-filesystem to enable verified local/NFS-backed upload finalization; it requires Cdn__RootPath and is disabled when unset.
• Upload__MaxBytes sets the maximum declared upload size (environment form of Upload:MaxBytes; default 268435456 bytes). Upload draft creation returns 400 for non-positive or over-limit sizes and for an invalid supplied TopicId (must match [A-Za-z0-9_-]{1,64}).