Ahsoka.Extensions.VideoPlayer 5.9.0

Ahsoka.Extensions.VideoPlayer

Display live camera video on an OpenView Pro (AM67 / j722s) panel from a C# app, with a small service API. Three camera families are supported today, all live streams:

Not yet supported on target: InputType.USB and InputType.FileSource are reserved enum values with no working pipeline yet. File-transport ops (Pause/Resume/Position) are no-ops for live cameras — see API reference.

The model (read this first)

Four ideas cover everything:

1. Players are configured, not constructed. You set up each video feed as a player in the OpenPV Developer Toolkit (its input type, source, and display surface) — no hand-edited config files. Your app never builds a pipeline; it asks the service for the configured players and plays them by ID.

2. One service, one client. The VideoService runs in the app's service host; you talk to it with a VideoServiceClient. PlayVideo starts a feed, StopVideo stops it, StatusUpdated pushes state.

3. The extension decodes; the compositor places. Each playing feed renders to its own IVI surface (via glimagesink + GLIMAGESINK_SURFACE_ID). Where that surface sits on screen (x/y/size/z) is owned by the LayerManager from your package SCREEN config — not by this extension. The extension only needs the SurfaceID.

4. GMSL/Analog vs VCIC differ only in bring-up and control. GMSL/Analog open a local v4l2 capture node that platform bring-up prepared at boot. VCIC has no local node: Play = SOME/IP Subscribe (start the RTP stream) and Stop = UnSubscribe. The C# API you call is identical for all three.

Your App ── VideoServiceClient ──▶ VideoService ──▶ platform impl ──▶ native Ahsoka.VideoPlayer ──▶ GStreamer
   ▲                                                                                                    │
   └────────────── StatusUpdated notifications ◀───────────────────────────────────────────────────────┘
        (feeds render to IVI surfaces; LayerManager places them per your SCREEN config)

Quick start (the whole basic API)

using Ahsoka.Services.Video;

// 0. Host the services in-process (the app that owns the video service must start the service
//    manager). Without it the client connects optimistically and the first request blocks forever.
AhsokaRuntime.StartLocalServiceManager();

var client = new VideoServiceClient();
client.Start();

// 1. The players you configured in the OpenPV Developer Toolkit.
VideoPlayerInfo info = client.GetVideoPlayers();

// 2. Start one (by ID). The source + surface come from its config; you don't pass a URL.
client.PlayVideo(new VideoPlaybackRequest { VideoPlayerID = id, StatusUpdateInMs = 1000 });

// 3. Stop it.
client.StopVideo(new VideoPlayer { VideoPlayerID = id });

Illustrative, not copy-paste-and-run: play the subset of players you actually want (playing every configured player at once would put GMSL, Analog, and VCIC all on screen over each other). That is the complete model for GMSL/Analog. VCIC adds exactly one call (GetVcicStatus, below).

Using GMSL / Analog cameras

The simple case: each camera is a configured player that the service plays on demand.

1. Configure your cameras in the OpenPV Developer Toolkit. Add a video player per camera and set its input type (GMSL or Analog), its source, and its display surface — all in the Toolkit. (For analog cameras, turn on deinterlace; set a horizontal/vertical flip if a camera is mounted mirrored.) You don't edit any config files by hand.

2. Bring-up is automatic. The platform prepares the camera hardware at boot — there's nothing for your app to do. A camera that isn't attached (or is below the supported firmware) is simply skipped, and its player reports Unavailable.

3. Play / Stop by ID. Exactly the Quick Start above — PlayVideo / StopVideo by VideoPlayerID. Deinterlace and flip (if you enabled them) are applied for you.

Using VCIC (Orlaco IP) cameras

Same Play/Stop API, but the feed is an IP camera reached over the network, so two things are added:

1. Cameras must be commissioned once (offline, run by hand — not in your app). Factory Orlaco cameras all share one IP/broadcast identity; commissioning gives each a unique IP (192.168.40.91…94) and stream port. Your host must be on the camera subnet (192.168.40.100). See the bench tooling (orlaco-commission.sh, documented in Tools/README.md); this is a provisioning step, not an API.

2. Gate the UI with GetVcicStatus(). It reports whether the host is on the camera subnet and which cameras are actually present, so you only offer / lay out the attached ones:

VcicStatus s = client.GetVcicStatus();
if (s.HostNetworkReady)
    foreach (var cam in s.Cameras.Where(c => c.Present))
        // play the player whose VideoSource == cam.Source ("VCIC0".."VCIC3")

3. Play / Stop are the same call — but under the hood Play sends a SOME/IP Subscribe (starts the RTP stream), the feed is HW-decoded (v4l2h264dec/wave5) and rendered with glimagesink, and Stop sends UnSubscribe. Cameras stream nothing until you Play them (No-Stream-At-Boot).

Minimal VCIC (no dynamic layout). That's the whole basic model — gate, then Play/Stop the present cameras. They render at the surfaces in your PackageInfo (a static quad); your app never touches the LayerManager. See Demos/Video/Ahsoka.CS.VideoPlayer/VcicSimpleExample.cs for a self-contained copy of exactly this. The promote/quad/2-up reflow in Program.Vcic.cs is optional UX built on top and can be ignored entirely.

4. (Advanced) Tile size drives quality. For VCIC, VideoPlaybackRequest.RenderWidth/Height = the on-screen size the camera is shown at. The extension maps a bigger tile to a higher-bitrate camera ROI (PickVcicRoi) and renders at that size. To change a camera's on-screen size at runtime you re-Play it at the new RenderWidth (a live glimagesink surface can't be resized in place). This is what the demo's promote/quad layout does — it is app-side UX, not part of the video API.

Expect a brief black on a re-played tile. Because a size change is a Stop+Play (the pipeline relaunches and the decoder re-initializes), the affected tile goes black for roughly a second during a promote/demote. Tiles whose size does not change are untouched. (A warm, resize-in-place path is a known future improvement; today, layout changes re-play.)

Rendering & surfaces

• Each feed renders to the IVI surface named by its SurfaceID. The extension exports that surface; it does not know or set the on-screen rectangle.
• The on-screen rectangle (x/y/w/h/z, visibility) lives in your package SCREEN config and is driven at runtime by the LayerManager (LayerManagerServiceClient). The demo moves/reveals surfaces there.
• VideoPlayer.RenderWidth/Height (config default) and VideoPlaybackRequest.RenderWidth/Height (per-play override, wins when set) are the present size — the resolution the GPU downscales to before handing the frame to the compositor. 0 = present native.
• When to set it: GMSL/Analog leave it 0 (present native) — the Core LayerManager source-crops the surface to its on-screen rectangle, so one or two feeds at native size are fine (that's what the shipped config does). VCIC sets it per tile because it runs up to four feeds at once: rendering each at its small on-screen size (not native 960p) keeps all four near frame rate and avoids the display plane re-reading a large source into a small tile. So: leave it 0 for GMSL/Analog; let the VCIC layout drive it for four-up.

API reference

VideoServiceClient:

StatusUpdated notification carries a VideoStatus; its VideoState is a PlaybackState (Unavailable/Ready/Playing/Stopped for live cameras — Paused is FILE-only). TotalTimeInMs/PositionInMs are meaningful only for FILE playback (0 for live cameras).

Not applied: VideoPlayer.Brightness/Contrast/Saturation are currently reserved — set them and nothing happens (see API-REVIEW.md).

The demo (Demos/Video/Ahsoka.CS.VideoPlayer)

A 3-page app: Home → GMSL/Analog page → VCIC page. Entering a page plays that page's cameras; leaving stops them. Read it in this order:

• Program.cs — the shell + the simple model: connect, GetVideoPlayers, page routing, and the GMSL/Analog page (play the page's players, reveal the tiles together, stop on leave). Start here — this is the minimal, copyable pattern.
• Program.Vcic.cs — the advanced VCIC page, kept separate on purpose: the GetVcicStatus gate, present-set resolution, and the promote/quad/2-up dynamic layout driven app-side via LayerManager + re-play-on-tier-change. None of this is required just to show cameras — it's polished UX.

Platforms

• OpenViewLinuxPro (AM67 / j722s, ARM64) — the target for GMSL/Analog/VCIC.
• Windows64 / Ubuntu64 — development. Camera hardware isn't present; the VCIC page is allowed for UI testing and the network probe is skipped.

Troubleshooting

More about developing in OpenPV →