Encoder OSD Module
Supported ICs: [RTL8735C]
Overview
The encoder OSD component (video_osd) lets an AP application draw a timestamp, an ARGB logo, and polygon privacy masks directly on the H.264/HEVC encoded output, and lets NN detection results share the same overlay hardware without colliding with them. The public headers are:
video_osd.h— timestamp and logo configuration, placement, and start/stop/show/hide APIs.video_osd_stream.h— the adaptive stream manager: init, status, deinit, memory budget, and error codes.video_osd_privacy.h— polygon privacy mask scene, capabilities, and apply/get/status/deinit APIs.nn_osd.h(invideo_nn_utils) — NN box/label/landmark drawing and canvas-group helpers.
These four headers are the only APIs an application needs. Internal headers such as video_osd_internal.h, video_osd_plan.h, and the compositor/polygon rasterizer sources implement the region planner and pixel blending described in Coexistence and Region Planning; do not call them directly.
All four features draw onto one encoded stream (H.264 or HEVC). Snapshot/JPEG output, raw NV12/RGB streams (V4/V5), and any pre-encoder recording path are not covered — see Limitations and Security.
AP / VP video pipeline. OSD composition happens on the VP encoder side; the AP side only builds the region list and submits it.
Quick Start
The reference example is mmf2_video_example_v1_osd_init.c, which exports two VIDEO app table entries:
mmf2_video_example_v1_osd_uvc_init— 720p H.264 over UVC (requiresCONFIG_USBD_UVC).mmf2_video_example_v1_osd_rtsp_init— the same scene over RTSP (requires networking).
It also exposes a console command, VAPP_OSD, with two sub-commands:
VAPP_OSD update— moves the concave privacy polygon at runtime.VAPP_OSD stop— blocks output and releases every OSD resource.
Three feature switches near the top of the example turn timestamp, logo, and privacy on or off independently:
#define V1_PRIVACY_ENA 1
#define V1_TIMESTAMP_ENA 1
#define V1_LOGO_ENA 1
All three can be disabled at once; the manager itself does not require any client. NN is not one of these three switches — it is a separate producer (nn_osd) that shares the same manager once a stream is initialized; see NN Coexistence.
Complete start/stop flow
#include "video_osd.h"
#include "video_osd_privacy.h"
#include "video_osd_stream.h"
/* 1. Open video_module with output blocked, then CMD_VIDEO_APPLY.
* The encoder must be running before OSD can be configured, but no
* unmasked frame may reach the sink while the gate is still closed. */
video_output_gate_t output_gate = { .blocked = 1 };
mm_context_t *video_v1_ctx = mm_module_open(&video_module);
mm_module_ctrl(video_v1_ctx, CMD_VIDEO_SET_OUTPUT_GATE, (int)&output_gate);
mm_module_ctrl(video_v1_ctx, CMD_VIDEO_SET_PARAMS, (int)&video_v1_params);
mm_module_ctrl(video_v1_ctx, MM_CMD_SET_QUEUE_LEN, 6);
mm_module_ctrl(video_v1_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
mm_module_ctrl(video_v1_ctx, CMD_VIDEO_APPLY, 0);
/* 2. Initialize the shared stream manager BEFORE any client. */
video_osd_stream_t osd_stream = {0};
video_osd_stream_config_t stream_config = {
.target = v1_osd_target, /* stream_id/codec/width/height, must match video_v1_params */
.memory_budget = 8U * 1024U * 1024U, /* 0 = unlimited; see Memory Model */
};
video_osd_stream_init(&osd_stream, &stream_config);
/* 3. Add whichever clients are needed: privacy, logo, timestamp (any subset).
* video_osd_privacy_apply_sync() retries EBUSY before acceptance, then
* waits for the encoder to apply that exact revision -- no manual polling. */
video_osd_privacy_t privacy = {0};
video_osd_privacy_config_t privacy_config = { .target = v1_osd_target };
video_osd_privacy_init(&privacy, &privacy_config);
video_osd_privacy_scene_t scene;
video_osd_privacy_scene_default(&scene);
/* ... fill scene.polygons[] ... */
uint32_t revision = 0;
int ret = video_osd_privacy_apply_sync(&privacy, &scene, &revision, 5000);
if (ret != 0) {
/* Do not report the update as successful; keep the context and handle
* the error (see Error Handling and Shutdown for the ETIMEDOUT case). */
}
/* Logo/timestamp use the matching *_sync entries: each already waits for
* its own initial revision, so no extra per-feature wait is needed here. */
video_osd_logo_t logo = {0};
video_osd_logo_config_t logo_config;
video_osd_logo_config_default(&logo_config);
logo_config.target = v1_osd_target;
logo_config.argb8888 = my_logo_argb;
logo_config.width = MY_LOGO_WIDTH;
logo_config.height = MY_LOGO_HEIGHT;
logo_config.stride_bytes = MY_LOGO_WIDTH * sizeof(uint32_t);
video_osd_logo_show_sync(&logo, &logo_config, 5000);
video_osd_timestamp_t timestamp = {0};
video_osd_timestamp_config_t ts_config;
video_osd_timestamp_config_default(&ts_config);
ts_config.target = v1_osd_target;
video_osd_timestamp_start_sync(×tamp, &ts_config, 5000);
/* 4. This final wait is not a required second confirmation for the three
* calls above -- it only obtains a common applied_timestamp for the
* output gate, also covering any clock/NN update that raced in between. */
video_osd_stream_status_t status;
video_osd_stream_wait_applied(&osd_stream, 0, 5000, &status);
/* status.last_error == 0 and status.applied_revision reached above expected */
/* 5. Open the sink, then open the output gate using the applied timestamp
* to drop any frame produced before the mask/logo/timestamp were live. */
output_gate.blocked = 0;
output_gate.filter_timestamp = status.applied_revision != 0;
output_gate.minimum_timestamp = status.applied_timestamp;
mm_module_ctrl(video_v1_ctx, CMD_VIDEO_SET_OUTPUT_GATE, (int)&output_gate);
mm_module_ctrl(video_v1_ctx, CMD_VIDEO_FORCE_IFRAME, 0);
/* 6. Shutdown: block output, stop the downstream sink/linker, then call the
* four *_sync removal APIs in reverse order, checking each return value.
* See Error Handling and Shutdown for the exact order and retry rules. */
Read mmf2_video_example_v1_osd_init.c for the full sectioned setup (mmf2_video_example_v1_osd_start()), including the UVC/RTSP sink selection and the SISO link to it. Copy the sectioned structure — output gate, stream manager, clients, sink, gate reopen — rather than only the individual API calls.
Public APIs
Only these APIs are customer-facing. Everything else in video_osd_internal.h, video_osd_plan.*, and video_osd_compositor.c is implementation detail for the region planner and pixel compositor.
Stream manager (video_osd_stream.h)
Function |
Description |
|---|---|
|
Create the shared manager for one H.264/HEVC stream. Call before any logo/timestamp/privacy/NN client on that stream. |
|
Read revisions, pending state, last error, memory counters, and region/index counts. |
|
Wait until |
|
Release the manager. Returns |
|
Retry |
Error codes returned by these and the privacy APIs:
Constant |
Meaning |
|---|---|
|
Invalid argument or geometry, a NULL context, or a sync removal called on an unmanaged direct-overlay client; the previous accepted state is unchanged. |
|
Allocation failed. |
|
Nothing was submitted; a previous transaction is still pending. Still returned by the asynchronous APIs; the |
|
The VP encoder rejected the region list, or a completion reports an error, at runtime. |
|
Not enough hardware overlay indices or memory budget for this request. |
|
A |
All error values are declared in video_osd.h; including video_osd_stream.h also makes them visible. video_osd_privacy.h provides matching VIDEO_OSD_PRIVACY_* aliases for every value, including VIDEO_OSD_PRIVACY_ETIMEDOUT.
Timestamp and logo (video_osd.h)
Function |
Description |
|---|---|
|
Fill camera defaults: local time, |
|
|
|
Start (or restart) the timestamp overlay without waiting for encoder confirmation; returns 0 once the request is accepted. |
|
Stop the clock task, remove the overlay, and wait for encoder-confirmed removal before returning 0. See Synchronous Convenience APIs. |
|
Stop the timestamp overlay. |
|
|
|
Show (or replace) the logo overlay without waiting for encoder confirmation. Copies every row; the caller’s ARGB buffer may be released once this call returns. |
|
Same completion/timeout contract as |
|
Hide the logo overlay. |
Privacy (video_osd_privacy.h)
Function |
Description |
|---|---|
|
Query polygon/vertex limits and a worst-case ARGB canvas size estimate for a target. |
|
Zero-initialize a scene with |
|
Create a privacy client on a stream. Auto-creates a stream manager if none exists yet, for compatibility; explicit stream ownership through |
|
Replace the entire scene, retrying |
|
Replace the entire scene without waiting for encoder confirmation. Returns 0 for accepted, not applied — poll status for |
|
Read back the last accepted (not necessarily applied) scene. |
|
Same fields as |
|
Remove masking and wait for safe retirement, including an auto-created manager if privacy owns it. See Synchronous Convenience APIs. |
|
Remove masking. May return |
NN overlay (nn_osd.h, in video_nn_utils)
Function |
Description |
|---|---|
|
Allocate one 1bpp canvas per color layer and assign OSD hardware indices. |
|
Protect canvases while a detection callback redraws them. |
|
Clear and draw rectangles, points, and text on a layer’s canvas. |
|
Submit all layer canvases for this frame. |
|
Feed the optional stale-overlay watchdog; call only when |
|
Release canvases, watchdog, and OSD indices. |
Timestamp
video_osd_timestamp_config_t fields:
Field |
Type |
Description |
|---|---|---|
|
|
|
|
|
Corner ( |
|
|
A |
|
|
|
|
|
Epoch seconds below which the clock is treated as unsynchronized; see Time Source. |
|
|
Rendered text buffer capacity. Default is |
|
|
1–3. The bitmap font is fixed-size; scale multiplies pixel dimensions. |
|
|
Text color and opacity, in YUV. Default is opaque white ( |
Format macros:
Macro |
Example output |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Important
The default max_chars (19) fits the 24-hour formats exactly. Any 12-hour format (_12 suffix) adds `` AM``/`` PM`` and needs at least 22 characters; the example sets V1_TIMESTAMP_MAX_CHARS explicitly for this reason. video_osd_timestamp_start_sync() rejects a format that does not fit max_chars.
Custom format strings are accepted as long as they fit within max_chars and are shorter than the internal format-string capacity; use a plain strftime() pattern such as "%H:%M" for a compact clock.
video_osd_timestamp_config_t config;
video_osd_timestamp_config_default(&config);
config.target = v1_osd_target;
config.placement.position = VIDEO_OSD_POSITION_LOWER_RIGHT;
config.placement.margin_x = 16;
config.placement.margin_y = 16;
config.format = VIDEO_OSD_TIMESTAMP_FORMAT_YMD_DASH_24;
config.time_zone = VIDEO_OSD_TIME_LOCAL;
config.max_chars = 19; /* use >= 22 for a _12 (AM/PM) format */
config.font_scale = 3;
video_osd_timestamp_start_sync(×tamp, &config, 5000);
Time source
video_osd_timestamp_start_sync() reads the system clock with time(). On this platform time() stays pinned at the Unix epoch until NTP (or another time source) sets it. Before synchronization, the timestamp overlay falls back to system uptime counted from 1970-01-01 00:00:00 rather than displaying a placeholder string. Synchronizing the clock is the application’s responsibility; the OSD component does not perform NTP itself.
Logo
video_osd_logo_config_t fields:
Field |
Type |
Description |
|---|---|---|
|
|
Same target struct as timestamp/privacy. |
|
|
Same corner/margin/custom placement model as timestamp. |
|
|
Pixel source, one |
|
|
Logo dimensions in pixels. |
|
|
Source row stride in bytes. Pass |
The logo format is fixed at ARGB8888; there is no bitmap or NV12 logo input path. Alpha is per-pixel, so a logo can mix fully opaque, fully transparent, and semi-transparent pixels in the same image. A logo drawn over a privacy mask blends with the mask’s opaque fill color, not with the camera pixels underneath it — see Coexistence and Region Planning.
video_osd_logo_config_t config;
video_osd_logo_config_default(&config);
config.target = v1_osd_target;
config.placement.position = VIDEO_OSD_POSITION_UPPER_LEFT;
config.placement.margin_x = 16;
config.placement.margin_y = 16;
config.argb8888 = my_logo_argb; /* uint32_t 0xAARRGGBB, ARGB8888 only */
config.width = MY_LOGO_WIDTH;
config.height = MY_LOGO_HEIGHT;
config.stride_bytes = MY_LOGO_WIDTH * sizeof(uint32_t);
video_osd_logo_show_sync(&logo, &config, 5000);
/* my_logo_argb may now be freed or reused. */
To replace the asset on a logo that is already showing, call video_osd_logo_hide_sync() first, then video_osd_logo_show_sync() again with the new argb8888/width/height. The current SDK example ships logo_samples/realtek_logo_argb.h as its sample asset — this is a genuine ARGB logo image, not an RGB test bar; substitute it with the customer’s own ARGB pixel data.
Privacy
A privacy scene masks up to VIDEO_OSD_PRIVACY_MAX_POLYGONS (8) polygons, each with 3 to VIDEO_OSD_PRIVACY_MAX_VERTICES (16) vertices. Convex and concave shapes are both supported in either winding order; a polygon must follow its perimeter without self-intersections or holes.
video_osd_privacy_scene_t fields:
Field |
Type |
Description |
|---|---|---|
|
|
Set by |
|
|
|
|
|
|
|
|
Only used when |
|
|
One |
|
|
Number of entries in |
|
|
Each has |
Coordinate systems
Coordinates always refer to image edges: origin at the top-left, x increasing right, y increasing down.
VIDEO_OSD_PRIVACY_NORMALIZED(default): each axis is an integer in[0, 10000](VIDEO_OSD_PRIVACY_NORMALIZED_MAX);(0,0)is the top-left corner and(10000,10000)is the bottom-right corner at any stream resolution.VIDEO_OSD_PRIVACY_REFERENCE_PIXELS: each axis is a pixel coordinate in[0, reference_width]/[0, reference_height]. Use this when the application already works in the pixel space of a reference image (e.g. a snapshot the user drew on) instead of normalized units.
In both systems the right/bottom edge always maps to the encoder’s current output width/height; the component performs no scaling beyond that single mapping. Letterboxing, cropping, mirroring, rotation, and PTZ/world-coordinate transforms are the application’s responsibility before it builds the scene.
Example: normalized polygon
static const video_osd_privacy_polygon_t polygons[] = {
{
.id = 1,
.enabled = 1,
.vertex_count = 4,
.vertices = {
{1000, 1000}, {4000, 1000}, {4000, 4000}, {1000, 4000},
},
},
};
video_osd_privacy_scene_t scene;
video_osd_privacy_scene_default(&scene);
scene.fill_rgb = 0x000000U;
scene.polygon_count = 1;
memcpy(scene.polygons, polygons, sizeof(polygons));
uint32_t revision = 0;
video_osd_privacy_apply_sync(&privacy, &scene, &revision, 5000);
Example: reference-pixel polygon
video_osd_privacy_scene_t scene;
video_osd_privacy_scene_default(&scene);
scene.coordinates = VIDEO_OSD_PRIVACY_REFERENCE_PIXELS;
scene.reference_width = 1920; /* pixel space the app's vertices were drawn in */
scene.reference_height = 1080;
scene.fill_rgb = 0x000000U;
scene.polygon_count = 1;
scene.polygons[0] = (video_osd_privacy_polygon_t) {
.id = 1, .enabled = 1, .vertex_count = 4,
.vertices = { {200, 150}, {800, 150}, {800, 600}, {200, 600} },
};
uint32_t revision = 0;
video_osd_privacy_apply_sync(&privacy, &scene, &revision, 5000);
Add, move, delete
video_osd_privacy_apply() always replaces the entire scene; there is no incremental add/remove call. Build the next scene from the application’s own polygon list (or from video_osd_privacy_get()) and:
Add a polygon: append it to
polygons[]and incrementpolygon_count.Move a polygon: keep its
id, change itsvertices[], and re-submit — this is how the example’supdateentry works.Disable one polygon without forgetting it: set its
enabled = 0.Delete one polygon: omit its
idfrom the next scene.Remove all masks: submit a scene with
polygon_count = 0, or setscene.enabled = 0to hide every polygon while keeping their definitions for a later re-enable.
These are explicit unmasking operations by design, not error fallbacks — the component never silently drops a mask on a partial failure; a rejected apply() leaves the previous accepted scene active instead.
Every apply() call copies the scene it is given, so the caller’s local scene variable can be reused or go out of scope immediately after the call returns. video_osd_privacy_get() returns the last accepted scene, which may not yet be the applied one — check video_osd_privacy_get_status() (or the shared stream status) to distinguish the two.
Coexistence and Region Planning
Composition order, back to front, is fixed: privacy → NN → logo → timestamp. Logo and timestamp may overlap each other or a privacy mask; a translucent logo drawn over a privacy mask blends with that mask’s opaque fill color, not with the camera pixels the mask is covering.
Region format selection
The manager rasterizes only aligned bounding regions, not full-frame overlays by default. Each of the four features contributes a distinct format choice into the planner’s raw table before any merging happens:
Enabled |
Initial output choice |
|---|---|
None |
No pixel buffer or hardware overlay index is used. |
Logo (L) |
Cropped ARGB region (fixed format for logo). |
Timestamp (T) |
Cropped bitmap region. |
NN (N) |
One cropped bitmap region per NN color layer. |
Privacy (P) |
Cropped bitmap region(s) per polygon. |
The planner then merges regions that share an encoder coding block — 64×16 for H.264, 64×64 for HEVC — repeating the merge until no two output regions share a block. Two opaque bitmap regions of the same YUV color remain bitmap after merging; mixed colors, any translucent overlap, or a region that includes an ARGB source (logo) require a localized ARGB8888 region for the merged group. There is no unconditional full-frame ARGB allocation, but a single large connected overlap group can still grow to cover the whole frame — the planner does not guarantee a small region in every layout.
The full 16-combination table (L/T/N/P, each optionally present) collapses to the same block-disjoint pattern above; every combination row is subject to the same merge/arbitration rules. This is a bounded greedy planner: it merges when doing so reduces total pixel payload, or when hardware overlay indices are running low, choosing the option with the smallest payload increase and re-checking for new block conflicts after each merge. It is not a per-block tiling scheme and not a globally optimal packing. NV12 is never selected as an OSD region format by this manager, even though the encoder driver’s native overlay formats include ARGB8888, NV12, and bitmap.
NN arbitration
NN and privacy/logo/timestamp regions are arbitrated per encoder coding block. If any region belonging to one NN group shares a block with a protected logo/timestamp region, or with a privacy polygon’s conservative bounding rectangle, the entire NN group for that callback is omitted — not just the conflicting box. The next NN callback that does not conflict restores the group automatically. This conservative check also applies to the empty notch of a concave polygon; arbitration works on the polygon’s bounding rectangle, not pixel-exact coverage, so a box that only overlaps a concave notch can still be suppressed.
Memory Model
Per-region cost
A bitmap region costs
aligned_width * height / 8payload bytes.An ARGB region costs
width * height * 4bytes.
Output region lifetime
Output regions are immutable while the encoder is reading them. A region whose content did not change reuses its existing buffer. A region that did change keeps both its old and new buffers only until the frame-boundary completion for that update is acknowledged, then the old allocation is retired. A static, unchanged region therefore never holds a second output canvas.
Input/source storage (in addition to the output regions above)
Logo keeps an owned copy of its ARGB source.
Timestamp keeps two small bitmap sources (double-buffered for atomic text updates).
NN keeps its original full-frame 1-bit drawing canvas; the managed path additionally copies a strided view into one tight manager-owned shadow buffer rather than an intermediate cropped buffer, but this does not eliminate the original canvas.
Status fields (video_osd_stream_get_status() / video_osd_privacy_get_status(), both return the same video_osd_stream_status_t layout)
Field |
Meaning |
|---|---|
|
Revision counters shared by every client on the stream; |
|
Non-zero while one transaction is in flight. Only one transaction may be pending at a time. |
|
The encoder-side frame timestamp at which the applied revision took effect; use it as |
|
0 on success; otherwise one of the |
|
Manager, descriptor, and live/staged output region allocations. |
|
Sampled registered client source storage, including NN working canvases. |
|
Largest observed sum of |
|
The budget passed to |
|
Reserved physical hardware overlay indices, including one pending update; not the same as the number of logical client handles. |
|
Currently applied output region counts by format. |
Budget semantics
memory_budget = 0 disables the limit entirely. A non-zero budget is checked at init and at every commit: the manager rejects an update that would push allocated_bytes + source_bytes (including in-flight old/new output buffers) over the budget. It is not a pre-allocation and not a hard cap on the whole process heap:
Client source allocations (e.g. a logo’s owned ARGB copy) can happen before the client registers with the manager, and are not counted until registration/commit.
RTOS task/semaphore stacks, allocator metadata, and unrelated buffers are excluded.
The demo’s
V1_OSD_MEMORY_BUDGETof 8 MiB is a ceiling the manager will refuse to exceed, not 8 MiB of memory reserved up front.
video_osd_privacy_get_caps().canvas_bytes is a separate, compatibility-only worst-case estimate — two full-frame ARGB canvases’ worth of bytes — and is not what video_osd_privacy_init() actually allocates. Query video_osd_stream_get_status() for real usage.
Hardware index pool
OSD and mosaic share one 12-index-per-encoder-channel hardware overlay pool. The stream manager is the only party that reserves physical indices from this pool; every managed helper (logo, timestamp, privacy, NN through nn_osd) uses a logical handle internally and does not make its own physical reservation. Enabling mosaic on an index disables OSD on that index and vice versa, so an application that also uses video_set_mosaic()/video_disable_mosaic() directly on the same channel is sharing the same 12 slots with the managed OSD stream.
Error Handling and Shutdown
Synchronous convenience APIs
The eight APIs below hide the status-polling and EBUSY-retry loops that a caller would otherwise write by hand. They do not add a session object, worker task, queue, or new AP/VP protocol; they call the underlying plain APIs described later in this section. Prefer them over manual polling on any managed stream.
API |
Success (return 0) means |
|---|---|
|
The initial logo update was accepted and then encoder-applied. |
|
The clock started and its initial update was accepted and then encoder-applied. |
|
The applied revision reached the fixed target, with no reported encoder error. |
|
A complete scene replacement was accepted and then encoder-applied. |
|
The clock task stopped, removal was confirmed, and the context was released. |
|
Removal was confirmed and the context was released. |
|
Privacy was deinitialized, including an auto-owned stream manager if privacy created one. |
|
The manager was released, after all of its clients were already removed. |
show_sync / start_sync initial-activation semantics
A matching stream manager must already exist — same
stream_id,codec, and frame width/height as the target passed tovideo_osd_stream_init().ctxmust be zero-initialized or fully cleaned up already. A non-empty context (one that is live, or partway through removal) returnsVIDEO_OSD_EINVAL— calling this again on the same context is not automatically treated as success the way the removal APIs are.Before acceptance, the call retries
EBUSYinternally. After acceptance, it waits only for that one initial revision, which is captured under the manager lock so a later timestamp/NN update cannot change what this call is waiting for.Returning 0 means the initial content was already encoder-applied — there is no need to call
video_osd_stream_wait_applied()a second time just to confirm this particular client came up.config, its format string (timestamp), and the logo’s source pixels must remain valid and unmodified for the duration of the call. The API copies the format string and pixel rows internally, so the caller’s inputs can be released after the call returns — even if that return is a timeout.timeout_mscovers both the pre-acceptance retry and the post-acceptance wait as one budget, and is still not a hard real-time deadline.A timeout or encoder error can happen after acceptance: the context may still be live, the logo may already be visible, or the clock may already be refreshing. None of this rolls back automatically — do not
memset()/free the context, and do not callshow()/start()again on the same context.Recovery is the same in every case: call the matching
stop_sync/hide_sync(safe even on an already-empty context) and only then attemptshow_sync/start_syncagain.The call takes three logical arguments (context, config, timeout) and has no output
revisionparameter. If the timeout happened before acceptance, a later successfulvideo_osd_stream_wait_applied()on some other revision does not prove this particular logo/clock was accepted; use the stop_sync-then-retry pattern above instead of trying to infer acceptance from an unrelated wait.A failure can be
VIDEO_OSD_EINVAL,VIDEO_OSD_ENOMEM,VIDEO_OSD_ENOSPC,VIDEO_OSD_EENCODER,VIDEO_OSD_EBUSY, orVIDEO_OSD_ETIMEDOUTdepending on where the call failed.
wait_applied revision semantics
revision > 0: waits forapplied_revision >= revision. This is a specific target, not “wait for the whole stream to go idle.”revision = 0: on the first successful status read, the call snapshots the currentrequested_revisionand uses that as a fixed target. A later timestamp or NN update does not move this target forward.Because of that fixed target, a successful return can still leave
pending == 1in the returned status — the originally targeted revision completed, but a newer, unrelated update is now in flight.Passing a revision that has not been requested yet on this manager returns
VIDEO_OSD_EINVAL. A revision value is only meaningful for the lifetime of thevideo_osd_stream_tit came from.statusmay beNULL. When it is not, it receives the last successfully read status, includingapplied_timestamp. If no status read ever succeeded, the caller’s output is left untouched — do not read it after a failed call.If
last_erroris set in status and no recovery transaction is currently pending, that error is returned. If a recovery transaction is pending, the call keeps waiting instead of failing immediately.Success here still only means encoder-applied, not that any network client or player has displayed the frame.
apply_sync scene-replacement semantics
Replaces the entire scene in one call; there is no incremental add/remove variant.
Before acceptance, the call retries an
EBUSYresult internally within thetimeout_msbudget.After acceptance, it waits only for that one revision; it never resubmits the scene.
Returning 0 means the scene was encoder-applied, not merely that the command was accepted.
revisionis an optional output; passNULLfor the common case. When non-NULL, it is cleared to 0 at the start of the call and filled in as soon as the scene is accepted — that value is retained even if the call later times out, so the caller can keep waiting on it (see the retry pattern below).The
sceneargument must remain valid and must not be modified by another task for the entire duration of the call.timeout_msis one budget covering both the pre-acceptance retry and the post-acceptance wait; it is not restarted between the two phases.
stop_sync / hide_sync / deinit_sync removal semantics
Returning 0 means that resource has finished being cleaned up. Calling one of these again on an already-removed or zero-initialized context also returns 0.
A
NULLcontext pointer returnsVIDEO_OSD_EINVAL— this is a caller error, not success.These sync removals apply only to a managed stream. Calling one on an unmanaged direct-overlay client (no stream manager) returns
VIDEO_OSD_EINVAL, because that path has no encoder-completion protocol to wait on — it never fakes a hardware confirmation.For timestamp removal, the update task is stopped as soon as removal begins; the clock does not resume automatically even if the call times out.
On error or timeout, keep the context and retry the same
_syncAPI. Do not call thevoidstop()/hide()instead, do not zero out or manually clear the context, do not free it, and do not reuse the same not-yet-removed context tostart()/show()again.video_osd_stream_deinit_sync()does not destroy other clients automatically — logo, timestamp, privacy, and NN must already be removed first.If privacy auto-created its stream manager (see Public APIs),
video_osd_privacy_deinit_sync()also releases that manager, but only after every other client on the stream has been removed.
Accepted vs. applied on the plain APIs
video_osd_privacy_apply(), video_osd_logo_show(), and video_osd_timestamp_start() returning 0 means the request was accepted into the pending transaction, not that it is visible on encoded output yet. Poll video_osd_stream_get_status() (or the privacy-specific alias) until pending == 0; then check applied_revision >= revision and last_error == 0 before treating the change as live — or call video_osd_stream_wait_applied() instead of writing this loop by hand. The “applied” completion itself only means the VP encoder’s preprocessing step selected the full region list for one input frame and retired the previous buffers — it is not a network/display acknowledgment to any downstream client.
Errors returned by the plain and sync APIs
A non-zero return from
apply()/show()/start()(other thanVIDEO_OSD_EBUSY) means nothing was submitted or the request was rejected outright; the previous accepted state is unchanged.VIDEO_OSD_EBUSYmeans a previous transaction is still pending — nothing new was submitted. The plain APIs return it directly to the caller; the_synchelpers retry it internally within their timeout budget instead. Coalesce rapid successive updates (e.g. from a dragging UI) into one replacement rather than submitting every intermediate one.A runtime rejection at the VP encoder surfaces through
last_errorin status (or asVIDEO_OSD_EENCODERfrom a_sync/wait_appliedcall), not through the initial return value. While that error is set, encoded output is intentionally stopped rather than falling back to an unmasked frame; submit a new valid scene/logo/timestamp to recover. Do not open the output gate while status reports an error.
Timeouts
timeout_msbounds one polling/retry budget for that single call, measured against a 64-bit monotonic RTOS clock so it is unaffected by camera time synchronization.EBUSYis retried at intervals of up to 20 ms; the manager mutex is never held across a sleep.timeout_ms = 0still makes one attempt — if that attempt is busy, the call returnsVIDEO_OSD_ETIMEDOUTimmediately rather than blocking.UINT32_MAXis a large but finite number of milliseconds, not “wait forever.”An in-flight IPC call, mutex wait, or scheduling delay can make the call return later than
timeout_ms— none of these APIs are hard real-time.VIDEO_OSD_ETIMEDOUTdoes not cancel a submitted command, does not roll back to the previous scene, and does not free a buffer whose completion is still unconfirmed. Keep the context alive and keep retrying/waiting — never free a context or treat it as safe to release on a timeout. If the VP side stops responding entirely, keep the output gate blocked and recover or restart the pipeline under application control; do not assume that unmasking or buffer retirement quietly succeeded.Each API call has its own independent timeout budget. The four sequential removal calls in the shutdown sequence below can therefore consume up to four separate timeout budgets, not one shared budget for the whole stop sequence.
An application that calls the plain APIs directly is responsible for its own status polling, as described above. Check the int return value of the _sync APIs to decide whether an operation succeeded; do not inspect ctx->priv.
Recovering from apply_sync timing out after acceptance
uint32_t revision = 0;
int ret = video_osd_privacy_apply_sync(&privacy, &scene, &revision, 5000);
if (ret == VIDEO_OSD_ETIMEDOUT && revision != 0) {
/* Already accepted, completion not yet confirmed: keep waiting on the
* same revision instead of resubmitting the scene. */
ret = video_osd_stream_wait_applied(&stream, revision, 5000, NULL);
}
/* ret != 0: do not report the update as successful; keep the context and
* handle the error. Submitting the scene again is a new replacement, not
* a way to cancel the one still in flight. */
If the caller does not need to distinguish “not yet accepted” from “accepted but not yet confirmed,” pass NULL for revision and query video_osd_privacy_get_status()/video_osd_stream_get_status() afterward instead.
Startup confirmation and the output gate
video_osd_stream_status_t status;
int ret = video_osd_stream_wait_applied(&stream, 0, 5000, &status);
if (ret != 0) {
/* Keep the output gate blocked and go to error handling. */
return ret;
}
/* Use status.applied_timestamp to configure the video_module output gate;
* enable filter_timestamp based on applied_revision != 0. */
The output gate stays in the MMF example, not in the video_osd core — the core has no dependency on UVC/RTSP. The logo/timestamp sync helpers above do not need this call to confirm their own activation; this one wait obtains a shared status (including any clock/NN update that raced in during startup), not a required second wait per feature.
Consistent activation across the three features
/* Encoder already started, output gate already closed, and the stream
* manager / privacy context already initialized. */
int ret = video_osd_privacy_apply_sync(&privacy, &scene, NULL, 5000);
if (!ret) {
ret = video_osd_logo_show_sync(&logo, &logo_config, 5000);
}
if (!ret) {
ret = video_osd_timestamp_start_sync(&clock, &clock_config, 5000);
}
if (ret != 0) {
/* Follow the synchronous cleanup path; do not assume a failure here
* means nothing was accepted. */
return ret;
}
/* All three are now applied; fetch the shared status.applied_timestamp
* and open the output gate safely. */
There is no required activation order between the three clients. The example starts privacy first, then logo, then timestamp — so masking is in place before the periodic clock refresh begins — but composition order is decided by the manager, not by call order (see Coexistence and Region Planning).
Shutdown order
Reverse of startup, using the sync removal APIs and checking each return value:
/* Block the output gate and stop the downstream sink/linker first.
* Keep the encoder running through all four removals below. */
int ret = video_osd_timestamp_stop_sync(&clock, 5000);
if (!ret) {
ret = video_osd_logo_hide_sync(&logo, 5000);
}
if (!ret) {
ret = video_osd_privacy_deinit_sync(&privacy, 5000);
}
if (!ret) {
ret = video_osd_stream_deinit_sync(&stream, 5000);
}
if (ret != 0) {
/* Keep output blocked and the encoder running; keep every context
* intact and retry this whole stop sequence later. */
return ret;
}
/* Only stop the encoder once every removal above returned 0. */
Block the output gate (
CMD_VIDEO_SET_OUTPUT_GATEwithblocked = 1) and stop the downstream linker/sink first.Call
video_osd_timestamp_stop_sync(), thenvideo_osd_logo_hide_sync(). If NN is in use, stop the NN producer and deinit its NN group(s) through thenn_osdAPI before continuing.Call
video_osd_privacy_deinit_sync().Call
video_osd_stream_deinit_sync(). The encoder must still be running through steps 2–4 so it can acknowledge each retirement.Only after every removal above returns 0 should the encoder itself be stopped/closed.
An auto-created manager (from privacy’s compatibility auto-init path) is released by privacy’s own deinit_sync() once every other client on that stream is gone — do not call video_osd_stream_deinit_sync() a second time for it. Do not stop or reconfigure the encoder externally while any of its OSD clients are still being torn down. None of these synchronous APIs may be called from an ISR, from an encoder/IPC callback, or while holding an NN group lock that a background callback needs.
NN Coexistence
NN detection overlays use nn_osd_group_init()/nn_osd_group_flush() from video_nn_utils, not this component’s client APIs directly, but they draw onto the same shared stream manager once video_osd_stream_init() has run. Each nn_osd layer uses one hardware OSD index; use one layer when boxes, labels, and landmarks can share a single color, and multiple layers only when different colors are required, since different-index bitmap regions can overlap.
Do not enable the nn_osd stale-overlay watchdog by default when a detection callback keeps refreshing the overlay every frame — pass 0 for enable_watchdog and skip nn_osd_group_kick() in that case. Enable the watchdog only when a callback may stop arriving while objects are still notionally present (so stale boxes would otherwise linger on screen); if enabled, comment why.
See Coexistence and Region Planning above for how an NN group is suppressed and restored when it conflicts with a protected logo/timestamp region or a privacy polygon’s bounding rectangle.
Limitations and Security
This component protects only the one configured encoded stream it was initialized on. JPEG snapshots, raw NV12/RGB streams used for NN input (V4/V5), other simultaneously running encoded streams, and any pre-encoder recording path are not automatically masked. A production design must apply masking — or an equivalent decision — to every output path it exposes, including at startup, on snapshot capture, and during error recovery, not only to the one stream this example configures.
Application-side transport of scene updates (HTTP/MQTT/P2P or any other channel), authorization/access control, coordinate-system translation from a UI, and persistence of the last-applied scene across reboots are all the caller’s responsibility; none of it is provided by
video_osd/video_osd_privacy.Low-latency/slice mode, look-ahead, multi-pass encoding, and combined H.264/HEVC + JPEG dual encoding are not supported by the managed transaction; this implementation targets the SDK’s single-core, single-pass synchronous encode path. Confirm against the VP encoder driver before relying on any of these together with managed OSD.
Do not open direct
video_set_osd_bitmap()/video_set_mosaic()(or matching disable) clients on a stream that already has a stream manager; once scene ownership is active, VP rejects direct-overlay writes on that stream. Mosaic remains a separate feature that shares the same 12-index hardware pool, not a backend for privacy polygons.AP and VP firmware — including prebuilt multimedia libraries — must be updated together, since the OSD scene protocol is a shared AP/VP structure (
video_osd_scene_t).
