Multimedia Framework

Supported ICs: [RTL8735C]

Overview

The SDK includes a lightweight Multimedia Framework (MMF) designed to make audio and video pipeline development modular and intuitive. Each functional component — video capture, audio capture, H.264/HEVC encoding, AAC encoding/decoding, RTSP streaming, USB UVC output, MP4 recording, and neural network inference — is implemented as an independent module. Developers build a pipeline by opening the required modules, configuring their parameters, and connecting them with linkers.

This modular approach provides several practical benefits:

Each module can be developed, tested, and replaced independently.
Pipelines of varying complexity are built using the same API pattern.
Swapping an output sink (for example, from RTSP to UVC, or from RTSP to MP4) requires only changing which modules are linked.
Adding NN inference to an existing pipeline requires only opening the VIPNN module and inserting it into the data path.

The MMF runs on FreeRTOS and uses a producer-consumer model: modules place processed frames into output queues, and linker tasks drain those queues and push frames into the next module’s input. All inter-module data transfer is therefore asynchronous and thread-safe.

Video Pipeline Architecture

For video applications, the platform uses two cores internally. The AP core (CA32) runs the user media application, such as RTSP, UVC, MP4 recording, or NN examples. The VP core (KM4) runs the Realtek-provided video firmware, which owns the camera, ISP, encoder, and video frame export path.

From the application developer’s point of view, the AP-side video_module is the main interface. Developers configure stream parameters, open MMF modules, and connect them with linkers on the AP side. The low-level ISP and encoder driver flow is handled inside the VP firmware.

Internally, the AP sends video control commands to the VP, and the VP pushes produced frames back to the AP-side MMF pipeline. Most applications do not need to call the IPC APIs directly.

../../_images/multimedia_pro_mmf_video_pipeline_ap_vp.drawio.svg

Note

Application developers only need to focus on AP-side application development. The IPC mechanism and VP-side firmware are provided by Realtek. This diagram is intended only to help users understand the overall video architecture.

Framework Architecture

The following diagram shows how data flows through a typical pipeline:

┌─────────────┐   mm_queue_item_t   ┌──────────────┐   mm_queue_item_t   ┌──────────────┐
│   Module A  │ ──────────────────> │    Linker    │ ──────────────────> │   Module B   │
│  (Producer) │                     │  (SISO/MIMO) │                     │  (Consumer)  │
└─────────────┘                     └──────────────┘                     └──────────────┘

Each module maintains an internal output queue. When a module finishes processing a frame, it places the resulting mm_queue_item_t into its output queue. The linker task continuously drains that queue and forwards items to the next module in the pipeline. Frame buffers are recycled internally by the framework, so application code does not need to manage buffer lifetimes.

Key Concepts

Module (``mm_module_t``)

A module is a self-contained processing unit defined by the mm_module_t structure. It exposes a standard interface: create, destroy, control, and handle. The Available Modules section lists all modules and their roles.

Module Context (``mm_context_t``)

Calling mm_module_open() allocates and returns an mm_context_t handle. This opaque handle holds the module’s internal state, its queue pair, and a pointer to private instance data. All subsequent configuration and linking operations use this handle.

mm_context_t *ctx = mm_module_open(&video_module);

Queue Items (``mm_queue_item_t``)

Data flows between modules as mm_queue_item_t items. The fields most relevant to application developers are:

Field	Description
`data_addr`	Pointer to the frame payload (encoded bitstream, raw PCM, etc.)
`size`	Payload size in bytes
`type`	Codec or data type (`AV_CODEC_ID_H264`, `AV_CODEC_ID_RGB888`, etc.)
`timestamp`	Frame timestamp (ms)
`flag`	Allocation mode: `MMQI_FLAG_STATIC` or `MMQI_FLAG_DYNAMIC`

STATIC vs DYNAMIC allocation

When initialising a module’s queue (MM_CMD_INIT_QUEUE_ITEMS), you choose between two allocation modes:

MMQI_FLAG_STATIC — all queue item buffers are pre-allocated once at startup. Buffer size is fixed. Best for audio (constant frame size, deterministic latency).
MMQI_FLAG_DYNAMIC — buffers are allocated from the heap on demand. Required for video streams where frame size varies per frame (e.g. compressed H.264). More flexible but uses heap memory.

As a rule of thumb: use MMQI_FLAG_STATIC for audio modules and MMQI_FLAG_DYNAMIC for video modules.

Linkers

A linker is a FreeRTOS task that continuously reads items from one module’s output queue and pushes them into the next module’s input queue. Four linker topologies are available:

Type	Topology	Typical use
SISO	1 input → 1 output	Single-stream segments
MIMO	Up to 4 inputs → up to 4 outputs (with per-output dependency masks)	Multiplexed A/V recording + streaming from shared sources
MISO	Multiple inputs → 1 output	Audio mixing
SIMO	1 input → multiple outputs	Duplicating a stream to multiple consumers

Module Lifecycle

All modules follow the same initialisation sequence:

// 1. Open the module — returns a context handle
mm_context_t *ctx = mm_module_open(&xxx_module);
if (!ctx) goto error;

// 2. Configure parameters
mm_module_ctrl(ctx, CMD_XXX_GET_PARAMS, (int)&params);  // read defaults (optional)
params.field = value;
mm_module_ctrl(ctx, CMD_XXX_SET_PARAMS, (int)&params);

// 3. Set queue depth and allocate queue items
mm_module_ctrl(ctx, MM_CMD_SET_QUEUE_LEN, 6);
mm_module_ctrl(ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_STATIC);  // or MMQI_FLAG_DYNAMIC

// 4. Apply (start the module)
mm_module_ctrl(ctx, CMD_XXX_APPLY, 0);

Standard control commands (accepted by every module):

Command	Description
`MM_CMD_SET_QUEUE_LEN`	Set queue depth (number of items)
`MM_CMD_INIT_QUEUE_ITEMS`	Allocate queue items (arg: `MMQI_FLAG_STATIC` or `MMQI_FLAG_DYNAMIC`)
`MM_CMD_CLEAR_QUEUE_ITEMS`	Flush queued items

Queue depth guidelines:

Video modules (encoded output): 60 items with MMQI_FLAG_DYNAMIC
Audio modules: 6 items with MMQI_FLAG_STATIC
NN / post-processing: 6 items with MMQI_FLAG_DYNAMIC

Using a deeper video queue absorbs burst encoder output and prevents frame drops when downstream modules are momentarily busy.

Teardown sequence

Always tear down linkers before closing modules:

// 1. Stop and delete linkers first
siso_stop(siso);
siso_delete(siso);

// 2. Then close modules
mm_module_close(ctx);

Closing a module while a linker is still running will cause the linker task to access freed memory.

Linker API

SISO

SISO connects one module output to one module input. Each SISO instance spawns one FreeRTOS task.

mm_siso_t *siso = siso_create();

siso_ctrl(siso, MMIC_CMD_ADD_INPUT,  (uint32_t)src_ctx,  0);
siso_ctrl(siso, MMIC_CMD_ADD_OUTPUT, (uint32_t)dst_ctx,  0);

// Optional tuning
siso_ctrl(siso, MMIC_CMD_SET_STACKSIZE,    32 * 1024, 0);  // stack size in bytes
siso_ctrl(siso, MMIC_CMD_SET_TASKPRIORITY, 3,         0);  // FreeRTOS priority

siso_start(siso);

Additional SISO operations:

siso_pause(siso);    // temporarily halt data flow
siso_resume(siso);   // resume
siso_stop(siso);     // stop the task
siso_delete(siso);   // free all resources

MIMO

MIMO connects multiple inputs to multiple outputs. Each output spawns its own FreeRTOS task, and the output_dep bitmask controls which inputs must be available before the output task forwards data. This lets a single MIMO instance handle both MP4 recording (V1 + audio) and RTSP streaming (V2 + audio) from three shared input sources.

mm_mimo_t *mimo = mimo_create();

mimo_ctrl(mimo, MMIC_CMD_ADD_INPUT0, (uint32_t)video_v1_ctx, 0);  // input 0: V1 stream
mimo_ctrl(mimo, MMIC_CMD_ADD_INPUT1, (uint32_t)video_v2_ctx, 0);  // input 1: V2 stream
mimo_ctrl(mimo, MMIC_CMD_ADD_INPUT2, (uint32_t)aac_ctx,      0);  // input 2: audio

// Output 0 (MP4): waits for V1 and audio to both be ready
mimo_ctrl(mimo, MMIC_CMD_ADD_OUTPUT0, (uint32_t)mp4_ctx,
          MMIC_DEP_INPUT0 | MMIC_DEP_INPUT2);

// Output 1 (RTSP): waits for V2 and audio to both be ready
mimo_ctrl(mimo, MMIC_CMD_ADD_OUTPUT1, (uint32_t)rtsp_ctx,
          MMIC_DEP_INPUT1 | MMIC_DEP_INPUT2);

mimo_start(mimo);

Dependency bitmask constants:

Constant	Bit
`MMIC_DEP_INPUT0`	0x01
`MMIC_DEP_INPUT1`	0x02
`MMIC_DEP_INPUT2`	0x04
`MMIC_DEP_INPUT3`	0x08

MIMO supports selective pause and resume:

mimo_pause(mimo, pause_mask);  // pause specific inputs or outputs
mimo_resume(mimo);             // resume all outputs

MISO

MISO (Multiple Input, Single Output) collects frames from several sources and feeds them into one downstream module. The typical use case is audio mixing — combining multiple PCM sources before encoding.

mm_miso_t *miso = miso_create();
miso_ctrl(miso, MMIC_CMD_ADD_INPUT0, (uint32_t)audio_ctx_a, 0);
miso_ctrl(miso, MMIC_CMD_ADD_INPUT1, (uint32_t)audio_ctx_b, 0);
miso_ctrl(miso, MMIC_CMD_ADD_OUTPUT, (uint32_t)aac_ctx,     0);
miso_start(miso);

SIMO

SIMO (Single Input, Multiple Output) delivers one module’s output to multiple downstream consumers. The framework handles buffer sharing and recycling automatically — application code connects the modules and starts the linker as usual.

mm_simo_t *simo = simo_create();
simo_ctrl(simo, MMIC_CMD_ADD_INPUT,   (uint32_t)video_v1_ctx, 0);
simo_ctrl(simo, MMIC_CMD_ADD_OUTPUT0, (uint32_t)rtsp_ctx,     0);
simo_ctrl(simo, MMIC_CMD_ADD_OUTPUT1, (uint32_t)uvcd_ctx,     0);
simo_start(simo);

Common linker control commands

Command	Description
`MMIC_CMD_ADD_INPUT[0–3]`	Connect input module on port N
`MMIC_CMD_ADD_OUTPUT[0–3]`	Connect output module on port N
`MMIC_CMD_SET_STACKSIZE`	FreeRTOS task stack size (bytes)
`MMIC_CMD_SET_TASKPRIORITY`	FreeRTOS task priority
`MMIC_CMD_SET_CTRL_TIMEOUT`	Control command timeout (ms)

Stack size reference

The linker task’s stack must be large enough for the downstream module’s handler. If the stack is too small, FreeRTOS will stack-overflow and reset the system.

Scenario	Recommended size
Simple pass-through (mux/demux)	8 – 16 KB
Video encoding (H.264 / HEVC)	32 KB
Audio encoding (AAC)	44 KB
NN inference (VIPNN)	32 – 64 KB

Task priority guidance

Linker tasks are created with a default priority of 1 (tskIDLE_PRIORITY + 1). Adjust the priority according to your system’s scheduling requirements. Do not raise linker task priority above 5, as higher priorities may starve ISP and encoder driver tasks and cause frame drops.

Available Modules

The following modules are available in the AP-side multimedia framework:

Module	Header	Description
`video_module`	`module_video.h`	Video capture and encoding (H.264 / HEVC / JPEG / RGB)
`audio_module`	`module_audio.h`	Audio capture and playback via AMIC / DMIC / DAC
`aac_module`	`module_aac.h`	AAC audio encoder
`aad_module`	`module_aad.h`	AAC audio decoder (for playback)
`rtp_module`	`module_rtp.h`	RTP audio receiver (for two-way audio)
`rtsp2_module`	`module_rtsp2.h`	RTSP server for live audio/video streaming over network
`uvcd_module`	`module_uvcd.h`	USB UVC device — streams video to a USB host
`mp4_module`	`module_mp4.h`	MP4 file muxer for SD card recording
`vipnn_module`	`module_vipnn.h`	VIP neural network inference (object / face detection)
`facerecog_module`	`module_facerecog.h`	Face recognition: compare MobileFaceNet embeddings against a stored identity database
`array_module`	`module_array.h`	Synthetic data source for offline testing (no ISP needed)
`filesaver_module`	`module_filesaver.h`	Generic file saver

Key module parameters are covered in the individual User Guides. The sections below summarise the most important configuration points.

video_module

video_module wraps the ISP and hardware encoder. It handles both raw (RGB / NV12) and compressed (H.264 / HEVC / JPEG) output depending on the format field.

The ISP supports up to five simultaneous output channels. Each video_module instance occupies one channel, selected by the stream_id parameter:

Channel	Typical format	Typical role
V1	H.264 or HEVC, high-res	Recording (MP4) or primary RTSP stream
V2	H.264, mid-res	Secondary RTSP stream or UVC
V3	H.264 or JPEG, low-res	Thumbnail, snapshot
V4	JPEG	Still capture
V5	RGB888 or NV12	Neural network inference input

V5 is special: the hardware encoder is bypassed and raw RGB or NV12 frames are passed directly downstream — typically to vipnn_module.

video_params_t video_params = {
    .stream_id = V1_STREAM_ID,       // 0–4, selects the ISP output channel
    .format    = VIDEO_H264,         // VIDEO_HEVC, VIDEO_JPEG, VIDEO_RGB, VIDEO_NV12
    .width     = 1920,
    .height    = 1080,
    .fps       = 30,
    .gop       = 30,                 // keyframe interval (frames)
    .bps       = 2000000,            // target bitrate in bps
    .rc_mode   = ENC_VBR,            // ENC_VBR, ENC_CBR, ENC_ABR
};
mm_module_ctrl(video_ctx, CMD_VIDEO_SET_PARAMS, (int)&video_params);
mm_module_ctrl(video_ctx, MM_CMD_SET_QUEUE_LEN, 60);
mm_module_ctrl(video_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
mm_module_ctrl(video_ctx, CMD_VIDEO_APPLY, 0);

Runtime adjustment — after the module is running, bitrate and frame rate can be changed on-the-fly without restarting the pipeline:

mm_module_ctrl(video_ctx, CMD_VIDEO_BITRATE, 1500000);  // change target bitrate (bps)
mm_module_ctrl(video_ctx, CMD_VIDEO_FPS,     20);       // change frame rate

Force keyframe — call this when a new RTSP client connects so it can start decoding immediately without waiting for the next natural keyframe:

mm_module_ctrl(video_ctx, CMD_VIDEO_FORCE_IFRAME, 0);

audio_module

audio_module handles both capture (microphone) and playback (DAC) paths on the same module instance.

audio_params_t audio_params;
mm_module_ctrl(audio_ctx, CMD_AUDIO_GET_PARAMS, (int)&audio_params);  // read defaults
audio_params.sample_rate    = ASR_16KHZ;    // ASR_8KHZ, ASR_16KHZ, ASR_48KHZ
audio_params.channel        = 1;
audio_params.use_mic_type   = USE_AUDIO_AMIC;  // or DMIC_LEFT, DMIC_RIGHT
audio_params.mic_gain       = MIC_20DB;
audio_params.enable_record  = 1;
audio_params.avsync_en      = 1;            // enable A/V sync timestamps (required for MP4)
mm_module_ctrl(audio_ctx, CMD_AUDIO_SET_PARAMS, (int)&audio_params);
mm_module_ctrl(audio_ctx, MM_CMD_SET_QUEUE_LEN, 6);
mm_module_ctrl(audio_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_STATIC);
mm_module_ctrl(audio_ctx, CMD_AUDIO_APPLY, 0);

Note

Set avsync_en = 1 whenever audio is combined with video in a MIMO pipeline (MP4 recording, A/V RTSP). This enables hardware-aligned timestamps on each audio frame so the muxer can synchronise audio and video correctly. Without it, MP4 files may have audio/video drift.

aac_module

aac_module encodes PCM audio to AAC. It requires explicit memory pool initialisation after setting parameters.

aac_params_t aac_params = {
    .trans_type  = AAC_TYPE_ADTS,   // AAC_TYPE_RAW or AAC_TYPE_ADTS
    .object_type = AAC_AOT_LC,
    .sample_rate = 16000,
    .channel     = 1,
    .bitrate     = 32000,
};
mm_module_ctrl(aac_ctx, CMD_AAC_SET_PARAMS, (int)&aac_params);
mm_module_ctrl(aac_ctx, CMD_AAC_INIT_MEM_POOL, 0);  // must call after SET_PARAMS
mm_module_ctrl(aac_ctx, MM_CMD_SET_QUEUE_LEN, 6);
mm_module_ctrl(aac_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
mm_module_ctrl(aac_ctx, CMD_AAC_APPLY, 0);

Note

The SISO linker connecting audio_module → aac_module must set a large stack size (44 KB) because the AAC encoder runs in the linker task context.

siso_ctrl(siso_audio_aac, MMIC_CMD_SET_STACKSIZE, 44 * 1024, 0);

rtsp2_module

rtsp2_module streams video and audio over RTSP. Video and audio streams are configured separately using CMD_RTSP2_SELECT_STREAM.

// Configure video stream (stream 0)
rtsp2_params_t rtsp2_v = {
    .type = AVMEDIA_TYPE_VIDEO,
    .u.v  = { .codec_id = AV_CODEC_ID_H264, .fps = 30 },
};
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SELECT_STREAM, 0);
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SET_PARAMS, (int)&rtsp2_v);
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SET_APPLY, 0);

// Configure audio stream (stream 1)
rtsp2_params_t rtsp2_a = {
    .type = AVMEDIA_TYPE_AUDIO,
    .u.a  = { .codec_id = AV_CODEC_ID_MP4A_LATM, .channel = 1, .samplerate = 16000 },
};
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SELECT_STREAM, 1);
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SET_PARAMS, (int)&rtsp2_a);
mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SET_APPLY, 0);

mm_module_ctrl(rtsp_ctx, CMD_RTSP2_SET_STREAMMING, ON);

mp4_module

mp4_module muxes video and audio into MP4 files on the SD card. Recording starts when CMD_MP4_START is issued.

mp4_params_t mp4_params = {
    .fps              = 30,
    .gop              = 30,
    .width            = 1920,
    .height           = 1080,
    .sample_rate      = 16000,
    .channel          = 1,
    .record_length    = 60,         // seconds per file
    .record_file_num  = 5,
    .record_type      = STORAGE_ALL,  // video + audio
    .record_file_name = "mmc:/test",
    .fatfs_buf_size   = 32 * 1024,
};
mm_module_ctrl(mp4_ctx, CMD_MP4_SET_PARAMS, (int)&mp4_params);
mm_module_ctrl(mp4_ctx, CMD_MP4_LOOP_MODE, 0);
mm_module_ctrl(mp4_ctx, CMD_MP4_START, mp4_params.record_file_num);

vipnn_module

vipnn_module runs neural network inference on the NPU. See the NN User Guide for full documentation. The key integration point is the result callback registered with CMD_VIPNN_SET_DISPPOST:

static nn_data_param_t nn_input_params = {
    .img = {
        .width  = NN_WIDTH,   // incoming RGB/NV12 frame width
        .height = NN_HEIGHT,  // incoming RGB/NV12 frame height
    },
    .codec_type = AV_CODEC_ID_RGB888,
};

mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_MODEL,           (int)&NN_MODEL_OBJ);
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_MODEL_FILE_NAME, (int)"vfs:/model.nb");
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_IN_PARAMS,       (int)&nn_input_params);
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_DISPPOST,        (int)nn_display_cb);
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_RES_SIZE,        sizeof(objdetect_res_t));
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_SET_RES_MAX_zhT,     32);
mm_module_ctrl(vipnn_ctx, CMD_VIPNN_APPLY, 0);

nn_data_param_t.img.width and height describe the actual frame received by VIPNN. Recent SDK examples use the whole frame as the NN input; per-frame ROI is no longer configured in this structure.

array_module

array_module replays a pre-loaded binary buffer as a video or audio source. This is useful for offline testing without a camera or microphone. The buffer is a raw encoded stream (e.g. H.264 or RGB frames) loaded into memory at compile time.

array_params_t array_params = {
    .type     = AVMEDIA_TYPE_VIDEO,
    .codec_id = AV_CODEC_ID_RGB888,
    .fps      = 15,
    .loop_en  = 1,
    .u        = { .v = { .width = 416, .height = 416 } },
};
mm_module_ctrl(array_ctx, CMD_ARRAY_SET_PARAMS, (int)&array_params);
mm_module_ctrl(array_ctx, MM_CMD_SET_QUEUE_LEN, 6);
mm_module_ctrl(array_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
mm_module_ctrl(array_ctx, CMD_ARRAY_APPLY, 0);
mm_module_ctrl(array_ctx, CMD_ARRAY_STREAMING, 1);  // start playback

Pipeline Examples

The following diagrams illustrate common pipeline topologies. For complete source code, build instructions, and compile-time options for each example, see Video User Guide .

Example 1: Single stream — V1 → RTSP

The minimal useful pipeline: one H.264 channel streamed live over RTSP. Start here if you are new to the MMF.

[video_module V1: H.264 1920×1080 @ 30fps] ──SISO──> [rtsp2_module]

Example 2: NN inference — array → VIPNN

Offline model verification without a camera. A synthetic image buffer replays test frames through the NPU via a SISO linker. Useful for validating model integration before connecting a live ISP channel.

[array_module (RGB test image)] ──SISO──> [vipnn_module]
                                                |
                                         nn_display_cb()

Example 3: Dual-stream — V1 RTSP + V5 face detection

Two independent SISO pipelines share the ISP simultaneously. V1 encodes H.264 for RTSP streaming; V5 delivers raw RGB frames to the face detection model. The two pipelines run at different frame rates with no interference.

[video_module V1: H.264 1920×1080 @ 30fps] ──SISO──> [rtsp2_module]
[video_module V5: RGB   576×320  @ 15fps ] ──SISO──> [vipnn_module]
                                                           |
                                                    nn_display_cb()

Example 4: Full A/V — MP4 recording + RTSP + NN

Three video channels, AAC-encoded audio, MP4 file recording, RTSP live streaming, and YOLO object detection — all running simultaneously using a MIMO linker.

                                       ┌──MIMO──> [mp4_module]   (V1 + AAC)
[video_module V1: HEVC 1920×1080] ──┐  │
[video_module V2: H264  1280×720] ──┼──┤
[audio_module] ─SISO─> [aac_module]─┘  └──MIMO──> [rtsp2_module] (V2 + AAC)

[video_module V5: RGB 416×416] ──SISO──> [vipnn_module]
                                               |
                                        nn_display_cb()

The MIMO linker has 3 inputs and 2 outputs. Each output specifies which inputs it depends on via a bitmask (see Linker API → MIMO):

Output 0 (MP4): MMIC_DEP_INPUT0 | MMIC_DEP_INPUT2 — V1 + AAC
Output 1 (RTSP): MMIC_DEP_INPUT1 | MMIC_DEP_INPUT2 — V2 + AAC

The V5 NN chain runs entirely independently and does not affect recording or streaming throughput.

Error Handling Pattern

All module open and linker create calls can fail. The standard pattern used across all examples checks the return value and jumps to a cleanup label on failure:

mm_context_t *video_ctx = mm_module_open(&video_module);
if (!video_ctx) {
    RTK_LOGE(TAG, "video open fail\n\r");
    goto error;
}
mm_module_ctrl(video_ctx, CMD_VIDEO_SET_PARAMS,    (int)&video_params);
mm_module_ctrl(video_ctx, MM_CMD_SET_QUEUE_LEN,    60);
mm_module_ctrl(video_ctx, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
mm_module_ctrl(video_ctx, CMD_VIDEO_APPLY,         0);

mm_siso_t *siso = siso_create();
if (!siso) {
    RTK_LOGE(TAG, "siso create fail\n\r");
    goto error;
}
siso_ctrl(siso, MMIC_CMD_ADD_INPUT,  (uint32_t)video_ctx, 0);
siso_ctrl(siso, MMIC_CMD_ADD_OUTPUT, (uint32_t)rtsp_ctx,  0);
siso_start(siso);
return;

error:
if (siso)      { siso_stop(siso);   siso_delete(siso); }
if (video_ctx) { mm_module_close(video_ctx); }
return;

Debug Output

The MMF provides per-subsystem debug logging controlled at runtime. Enable or disable specific categories using the macros in mmf2_dbg.h:

DBG_MMF_DEBUG_MSG_ON(_MMF_DBG_NN_);      // enable NN module debug
DBG_MMF_DEBUG_MSG_ON(_MMF_DBG_LINKER_);  // enable linker debug
DBG_MMF_INFO_MSG_OFF(0xFFFFFFFF);        // suppress all INFO messages

Commonly used flags:

Flag	Module
`_MMF_DBG_VIDEO_`	video_module
`_MMF_DBG_AUDIO_`	audio_module
`_MMF_DBG_RTSP_`	rtsp2_module
`_MMF_DBG_NN_`	vipnn_module
`_MMF_DBG_MODULE_`	Core module framework
`_MMF_DBG_LINKER_`	All linker types (SISO, MIMO, etc.)
`_MMF_DBG_ISP_`	ISP driver
`_MMF_DBG_ENCODER_`	Hardware encoder