MuxerMPG Class Reference

MPEG-1 System (Program) Stream Encoder, as defined by ISO/IEC 11172-1: muxes an already-encoded MPEG-1 video elementary stream (ISO/IEC 11172-2) and an optional MPEG-1 audio elementary stream (ISO/IEC 11172-3, Layer I/II/III) into pack_header/system_header/PES_packet framing written to a Print (a local File to record, or e.g. a network Client to publish a live stream). More...

#include <ContainerMPG.h>

Inheritance diagram for MuxerMPG:

Public Member Functions
	MuxerMPG ()
	MuxerMPG (Print &out)
size_t	addAudioFrame (const uint8_t *data, size_t len) override
size_t	addI420Frame (const uint8_t *data, size_t len) override
size_t	addJpegFrame (const uint8_t *data, size_t len) override
size_t	addRGB565Frame (const uint8_t *data, size_t len) override
size_t	addVideoFrame (const uint8_t *data, size_t len, bool isKeyFrame=true) override
size_t	addYUV422Frame (const uint8_t *data, size_t len) override
uint32_t	audioFrameCount ()
AudioInfoFormat &	audioInfo () override
	Provides read/write access to the audio track's AudioInfoFormat.
bool	begin () override
void	end () override
	Writes the MPEG_program_end_code trailer and closes the muxer.
virtual void	flush ()
float	getAudioSamplesPerVideoFrame ()
MuxerVideoConfig	getVideoInfo () override
	Provides the video track configuration.
virtual uint32_t	getWriteTimeMs () const
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
virtual bool	isKeyFrame (const uint8_t *data, size_t len)
const char *	mimeVideo () override
	operator bool () override
void	setAudioInfo (AudioInfoFormat info) override
void	setOutput (Print &out) override
	Defines the output: e.g. a local File or a network Client.
virtual void	setSkipRender (bool skip)
void	setStreamType (StreamContentType type) override
	Selects whether write() feeds the video or the audio track.
void	setVideoInfo (MuxerVideoConfig config) override
virtual void	setVideoInfoSource (VideoInfoSource &source)
StreamContentType	streamType () override
	The track write() currently targets (see setStreamType())
virtual uint64_t	totalDecodeMs () const
uint32_t	videoFrameCount ()
size_t	write (const uint8_t *data, size_t len) override

Protected Member Functions
uint64_t	audioPts ()
uint64_t	clampScr (uint64_t scr)
uint64_t	videoPts ()
size_t	writeAccessUnit (uint8_t stream_id, const uint8_t *data, size_t len, uint64_t pts)
size_t	writeElementaryChunk (uint8_t stream_id, const uint8_t *data, size_t len, bool withPts, uint64_t pts)
	Writes one pack_header + PES header + payload chunk (<= a few KB).
void	writePackHeader (uint64_t scr_in)
void	writeStreamEntry (uint8_t id, bool isVideo)
void	writeSystemHeader ()
void	writeTimestampField (uint8_t id4, uint64_t ts)

Protected Attributes
uint32_t	audio_frame_count = 0
AudioInfoFormat	audio_info
bool	has_audio = false
bool	is_open = false
uint64_t	last_scr = 0
Print *	p_out = nullptr
Print *	p_print = nullptr
MuxerVideoConfig	video_cfg
uint32_t	video_frame_count = 0
StreamContentType	write_stream_type = StreamContentType::Video

Static Protected Attributes
static const uint32_t	AUDIO_SAMPLES_PER_FRAME = 1152
	MPEG-1 Layer II/III audio frames are always 1152 samples.
static const size_t	MAX_PES_PAYLOAD = 4096
static const uint32_t	MUX_RATE = 0x3FFFFF

Detailed Description

MPEG-1 System (Program) Stream Encoder, as defined by ISO/IEC 11172-1: muxes an already-encoded MPEG-1 video elementary stream (ISO/IEC 11172-2) and an optional MPEG-1 audio elementary stream (ISO/IEC 11172-3, Layer I/II/III) into pack_header/system_header/PES_packet framing written to a Print (a local File to record, or e.g. a network Client to publish a live stream).

This is a streaming writer, like MuxerAVI: mux_rate is written as an unspecified placeholder (all bits set) and there is no index/seek table - sufficient for sequential playback but not for random access. Each addVideoFrame()/addAudioFrame() call is one complete access unit; large video frames are automatically split across multiple pack/PES units (PES payload is limited to 16 bits) - only the first fragment of an access unit carries a PTS, exactly what DemuxerMPG uses to find frame boundaries again. SCR/PTS are derived from the frame/sample counters and the configured fps/sample_rate (MPEG-1 audio access units are assumed to be 1152 samples, true for Layer II and Layer III), not a wall clock.

Usage:

MuxerMPG mpg(client); // any Print: File, WiFiClient, ...
MuxerVideoConfig cfg;
cfg.width = 352;
cfg.height = 288;
cfg.fps = 25;
cfg.format = VideoFormat::MPEG1;
mpg.setVideoInfo(cfg);
mpg.begin();
// for each encoded MPEG-1 picture (a run of ES bytes up to the next
// picture_start_code):
mpg.addVideoFrame(mpeg1_picture_data, len);

An MPEG audio track can be fed directly from an AudioEncoder (e.g. MP3EncoderLAME) - Muxer is itself a Print, so pointing the encoder's output at the muxer and toggling setStreamType() around the PCM write() is all that is needed:

mpg.setAudioInfo(AudioInfoFormat(44100, 2, 16, AudioFormat::MP3));
MP3EncoderLAME mp3;
mp3.setOutput(mpg);
mp3.begin(AudioInfo(44100, 2, 16));
mpg.setStreamType(StreamContentType::Audio);  // mpg.write() -> addAudioFrame()
mp3.write(pcm_data, len);

(VideoMuxer/VideoMuxerWithTasks do this setStreamType() toggling, and the video-side driving, automatically - MuxerMPG implements the same Muxer interface as MuxerAVI/MuxerMP4, so it plugs into them unmodified.) Note that the SCR/PTS this muxer writes assume each addAudioFrame()/write() call is exactly one 1152-sample MPEG audio frame; an encoder whose write() calls don't align to real frame boundaries (LAME's often don't) will still produce a byte-correct, fully decodable elementary stream - only the fine-grained timestamp hinting in the file becomes approximate.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

MuxerMPG() [1/2]

MuxerMPG ( )

inline

MuxerMPG() [2/2]

MuxerMPG ( Print &  out )

inline

Member Function Documentation

addAudioFrame()

size_t addAudioFrame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Writes one complete MPEG-1 audio frame (e.g. one Layer II/III frame, starting at its sync word) as a single pack/PES unit. Ignored (returns 0) if no audio track was configured via setAudioInfo().

Implements Muxer.

addI420Frame()

size_t addI420Frame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Writes one planar 4:2:0 YUV (I420/IYUV) frame. Expects exactly width*height*3/2 bytes.

Implements Muxer.

addJpegFrame()

size_t addJpegFrame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Program Stream has no defined mapping for raw/JPEG pixel data - writes the buffer through as if it were an MPEG-1 video ES fragment, with a warning, purely to satisfy the Muxer interface.

Implements Muxer.

addRGB565Frame()

size_t addRGB565Frame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Writes one uncompressed RGB565 (16-bit, 5-6-5) frame. Expects exactly width*height*2 bytes.

Implements Muxer.

addVideoFrame()

size_t addVideoFrame ( const uint8_t *  data, size_t  len, bool  isKeyFrame = true  )

inlineoverridevirtual

Writes one complete MPEG-1 picture (a run of video ES bytes, normally starting at its picture_start_code) as one or more pack/PES units.

Parameters

isKeyFrame

accepted for interface compatibility with Muxer::addVideoFrame() - the MPEG-1 video ES already encodes I/P/B picture_coding_type itself, so there is no separate container-level flag to write it into (same convention as MuxerAVI).

Implements Muxer.

addYUV422Frame()

size_t addYUV422Frame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Writes one packed 4:2:2 YUV frame (YUY2/YUYV byte order). Expects exactly width*height*2 bytes.

Implements Muxer.

audioFrameCount()

uint32_t audioFrameCount ( )

inline

audioInfo()

AudioInfoFormat & audioInfo ( )

inlineoverridevirtual

Provides read/write access to the audio track's AudioInfoFormat.

Implements Muxer.

audioPts()

uint64_t audioPts ( )

inlineprotected

begin()

bool begin ( )

inlineoverridevirtual

Writes the initial pack_header + system_header. Call after configuring video (and audio, if any) and before writing any frames.

Implements Muxer.

clampScr()

uint64_t clampScr ( uint64_t  scr )

inlineprotected

end()

void end ( )

inlineoverridevirtual

Writes the MPEG_program_end_code trailer and closes the muxer.

Implements Muxer.

flush()

virtual void flush ( )

inlinevirtualinherited

Finalizes the frame most recently written via one or more write() calls - see class comment. Default no-op for implementations that display/decode synchronously in write() instead.

Reimplemented from VideoOutput.

getAudioSamplesPerVideoFrame()

float getAudioSamplesPerVideoFrame ( )

inlineinherited

Average number of audio samples per video frame, derived from audioInfo().sample_rate and getVideoInfo().fps - the natural audio chunk size to write once per video frame if you want to keep both tracks advancing at roughly the same pace as you write them (not a hard requirement - see addAudioFrame()/addVideoFrame()). 0 if fps hasn't been set.

getVideoInfo()

MuxerVideoConfig getVideoInfo ( )

inlineoverridevirtual

Provides the video track configuration.

Implements Muxer.

getWriteTimeMs()

virtual uint32_t getWriteTimeMs ( ) const

inlinevirtualinherited

Optional: returns the time (ms) spent in the last write() call.

Reimplemented in OutputTFT_eSPI, OutputTinyGPU, and OutputOpenCV.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

virtual bool isKeyFrame ( const uint8_t *  data, size_t  len  )

inlinevirtualinherited

True if data (one complete encoded frame, as handed to write()) is a keyframe/sync-sample - self-contained, decodable without any earlier frame. Used e.g. by PacedVideoOutput to decide which frames are safe to drop, and whether it's safe to resume decoding after abandoning a backlog (see its own class comment). Default false: a plain VideoOutput doesn't know or care about codec structure - override this in a decoder for the bitstream format it actually parses (see H264Decoder/H264DecoderESP32S3's isH264KeyFrame()-based override, MPGDecoder's isMpeg1KeyFrame()- based one). Getting this right matters beyond bookkeeping: a target whose frames are never recognized as keyframes can leave a caller like PacedVideoOutput unable to ever resume after a resync.

Reimplemented in PacedVideoOutput, H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, and OutputFPSMeter.

mimeVideo()

const char * mimeVideo ( )

inlineoverridevirtual

The container's MIME type (e.g. "video/avi", "video/mp4") - useful for e.g. an HTTP Content-Type header when streaming the muxed output to a client.

Implements Muxer.

operator bool()

operator bool ( )

inlineoverridevirtual

Implements Muxer.

setAudioInfo()

void setAudioInfo ( AudioInfoFormat  info )

inlineoverridevirtual

Adds an (optional) audio track. info.format selects the codec tag (default/only meaningful choice: AudioFormat::MP3, used generically here for any MPEG-1 Layer I/II/III elementary stream). Call before begin().

Implements Muxer.

setOutput()

void setOutput ( Print &  out )

inlineoverridevirtual

Defines the output: e.g. a local File or a network Client.

Implements Muxer.

setSkipRender()

virtual void setSkipRender ( bool  skip )

inlinevirtualinherited

Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.

Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

setStreamType()

void setStreamType ( StreamContentType  type )

inlineoverridevirtual

Selects whether write() feeds the video or the audio track.

Implements Muxer.

setVideoInfo()

void setVideoInfo ( MuxerVideoConfig  config )

inlineoverridevirtual

Defines the video track configuration - call before begin(). Set config.format = VideoFormat::MPEG1 (the only format this container carries as a real video track).

Implements Muxer.

setVideoInfoSource()

virtual void setVideoInfoSource ( VideoInfoSource &  source )

inlinevirtualinherited

Optional: registers where width/height/fps/format (VideoInfo) come from - e.g. the demuxer feeding this output, so it can size its own buffers/panel setup without the caller having to duplicate that call per sketch (VideoPlayer::begin() does this automatically for whichever VideoOutput it was given). Default no-op: only implementations that actually need VideoInfo (OutputTinyGPU/ OutputOpenCV/OutputTFT_eSPI) override this.

Reimplemented in MultiVideoDecoder, OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.

streamType()

StreamContentType streamType ( )

inlineoverridevirtual

The track write() currently targets (see setStreamType())

Implements Muxer.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

videoFrameCount()

uint32_t videoFrameCount ( )

inline

videoPts()

uint64_t videoPts ( )

inlineprotected

write()

size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

VideoOutput API / generic sink: writes one complete frame to whichever track streamType() currently selects.

Implements VideoOutput.

writeAccessUnit()

size_t writeAccessUnit ( uint8_t  stream_id, const uint8_t *  data, size_t  len, uint64_t  pts  )

inlineprotected

Splits one access unit (a full encoded video picture, or one audio frame) across as many pack/PES units as needed; only the first carries a PTS.

writeElementaryChunk()

size_t writeElementaryChunk ( uint8_t  stream_id, const uint8_t *  data, size_t  len, bool  withPts, uint64_t  pts  )

inlineprotected

Writes one pack_header + PES header + payload chunk (<= a few KB).

writePackHeader()

void writePackHeader ( uint64_t  scr_in )

inlineprotected

writeStreamEntry()

void writeStreamEntry ( uint8_t  id, bool  isVideo  )

inlineprotected

writeSystemHeader()

void writeSystemHeader ( )

inlineprotected

writeTimestampField()

void writeTimestampField ( uint8_t  id4, uint64_t  ts  )

inlineprotected

Member Data Documentation

audio_frame_count

uint32_t audio_frame_count = 0

protected

audio_info

AudioInfoFormat audio_info

protected

AUDIO_SAMPLES_PER_FRAME

const uint32_t AUDIO_SAMPLES_PER_FRAME = 1152

staticprotected

MPEG-1 Layer II/III audio frames are always 1152 samples.

has_audio

bool has_audio = false

protected

is_open

bool is_open = false

protected

last_scr

uint64_t last_scr = 0

protected

MAX_PES_PAYLOAD

const size_t MAX_PES_PAYLOAD = 4096

staticprotected

Video/audio PES payload is chunked at this size so the 16-bit PES_packet_length field (and the optional-header bytes) always fit, with margin to spare - real encoders commonly interleave in similarly sized packets rather than one huge PES per access unit.

MUX_RATE

const uint32_t MUX_RATE = 0x3FFFFF

staticprotected

mux_rate: 22-bit field, unit 50 bytes/sec - all-ones is used here as an "unspecified/streaming" placeholder, the same convention MuxerAVI uses for its unknown RIFF/movi sizes.

p_out

Print* p_out = nullptr

protected

p_print

Print* p_print = nullptr

protectedinherited

video_cfg

MuxerVideoConfig video_cfg

protected

video_frame_count

uint32_t video_frame_count = 0

protected

write_stream_type

StreamContentType write_stream_type = StreamContentType::Video

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/AudioCodecs/ContainerMPG.h