H264DecoderESP32S3< Alloc > Class Template Reference

H.264 video decoder for ESP32-S3: wraps esp_h264::H264Decoder (https://github.com/pschatzmann/ESP32S3-h264) as a VideoOutput, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/ DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h, the portable TinyH264-based decoder) plays, for boards where the ESP32-S3-specific esp_h264 backend is preferred instead. Consumes an Annex-B H.264 bitstream via write() and, once a complete picture has been decoded, writes it - converted to setVideoFormat()'s format (RGB565 by default, the common TFT wire format) - to the Print target configured via setOutput(), one write() call per decoded picture. More...

#include <CodecH264ESP32S3.h>

Inheritance diagram for H264DecoderESP32S3< Alloc >:

Public Member Functions
	H264DecoderESP32S3 ()
	H264DecoderESP32S3 (Print &out)
	H264DecoderESP32S3 (VideoOutput &out)
bool	begin () override
	Initializes the decoder - see esp_h264::H264Decoder::begin().
VideoFormat	codecFormat () override
uint32_t	decodeErrors () const
	Number of decode errors since begin().
esp_h264::H264Decoder< Alloc > &	driver ()
	Direct access to the wrapped esp_h264::H264Decoder.
void	end () override
	Releases the decoder's resources - see esp_h264::H264Decoder::end().
void	flush () override
uint32_t	frameCount () const
	Number of frames successfully decoded since begin().
virtual uint32_t	getWriteTimeMs () const
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
bool	isKeyFrame (const uint8_t *data, size_t len) override
	See VideoOutput::isKeyFrame() - scans for an Annex-B IDR slice NAL.
virtual bool	isValid (const uint8_t *data, size_t len)
void	setInputBufferSize (size_t size)
void	setOutput (Print &out) override
void	setOutput (VideoOutput &out) override
void	setOutputBufferSize (size_t size)
virtual void	setSkipRender (bool skip)
void	setVideoFormat (VideoFormat format) override
virtual void	setVideoInfoSource (VideoInfoSource &source)
virtual uint64_t	totalDecodeMs () const
VideoInfo	videoInfo () override
size_t	write (const uint8_t *data, size_t len) override

Protected Attributes
esp_h264::H264Decoder< Alloc >::Config	config_
esp_h264::H264Decoder< Alloc >	decoder_
Print *	p_out = nullptr
VideoOutput *	p_out_video = nullptr

Detailed Description

template<typename Alloc = H264_DEFAULT_ALLOCATOR>
class audio_tools::H264DecoderESP32S3< Alloc >

H.264 video decoder for ESP32-S3: wraps esp_h264::H264Decoder (https://github.com/pschatzmann/ESP32S3-h264) as a VideoOutput, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/ DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h, the portable TinyH264-based decoder) plays, for boards where the ESP32-S3-specific esp_h264 backend is preferred instead. Consumes an Annex-B H.264 bitstream via write() and, once a complete picture has been decoded, writes it - converted to setVideoFormat()'s format (RGB565 by default, the common TFT wire format) - to the Print target configured via setOutput(), one write() call per decoded picture.

write()/flush() implement the VideoOutput contract (see Video.h): write() may be called one or more times per frame - bytes are forwarded to the wrapped decoder's decode() immediately, which invokes the frame callback synchronously for every completed picture found in that call, not deferred to flush(). flush() is a no-op, kept only because some producers (e.g. DemuxerAVI/DemuxerMP4) call it unconditionally after each frame.

No setup beyond setOutput() is required before begin() - unlike esp_h264::H264Decoder's own defaultConfig() (sized for VGA), this wrapper defaults its input/output buffers to sizes that fit typical small-panel use directly; setInputBufferSize()/setOutputBufferSize() exist only as an optional performance tweak (see their own comments).

The Alloc template parameter is forwarded to esp_h264::H264Decoder - see that class's own file comment (H264Decoder.h) for the PSRAM- vs. internal-RAM allocator choice (H264DecoderPSRAM/H264DecoderRAM aliases). Defaults to H264_DEFAULT_ALLOCATOR.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

H264DecoderESP32S3() [1/3]

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

H264DecoderESP32S3 ( )

inline

H264DecoderESP32S3() [2/3]

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

H264DecoderESP32S3 ( Print & out )

inline

H264DecoderESP32S3() [3/3]

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

H264DecoderESP32S3 ( VideoOutput & out )

inline

Member Function Documentation

begin()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

bool begin ( )

inlineoverridevirtual

Initializes the decoder - see esp_h264::H264Decoder::begin().

Implements VideoDecoder.

codecFormat()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

VideoFormat codecFormat ( )

inlineoverridevirtual

The compressed bitstream codec this decoder consumes (e.g. VideoFormat::H264, VideoFormat::MJPEG) - NOT the same axis as setVideoFormat()/videoInfo().format below, which is the decoded pixel format this decoder produces. Used by MultiVideoDecoder::addDecoder() to register a decoder under its own codec instead of the caller having to pass it separately (and possibly get it wrong).

Implements VideoDecoder.

decodeErrors()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

uint32_t decodeErrors ( ) const

inline

Number of decode errors since begin().

driver()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

esp_h264::H264Decoder< Alloc > & driver ( )

inline

Direct access to the wrapped esp_h264::H264Decoder.

end()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void end ( )

inlineoverridevirtual

Releases the decoder's resources - see esp_h264::H264Decoder::end().

Implements VideoDecoder.

flush()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void flush ( )

inlineoverridevirtual

No-op - kept to satisfy the VideoOutput/Print contract; decoding already happens synchronously in write() (see class comment).

Reimplemented from VideoOutput.

frameCount()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

uint32_t frameCount ( ) const

inline

Number of frames successfully decoded since begin().

getWriteTimeMs()

virtual uint32_t getWriteTimeMs ( ) const

inlinevirtualinherited

Optional: returns the time (ms) spent in the last write() call.

Reimplemented in OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

bool isKeyFrame ( const uint8_t * data, size_t len )

inlineoverridevirtual

See VideoOutput::isKeyFrame() - scans for an Annex-B IDR slice NAL.

Reimplemented from VideoOutput.

isValid()

virtual bool isValid ( const uint8_t * data, size_t len )

inlinevirtualinherited

True if data (the start of an access unit, as handed to write()) looks like this decoder's own bitstream format - content-sniffing, not a guarantee (see the concrete class for exactly what's checked). Used by MultiVideoDecoder to auto-select a registered decoder when no VideoInfoSource answer is available (see its own class comment); not otherwise part of the write()/flush() decode path. Default false, matching VideoOutput::isKeyFrame()'s own default - a decoder not meant to be auto-detected this way (e.g. a hardware-accelerated backend not registered by default, still usable via an explicit VideoInfoSource-based selection) simply never overrides it.

Reimplemented in H264Decoder, MJPEGDecoder, and MPGDecoder.

setInputBufferSize()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void setInputBufferSize ( size_t size )

inline

Optional: pre-sizes the input buffer the wrapped decoder copies write()'s data into - purely a performance knob, not a requirement: esp_h264::H264Decoder grows it on demand if a frame ever exceeds it (see H264Decoder::decode()), so a too-small (or never-called) value still decodes correctly, just with an occasional reallocation. Defaults to 64KB (see the constructor's comment). Call before begin().

setOutput() [1/2]

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void setOutput ( Print & out )

inlineoverridevirtual

Defines the target the decoded picture is written to, one write() call per decoded picture, in the format selected via setVideoFormat() (RGB565 by default). Call before begin().

Implements VideoDecoder.

setOutput() [2/2]

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void setOutput ( VideoOutput & out )

inlineoverridevirtual

Defines the target each decoded picture is written to - the VideoOutput-specific counterpart of setOutput(Print&), for a target like OutputTinyGPU/OutputOpenCV that implements VideoOutput but not Print (the two are unrelated types - see VideoFrameMeter's own comment). Part of the base interface (unlike other decoder-specific extras) so code holding only a VideoDecoder& (e.g. VideoPlayer) can wire it to an arbitrary VideoOutput& without knowing the concrete decoder type - every current decoder already implemented this identically before it was pulled up into the interface.

Implements VideoDecoder.

setOutputBufferSize()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void setOutputBufferSize ( size_t size )

inline

Optional: pre-sizes the output buffer the wrapped decoder allocates for the decoded/converted picture - purely a performance knob, not a requirement: it grows on demand to fit width*height*bytes-per- pixel for setVideoFormat()'s format if this proves too small (see H264Decoder::processDecodedFrame()). Defaults to 100KB (see the constructor's comment). Call before begin().

setSkipRender()

virtual void setSkipRender ( bool skip )

inlinevirtualinherited

Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.

Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

setVideoFormat()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

void setVideoFormat ( VideoFormat format )

inlineoverridevirtual

Selects the pixel format written to setOutput()'s target - this backend (esp_h264) only supports VideoFormat::RGB565 (the default) and VideoFormat::I420; any other value is logged and ignored (the previously selected format stays in effect) - use driver() and esp_h264_types.h's other ESP_H264_RAW_FMT_* values directly if you need one of those instead. Call before begin().

Implements VideoDecoder.

setVideoInfoSource()

virtual void setVideoInfoSource ( VideoInfoSource & source )

inlinevirtualinherited

Optional: registers where width/height/fps/format (VideoInfo) come from - e.g. the demuxer feeding this output, so it can size its own buffers/panel setup without the caller having to duplicate that call per sketch (VideoPlayer::begin() does this automatically for whichever VideoOutput it was given). Default no-op: only implementations that actually need VideoInfo (OutputTinyGPU/ OutputOpenCV/OutputTFT_eSPI) override this.

Reimplemented in MultiVideoDecoder, OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

videoInfo()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

VideoInfo videoInfo ( )

inlineoverridevirtual

Reports the format/dimensions of the picture written to setOutput()'s target - format is setVideoFormat()'s most recently selected value (RGB565 if never called); width/height are the most recently decoded picture's (0 before any picture has been decoded).

Implements VideoDecoder.

write()

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

size_t write ( const uint8_t * data, size_t len )

inlineoverridevirtual

Feeds one chunk of Annex-B H.264 data - may be called more than once per frame. Decodes immediately, invoking setOutput()'s Print once per completed picture from within this call (not deferred to flush()).

Implements VideoOutput.

Member Data Documentation

config_

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

esp_h264::H264Decoder<Alloc>::Config config_

protected

decoder_

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

esp_h264::H264Decoder<Alloc> decoder_

protected

p_out

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

Print* p_out = nullptr

protected

p_out_video

template<typename Alloc = H264_DEFAULT_ALLOCATOR>

VideoOutput* p_out_video = nullptr

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/CodecH264ESP32S3.h