H.264 video decoder for ESP32-S3: wraps esp_h264::H264Decoder (https://github.com/pschatzmann/ESP32S3-h264) as a VideoOutput, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/ DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h, the portable TinyH264-based decoder) plays, for boards where the ESP32-S3-specific esp_h264 backend is preferred instead. Consumes an Annex-B H.264 bitstream via write() and, once a complete picture has been decoded, writes it - converted to setVideoFormat()'s format (RGB565 by default, the common TFT wire format) - to the Print target configured via setOutput(), one write() call per decoded picture. More...
#include <CodecH264ESP32S3.h>
Public Member Functions | |
| H264DecoderESP32S3 () | |
| H264DecoderESP32S3 (Print &out) | |
| H264DecoderESP32S3 (VideoOutput &out) | |
| bool | begin () override |
| Initializes the decoder - see esp_h264::H264Decoder::begin(). | |
| VideoFormat | codecFormat () override |
| uint32_t | decodeErrors () const |
| Number of decode errors since begin(). | |
| esp_h264::H264Decoder< Alloc > & | driver () |
| Direct access to the wrapped esp_h264::H264Decoder. | |
| void | end () override |
| Releases the decoder's resources - see esp_h264::H264Decoder::end(). | |
| void | flush () override |
| uint32_t | frameCount () const |
| Number of frames successfully decoded since begin(). | |
| virtual uint32_t | getWriteTimeMs () const |
| Optional: returns the time (ms) spent in the last write() call. | |
| virtual bool | hadOutput () const |
| bool | isKeyFrame (const uint8_t *data, size_t len) override |
| See VideoOutput::isKeyFrame() - scans for an Annex-B IDR slice NAL. | |
| virtual bool | isValid (const uint8_t *data, size_t len) |
| void | setInputBufferSize (size_t size) |
| void | setOutput (Print &out) override |
| void | setOutput (VideoOutput &out) override |
| void | setOutputBufferSize (size_t size) |
| virtual void | setSkipRender (bool skip) |
| void | setVideoFormat (VideoFormat format) override |
| virtual void | setVideoInfoSource (VideoInfoSource &source) |
| virtual uint64_t | totalDecodeMs () const |
| VideoInfo | videoInfo () override |
| size_t | write (const uint8_t *data, size_t len) override |
Protected Attributes | |
| esp_h264::H264Decoder< Alloc >::Config | config_ |
| esp_h264::H264Decoder< Alloc > | decoder_ |
| Print * | p_out = nullptr |
| VideoOutput * | p_out_video = nullptr |
Detailed Description
class audio_tools::H264DecoderESP32S3< Alloc >
H.264 video decoder for ESP32-S3: wraps esp_h264::H264Decoder (https://github.com/pschatzmann/ESP32S3-h264) as a VideoOutput, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/ DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h, the portable TinyH264-based decoder) plays, for boards where the ESP32-S3-specific esp_h264 backend is preferred instead. Consumes an Annex-B H.264 bitstream via write() and, once a complete picture has been decoded, writes it - converted to setVideoFormat()'s format (RGB565 by default, the common TFT wire format) - to the Print target configured via setOutput(), one write() call per decoded picture.
write()/flush() implement the VideoOutput contract (see Video.h): write() may be called one or more times per frame - bytes are forwarded to the wrapped decoder's decode() immediately, which invokes the frame callback synchronously for every completed picture found in that call, not deferred to flush(). flush() is a no-op, kept only because some producers (e.g. DemuxerAVI/DemuxerMP4) call it unconditionally after each frame.
No setup beyond setOutput() is required before begin() - unlike esp_h264::H264Decoder's own defaultConfig() (sized for VGA), this wrapper defaults its input/output buffers to sizes that fit typical small-panel use directly; setInputBufferSize()/setOutputBufferSize() exist only as an optional performance tweak (see their own comments).
The Alloc template parameter is forwarded to esp_h264::H264Decoder - see that class's own file comment (H264Decoder.h) for the PSRAM- vs. internal-RAM allocator choice (H264DecoderPSRAM/H264DecoderRAM aliases). Defaults to H264_DEFAULT_ALLOCATOR.
- Copyright
- GPLv3
Constructor & Destructor Documentation
◆ H264DecoderESP32S3() [1/3]
|
inline |
◆ H264DecoderESP32S3() [2/3]
|
inline |
◆ H264DecoderESP32S3() [3/3]
|
inline |
Member Function Documentation
◆ begin()
|
inlineoverridevirtual |
Initializes the decoder - see esp_h264::H264Decoder::begin().
Implements VideoDecoder.
◆ codecFormat()
|
inlineoverridevirtual |
The compressed bitstream codec this decoder consumes (e.g. VideoFormat::H264, VideoFormat::MJPEG) - NOT the same axis as setVideoFormat()/videoInfo().format below, which is the decoded pixel format this decoder produces. Used by MultiVideoDecoder::addDecoder() to register a decoder under its own codec instead of the caller having to pass it separately (and possibly get it wrong).
Implements VideoDecoder.
◆ decodeErrors()
|
inline |
Number of decode errors since begin().
◆ driver()
|
inline |
Direct access to the wrapped esp_h264::H264Decoder.
◆ end()
|
inlineoverridevirtual |
Releases the decoder's resources - see esp_h264::H264Decoder::end().
Implements VideoDecoder.
◆ flush()
|
inlineoverridevirtual |
No-op - kept to satisfy the VideoOutput/Print contract; decoding already happens synchronously in write() (see class comment).
Reimplemented from VideoOutput.
◆ frameCount()
|
inline |
Number of frames successfully decoded since begin().
◆ getWriteTimeMs()
|
inlinevirtualinherited |
Optional: returns the time (ms) spent in the last write() call.
Reimplemented in OutputTFT_eSPI, OutputTinyGPU, and OutputOpenCV.
◆ hadOutput()
|
inlinevirtualinherited |
True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.
Reimplemented in MPGDecoder.
◆ isKeyFrame()
|
inlineoverridevirtual |
See VideoOutput::isKeyFrame() - scans for an Annex-B IDR slice NAL.
Reimplemented from VideoOutput.
◆ isValid()
|
inlinevirtualinherited |
True if data (the start of an access unit, as handed to write()) looks like this decoder's own bitstream format - content-sniffing, not a guarantee (see the concrete class for exactly what's checked). Used by MultiVideoDecoder to auto-select a registered decoder when no VideoInfoSource answer is available (see its own class comment); not otherwise part of the write()/flush() decode path. Default false, matching VideoOutput::isKeyFrame()'s own default - a decoder not meant to be auto-detected this way (e.g. a hardware-accelerated backend not registered by default, still usable via an explicit VideoInfoSource-based selection) simply never overrides it.
Reimplemented in H264Decoder, MJPEGDecoder, and MPGDecoder.
◆ setInputBufferSize()
|
inline |
Optional: pre-sizes the input buffer the wrapped decoder copies write()'s data into - purely a performance knob, not a requirement: esp_h264::H264Decoder grows it on demand if a frame ever exceeds it (see H264Decoder::decode()), so a too-small (or never-called) value still decodes correctly, just with an occasional reallocation. Defaults to 64KB (see the constructor's comment). Call before begin().
◆ setOutput() [1/2]
|
inlineoverridevirtual |
Defines the target the decoded picture is written to, one write() call per decoded picture, in the format selected via setVideoFormat() (RGB565 by default). Call before begin().
Implements VideoDecoder.
◆ setOutput() [2/2]
|
inlineoverridevirtual |
Defines the target each decoded picture is written to - the VideoOutput-specific counterpart of setOutput(Print&), for a target like OutputTinyGPU/OutputOpenCV that implements VideoOutput but not Print (the two are unrelated types - see VideoFrameMeter's own comment). Part of the base interface (unlike other decoder-specific extras) so code holding only a VideoDecoder& (e.g. VideoPlayer) can wire it to an arbitrary VideoOutput& without knowing the concrete decoder type - every current decoder already implemented this identically before it was pulled up into the interface.
Implements VideoDecoder.
◆ setOutputBufferSize()
|
inline |
Optional: pre-sizes the output buffer the wrapped decoder allocates for the decoded/converted picture - purely a performance knob, not a requirement: it grows on demand to fit width*height*bytes-per- pixel for setVideoFormat()'s format if this proves too small (see H264Decoder::processDecodedFrame()). Defaults to 100KB (see the constructor's comment). Call before begin().
◆ setSkipRender()
|
inlinevirtualinherited |
Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.
Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.
◆ setVideoFormat()
|
inlineoverridevirtual |
Selects the pixel format written to setOutput()'s target - this backend (esp_h264) only supports VideoFormat::RGB565 (the default) and VideoFormat::I420; any other value is logged and ignored (the previously selected format stays in effect) - use driver() and esp_h264_types.h's other ESP_H264_RAW_FMT_* values directly if you need one of those instead. Call before begin().
Implements VideoDecoder.
◆ setVideoInfoSource()
|
inlinevirtualinherited |
Optional: registers where width/height/fps/format (VideoInfo) come from - e.g. the demuxer feeding this output, so it can size its own buffers/panel setup without the caller having to duplicate that call per sketch (VideoPlayer::begin() does this automatically for whichever VideoOutput it was given). Default no-op: only implementations that actually need VideoInfo (OutputTinyGPU/ OutputOpenCV/OutputTFT_eSPI) override this.
Reimplemented in MultiVideoDecoder, OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.
◆ totalDecodeMs()
|
inlinevirtualinherited |
Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.
Reimplemented in H264Decoder, and MultiVideoDecoder.
◆ videoInfo()
|
inlineoverridevirtual |
Reports the format/dimensions of the picture written to setOutput()'s target - format is setVideoFormat()'s most recently selected value (RGB565 if never called); width/height are the most recently decoded picture's (0 before any picture has been decoded).
Implements VideoDecoder.
◆ write()
|
inlineoverridevirtual |
Feeds one chunk of Annex-B H.264 data - may be called more than once per frame. Decodes immediately, invoking setOutput()'s Print once per completed picture from within this call (not deferred to flush()).
Implements VideoOutput.
Member Data Documentation
◆ config_
|
protected |
◆ decoder_
|
protected |
◆ p_out
|
protected |
◆ p_out_video
|
protected |
The documentation for this class was generated from the following file:
- src/AudioTools/Video/CodecH264ESP32S3.h
