VideoDecoder Class Reference

Common interface for video decoders (e.g. H264Decoder, H264DecoderESP32S3 - CodecH264.h/CodecH264ESP32S3.h) - standardizes lifecycle (begin()/end()), the Print target decoded pictures are written to, and the pixel format they're written in (setVideoFormat()), on top of VideoOutput's write()/flush() (the encoded-bitstream input side, inherited unchanged). Concrete decoders may still expose their own additional config knobs beyond this shared surface. More...

#include <CodecVideo.h>

Inheritance diagram for VideoDecoder:

Collaboration diagram for VideoDecoder:

Public Member Functions
virtual	~VideoDecoder ()=default
virtual bool	begin ()=0
	Initializes the decoder (allocates its picture buffers, etc).
virtual VideoFormat	codecFormat ()=0
virtual void	end ()=0
	Releases the decoder's resources.
virtual void	flush ()
virtual uint32_t	getWriteTimeMs () const
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
virtual bool	isKeyFrame (const uint8_t *data, size_t len)
virtual bool	isValid (const uint8_t *data, size_t len)
virtual void	setOutput (Print &out)=0
	Defines the target each decoded picture is written to.
virtual void	setOutput (VideoOutput &out)=0
virtual void	setSkipRender (bool skip)
virtual void	setVideoFormat (VideoFormat format)=0
virtual void	setVideoInfoSource (VideoInfoSource &source)
virtual uint64_t	totalDecodeMs () const
virtual VideoInfo	videoInfo ()=0
virtual size_t	write (const uint8_t *data, size_t len)=0

Detailed Description

Common interface for video decoders (e.g. H264Decoder, H264DecoderESP32S3 - CodecH264.h/CodecH264ESP32S3.h) - standardizes lifecycle (begin()/end()), the Print target decoded pictures are written to, and the pixel format they're written in (setVideoFormat()), on top of VideoOutput's write()/flush() (the encoded-bitstream input side, inherited unchanged). Concrete decoders may still expose their own additional config knobs beyond this shared surface.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

~VideoDecoder()

virtual ~VideoDecoder ( )

virtualdefault

Member Function Documentation

begin()

virtual bool begin ( )

pure virtual

Initializes the decoder (allocates its picture buffers, etc).

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

codecFormat()

virtual VideoFormat codecFormat ( )

pure virtual

The compressed bitstream codec this decoder consumes (e.g. VideoFormat::H264, VideoFormat::MJPEG) - NOT the same axis as setVideoFormat()/videoInfo().format below, which is the decoded pixel format this decoder produces. Used by MultiVideoDecoder::addDecoder() to register a decoder under its own codec instead of the caller having to pass it separately (and possibly get it wrong).

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

end()

virtual void end ( )

pure virtual

Releases the decoder's resources.

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

flush()

virtual void flush ( )

inlinevirtualinherited

Finalizes the frame most recently written via one or more write() calls - see class comment. Default no-op for implementations that display/decode synchronously in write() instead.

Reimplemented in Muxer, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, OutputFPSMeter, OutputOpenCV, PacedVideoOutput, and VideoFrameMeter.

getWriteTimeMs()

virtual uint32_t getWriteTimeMs ( ) const

inlinevirtualinherited

Optional: returns the time (ms) spent in the last write() call.

Reimplemented in OutputTFT_eSPI, OutputTinyGPU, and OutputOpenCV.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

virtual bool isKeyFrame ( const uint8_t *  data, size_t  len  )

inlinevirtualinherited

True if data (one complete encoded frame, as handed to write()) is a keyframe/sync-sample - self-contained, decodable without any earlier frame. Used e.g. by PacedVideoOutput to decide which frames are safe to drop, and whether it's safe to resume decoding after abandoning a backlog (see its own class comment). Default false: a plain VideoOutput doesn't know or care about codec structure - override this in a decoder for the bitstream format it actually parses (see H264Decoder/H264DecoderESP32S3's isH264KeyFrame()-based override, MPGDecoder's isMpeg1KeyFrame()- based one). Getting this right matters beyond bookkeeping: a target whose frames are never recognized as keyframes can leave a caller like PacedVideoOutput unable to ever resume after a resync.

Reimplemented in PacedVideoOutput, H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, and OutputFPSMeter.

isValid()

virtual bool isValid ( const uint8_t *  data, size_t  len  )

inlinevirtual

True if data (the start of an access unit, as handed to write()) looks like this decoder's own bitstream format - content-sniffing, not a guarantee (see the concrete class for exactly what's checked). Used by MultiVideoDecoder to auto-select a registered decoder when no VideoInfoSource answer is available (see its own class comment); not otherwise part of the write()/flush() decode path. Default false, matching VideoOutput::isKeyFrame()'s own default - a decoder not meant to be auto-detected this way (e.g. a hardware-accelerated backend not registered by default, still usable via an explicit VideoInfoSource-based selection) simply never overrides it.

Reimplemented in H264Decoder, MJPEGDecoder, and MPGDecoder.

setOutput() [1/2]

virtual void setOutput ( Print &  out )

pure virtual

Defines the target each decoded picture is written to.

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

setOutput() [2/2]

virtual void setOutput ( VideoOutput &  out )

pure virtual

Defines the target each decoded picture is written to - the VideoOutput-specific counterpart of setOutput(Print&), for a target like OutputTinyGPU/OutputOpenCV that implements VideoOutput but not Print (the two are unrelated types - see VideoFrameMeter's own comment). Part of the base interface (unlike other decoder-specific extras) so code holding only a VideoDecoder& (e.g. VideoPlayer) can wire it to an arbitrary VideoOutput& without knowing the concrete decoder type - every current decoder already implemented this identically before it was pulled up into the interface.

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

setSkipRender()

virtual void setSkipRender ( bool  skip )

inlinevirtualinherited

Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.

Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

setVideoFormat()

virtual void setVideoFormat ( VideoFormat  format )

pure virtual

Selects the pixel format written to setOutput()'s target - e.g. VideoFormat::RGB565 (the common TFT wire format), RGB666/RGB888 for higher color depth displays, or I420 to pass the decoded planes through unconverted. Not every decoder backend supports every value (e.g. RGB666/RGB888 are TinyH264-only, not available on the esp_h264 backend) - unsupported values are logged and ignored (the previously selected format stays in effect); see the concrete class for exactly which ones it supports. Call before begin().

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

setVideoInfoSource()

virtual void setVideoInfoSource ( VideoInfoSource &  source )

inlinevirtualinherited

Optional: registers where width/height/fps/format (VideoInfo) come from - e.g. the demuxer feeding this output, so it can size its own buffers/panel setup without the caller having to duplicate that call per sketch (VideoPlayer::begin() does this automatically for whichever VideoOutput it was given). Default no-op: only implementations that actually need VideoInfo (OutputTinyGPU/ OutputOpenCV/OutputTFT_eSPI) override this.

Reimplemented in MultiVideoDecoder, OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

videoInfo()

virtual VideoInfo videoInfo ( )

pure virtual

Reports the format/dimensions of the picture written to setOutput()'s target - VideoInfo::format is always the format most recently selected via setVideoFormat() (RGB565 if never called), the reliable way to determine it (rather than assuming); width/ height reflect the most recently decoded picture, 0 before any picture has been decoded.

Implemented in H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, and MultiVideoDecoder.

write()

virtual size_t write ( const uint8_t *  data, size_t  len  )

pure virtualinherited

Implemented in MuxerAVI, MuxerMP4, MuxerMPG, MuxerMTS, H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, OutputFPSMeter, OutputOpenCV, OutputTFT_eSPI, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/CodecVideo.h