OutputOpenCV Class Reference

Display a video frame with OpenCV, to be used on the desktop - VideoFormat::MJPEG (the default) expects one complete, already-encoded JPEG image assembled across write() calls and closed off by flush(), as produced by e.g. DemuxerAVI. Any other setVideoFormat() value (e.g. RGB565, the common raw picture format decoders like H264Decoder produce) expects one complete, already-decoded picture per write() call instead - setSize() must be called too in that case, since unlike JPEG a raw picture doesn't carry its own dimensions. More...

#include <OutputOpenCV.h>

Inheritance diagram for OutputOpenCV:

Public Member Functions
	OutputOpenCV ()=default
void	flush () override
uint32_t	getWriteTimeMs () const override
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
virtual bool	isKeyFrame (const uint8_t *data, size_t len)
void	setSize (uint16_t width, uint16_t height)
virtual void	setSkipRender (bool skip)
void	setVideoFormat (VideoFormat format)
	Selects the format of the data passed to write() - see class comment.
void	setVideoInfoSource (VideoInfoSource &source) override
virtual uint64_t	totalDecodeMs () const
size_t	write (const uint8_t *data, size_t len) override

Protected Member Functions
void	displayJpeg ()
void	displayRaw (const uint8_t *data, size_t len)
void	ensureWindow ()

Protected Attributes
bool	create_window = true
uint16_t	height = 0
std::vector< uint8_t >	img_vector
VideoInfoSource *	p_info = nullptr
size_t	pos = 0
uint64_t	start = 0
std::vector< uint8_t >	swap_buffer
VideoFormat	video_format = VideoFormat::MJPEG
uint16_t	width = 0
const char *	window = "Movie"
uint32_t	write_time_ms = 0

Detailed Description

Display a video frame with OpenCV, to be used on the desktop - VideoFormat::MJPEG (the default) expects one complete, already-encoded JPEG image assembled across write() calls and closed off by flush(), as produced by e.g. DemuxerAVI. Any other setVideoFormat() value (e.g. RGB565, the common raw picture format decoders like H264Decoder produce) expects one complete, already-decoded picture per write() call instead - setSize() must be called too in that case, since unlike JPEG a raw picture doesn't carry its own dimensions.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

OutputOpenCV()

OutputOpenCV ( )

default

Member Function Documentation

displayJpeg()

void displayJpeg ( )

inlineprotected

displayRaw()

void displayRaw ( const uint8_t *  data, size_t  len  )

inlineprotected

Wraps one complete, already-decoded raw picture (no copy - OpenCV reads directly out of 'data') and converts it to OpenCV's native BGR for display. Not every VideoFormat OpenCV could conceivably display is covered here - see the switch below for exactly which ones are.

ensureWindow()

void ensureWindow ( )

inlineprotected

flush()

void flush ( )

inlineoverridevirtual

Displays the assembled jpeg image, then resets for the next frame - only relevant for VideoFormat::MJPEG; raw formats display immediately from write() instead (one complete picture per call), so this is a no-op for those.

Reimplemented from VideoOutput.

getWriteTimeMs()

uint32_t getWriteTimeMs ( ) const

inlineoverridevirtual

Optional: returns the time (ms) spent in the last write() call.

Reimplemented from VideoOutput.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

virtual bool isKeyFrame ( const uint8_t *  data, size_t  len  )

inlinevirtualinherited

True if data (one complete encoded frame, as handed to write()) is a keyframe/sync-sample - self-contained, decodable without any earlier frame. Used e.g. by PacedVideoOutput to decide which frames are safe to drop, and whether it's safe to resume decoding after abandoning a backlog (see its own class comment). Default false: a plain VideoOutput doesn't know or care about codec structure - override this in a decoder for the bitstream format it actually parses (see H264Decoder/H264DecoderESP32S3's isH264KeyFrame()-based override, MPGDecoder's isMpeg1KeyFrame()- based one). Getting this right matters beyond bookkeeping: a target whose frames are never recognized as keyframes can leave a caller like PacedVideoOutput unable to ever resume after a resync.

Reimplemented in PacedVideoOutput, H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, and OutputFPSMeter.

setSize()

void setSize ( uint16_t  width, uint16_t  height  )

inline

Defines the picture size - only needed (and used) for raw, non-MJPEG formats (see class comment). Call before the first write(), or use setVideoInfoSource() instead to pick it up automatically.

setSkipRender()

virtual void setSkipRender ( bool  skip )

inlinevirtualinherited

Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.

Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

setVideoFormat()

void setVideoFormat ( VideoFormat  format )

inline

Selects the format of the data passed to write() - see class comment.

setVideoInfoSource()

void setVideoInfoSource ( VideoInfoSource &  source )

inlineoverridevirtual

Refreshes width/height from 'source' on every write() instead of requiring the caller to poll it (e.g. a DemuxerMP4/DemuxerAVI, or a decoder like H264Decoder) - removes the need for setSize() once the source knows its own dimensions. Deliberately only syncs width/height, not 'format': a container-level source (e.g. a Demuxer) reports its own encoded codec (H264, MJPEG, ...), which is not necessarily what actually gets written here once a decoder sits in between - format still comes from setVideoFormat().

Reimplemented from VideoOutput.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

write()

size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Implements VideoOutput.

Member Data Documentation

create_window

bool create_window = true

protected

height

uint16_t height = 0

protected

img_vector

std::vector<uint8_t> img_vector

protected

p_info

VideoInfoSource* p_info = nullptr

protected

pos

size_t pos = 0

protected

start

uint64_t start = 0

protected

swap_buffer

std::vector<uint8_t> swap_buffer

protected

video_format

VideoFormat video_format = VideoFormat::MJPEG

protected

width

uint16_t width = 0

protected

window

const char* window = "Movie"

protected

write_time_ms

uint32_t write_time_ms = 0

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/OutputOpenCV.h