VideoPlayer Class Reference

High-level video playback pipeline and controller - the video counterpart of AudioPlayer (CoreAudio/AudioPlayer.h). Wraps every class a video playback example otherwise wires by hand (a container Demuxer, a VideoDecoder driven through a PacedVideoOutput, and optionally an audio decode chain synced against it) behind one object driven by a single copy() call per loop() iteration. More...

#include <VideoPlayer.h>

Inheritance diagram for VideoPlayer:

Public Member Functions
	VideoPlayer ()=default
	VideoPlayer (Demuxer &demuxer, VideoOutput &videoOutput)
	VideoPlayer (Demuxer &demuxer, VideoOutput &videoOutput, AudioOutput &audioOutput)
	VideoPlayer (Demuxer &demuxer, VideoOutput &videoOutput, AudioStream &audioOutput)
	VideoPlayer (Demuxer &demuxer, VideoOutput &videoOutput, Print &audioOutput)
	VideoPlayer (VideoPlayer const &)=delete
void	addAudioDecoder (AudioDecoder &decoder, const char *mime=nullptr)
void	addDemuxer (Demuxer &demuxer)
void	addVideoDecoder (VideoDecoder &decoder)
MultiDecoder &	audioDecoder ()
bool	begin (Stream &source)
size_t	copy ()
size_t	copyAll ()
	Copies until the source is exhausted (blocking).
MultiVideoDemuxer &	demuxer ()
void	end ()
Stream *	getStream ()
	The Stream given to begin() - nullptr before the first begin() call.
bool	isActive () const
	operator bool () const
VideoPlayer &	operator= (VideoPlayer const &)=delete
void	setActive (bool isActive)
void	setAudioOutput (AudioOutput &out)
void	setAudioOutput (AudioStream &out)
void	setAudioOutput (Print &out)
void	setBufferSize (int size)
void	setUseAudioClock (bool active)
void	setVideoOutput (VideoOutput &out)
bool	useAudioClock () const
MultiVideoDecoder &	videoDecoder ()
PacedVideoOutput &	videoSyncTask ()

Protected Attributes
bool	active = false
AudioTimeSourceStream	audio_clock
EncodedAudioStream	audio_out
int	buffer_size = 1024
MultiDecoder	default_audio_decoder
MultiVideoDemuxer	default_demuxer
MultiVideoDecoder	default_video_decoder
Print *	p_audio_output = nullptr
AudioOutput *	p_audio_output_typed = nullptr
AudioStream *	p_audio_stream = nullptr
Stream *	p_source = nullptr
VideoOutput *	p_video_output = nullptr
bool	use_audio_clock = false
PacedVideoOutput	video_sync {default_video_decoder}

Detailed Description

High-level video playback pipeline and controller - the video counterpart of AudioPlayer (CoreAudio/AudioPlayer.h). Wraps every class a video playback example otherwise wires by hand (a container Demuxer, a VideoDecoder driven through a PacedVideoOutput, and optionally an audio decode chain synced against it) behind one object driven by a single copy() call per loop() iteration.

Demuxes the container through a built-in MultiVideoDemuxer, decodes video through a built-in MultiVideoDecoder, and decodes audio through a built-in MultiDecoder - all three start with NOTHING registered (see each one's own class comment) and none of them pulls in any container/codec library on its own, so including this header alone adds no external dependency at all. Register whatever your content actually needs via addDemuxer()/addVideoDecoder()/addAudioDecoder(), exactly as MultiVideoDemuxerFull's/MultiVideoDecoderFull's/ DecoderHelix's own constructors do internally:

DemuxerAVI aviDemuxer;
VideoPlayer player(aviDemuxer, tftOutput, audioOut);
H264Decoder h264Decoder;
MJPEGDecoder mjpegDecoder;
MP3DecoderHelix mp3Decoder;
AACDecoderHelix aacDecoder;
void setup() {
  ...
  player.addVideoDecoder(h264Decoder);
  player.addVideoDecoder(mjpegDecoder);
  player.addAudioDecoder(mp3Decoder, "audio/mpeg");
  player.addAudioDecoder(aacDecoder, "audio/aac");
  player.begin(file);
}
void loop() {
  if (player.copy() == 0) { file.close(); ... }
}

matching sd-avi-mjpg-video.ino/decode-avi.ino's own pipeline - demuxer is just a convenience for this common single-format case (it's registered exactly the way an addDemuxer() call would be); call addDemuxer() separately, as many times as needed, for additional container formats. Use VideoPlayerFull (VideoPlayerFull.h) instead for the "just point it at a file, any common format just works" convenience with no registration code of your own needed - it pre-registers every container format (AVI/MP4/MPG), every video codec (H264/MJPEG/MPEG-1), and the most common audio ones (MP3/AAC/MP2), at the cost of pulling in every one of their parser/codec libraries unconditionally (see its own class comment).

addAudioDecoder() also covers a codec the DemuxerXxx::mime() the selected demuxer reports precisely but a plain byte-sniffing MimeDetector wouldn't otherwise recognize, e.g. MP2Decoder (TinyMP2) for MPEG-1 Layer II audio, the same way sd-mpg-video.ino's hand-wired pipeline does:

player.addAudioDecoder(mp2Decoder, "audio/mpeg; codecs=\"mpeg1-layer2\"");

addVideoDecoder() similarly covers a hardware-accelerated decoder like H264DecoderESP32S3 (CodecH264ESP32S3.h) in place of a portable one - note its VideoDecoder::isValid() isn't overridden (see the class comment on that method), so it only gets selected via a setVideoInfoSource() answer, not the content-sniffing fallback; set one (typically the demuxer feeding this player) if you rely on it. There is no way to replace any of the three built-in multi-selectors wholesale (e.g. with a single-format demuxer/decoder) - add to what's already registered instead. A video-only stream needs no audio decoder registered at all - see setAudioOutput()/the class's video-only constructor.

Pipeline: Stream (source) -> copy() (CodecCopy-equivalent) -> Demuxer (demux) -> EncodedAudioStream (AudioDecoder) -> [AudioTimeSourceStream (audio clock), see setUseAudioClock()] -> audio output -> PacedVideoOutput (buffer + schedule) -> VideoDecoder -> video output (own background task)

Audio clock: scheduling video against real playback progress (PacedVideoOutput::setAudioClock()) needs an audio clock that's actually advancing - wiring one against a track that never delivers any bytes (silent/absent audio) would stall video forever waiting for a clock that never moves (see decode-mp4.ino's own history for exactly this bug). setUseAudioClock() therefore defaults to false: video is scheduled against wall-clock time unless you explicitly opt in once an audio output is wired AND you know the content actually has a real audio track.

Only covers the common "feed a Demuxer straight from a Stream" case, matching the RAM-backed sample-table default every Demuxer uses out of the box - the seek-backed/spooled MP4 strategies (see decode-mp4-file.ino/decode-mp4-spooled.ino) feed their Demuxer through their own FileSeekableSource/SpoolStorageFactory machinery instead, for their own memory-optimization reasons, and are not wrapped here; keep your own reference to the concrete DemuxerMP4 you construct and pass to addDemuxer() (it stays fully usable on its own - MultiVideoDemuxer just forwards write()/setOutputAudio()/setOutputVideo()/... to it once selected) and drive it directly instead of calling copy()/copyAll() on this class.

Operation model: call copy() regularly (non-blocking) in loop(), or copyAll() for blocking end-to-end playback.

Dependencies: none - this header itself doesn't pull in any container/codec library (see above). Bring whichever demuxers and video/audio codec libraries your content actually needs and register them via addDemuxer()/addVideoDecoder()/addAudioDecoder(), or use VideoPlayerFull for all of them at once.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

VideoPlayer() [1/6]

VideoPlayer ( )

default

VideoPlayer() [2/6]

VideoPlayer ( Demuxer &  demuxer, VideoOutput &  videoOutput  )

inline

Video-only playback (no audio track) - auto-detected video codec. demuxer is registered the same way an addDemuxer() call would (this is just a convenience for the common single-format case - call addDemuxer() separately, as many times as needed, for additional container formats).

VideoPlayer() [3/6]

VideoPlayer ( Demuxer &  demuxer, VideoOutput &  videoOutput, AudioOutput &  audioOutput  )

inline

Video + audio playback (AudioOutput target, e.g. AudioBoardStream/ I2SStream) with auto-detected codecs. See the class comment's "Audio clock" section - setUseAudioClock(true) still needs to be called explicitly if the content actually has a real audio track and should be scheduled against it. See the video-only constructor above for demuxer.

VideoPlayer() [4/6]

VideoPlayer ( Demuxer &  demuxer, VideoOutput &  videoOutput, Print &  audioOutput  )

inline

Video + audio playback (a generic Print target) with auto-detected codecs. See the class comment's "Audio clock" section, and the video-only constructor above for demuxer.

VideoPlayer() [5/6]

VideoPlayer ( Demuxer &  demuxer, VideoOutput &  videoOutput, AudioStream &  audioOutput  )

inline

Video + audio playback (AudioStream target, e.g. PortAudioStream) with auto-detected codecs. See the class comment's "Audio clock" section, and the video-only constructor above for demuxer.

VideoPlayer() [6/6]

VideoPlayer ( VideoPlayer const &  )

delete

Non-copyable: video_sync/audio_out below are wired against this object's own member addresses - copying would leave the copy's internal wiring pointing at the original.

Member Function Documentation

addAudioDecoder()

void addAudioDecoder ( AudioDecoder &  decoder, const char *  mime = nullptr  )

inline

Registers an audio codec with the built-in (initially empty) MultiDecoder - see MultiDecoder::addDecoder() (the mime-only overload). Nothing is registered by default (see the class comment), so this needs at least one call per codec your content's audio track actually uses. Call before begin().

addDemuxer()

void addDemuxer ( Demuxer &  demuxer )

inline

Registers a container demuxer (DemuxerAVI/DemuxerMP4/DemuxerMPG/...) with the built-in (initially empty) MultiVideoDemuxer, under its own Demuxer::mimeVideo() - see MultiVideoDemuxer::addDemuxer(). Nothing is registered by default (see the class comment), so this needs at least one call per container format your content actually uses. Call before begin().

addVideoDecoder()

void addVideoDecoder ( VideoDecoder &  decoder )

inline

Registers a video codec with the built-in (initially empty) MultiVideoDecoder, under its own VideoDecoder::codecFormat() - see MultiVideoDecoder::addDecoder(). Nothing is registered by default (see the class comment), so this needs at least one call per codec your content's video track actually uses. Call before begin().

audioDecoder()

MultiDecoder & audioDecoder ( )

inline

The built-in audio multi-decoder (empty until addAudioDecoder() is called - see the class comment) - also useful for diagnostics (e.g. selectedMime()).

begin()

bool begin ( Stream &  source )

inline

Wires the full pipeline (video decoder -> output via PacedVideoOutput; audio decoder -> [audio clock ->] output, if an audio output was given) and starts the demuxer. source (e.g. an open File) is read from on every subsequent copy() call - the caller owns it and is responsible for closing it once copy()/copyAll() signal end of stream (return 0).

copy()

size_t copy ( )

inline

Reads and demuxes one chunk (setBufferSize(), default 1024 bytes) from the source given to begin() - call this every loop() iteration. Also keeps the video schedule's fps in sync with the demuxer's own parsed rate, since VideoInfo::fps only becomes known partway through (once the container's header has been parsed) - no separate polling needed in the caller's loop(), unlike the *.ino examples this class replaces.

Returns

bytes actually copied - 0 once the source is exhausted (or playback is inactive/not begin()'d), matching CodecCopy::copy()'s own "0 means stop" convention every current *.ino example already checks for.

copyAll()

size_t copyAll ( )

inline

Copies until the source is exhausted (blocking).

demuxer()

MultiVideoDemuxer & demuxer ( )

inline

The built-in demuxer multi-selector (empty until addDemuxer() is called - see the class comment) - also useful for diagnostics (e.g. selectedDemuxer()/mimeVideo()).

end()

void end ( )

inline

Stops playback and releases the video/audio decoders + the sync task's background render thread. The Stream given to begin() is left for the caller to close.

getStream()

Stream * getStream ( )

inline

The Stream given to begin() - nullptr before the first begin() call.

isActive()

bool isActive ( ) const

inline

True between a successful begin() and end()/setActive(false)/source exhaustion.

operator bool()

operator bool ( ) const

inline

operator=()

VideoPlayer & operator= ( VideoPlayer const &  )

delete

setActive()

void setActive ( bool  isActive )

inline

Halts copy()/copyAll() without tearing down the pipeline (unlike end()) - resume with setActive(true).

setAudioOutput() [1/3]

void setAudioOutput ( AudioOutput &  out )

inline

setAudioOutput() [2/3]

void setAudioOutput ( AudioStream &  out )

inline

setAudioOutput() [3/3]

void setAudioOutput ( Print &  out )

inline

Defines the final audio output target and enables the audio path - leave unset entirely for video-only content. Must already be configured/begin()'d. Call before begin(). See the class comment's "Audio clock" section for setUseAudioClock()'s default.

Three overloads, mirroring AudioPlayer::setOutput() - AudioOutput and AudioStream are unrelated types (both derive from Print, neither from the other - see AudioOutput.h's own class comment), so the most specific one actually available should be used: it lets begin() wire the audio clock/EncodedAudioStream via the matching setOutput( AudioOutput&)/setStream(AudioStream&) overload instead of falling back to the generic Print& one, which is what makes e.g. audio-info change notifications reach the real output.

setBufferSize()

void setBufferSize ( int  size )

inline

Sets the read-buffer size (bytes) copy() uses per call (default 1024, same as CodecCopy's own default). Call before begin().

setUseAudioClock()

void setUseAudioClock ( bool  active )

inline

See the class comment's "Audio clock" section. Off by default; setAudioOutput() does not change it - opt in explicitly once you know the content has a real audio track. Call before begin().

setVideoOutput()

void setVideoOutput ( VideoOutput &  out )

inline

Defines the final video display target (e.g. OutputTinyGPU, OutputOpenCV) - must already be configured/begin()'d (board/panel init is outside this class's scope, same as AudioPlayer's AudioOutput& argument). Call before begin().

useAudioClock()

bool useAudioClock ( ) const

inline

videoDecoder()

MultiVideoDecoder & videoDecoder ( )

inline

The built-in video multi-decoder (empty until addVideoDecoder() is called - see the class comment) - also useful for diagnostics (e.g. selectedFormat()).

videoSyncTask()

PacedVideoOutput & videoSyncTask ( )

inline

Escape hatch for tuning beyond this class's own surface - e.g. setTaskParameters()/setQueueBytes()/setMaxQueuedIFrames()/ setIgnorePFrames()/setSchedulingDelayMs(), or the frameCount()/ avgFrameMs()/outputFPS() family of diagnostics.

Member Data Documentation

active

bool active = false

protected

audio_clock

AudioTimeSourceStream audio_clock

protected

audio_out

EncodedAudioStream audio_out

protected

buffer_size

int buffer_size = 1024

protected

default_audio_decoder

MultiDecoder default_audio_decoder

protected

default_demuxer

MultiVideoDemuxer default_demuxer

protected

default_video_decoder

MultiVideoDecoder default_video_decoder

protected

p_audio_output

Print* p_audio_output = nullptr

protected

p_audio_output_typed

AudioOutput* p_audio_output_typed = nullptr

protected

p_audio_stream

AudioStream* p_audio_stream = nullptr

protected

p_source

Stream* p_source = nullptr

protected

p_video_output

VideoOutput* p_video_output = nullptr

protected

use_audio_clock

bool use_audio_clock = false

protected

video_sync

PacedVideoOutput video_sync {default_video_decoder}

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/VideoPlayer.h