MJPEGDecoder Class Reference

Motion-JPEG video decoder: wraps TinyJPEGDecoder (https://github.com/pschatzmann/TinyJPEG, a header-only port of ChaN's TJpgDec) as a VideoDecoder, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h) and MPGDecoder (CodecMPG.h) play for their own codecs. Named "MJPEGDecoder" rather than reusing TinyJPEGDecoder's own name to avoid shadowing that class from within this one. More...

#include <CodecJPEG.h>

Inheritance diagram for MJPEGDecoder:

Public Member Functions
	MJPEGDecoder ()
	MJPEGDecoder (Print &out)
	MJPEGDecoder (VideoOutput &out)
bool	begin () override
	Initializes the decoder (allocates its picture buffers, etc).
bool	byteSwap () const
VideoFormat	codecFormat () override
void	end () override
	Releases the decoder's resources.
void	flush () override
virtual uint32_t	getWriteTimeMs () const
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
bool	isKeyFrame (const uint8_t *data, size_t len) override
bool	isValid (const uint8_t *data, size_t len) override
void	setByteSwap (bool active)
void	setOutput (Print &out) override
void	setOutput (VideoOutput &out) override
virtual void	setSkipRender (bool skip)
void	setVideoFormat (VideoFormat format) override
virtual void	setVideoInfoSource (VideoInfoSource &source)
virtual uint64_t	totalDecodeMs () const
VideoInfo	videoInfo () override
size_t	write (const uint8_t *data, size_t len) override

Protected Member Functions
bool	writeBlock (int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t *data)
size_t	writeToOutput (const uint8_t *data, size_t len)

Static Protected Member Functions
static bool	onBlock (tinyjpeg::TinyJPEGDecoder &decoder, int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t *data)

Protected Attributes
bool	byte_swap = true
tinyjpeg::TinyJPEGDecoder	decoder_
Vector< uint8_t >	frame_buffer
uint16_t	height_ = 0
Vector< uint8_t >	img_vector
Print *	p_out = nullptr
VideoOutput *	p_out_video = nullptr
size_t	pos = 0
uint64_t	start_ms = 0
uint16_t	width_ = 0

Detailed Description

Motion-JPEG video decoder: wraps TinyJPEGDecoder (https://github.com/pschatzmann/TinyJPEG, a header-only port of ChaN's TJpgDec) as a VideoDecoder, so it can be plugged directly into e.g. DemuxerAVI::setOutputVideo()/DemuxerMP4::setOutputVideo() - the same role H264Decoder (CodecH264.h) and MPGDecoder (CodecMPG.h) play for their own codecs. Named "MJPEGDecoder" rather than reusing TinyJPEGDecoder's own name to avoid shadowing that class from within this one.

write()/flush() implement the VideoOutput contract (see Video.h), but unlike H264Decoder/MPGDecoder - which decode incrementally as Annex-B/ PES bytes arrive, with no separate "frame complete" signal needed - a bare JPEG byte stream has no equivalent self-framing a demuxer can use to decode early. write() therefore just accumulates one JPEG image's bytes, and flush() (called by e.g. DemuxerAVI/DemuxerMP4 at each frame boundary) decodes the assembled image directly into a full-frame RGB565 buffer and writes it to the target configured via setOutput(), one write() call per decoded picture - same convention as H264Decoder/ MPGDecoder, so any VideoOutput (OutputTFT_eSPI, OutputTinyGPU, OutputOpenCV, ...) works unchanged regardless of which codec produced the picture.

Owns a plain, private tinyjpeg::TinyJPEGDecoder member - unlike Bodmer's JPEGDecoder (this class's previous backend), TinyJPEGDecoder threads its per-decode context through the decoder's own instance instead of a global singleton, so multiple MJPEGDecoder instances (or any other concurrent TinyJPEGDecoder use elsewhere in the same sketch) coexist safely.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

MJPEGDecoder() [1/3]

MJPEGDecoder ( )

inline

MJPEGDecoder() [2/3]

MJPEGDecoder ( Print &  out )

inline

MJPEGDecoder() [3/3]

MJPEGDecoder ( VideoOutput &  out )

inline

Member Function Documentation

begin()

bool begin ( )

inlineoverridevirtual

Initializes the decoder (allocates its picture buffers, etc).

Implements VideoDecoder.

byteSwap()

bool byteSwap ( ) const

inline

codecFormat()

VideoFormat codecFormat ( )

inlineoverridevirtual

The compressed bitstream codec this decoder consumes (e.g. VideoFormat::H264, VideoFormat::MJPEG) - NOT the same axis as setVideoFormat()/videoInfo().format below, which is the decoded pixel format this decoder produces. Used by MultiVideoDecoder::addDecoder() to register a decoder under its own codec instead of the caller having to pass it separately (and possibly get it wrong).

Implements VideoDecoder.

end()

void end ( )

inlineoverridevirtual

Releases the decoder's resources.

Implements VideoDecoder.

flush()

void flush ( )

inlineoverridevirtual

Decodes the assembled JPEG image and writes the resulting RGB565 frame to setOutput()'s target, then resets for the next frame - a no-op if write() hasn't accumulated anything since the last call. Reads the picture's width/height first (getJpgSize() - a cheap, header-only parse) so frame_buffer can be sized before the per-MCU-block callback (onBlock()/writeBlock()) starts firing, since TinyJPEGDecoder streams blocks out via callback as it decodes rather than handing back a whole decoded picture at once.

Reimplemented from VideoOutput.

getWriteTimeMs()

virtual uint32_t getWriteTimeMs ( ) const

inlinevirtualinherited

Optional: returns the time (ms) spent in the last write() call.

Reimplemented in OutputTFT_eSPI, OutputTinyGPU, and OutputOpenCV.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

bool isKeyFrame ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

See VideoOutput::isKeyFrame() - always true: Motion-JPEG is an all-intra format, every frame is a complete, independently decodable image with no inter-frame prediction (the same property that makes an H.264 IDR frame or an MPEG-1 I-picture "key", just true of every frame here instead of periodically). This matters beyond classification - PacedVideoOutput's awaiting_keyframe recovery latch (set after a resync) only clears once a frame reports isKeyFrame()==true; returning false here would leave every frame classified as non-key, so once any resync ever fired, that latch would never clear and rendering would silently stall forever.

Reimplemented from VideoOutput.

isValid()

bool isValid ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

True if data looks like the start of a JPEG image (SOI marker FF D8) - unambiguous, and reliable even on the very first write() of a stream since a JPEG image always begins at FF D8 regardless of framing - see VideoDecoder::isValid().

Reimplemented from VideoDecoder.

onBlock()

static bool onBlock ( tinyjpeg::TinyJPEGDecoder &  decoder, int16_t  x, int16_t  y, uint16_t  w, uint16_t  h, uint16_t *  data  )

inlinestaticprotected

Per-MCU-block callback registered on decoder_ - see tinyjpeg::SketchCallback. decoder's getUserData() is this instance (set in the constructor), routing back to writeBlock().

setByteSwap()

void setByteSwap ( bool  active )

inline

Swaps the two bytes of every decoded RGB565 pixel before it reaches setOutput()'s target - on by default. TinyJPEGDecoder packs each pixel in the CPU's native uint16_t byte order (little-endian on ESP32/most Arduino targets) unless told otherwise, but SPI TFT panels (ILI9341 etc.) expect RGB565 transmitted big-endian - without the swap, colors come out visibly wrong (channels scrambled, not just a hue shift). Forwards straight to tinyjpeg::TinyJPEGDecoder::setSwapBytes(). Turn off only if your VideoOutput target already performs its own byte-swap (e.g. some display libraries' pushColors() variants).

setOutput() [1/2]

void setOutput ( Print &  out )

inlineoverridevirtual

Defines the target the decoded picture is written to, one write() call per decoded picture (RGB565 - the only format TinyJPEGDecoder produces).

Implements VideoDecoder.

setOutput() [2/2]

void setOutput ( VideoOutput &  out )

inlineoverridevirtual

Defines the target each decoded picture is written to - the VideoOutput-specific counterpart of setOutput(Print&), for a target like OutputTinyGPU/OutputOpenCV that implements VideoOutput but not Print (the two are unrelated types - see VideoFrameMeter's own comment). Part of the base interface (unlike other decoder-specific extras) so code holding only a VideoDecoder& (e.g. VideoPlayer) can wire it to an arbitrary VideoOutput& without knowing the concrete decoder type - every current decoder already implemented this identically before it was pulled up into the interface.

Implements VideoDecoder.

setSkipRender()

virtual void setSkipRender ( bool  skip )

inlinevirtualinherited

Hint to skip the expensive part of displaying the next frame(s) (e.g. the panel refresh) while still accepting and fully processing write() calls - used to recover from falling behind the playback schedule without breaking a codec's decode state (e.g. H.264 inter-prediction reference chain, which requires every frame to still be decoded even if it's never shown). Default no-op: implementations that can't skip rendering cheaply just ignore it and always render.

Reimplemented in MultiVideoDecoder, OutputFPSMeter, OutputTinyGPU, PacedVideoOutput, and VideoFrameMeter.

setVideoFormat()

void setVideoFormat ( VideoFormat  format )

inlineoverridevirtual

TinyJPEGDecoder only ever produces RGB565 - any other value is logged and ignored.

Implements VideoDecoder.

setVideoInfoSource()

virtual void setVideoInfoSource ( VideoInfoSource &  source )

inlinevirtualinherited

Optional: registers where width/height/fps/format (VideoInfo) come from - e.g. the demuxer feeding this output, so it can size its own buffers/panel setup without the caller having to duplicate that call per sketch (VideoPlayer::begin() does this automatically for whichever VideoOutput it was given). Default no-op: only implementations that actually need VideoInfo (OutputTinyGPU/ OutputOpenCV/OutputTFT_eSPI) override this.

Reimplemented in MultiVideoDecoder, OutputOpenCV, OutputTFT_eSPI, and OutputTinyGPU.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

videoInfo()

VideoInfo videoInfo ( )

inlineoverridevirtual

Reports the dimensions of the most recently decoded picture - 0 before any picture has been decoded.

Implements VideoDecoder.

write()

size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Accumulates one JPEG image's bytes - may be called more than once per frame (e.g. from a demuxer that hands over payload in pieces). Decoding happens on flush(), not here.

Implements VideoOutput.

writeBlock()

bool writeBlock ( int16_t  x, int16_t  y, uint16_t  w, uint16_t  h, uint16_t *  data  )

inlineprotected

Copies one decoded MCU block into frame_buffer at (x, y) - a right/bottom edge block can be narrower/shorter than a full MCU (w/h already reflect that; TinyJPEGDecoder, unlike the old JPEGDecoder backend, never hands back an over-wide/over-tall edge block in the first place), so no separate clamping is needed beyond the defensive min() below.

writeToOutput()

size_t writeToOutput ( const uint8_t *  data, size_t  len  )

inlineprotected

Member Data Documentation

byte_swap

bool byte_swap = true

protected

decoder_

tinyjpeg::TinyJPEGDecoder decoder_

protected

frame_buffer

Vector<uint8_t> frame_buffer

protected

height_

uint16_t height_ = 0

protected

img_vector

Vector<uint8_t> img_vector

protected

p_out

Print* p_out = nullptr

protected

p_out_video

VideoOutput* p_out_video = nullptr

protected

pos

size_t pos = 0

protected

start_ms

uint64_t start_ms = 0

protected

width_

uint16_t width_ = 0

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/CodecJPEG.h