OutputTinyGPU Class Reference

Bridges VideoDecoder's write() calls to a TinyGPU DisplayDriver (e.g. ILI9341Driver, or any board wrapped in an LCDBoard - see the LCDBoard constructor) - VideoDecoder always hands over one complete frame per write() call (see its class comment), so this can push the whole frame in one go. Usually the data is in RGB565 format, but other formats are supported as well. More...

#include <OutputTinyGPU.h>

Inheritance diagram for OutputTinyGPU:

Public Member Functions
	OutputTinyGPU (DisplayDriver< RGB565 > &driver, int pinBacklight, int bandHeight=40)
	OutputTinyGPU (LCDBoard &board, int bandHeight=40)
bool	begin ()
void	clearScreen ()
virtual void	flush ()
uint32_t	getWriteTimeMs () const
	Optional: returns the time (ms) spent in the last write() call.
virtual bool	hadOutput () const
virtual bool	isKeyFrame (const uint8_t *data, size_t len)
void	setRotation (DisplayRotation rotation)
void	setScaleSingleBuffer (bool enable)
void	setScaleToFit (bool enable)
void	setSkipRender (bool skip) override
void	setVideoInfo (VideoInfo info)
	Defines the video format and dimensions - call before begin() if you.
void	setVideoInfoSource (VideoInfoSource &source) override
	Defines the source of the video information (width, height, fps, format)
virtual uint64_t	totalDecodeMs () const
void	updateScaleLuts (int srcW, int srcH, int width, int height)
size_t	write (const uint8_t *data, size_t len) override
void	writeI420 (const uint8_t *data, int srcW, int srcH)
void	writeScaled (const uint8_t *data, int srcW, int srcH)

Static Public Member Functions
static RGB565	yuvToRgb565 (uint8_t Y, uint8_t U, uint8_t V)

Protected Attributes
Vector< RGB565 >	i420_rgb_buf_ {0, DefaultESP32AllocatorRAM}
VideoInfo	info_
int	kBandHeight
int	kPinBacklight
LCDBoard *	p_board = nullptr
VideoInfoSource *	p_info = nullptr
Vector< RGB565 >	scale_frame_buf_ {0, DefaultESP32AllocatorRAM}
Vector< RGB565 >	scale_line_ {0, DefaultESP32AllocatorRAM}
int	scale_lut_dst_h_ = -1
int	scale_lut_dst_w_ = -1
int	scale_lut_src_h_ = -1
int	scale_lut_src_w_ = -1
bool	scale_single_buffer = true
bool	scale_to_fit = false
Vector< int >	scale_x_lut_ {0, DefaultESP32AllocatorRAM}
Vector< int >	scale_y_lut_ {0, DefaultESP32AllocatorRAM}
bool	skip_render = false
DisplayDriver< RGB565 > &	tftDriver
uint32_t	write_time_ms = 0

Detailed Description

Bridges VideoDecoder's write() calls to a TinyGPU DisplayDriver (e.g. ILI9341Driver, or any board wrapped in an LCDBoard - see the LCDBoard constructor) - VideoDecoder always hands over one complete frame per write() call (see its class comment), so this can push the whole frame in one go. Usually the data is in RGB565 format, but other formats are supported as well.

write() never allocates or copies a full-frame buffer of its own for the unscaled path: it wraps the caller's already-decoded frame in a SurfaceWithExternalBuffer (a TinyGPU view over externally-owned memory) and hands that straight to the display driver. Scaling (setScaleToFit()) and I420 conversion do need their own scratch buffers - see setScaleSingleBuffer() for the size/SPI-transaction-count tradeoff those involve.

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

OutputTinyGPU() [1/2]

OutputTinyGPU ( DisplayDriver< RGB565 > &  driver, int  pinBacklight, int  bandHeight = 40  )

inline

Parameters

driver	Any TinyGPU DisplayDriver<RGB565> - e.g. ILI9341Driver. Its own width()/height() (rotation-aware - see DisplayDriver.h) are queried fresh wherever needed (scale_to_fit's target, clearScreen()'s band loop), so a later setRotation() call is picked up automatically.
bandHeight	height (in pixels) of the strip used to clear the screen in begin() - kept small (default 40) so the clear buffer stays a small, single allocation instead of a full-screen framebuffer; see TinyGPU's bouncing-ball example for the same technique.

OutputTinyGPU() [2/2]

OutputTinyGPU ( LCDBoard &  board, int  bandHeight = 40  )

inline

Parameters

board

Bundles the display driver (and, on concrete boards, touch/I2S/LED pin assignments) behind one begin() call - see TinyGPU's LCDBoards.h. begin() calls board.begin() instead of separately managing a backlight pin and the driver's own begin(), since the board's begin() already brings up its backlight/bus/ display (and touch, if present) together.

Member Function Documentation

begin()

bool begin ( )

inline

clearScreen()

void clearScreen ( )

inline

flush()

virtual void flush ( )

inlinevirtualinherited

Finalizes the frame most recently written via one or more write() calls - see class comment. Default no-op for implementations that display/decode synchronously in write() instead.

Reimplemented in Muxer, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, OutputFPSMeter, OutputOpenCV, PacedVideoOutput, and VideoFrameMeter.

getWriteTimeMs()

uint32_t getWriteTimeMs ( ) const

inlinevirtual

Optional: returns the time (ms) spent in the last write() call.

Reimplemented from VideoOutput.

hadOutput()

virtual bool hadOutput ( ) const

inlinevirtualinherited

True if the most recent write()+flush() call actually produced a displayable picture - default true, matching every synchronous decoder (H264Decoder, MJPEGDecoder, ...), which always decodes and pushes pixels fully within that one call. Override this only if your decoder can legitimately accept/decode a frame's bytes without emitting a picture during that same call - e.g. MPGDecoder, whose B-picture display-order reordering can hold a just-decoded picture back and instead emit an earlier one (or nothing at all) from a given write(), see its own override. Used by PacedVideoOutput to avoid counting/timing a call that did no real rendering work as a rendered frame - without this, its outputFPS()/frameCountI()/ frameCountP()/avgFrameMs() would overcount for such a decoder.

Reimplemented in MPGDecoder.

isKeyFrame()

virtual bool isKeyFrame ( const uint8_t *  data, size_t  len  )

inlinevirtualinherited

True if data (one complete encoded frame, as handed to write()) is a keyframe/sync-sample - self-contained, decodable without any earlier frame. Used e.g. by PacedVideoOutput to decide which frames are safe to drop, and whether it's safe to resume decoding after abandoning a backlog (see its own class comment). Default false: a plain VideoOutput doesn't know or care about codec structure - override this in a decoder for the bitstream format it actually parses (see H264Decoder/H264DecoderESP32S3's isH264KeyFrame()-based override, MPGDecoder's isMpeg1KeyFrame()- based one). Getting this right matters beyond bookkeeping: a target whose frames are never recognized as keyframes can leave a caller like PacedVideoOutput unable to ever resume after a resync.

Reimplemented in PacedVideoOutput, H264Decoder, H264DecoderESP32S3< Alloc >, MJPEGDecoder, MPGDecoder, MultiVideoDecoder, and OutputFPSMeter.

setRotation()

void setRotation ( DisplayRotation  rotation )

inline

Changes the panel's rotation after begin() - forwards to the underlying driver (see DisplayDriver::setRotation()); a no-op if that driver doesn't support rotation. Nothing else to do here: scale_to_fit's target and clearScreen()'s band loop read tftDriver.width()/height() fresh each time, and those are already rotation-aware (see DisplayDriver.h), so they pick up the change automatically.

setScaleSingleBuffer()

void setScaleSingleBuffer ( bool  enable )

inline

Controls how writeScaled() (the actual scale_to_fit implementation) trades RAM for SPI transaction count. true (the default): scale the whole frame into one full output-size buffer (scale_frame_buf_ - width*height*sizeof(RGB565) bytes, e.g. 150KB at 320x240) and hand it to the driver in a single writeData() call, the same one-transaction shape as the unscaled path. false: the original one-row-at-a-time path - only a single row's worth of RAM (scale_line_), but one writeData() call per output row (e.g. 240 small SPI transactions instead of 1 at 320x240) - each pays its own fixed per-transaction overhead (chip-select, command/address setup, driver call overhead), which tends to dominate over the pixel math itself. Call before the first scaled frame arrives - changing it afterwards takes effect on the next writeScaled() call, not retroactively.

setScaleToFit()

void setScaleToFit ( bool  enable )

inline

When enabled, a decoded frame whose size doesn't match the panel's current width/height (tftDriver.width()/height(), rotation-aware - see DisplayDriver.h) is nearest-neighbor scaled to fill it exactly - the aspect ratio is NOT preserved, the source is stretched to the panel's own aspect ratio. Off by default (direct 1:1 write at (0,0), e.g. a QCIF video shown at native size in the corner of a larger panel). See setScaleSingleBuffer() for how the actual scaling is done.

setSkipRender()

void setSkipRender ( bool  skip )

inlineoverridevirtual

Skips the actual panel refresh (the SPI-bound part) on the next and subsequent write() calls, while still validating the frame and returning len as if it had been shown - used to recover from falling behind the playback schedule without breaking a codec's decode state (the caller must still get a normal "success" return so its own frame-complete bookkeeping isn't disturbed). Call again with false to resume rendering.

Reimplemented from VideoOutput.

setVideoInfo()

void setVideoInfo ( VideoInfo  info )

inline

Defines the video format and dimensions - call before begin() if you.

setVideoInfoSource()

void setVideoInfoSource ( VideoInfoSource &  source )

inlineoverridevirtual

Defines the source of the video information (width, height, fps, format)

Reimplemented from VideoOutput.

totalDecodeMs()

virtual uint64_t totalDecodeMs ( ) const

inlinevirtualinherited

Optional: sum of time (ms) spent purely decoding (excluding any surrounding convert/render/SPI work a subclass's write() also does) since begin() - see H264Decoder's own override for the only current implementation. Default 0: only meaningful for a decoder that separates decode time from render time internally: PacedVideoOutput:: logTo() prints a decode-vs-render split under "avg decode ms:" only when this returns nonzero.

Reimplemented in H264Decoder, and MultiVideoDecoder.

updateScaleLuts()

void updateScaleLuts ( int  srcW, int  srcH, int  width, int  height  )

inline

Rebuilds scale_x_lut_/scale_y_lut_ (the nearest-neighbor source column/row for each output column/row) - a no-op unless srcW/srcH/ width/height actually changed since the last call (the common case: same video, same panel, every frame), so the division this replaces runs at most once per resolution change instead of once per pixel.

write()

size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Implements VideoOutput.

writeI420()

void writeI420 ( const uint8_t *  data, int  srcW, int  srcH  )

inline

Converts one planar I420 frame (srcW x srcH: full-size Y plane followed by two (srcW/2)x(srcH/2) U/V planes) to RGB565 and writes it to the driver. The YUV->RGB conversion (several integer multiplies + clamps per pixel - not free) always runs exactly once per source pixel, into i420_rgb_buf_ at native resolution; scale_to_fit then reuses writeScaled()'s existing (conversion-free, just array indexing) RGB565 upscale path instead of redoing the YUV math once per output pixel - on this panel (320x240 vs a 176x144 source) that's the difference between ~25k and ~77k conversions per frame.

writeScaled()

void writeScaled ( const uint8_t *  data, int  srcW, int  srcH  )

inline

Nearest-neighbor scales a source frame (srcW x srcH, RGB565, tightly packed) up/down to exactly width x height - see setScaleSingleBuffer() for the two ways this actually reaches the driver. Each output pixel's source column/row comes from scale_x_lut_/scale_y_lut_ (see updateScaleLuts()) - a lookup, not a division - since srcW/srcH/ width/height are the same on every call for a given video+panel.

yuvToRgb565()

static RGB565 yuvToRgb565 ( uint8_t  Y, uint8_t  U, uint8_t  V  )

inlinestatic

BT.601 (limited-range) integer YUV->RGB565 conversion - the standard fixed-point formula (avoids per-pixel float math), same coefficients used by most embedded YUV converters.

Member Data Documentation

i420_rgb_buf_

Vector<RGB565> i420_rgb_buf_ {0, DefaultESP32AllocatorRAM}

protected

Native-resolution scratch buffer for writeI420()'s YUV->RGB565 conversion - unlike scale_line_ (one row), this holds a full frame, since the conversion must run before scaling can reuse writeScaled(). Only allocated once an I420 frame is actually written.

info_

VideoInfo info_

protected

kBandHeight

int kBandHeight

protected

kPinBacklight

int kPinBacklight

protected

p_board

LCDBoard* p_board = nullptr

protected

p_info

VideoInfoSource* p_info = nullptr

protected

scale_frame_buf_

Vector<RGB565> scale_frame_buf_ {0, DefaultESP32AllocatorRAM}

protected

All the scratch buffers below are forced onto internal RAM (via DefaultESP32AllocatorRAM - Allocator.h - instead of Vector's own default, which tries PSRAM first) rather than internal SRAM's default Vector allocator (which tries PSRAM first, via ps_malloc) - every one of them is read/written in a tight per-pixel loop, tens of thousands of times a frame, and PSRAM's higher per-access latency costs far more there than its extra capacity is worth for buffers this size. Full output-size scratch buffer for writeScaled()'s single-buffer path (see setScaleSingleBuffer(), the default) - only allocated once a scaled frame is actually written.

scale_line_

Vector<RGB565> scale_line_ {0, DefaultESP32AllocatorRAM}

protected

Single-row scratch buffer for writeScaled()'s row-at-a-time path (setScaleSingleBuffer(false)) - only allocated if that path is used.

scale_lut_dst_h_

int scale_lut_dst_h_ = -1

protected

scale_lut_dst_w_

int scale_lut_dst_w_ = -1

protected

scale_lut_src_h_

int scale_lut_src_h_ = -1

protected

scale_lut_src_w_

int scale_lut_src_w_ = -1

protected

scale_single_buffer

bool scale_single_buffer = true

protected

scale_to_fit

bool scale_to_fit = false

protected

scale_x_lut_

Vector<int> scale_x_lut_ {0, DefaultESP32AllocatorRAM}

protected

Nearest-neighbor source column/row for each output column/row - see updateScaleLuts(). scale_lut_* below records what resolution these were last built for, so a resolution change (not every frame) is what triggers a rebuild. Read once per output pixel (the innermost loop in writeScaled()) - the single hottest access of any buffer here, so keeping it off PSRAM matters most for these two.

scale_y_lut_

Vector<int> scale_y_lut_ {0, DefaultESP32AllocatorRAM}

protected

skip_render

bool skip_render = false

protected

tftDriver

DisplayDriver<RGB565>& tftDriver

protected

write_time_ms

uint32_t write_time_ms = 0

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Video/OutputTinyGPU.h