JPEGRtpEncoder Class Reference

JPEG RTP Encoder - Fragments JPEG frames per RFC 2435 for RTSP video streaming. More...

#include <JPEGRtpEncoder.h>

Inheritance diagram for JPEGRtpEncoder:

Collaboration diagram for JPEGRtpEncoder:

Public Member Functions
	JPEGRtpEncoder ()=default
	JPEGRtpEncoder (size_t maxFragmentSize)
	Constructor with maximum fragment size.
AudioInfo	audioInfo () override
	provides the actual input AudioInfo
virtual AudioInfo	audioInfoOut ()
bool	begin () override
virtual bool	begin (AudioInfo info)
void	end () override
virtual uint32_t	frameDurationUs ()
	Optional rtsp function: provide the frame duration in microseconds.
virtual int	frameSize ()
RTSPFormat &	getFormat () override
	Get the media format configuration.
const char *	mime () override
	Provides the mime type of the encoded result.
	operator bool () override
int	packetSize () override
	Size of the next queued, ready-to-send fragment, for sources that provide already RTP-payload-ready, self-delimited fragments (e.g. RFC 2435 JPEG) instead of a continuous byte stream. For these sources, readBytes() must always be called with exactly packetSize() as maxBytes, so each call retrieves exactly one complete fragment instead of the transport having to guess where one RTP payload ends and the next begins.
int	readBytes (void *dest, int maxBytes) override
virtual uint16_t	samplesPerFrame ()
	Optional rtsp function: provide samples per the frame.
void	setAudioInfo (AudioInfo info) override
	Defines the sample rate, number of channels and bits per sample.
void	setFormat (RTSPFormat &format) override
	Associates the RTSPFormat (e.g. RTSPFormatMJPEG) that supplies width/height.
void	setMaxFragmentSize (size_t size)
	Set the maximum RTP payload size per packet (header + data)
void	setMaxQueuedFrames (size_t frames)
	Maximum number of complete frames buffered when the consumer falls behind.
virtual void	setOutput (Print &out_stream) override
	Default output assignment (encoders may override to store Print reference)
void	start () override
	Initialize media source for streaming.
void	stop () override
	Cleanup media source after streaming.
size_t	write (const uint8_t *data, size_t len) override
	Process JPEG frame data and queue the resulting RTP fragments.

Protected Member Functions
void	appendFragment (const uint8_t *payload, size_t len, bool last)
void	buildJpegRtpHeader (uint8_t *buffer, size_t fragmentOffset, int width, int height, uint8_t type, uint8_t qValue)
void	clearQueue ()
uint8_t	detectJpegType (const uint8_t *data, size_t headerLen)
	Determine the RFC 2435 Type (0 = 4:2:2, 1 = 4:2:0) from the JPEG's SOF0/SOF1 marker so the receiver reconstructs the correct chroma subsampling. Defaults to 0 (4:2:2) if it cannot be determined.
void	dropOldestFrame ()
int	extractQuantTables (const uint8_t *data, size_t headerLen, uint8_t *out)
	Extract the (8-bit precision) quantization tables 0 and 1 from the JPEG DQT segments found in [data, data+headerLen). RFC 2435 expects table 0 (luma) followed by table 1 (chroma), each 64 bytes in zigzag order exactly as stored in the JPEG - no reordering needed.
size_t	findJpegDataStart ()
	Find the start of the entropy-coded scan data, i.e. the first byte after the SOS (Start Of Scan) marker segment. RFC 2435 requires everything before this point (SOI/APPn/DQT/SOF/SOS) to be stripped from the RTP payload.
bool	isCompleteJpegFrame ()
	Check if current buffer contains a complete JPEG frame.
void	processCompleteFrame ()
	Strip the JPEG headers, fragment the scan data and queue one RFC 2435 RTP payload per fragment.
int	queuePacketSize ()
int	queueReadBytes (void *dest, int maxBytes)
uint32_t	readQueueU32 (size_t pos)
void	writeBlocking (Print *out, uint8_t *data, size_t len)
void	writeQuantTableHeader (uint8_t *buffer, const uint8_t *qTable, int qTableLen)

Protected Attributes
RTSPFormatMJPEG	default_format
AudioInfo	info
Vector< uint8_t >	m_currentFrame
bool	m_frameComplete = true
bool	m_isStarted = false
size_t	m_maxFragmentSize = 1400
size_t	m_maxQueuedFrames = 1
Vector< uint8_t >	m_queue
size_t	m_queuedFrames = 0
size_t	m_queueHead = 0
Vector< uint8_t >	m_scratch
RTSPFormat *	p_format = nullptr

Detailed Description

JPEG RTP Encoder - Fragments JPEG frames per RFC 2435 for RTSP video streaming.

This class plays both roles needed to get a JPEG frame onto the wire:

• As an AudioEncoder, write() accepts complete JPEG frames (e.g. straight from a camera).
• As an IMediaSource, packetSize()/readBytes() hand out the resulting RTP payloads one fragment at a time, each one already carrying its RFC 2435 header (and, on the first fragment of a frame, an embedded quantization-table header).

Keeping fragmentation and header construction in one place means the RTP transport never has to re-derive fragment boundaries or rebuild a JPEG header of its own - it only wraps each queued fragment in the generic RTP header (sequence number, shared timestamp, marker bit set once packetSize() reports the queue empty).

Per RFC 2435, the payload must be the raw entropy-coded scan data only; SOI/APPn/DQT/SOF/SOS markers are stripped and reconstructed by the receiver from the RTP header fields, so header stripping is not optional.

Note

Restart intervals (DRI) are not supported.

Usage with RTSPOutput:

RTSPFormatMJPEG mjpegFormat(640, 480, 30.0f);
JPEGRtpEncoder jpegEncoder;
RTSPOutput<RTSPPlatformWiFi> rtspOutput(mjpegFormat, jpegEncoder);

Author

Phil Schatzmann

Constructor & Destructor Documentation

JPEGRtpEncoder() [1/2]

JPEGRtpEncoder ( )

default

JPEGRtpEncoder() [2/2]

JPEGRtpEncoder ( size_t  maxFragmentSize )

inline

Constructor with maximum fragment size.

Parameters

maxFragmentSize

Maximum RTP payload size per packet, including the RFC 2435 header (default: 1400 bytes)

Member Function Documentation

appendFragment()

void appendFragment ( const uint8_t *  payload, size_t  len, bool  last  )

inlineprotectedinherited

audioInfo()

AudioInfo audioInfo ( )

inlineoverridevirtualinherited

provides the actual input AudioInfo

Implements AudioInfoSupport.

Reimplemented in CodecNOP, MP3EncoderShine, BinaryContainerEncoder, and MP3ParserEncoder.

audioInfoOut()

virtual AudioInfo audioInfoOut ( )

inlinevirtualinherited

provides the actual output AudioInfo: this is usually the same as audioInfo() unless we use a transforming stream

Reimplemented in MP3EncoderShine, PureDataStream, PWMAudioOutput< PWMDriverT >, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, FormatConverterStream, Pipeline, ResampleStream, ResampleStreamT< TInterpolator >, and SupportedRatesStream.

begin() [1/2]

bool begin ( )

inlineoverridevirtual

Implements AudioWriter.

begin() [2/2]

virtual bool begin ( AudioInfo  info )

inlinevirtualinherited

Reimplemented in OpusOggEncoder, OggContainerEncoder, DecoderBasic, AudioDecoder, AACEncoderFAAC, AACEncoderFDK, AACEncoderVO, OggContainerDecoder, and MetaDataFilterEncoder.

buildJpegRtpHeader()

void buildJpegRtpHeader ( uint8_t *  buffer, size_t  fragmentOffset, int  width, int  height, uint8_t  type, uint8_t  qValue  )

inlineprotected

clearQueue()

void clearQueue ( )

inlineprotectedinherited

detectJpegType()

uint8_t detectJpegType ( const uint8_t *  data, size_t  headerLen  )

inlineprotected

Determine the RFC 2435 Type (0 = 4:2:2, 1 = 4:2:0) from the JPEG's SOF0/SOF1 marker so the receiver reconstructs the correct chroma subsampling. Defaults to 0 (4:2:2) if it cannot be determined.

dropOldestFrame()

void dropOldestFrame ( )

inlineprotectedinherited

end()

void end ( )

inlineoverridevirtual

Implements AudioWriter.

extractQuantTables()

int extractQuantTables ( const uint8_t *  data, size_t  headerLen, uint8_t *  out  )

inlineprotected

Extract the (8-bit precision) quantization tables 0 and 1 from the JPEG DQT segments found in [data, data+headerLen). RFC 2435 expects table 0 (luma) followed by table 1 (chroma), each 64 bytes in zigzag order exactly as stored in the JPEG - no reordering needed.

Some encoders emit a single quantization table shared by all components (SOF Tq=0 for luma and chroma alike) instead of two - a valid standalone JPEG, but not expressible in RFC 2435's wire format, whose receiver-side reconstruction (Appendix A) always assumes two distinct tables. Sending only the one table found would leave the reconstructed chroma table uninitialized on the receiver, corrupting decode; duplicating table 0 into table 1 is faithful to the source (both components really do use that table) and satisfies the two-table format.

Returns

Number of bytes written to out (0 or 128), 0 if no usable 8-bit-precision table was found

findJpegDataStart()

size_t findJpegDataStart ( )

inlineprotected

Find the start of the entropy-coded scan data, i.e. the first byte after the SOS (Start Of Scan) marker segment. RFC 2435 requires everything before this point (SOI/APPn/DQT/SOF/SOS) to be stripped from the RTP payload.

Returns

Offset to the scan data, or 0 if no SOS marker was found

frameDurationUs()

virtual uint32_t frameDurationUs ( )

inlinevirtualinherited

Optional rtsp function: provide the frame duration in microseconds.

Reimplemented in ADPCMEncoder, MP3EncoderShine, OpusOggEncoder, MP3ParserEncoder, and MetaDataFilterEncoder.

frameSize()

virtual int frameSize ( )

inlinevirtualinherited

Optional rtsp function: provide the encoded size (in bytes) of one frame, once known (e.g. after the encoder has seen real data). Lets an RTSPFormat size its RTP fragments to match one frame instead of a fixed guess, so throughput actually tracks frameDurationUs() - a fragment size covering several frames' worth of bytes sent at one-frame timing overruns real-time bandwidth and, over UDP (no flow control), causes packet loss. Returns 0 if not yet known/not applicable (fixed-frame-size codecs have no need to override this).

Reimplemented in ADPCMEncoder, EncoderALAC, AMRNBEncoder, AMRWBEncoder, and MP3ParserEncoder.

getFormat()

RTSPFormat & getFormat ( )

inlineoverridevirtual

Get the media format configuration.

Returns the RTSPFormat object that describes the media data characteristics including sample rate, bit depth, number of channels, and RTP packaging parameters. If no format has been explicitly set, creates a default 16-bit PCM format at 16kHz mono.

Returns

Pointer to RTSPFormat describing the media characteristics

Note

Default format: 16-bit PCM, 16000 Hz, 1 channel

See also

setFormat(), RTSPFormatPCM

Implements IMediaSource.

isCompleteJpegFrame()

bool isCompleteJpegFrame ( )

inlineprotected

Check if current buffer contains a complete JPEG frame.

Returns

true if frame is complete (ends with 0xFF 0xD9)

mime()

const char * mime ( )

inlineoverridevirtual

Provides the mime type of the encoded result.

Implements AudioEncoder.

operator bool()

operator bool ( )

inlineoverridevirtual

Implements AudioWriter.

packetSize()

int packetSize ( )

inlineoverridevirtual

Size of the next queued, ready-to-send fragment, for sources that provide already RTP-payload-ready, self-delimited fragments (e.g. RFC 2435 JPEG) instead of a continuous byte stream. For these sources, readBytes() must always be called with exactly packetSize() as maxBytes, so each call retrieves exactly one complete fragment instead of the transport having to guess where one RTP payload ends and the next begins.

Call it once before each readBytes() to size that call, and again right after to tell whether the fragment just read was the last one currently available - a 0 result means the caller should set the RTP marker bit on the fragment it just sent (and may advance the timestamp for the next frame).

Returns

-1 if this source is not packetized (use the plain readBytes() stream instead), 0 if packetized but nothing is queued right now, otherwise the size in bytes of the next fragment

Reimplemented from IMediaSource.

processCompleteFrame()

void processCompleteFrame ( )

inlineprotected

Strip the JPEG headers, fragment the scan data and queue one RFC 2435 RTP payload per fragment.

queuePacketSize()

int queuePacketSize ( )

inlineprotectedinherited

queueReadBytes()

int queueReadBytes ( void *  dest, int  maxBytes  )

inlineprotectedinherited

Retrieves the next queued fragment. Call queuePacketSize() first and pass its result as maxBytes so exactly one fragment is read.

readBytes()

int readBytes ( void *  dest, int  maxBytes  )

inlineoverridevirtual

Retrieves the next queued fragment. Call packetSize() first and pass its result as maxBytes so exactly one fragment is read.

Implements IMediaSource.

readQueueU32()

uint32_t readQueueU32 ( size_t  pos )

inlineprotectedinherited

samplesPerFrame()

virtual uint16_t samplesPerFrame ( )

inlinevirtualinherited

Optional rtsp function: provide samples per the frame.

Reimplemented in OpusAudioEncoder, OpusMultiStreamAudioEncoder, MP3EncoderShine, MP3ParserEncoder, and MetaDataFilterEncoder.

setAudioInfo()

void setAudioInfo ( AudioInfo  from )

inlineoverridevirtual

Defines the sample rate, number of channels and bits per sample.

Reimplemented from AudioEncoder.

setFormat()

void setFormat ( RTSPFormat &  format )

inlineoverridevirtual

Associates the RTSPFormat (e.g. RTSPFormatMJPEG) that supplies width/height.

Implements RTSPVideoEncoder.

setMaxFragmentSize()

void setMaxFragmentSize ( size_t  size )

inline

Set the maximum RTP payload size per packet (header + data)

setMaxQueuedFrames()

void setMaxQueuedFrames ( size_t  frames )

inline

Maximum number of complete frames buffered when the consumer falls behind.

setOutput()

virtual void setOutput ( Print &  out_stream )

inlineoverridevirtualinherited

Default output assignment (encoders may override to store Print reference)

Implements AudioWriter.

Reimplemented in EncoderBasic, DSFEncoder, WAVEncoder, EncoderBase64, CodecNOP, APTXEncoder, Codec2Encoder, CopyEncoder, G722Encoder, GSMEncoder, ILBCEncoder, LC3Encoder, MP3EncoderLAME, EncoderNetworkFormat, SBCEncoder, AACEncoderFAAC, AACEncoderFDK, AACEncoderVO, ADPCMEncoder, ADPCMEncoderXQ, EncoderALAC, AMRNBEncoder, AMRWBEncoder, CodecChain, FLACEncoder, EncoderFloat, G729Encoder, G7xxEncoder, EncoderL16, EncoderL8, MP3EncoderShine, OpusAudioEncoder, OpusMultiStreamAudioEncoder, MP3ParserEncoder, MetaDataFilterEncoder, BinaryContainerEncoder, OSCContainerEncoder, and OggContainerEncoder.

start()

void start ( )

inlineoverridevirtual

Initialize media source for streaming.

Called when streaming is about to begin. Implementations should use this to initialize hardware, allocate buffers, start media capture, or perform any other setup required for media data generation.

Note

Called by RTSPMediaStreamer.start()

Default implementation does nothing (suitable for sources that need no setup)

Reimplemented from IMediaSource.

stop()

void stop ( )

inlineoverridevirtual

Cleanup media source after streaming.

Called when streaming has ended. Implementations should use this to release hardware resources, deallocate buffers, stop media capture, or perform cleanup operations.

Note

Called by RTSPMediaStreamer.stop()

Default implementation does nothing (suitable for sources that need no cleanup)

Reimplemented from IMediaSource.

write()

size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

Process JPEG frame data and queue the resulting RTP fragments.

Parameters

data	Pointer to JPEG frame data
len	Length of JPEG frame data

Returns

Number of bytes processed

Implements AudioWriter.

writeBlocking()

void writeBlocking ( Print *  out, uint8_t *  data, size_t  len  )

inlineprotectedinherited

writeQuantTableHeader()

void writeQuantTableHeader ( uint8_t *  buffer, const uint8_t *  qTable, int  qTableLen  )

inlineprotected

RFC 2435 Quantization Table header (only present on the first fragment of a frame when Q >= 128)

Member Data Documentation

default_format

RTSPFormatMJPEG default_format

protected

info

AudioInfo info

protectedinherited

m_currentFrame

Vector<uint8_t> m_currentFrame

protected

m_frameComplete

bool m_frameComplete = true

protected

m_isStarted

bool m_isStarted = false

protected

m_maxFragmentSize

size_t m_maxFragmentSize = 1400

protected

m_maxQueuedFrames

size_t m_maxQueuedFrames = 1

protectedinherited

m_queue

Vector<uint8_t> m_queue

protectedinherited

m_queuedFrames

size_t m_queuedFrames = 0

protectedinherited

m_queueHead

size_t m_queueHead = 0

protectedinherited

m_scratch

Vector<uint8_t> m_scratch

protected

p_format

RTSPFormat* p_format = nullptr

protected

---

The documentation for this class was generated from the following file:

• src/AudioTools/Communication/RTSP/JPEGRtpEncoder.h