TfLiteAudioStream Class Reference

TfLiteAudioStream which uses Tensorflow Light to analyze the data. If it is used as a generator (where we read audio data) More...

#include <TfLiteAudioStream.h>

Inheritance diagram for TfLiteAudioStream:

Public Member Functions
	TfLiteAudioStream ()
	~TfLiteAudioStream ()
virtual void	addNotifyAudioChange (AudioInfoSupport &bi)
	Adds target to be notified about audio changes.
virtual AudioInfo	audioInfo () override
	provides the actual input AudioInfo
virtual AudioInfo	audioInfoOut ()
virtual int	available () override
	We can provide only some audio data when cfg.input is defined.
virtual int	availableForWrite () override
virtual int	availableToWrite () override
	Constant streaming.
virtual bool	begin ()
virtual bool	begin (TfLiteConfig config) override
	Start the processing.
virtual void	clearNotifyAudioChange ()
	Deletes all change notify subscriptions.
TfLiteConfig &	config () override
	Provides the TfLiteConfig information.
virtual TfLiteConfig	defaultConfig () override
virtual void	end ()
virtual void	flush () override
tflite::MicroInterpreter &	interpreter () override
	Provides the tf lite interpreter.
bool	isNotifyActive ()
	Checks if the automatic AudioInfo update is active.
int8_t *	modelInputBuffer () override
	Provides access to the model input buffer.
virtual	operator bool ()
virtual size_t	readBytes (uint8_t *data, size_t len) override
	provide audio data with cfg.input
virtual size_t	readSilence (uint8_t *buffer, size_t length)
	Source to generate silence: just sets the buffer to 0.
virtual bool	removeNotifyAudioChange (AudioInfoSupport &bi)
	Removes a target in order not to be notified about audio changes.
virtual void	setAudioInfo (AudioInfo newInfo) override
	Defines the input AudioInfo.
void	setInterpreter (tflite::MicroInterpreter *p_interpreter)
	Optionally define your own p_interpreter.
void	setNotifyActive (bool flag)
	Deactivate/Reactivate automatic AudioInfo updates: (default is active)
void	setWriteBufferSize (int size)
virtual size_t	write (const uint8_t *data, size_t len) override
	process the data in batches of max kMaxAudioSampleSize.
virtual size_t	write (uint8_t ch) override
virtual void	writeSilence (size_t len)
	Writes len bytes of silence (=0).

Protected Member Functions
virtual int	not_supported (int out, const char *msg="")
void	notifyAudioChange (AudioInfo info)
void	refillReadBuffer ()
	Refill small read buffer (e.g. 8 bytes) to avoid single byte reads when calling read()
virtual bool	setModel (const unsigned char *model)
virtual bool	setupInterpreter ()
virtual bool	setupWriter ()

Protected Attributes
int	_timeout = 10
TfLiteConfig	cfg
AudioInfo	info
bool	is_notify_active = true
bool	is_setup = false
Vector< AudioInfoSupport * >	notify_vector
tflite::MicroInterpreter *	p_interpreter = nullptr
const tflite::Model *	p_model = nullptr
TfLiteTensor *	p_tensor = nullptr
uint8_t *	p_tensor_arena = nullptr
int8_t *	p_tensor_buffer = nullptr
RingBuffer< uint8_t >	tmp_in {0}
RingBuffer< uint8_t >	tmp_out {0}
int	write_buffer_size = MAX_SINGLE_CHARS

Detailed Description

TfLiteAudioStream which uses Tensorflow Light to analyze the data. If it is used as a generator (where we read audio data)

Author

Phil Schatzmann

Copyright

GPLv3

Constructor & Destructor Documentation

TfLiteAudioStream()

TfLiteAudioStream ( )

inline

~TfLiteAudioStream()

~TfLiteAudioStream ( )

inline

Member Function Documentation

addNotifyAudioChange()

virtual void addNotifyAudioChange ( AudioInfoSupport &  bi )

inlinevirtualinherited

Adds target to be notified about audio changes.

Reimplemented in CodecNOP, EncodedAudioOutput, EncodedAudioStream, AACDecoderFDK, DecoderBasic, CodecChain, MP3DecoderHelix, MP3DecoderMAD, OggContainerDecoder, RTSPClient< TcpClient, UdpSocket >, Pipeline, and Pipeline::ModifyingStreamAdapter.

audioInfo()

virtual AudioInfo audioInfo ( )

inlineoverridevirtualinherited

provides the actual input AudioInfo

Implements AudioInfoSupport.

Reimplemented in JupyterAudioT< T >, MozziStream, TimerCallbackAudioStream, EncodedAudioStream, PureDataStream, AdapterAudioOutputToAudioStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, and InputMerge< T >.

audioInfoOut()

virtual AudioInfo audioInfoOut ( )

inlinevirtualinherited

provides the actual output AudioInfo: this is usually the same as audioInfo() unless we use a transforming stream

Reimplemented in PureDataStream, PWMAudioOutput, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, FormatConverterStream, Pipeline, ResampleStream, and ResampleStreamT< TInterpolator >.

available()

virtual int available ( )

inlineoverridevirtual

We can provide only some audio data when cfg.input is defined.

Reimplemented from BaseStream.

availableForWrite()

virtual int availableForWrite ( )

inlineoverridevirtualinherited

Reimplemented from Print.

Reimplemented in AudioLoRa, UDPStream, VBANStream, AudioStreamWrapper, ConverterStream< T >, LoRaStream, EncodedAudioStream, FaustStream< DSP >, AudioFFTBase, I2SCodecStream, MiniAudioStream, StdioStream, A2DPStream, ESPNowStream, HammingFEC< bytecount, block_t >, AbstractURLStream, URLStreamESP32, ReedSolomonFEC< bytecount, additional_bytes >, SPIAudioSlave, AnalogAudioArduino, AnalogAudioStream, AudioEffectStreamT< T >, Equalizer3Bands, Equalizer3BandsPerChannel, I2SStream, ReformatBaseStream, TimedStream, MemoryStream, RingBufferStream, MeasuringStream, ProgressStream, Throttle, FilteredStream< T, TF >, FilteredStream< int16_t, int16_t >, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, QueueStream< T >, QueueStream< uint8_t >, DynamicMemoryStream, FadeStream, FrequencyDetectorAutoCorrelation, FrequencyDetectorZeroCrossing, Pipeline, VolumeStream, AudioBLEClient, AudioBLEClient, AudioBLEServer, and AudioBLEServer.

availableToWrite()

virtual int availableToWrite ( )

inlineoverridevirtual

Constant streaming.

Implements TfLiteAudioStreamBase.

begin() [1/2]

virtual bool begin ( )

inlinevirtualinherited

Reimplemented in AudioOutputWithCallback, AudioMP34DT05, I2SCodecStream, MozziStream, PureDataStream, SPDIFOutput, VS1053Stream, WM8960Stream, AdaptiveResamplingStream, AudioLoRa, ESPNowStream, VBANStream, AudioEffectStreamT< T >, Equalizer3Bands, Equalizer3BandsPerChannel, I2SStream, AudioStreamWrapper, TimerCallbackAudioStream, FrequencyDetectorAutoCorrelation, GoertzelStream, ResampleStreamT< TInterpolator >, FileLoopT< FileType >, FileLoopT< File >, LoRaStream, EncodedAudioStream, AudioBoardStream, AudioFFTBase, AudioKitStream, MiniAudioStream, PortAudioStream, StdioStream, HLSStreamT< URLStream >, AnalogAudioArduino, AnalogAudioStream, AdapterAudioOutputToAudioStream, TimedStream, MemoryStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, MeasuringStream, ProgressStream, Throttle, InputMerge< T >, CallbackStream, FilteredStream< T, TF >, FilteredStream< int16_t, int16_t >, VolumeMeter, AudioInputMonitor, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, FormatConverterStream, CatStream, QueueStream< T >, QueueStream< uint8_t >, DynamicMemoryStream, Pipeline, Pipeline::ModifyingStreamAdapter, ResampleStream, VolumeStream, and EqualizerNBands< SampleT, AccT, NUM_TAPS, NUM_BANDS >.

begin() [2/2]

virtual bool begin ( TfLiteConfig  config )

inlineoverridevirtual

Start the processing.

Implements TfLiteAudioStreamBase.

clearNotifyAudioChange()

virtual void clearNotifyAudioChange ( )

inlinevirtualinherited

Deletes all change notify subscriptions.

Reimplemented in RTSPClient< TcpClient, UdpSocket >.

config()

TfLiteConfig & config ( )

inlineoverridevirtual

Provides the TfLiteConfig information.

Implements TfLiteAudioStreamBase.

defaultConfig()

virtual TfLiteConfig defaultConfig ( )

inlineoverridevirtual

Implements TfLiteAudioStreamBase.

end()

virtual void end ( )

inlinevirtualinherited

Reimplemented in FaustStream< DSP >, AudioMP34DT05, I2SCodecStream, MozziStream, PureDataStream, SPDIFOutput, VS1053Stream, WM8960Stream, AdaptiveResamplingStream, AudioLoRa, ESPNowStream, BufferedTaskStream, URLStreamBufferedT< T >, URLStreamBufferedT< ICYStream >, Equalizer3Bands, I2SStream, AudioStreamWrapper, TimerCallbackAudioStream, FormatConverterStream, GoertzelStream, FileLoopT< FileType >, FileLoopT< File >, EqualizerNBands< SampleT, AccT, NUM_TAPS, NUM_BANDS >, LoRaStream, EncodedAudioStream, AudioFFTBase, AudioKitStream, MiniAudioStream, PortAudioStream, StdioStream, A2DPStream, HLSStreamT< URLStream >, HttpRequest, ICYStreamT< T >, URLStream, URLStreamESP32, SPIAudioSlave, AnalogAudioArduino, AnalogAudioStream, AudioEffectStreamT< T >, Equalizer3BandsPerChannel, ReformatBaseStream, AdapterAudioOutputToAudioStream, MemoryStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, InputMixer< T, SumT >, InputMerge< T >, CallbackStream, FilteredStream< T, TF >, FilteredStream< int16_t, int16_t >, ChannelFormatConverterStream, NumberFormatConverterStream, CatStream, QueueStream< T >, QueueStream< uint8_t >, DynamicMemoryStream, Pipeline, Pipeline::ModifyingStreamAdapter, VolumeStream, AudioBLEClient, AudioBLEClient, AudioBLEServer, AudioBLEServer, AbstractURLStream, and AudioBLEStream.

flush()

virtual void flush ( )

inlineoverridevirtualinherited

Reimplemented from Print.

Reimplemented in PureDataStream, URLStreamBufferedT< T >, URLStreamBufferedT< ICYStream >, ReformatBaseStream, AudioStreamWrapper, ResampleStream, EncodedAudioStream, URLStream, BufferedTaskStream, I2SStream, MemoryStream, RingBufferStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, and BufferedStream.

interpreter()

tflite::MicroInterpreter & interpreter ( )

inlineoverridevirtual

Provides the tf lite interpreter.

Implements TfLiteAudioStreamBase.

isNotifyActive()

bool isNotifyActive ( )

inlineinherited

Checks if the automatic AudioInfo update is active.

modelInputBuffer()

int8_t * modelInputBuffer ( )

inlineoverridevirtual

Provides access to the model input buffer.

Implements TfLiteAudioStreamBase.

not_supported()

virtual int not_supported ( int  out, const char *  msg = ""  )

inlineprotectedvirtualinherited

notifyAudioChange()

void notifyAudioChange ( AudioInfo  info )

inlineprotectedinherited

operator bool()

virtual operator bool ( )

inlinevirtualinherited

Reimplemented in A2DPStream, ICYStreamT< T >, AudioBLEStream, AudioFFTBase, PortAudioStream, URLStream, I2SStream, AdapterAudioOutputToAudioStream, TimedStream, MemoryStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, and Pipeline.

readBytes()

virtual size_t readBytes ( uint8_t *  data, size_t  len  )

inlineoverridevirtual

provide audio data with cfg.input

Reimplemented from AudioStream.

readSilence()

virtual size_t readSilence ( uint8_t *  buffer, size_t  length  )

inlinevirtualinherited

Source to generate silence: just sets the buffer to 0.

refillReadBuffer()

void refillReadBuffer ( )

inlineprotectedinherited

Refill small read buffer (e.g. 8 bytes) to avoid single byte reads when calling read()

removeNotifyAudioChange()

virtual bool removeNotifyAudioChange ( AudioInfoSupport &  bi )

inlinevirtualinherited

Removes a target in order not to be notified about audio changes.

Reimplemented in RTSPClient< TcpClient, UdpSocket >.

setAudioInfo()

virtual void setAudioInfo ( AudioInfo  info )

inlineoverridevirtualinherited

Defines the input AudioInfo.

Implements AudioInfoSupport.

Reimplemented in VS1053Stream, WM8960Stream, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, VolumeStream, MiniAudioStream, PortAudioStream, I2SCodecStream, MozziStream, SPDIFOutput, AudioLoRa, AnalogAudioStream, I2SStream, TimerCallbackAudioStream, AudioBLEStream, LoRaStream, AudioFFTBase, AudioKitStream, SPIAudioSlave, VBANStream, AnalogAudioArduino, Equalizer3Bands, Equalizer3BandsPerChannel, AdapterAudioOutputToAudioStream, TimedStream, MeasuringStream, ProgressStream, CallbackStream, VolumeMeter, AudioInputMonitor, FormatConverterStream, FadeStream, GoertzelStream, Pipeline::ModifyingStreamAdapter, MDFEchoCancellationStream< Allocator >, EncodedAudioStream, PureDataStream, GeneratedSoundStream< T >, GeneratedSoundStream< int16_t >, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, Pipeline, ResampleStream, and ResampleStreamT< TInterpolator >.

setInterpreter()

void setInterpreter ( tflite::MicroInterpreter *  p_interpreter )

inlinevirtual

Optionally define your own p_interpreter.

Implements TfLiteAudioStreamBase.

setModel()

virtual bool setModel ( const unsigned char *  model )

inlineprotectedvirtual

setNotifyActive()

void setNotifyActive ( bool  flag )

inlineinherited

Deactivate/Reactivate automatic AudioInfo updates: (default is active)

setupInterpreter()

virtual bool setupInterpreter ( )

inlineprotectedvirtual

setupWriter()

virtual bool setupWriter ( )

inlineprotectedvirtual

setWriteBufferSize()

void setWriteBufferSize ( int  size )

inlineinherited

write() [1/2]

virtual size_t write ( const uint8_t *  data, size_t  len  )

inlineoverridevirtual

process the data in batches of max kMaxAudioSampleSize.

Implements TfLiteAudioStreamBase.

write() [2/2]

virtual size_t write ( uint8_t  ch )

inlineoverridevirtualinherited

Reimplemented in MemoryStream, AudioStreamWrapper, BufferedTaskStream, RingBufferStream, BufferedStream, and URLStream.

writeSilence()

virtual void writeSilence ( size_t  len )

inlinevirtualinherited

Writes len bytes of silence (=0).

Member Data Documentation

_timeout

int _timeout = 10

protectedinherited

cfg

TfLiteConfig cfg

protected

info

AudioInfo info

protectedinherited

is_notify_active

bool is_notify_active = true

protectedinherited

is_setup

bool is_setup = false

protected

notify_vector

Vector<AudioInfoSupport*> notify_vector

protectedinherited

p_interpreter

tflite::MicroInterpreter* p_interpreter = nullptr

protected

p_model

const tflite::Model* p_model = nullptr

protected

p_tensor

TfLiteTensor* p_tensor = nullptr

protected

p_tensor_arena

uint8_t* p_tensor_arena = nullptr

protected

p_tensor_buffer

int8_t* p_tensor_buffer = nullptr

protected

tmp_in

RingBuffer<uint8_t> tmp_in {0}

protectedinherited

tmp_out

RingBuffer<uint8_t> tmp_out {0}

protectedinherited

write_buffer_size

int write_buffer_size = MAX_SINGLE_CHARS

protectedinherited

---

The documentation for this class was generated from the following file:

• src/AudioTools/AudioLibs/TfLiteAudioStream.h