Abstract base class for effects that operate on audio in the frequency domain: incoming samples are transformed with a forward FFT, the resulting bins are modified by the effect() method that a subclass implements, and the modified spectrum is transformed back with an inverse FFT (overlap-add) and streamed to the output. More...

#include <FFTEffects.h>

Inheritance diagram for FFTEffect:

Public Member Functions
	FFTEffect (Print &out)

	FFTEffect (Print &out, AudioFFTBase &fft)

virtual void	addNotifyAudioChange (AudioInfoSupport &bi)
	Adds target to be notified about audio changes.

virtual AudioInfo	audioInfo () override
	provides the actual input AudioInfo

virtual AudioInfo	audioInfoOut ()

virtual int	availableForWrite () override

bool	begin () override

virtual bool	begin (AudioInfo info)

bool	begin (FFTEffectConfig info)

virtual void	clearNotifyAudioChange ()
	Deletes all change notify subscriptions.

FFTEffectConfig	defaultConfig ()

virtual void	end ()

virtual void	flush ()

virtual bool	isDeletable ()
	If true we need to release the related memory in the destructor.

bool	isNotifyActive ()
	Checks if the automatic AudioInfo update is active.

virtual	operator bool ()

virtual bool	removeNotifyAudioChange (AudioInfoSupport &bi)
	Removes a target in order not to be notified about audio changes.

virtual void	setAudioInfo (AudioInfo newInfo) override
	Defines the input AudioInfo.

void	setNotifyActive (bool flag)
	Deactivate/Reactivate automatic AudioInfo updates: (default is active)

size_t	write (const uint8_t *data, size_t len) override

virtual size_t	write (uint8_t ch) override

virtual void	writeSilence (size_t len)

Protected Member Functions
virtual void	effect (AudioFFTBase &fft)=0

void	notifyAudioChange (AudioInfo info)

void	processOutput ()

Static Protected Member Functions
static void	effect_callback (AudioFFTBase &fft)

Protected Attributes
int	_timeout = 10

BufferedWindow	buffered {&hann}

AudioInfo	cfg

StreamCopy	copier

AudioRealFFT	default_fft

AudioFFTConfig	fft_cfg {default_fft.defaultConfig(RXTX_MODE)}

Hann	hann

bool	is_active = false

bool	is_notify_active = true

Vector< AudioInfoSupport * >	notify_vector

AudioFFTBase *	p_fft = nullptr

Print *	p_out = nullptr

SingleBuffer< uint8_t >	tmp {MAX_SINGLE_CHARS}

int	tmpPos = 0

Detailed Description

Abstract base class for effects that operate on audio in the frequency domain: incoming samples are transformed with a forward FFT, the resulting bins are modified by the effect() method that a subclass implements, and the modified spectrum is transformed back with an inverse FFT (overlap-add) and streamed to the output.

Processing pipeline driven by write():

incoming samples are accumulated into windows of FFTEffectConfig::length samples, hopping by FFTEffectConfig::stride (stride < length gives overlapping windows)
a windowed forward FFT is calculated once a window is complete
effect(AudioFFTBase&) is invoked so the subclass can inspect/replace bins via fft.magnitude()/fft.getBin()/fft.setBin()
the inverse FFT is calculated and the result is overlap-added and copied to the output

Usage:

FFTPitchShift effect(out);  // or FFTRobotize, FFTWhisper, FFTNop, FFTEqualizer...
auto cfg = effect.defaultConfig();
cfg.copyFrom(info);  // sample_rate / channels / bits_per_sample
cfg.length = 1024;   // fft size, must be a power of two
cfg.stride = 512;    // hop size; < length gives overlap-add
effect.begin(cfg);

By default an internally owned AudioRealFFT performs the fft/ifft; pass a different AudioFFTBase-derived instance (e.g. AudioKissFFT, AudioCmsisFFT, AudioESP32FFT, AudioEspressifFFT, AudioFixedFFT) to the constructor to use a different implementation. The engine must outlive the effect.

To implement a new effect, subclass FFTEffect and override effect(AudioFFTBase&); see FFTRobotize, FFTWhisper, FFTPitchShift and FFTEqualizer for examples.

Note: This is quite processing time intensive: keep the sample rate low if the processor is not fast enough to keep up in real time or select an integer based FFT implementation (AudioFixedFFT) instead of the default AudioRealFFT.; Effectively mono only: the forward FFT analyzes just AudioFFTConfig::channel_used (default channel 0) and discards the other channels' input samples; the reconstructed signal is then written identically to every output channel. With stereo input/output this collapses the signal to dual-mono and loses the original stereo image - it does not process channels independently.

Author: phil schatzmann

Constructor & Destructor Documentation

◆ FFTEffect() [1/2]

FFTEffect ( Print & out )

inline

◆ FFTEffect() [2/2]

FFTEffect	(	Print &	out,
		AudioFFTBase &	fft
	)

inline

Member Function Documentation

◆ addNotifyAudioChange()

virtual void addNotifyAudioChange ( AudioInfoSupport & bi )

inlinevirtualinherited

Adds target to be notified about audio changes.

Reimplemented in CodecNOP, EncodedAudioOutput, EncodedAudioStream, AACDecoderFDK, DecoderBasic, CodecChain, MP3DecoderHelix, MP3DecoderMAD, OggContainerDecoder, RTSPClient< TcpClient, UdpSocket >, Pipeline, and Pipeline::ModifyingStreamAdapter.

◆ audioInfo()

virtual AudioInfo audioInfo ( )

inlineoverridevirtualinherited

provides the actual input AudioInfo

Implements AudioInfoSupport.

Reimplemented in AdapterPrintToAudioOutput, EncodedAudioOutput, and AdapterAudioStreamToAudioOutput.

◆ audioInfoOut()

virtual AudioInfo audioInfoOut ( )

inlinevirtualinherited

provides the actual output AudioInfo: this is usually the same as audioInfo() unless we use a transforming stream

Reimplemented in MP3EncoderShine, PureDataStream, PWMAudioOutput< PWMDriverT >, ChannelFormatConverterStreamT< T >, ChannelFormatConverterStream, NumberFormatConverterStreamT< TFrom, TTo >, NumberFormatConverterStream, FormatConverterStream, Pipeline, ResampleStream, and ResampleStreamT< TInterpolator >.

◆ availableForWrite()

virtual int availableForWrite ( )

inlineoverridevirtualinherited

Reimplemented from Print.

Reimplemented in AudioESP32ULP, RTSPOutput< Platform >, SPIAudioMaster, EncodedAudioOutput, AudioServerEx, AudioSyncWriter, AdapterAudioStreamToAudioOutput, CsvOutput< T >, MemoryOutput, and PWMAudioOutput< PWMDriverT >.

◆ begin() [1/3]

bool begin ( )

inlineoverridevirtual

Reimplemented from AudioOutput.

Reimplemented in FFTPitchShift, and FFTEqualizer.

◆ begin() [2/3]

virtual bool begin ( AudioInfo info )

inlinevirtualinherited

Reimplemented in AudioESP32ULP, RTSPOutput< Platform >, RTTTLOutput< T >, ChannelsSelectOutput, CsvOutput< T >, and EncodedAudioOutput.

◆ begin() [3/3]

bool begin ( FFTEffectConfig info )

inline

◆ clearNotifyAudioChange()

virtual void clearNotifyAudioChange ( )

inlinevirtualinherited

Deletes all change notify subscriptions.

Reimplemented in RTSPClient< TcpClient, UdpSocket >.

◆ defaultConfig()

FFTEffectConfig defaultConfig ( )

inline

◆ effect()

virtual void effect ( AudioFFTBase & fft )

protectedpure virtual

Implemented in FFTRobotize, FFTWhisper, FFTNop, FFTPitchShift, and FFTEqualizer.

◆ effect_callback()

static void effect_callback ( AudioFFTBase & fft )

inlinestaticprotected

bool isNotifyActive ( )

inlineinherited

Checks if the automatic AudioInfo update is active.

◆ notifyAudioChange()

void notifyAudioChange ( AudioInfo info )

inlineprotectedinherited

◆ operator bool()

virtual operator bool ( )

inlinevirtualinherited

Reimplemented in RTSPOutput< Platform >, EncodedAudioOutput, and AdapterAudioStreamToAudioOutput.

◆ processOutput()

void processOutput ( )

inlineprotected

◆ removeNotifyAudioChange()

virtual bool removeNotifyAudioChange ( AudioInfoSupport & bi )

inlinevirtualinherited

Removes a target in order not to be notified about audio changes.

Reimplemented in RTSPClient< TcpClient, UdpSocket >.

◆ setAudioInfo()

virtual void setAudioInfo ( AudioInfo info )

inlineoverridevirtualinherited

Defines the input AudioInfo.

Implements AudioInfoSupport.

Reimplemented in ChannelsSelectOutput, AdapterPrintToAudioOutput, MultiOutput, SPIAudioMaster, AdapterAudioStreamToAudioOutput, CsvOutput< T >, PWMAudioOutput< PWMDriverT >, and EncodedAudioOutput.

◆ setNotifyActive()

void setNotifyActive ( bool flag )

inlineinherited

Deactivate/Reactivate automatic AudioInfo updates: (default is active)

◆ write() [1/2]

size_t write	(	const uint8_t *	data,
		size_t	len
	)

inlineoverridevirtual

Implements AudioOutput.

◆ write() [2/2]

virtual size_t write ( uint8_t ch )

inlineoverridevirtualinherited

Reimplemented in MetaDataOutput, and MultiOutput.

◆ writeSilence()

virtual void writeSilence ( size_t len )

inlinevirtualinherited

Writes n 0 values (= silence)

Parameters

int tmpPos = 0

protectedinherited

The documentation for this class was generated from the following file:

src/AudioTools/FFT/FFTEffects.h

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ FFTEffect() [1/2]

◆ FFTEffect() [2/2]

Member Function Documentation

◆ addNotifyAudioChange()

◆ audioInfo()

◆ audioInfoOut()

◆ availableForWrite()

◆ begin() [1/3]

◆ begin() [2/3]

◆ begin() [3/3]

◆ clearNotifyAudioChange()

◆ defaultConfig()

◆ effect()

◆ effect_callback()

◆ end()

◆ flush()

◆ isDeletable()

◆ isNotifyActive()

◆ notifyAudioChange()

◆ operator bool()

◆ processOutput()

◆ removeNotifyAudioChange()

◆ setAudioInfo()

◆ setNotifyActive()

◆ write() [1/2]

◆ write() [2/2]

◆ writeSilence()

Member Data Documentation

◆ _timeout

◆ buffered

◆ cfg

◆ copier

◆ default_fft

◆ fft_cfg

◆ hann

◆ is_active

◆ is_notify_active

◆ notify_vector

◆ p_fft

◆ p_out

◆ tmp

◆ tmpPos