SoundTouch音頻處理庫源碼分析及演算法提取(6)

TDStretch類的實現

SoundTouch類成員函數putSamples(const SAMPLETYPE *samples, uint nSamples)實現如下，根據上篇的分析rate是一個比率，大於1表示速度加快，小於1表示速度減慢，對於播放速度減慢這種情況。

……

#ifndef PREVENT_CLICK_AT_RATE_CROSSOVER

    else if (rate <= 1.0f)

    {

        // transpose the rate down, output the transposed sound to tempo changer buffer

        assert(output == pTDStretch);

        pRateTransposer->putSamples(samples, nSamples);

        pTDStretch->moveSamples(*pRateTransposer);

    }

    else

#endif

    {

        // evaluate the tempo changer, then transpose the rate up,

        assert(output == pRateTransposer);

        pTDStretch->putSamples(samples, nSamples);

        pRateTransposer->moveSamples(*pTDStretch);

    }

……

先通過pRateTransposer->putSamples(samples, nSamples);對聲音進行了重採樣，採用的是線性插值法，然後調用pTDStretch->moveSamples(*pRateTransposer);pTDStretch是TDStretch類的執行個體。TDStretch類定義如下：

/// Class that does the time-stretch (tempo change) effect for the processed

/// sound.

class TDStretch : public FIFOProcessor

{

protected:

    int channels;

    int sampleReq;

    float tempo;

 

    SAMPLETYPE *pMidBuffer;

    SAMPLETYPE *pRefMidBuffer;

    SAMPLETYPE *pRefMidBufferUnaligned;

    int overlapLength;

    int seekLength;

    int seekWindowLength;

    int overlapDividerBits;

    int slopingDivider;

    float nominalSkip;

    float skipFract;

    FIFOSampleBuffer outputBuffer;

    FIFOSampleBuffer inputBuffer;

    BOOL bQuickSeek;

//    int outDebt;

//    BOOL bMidBufferDirty;

 

    int sampleRate;

    int sequenceMs;

    int seekWindowMs;

    int overlapMs;

    BOOL bAutoSeqSetting;

    BOOL bAutoSeekSetting;

 

    void acceptNewOverlapLength(int newOverlapLength);

 

    virtual void clearCrossCorrState();

    void calculateOverlapLength(int overlapMs);

 

    virtual LONG_SAMPLETYPE calcCrossCorrStereo(const SAMPLETYPE *mixingPos, const SAMPLETYPE *compare) const;

    virtual LONG_SAMPLETYPE calcCrossCorrMono(const SAMPLETYPE *mixingPos, const SAMPLETYPE *compare) const;

 

    virtual int seekBestOverlapPositionStereo(const SAMPLETYPE *refPos);

    virtual int seekBestOverlapPositionStereoQuick(const SAMPLETYPE *refPos);

    virtual int seekBestOverlapPositionMono(const SAMPLETYPE *refPos);

    virtual int seekBestOverlapPositionMonoQuick(const SAMPLETYPE *refPos);

    int seekBestOverlapPosition(const SAMPLETYPE *refPos);

 

    virtual void overlapStereo(SAMPLETYPE *output, const SAMPLETYPE *input) const;

    virtual void overlapMono(SAMPLETYPE *output, const SAMPLETYPE *input) const;

 

    void clearMidBuffer();

    void overlap(SAMPLETYPE *output, const SAMPLETYPE *input, uint ovlPos) const;

 

    void precalcCorrReferenceMono();

    void precalcCorrReferenceStereo();

 

    void calcSeqParameters();

 

    /// Changes the tempo of the given sound samples.

    /// Returns amount of samples returned in the "output" buffer.

    /// The maximum amount of samples that can be returned at a time is set by

    /// the 'set_returnBuffer_size' function.

    void processSamples();

   

public:

    TDStretch();

    virtual ~TDStretch();

 

    /// Operator 'new' is overloaded so that it automatically creates a suitable instance

    /// depending on if we've a MMX/SSE/etc-capable CPU available or not.

    static void *operator new(size_t s);

 

    /// Use this function instead of "new" operator to create a new instance of this class.

    /// This function automatically chooses a correct feature set depending on if the CPU

    /// supports MMX/SSE/etc extensions.

    static TDStretch *newInstance();

   

    /// Returns the output buffer object

    FIFOSamplePipe *getOutput() { return &outputBuffer; };

 

    /// Returns the input buffer object

    FIFOSamplePipe *getInput() { return &inputBuffer; };

 

    /// Sets new target tempo. Normal tempo = 'SCALE', smaller values represent slower

    /// tempo, larger faster tempo.

    void setTempo(float newTempo);

 

    /// Returns nonzero if there aren't any samples available for outputting.

    virtual void clear();

 

    /// Clears the input buffer

    void clearInput();

 

    /// Sets the number of channels, 1 = mono, 2 = stereo

    void setChannels(int numChannels);

 

    /// Enables/disables the quick position seeking algorithm. Zero to disable,

    /// nonzero to enable

    void enableQuickSeek(BOOL enable);

 

    /// Returns nonzero if the quick seeking algorithm is enabled.

    BOOL isQuickSeekEnabled() const;

 

    /// Sets routine control parameters. These control are certain time constants

    /// defining how the sound is stretched to the desired duration.

    //

    /// 'sampleRate' = sample rate of the sound

    /// 'sequenceMS' = one processing sequence length in milliseconds

    /// 'seekwindowMS' = seeking window length for scanning the best overlapping

    ///      position

    /// 'overlapMS' = overlapping length

    void setParameters(int sampleRate,          ///< Samplerate of sound being processed (Hz)

                       int sequenceMS = -1,     ///< Single processing sequence length (ms)

                       int seekwindowMS = -1,   ///< Offset seeking window length (ms)

                       int overlapMS = -1       ///< Sequence overlapping length (ms)

                       );

 

    /// Get routine control parameters, see setParameters() function.

    /// Any of the parameters to this function can be NULL, in such case corresponding parameter

    /// value isn't returned.

    void getParameters(int *pSampleRate, int *pSequenceMs, int *pSeekWindowMs, int *pOverlapMs) const;

 

    /// Adds 'numsamples' pcs of samples from the 'samples' memory position into

    /// the input of the object.

    virtual void putSamples(

            const SAMPLETYPE *samples,  ///< Input sample data

            uint numSamples                         ///< Number of samples in 'samples' so that one sample

                                                    ///< contains both channels if stereo

            );

};

TDStretch類和基類的派生關係

FIFOSamplePipe-> FIFOProcessor->TDStretch

我們先看看他的建構函式

TDStretch::TDStretch() : FIFOProcessor(&outputBuffer)

{

    bQuickSeek = FALSE;

    channels = 2;

 

    pMidBuffer = NULL;

    pRefMidBufferUnaligned = NULL;

    overlapLength = 0;

 

    bAutoSeqSetting = TRUE;

    bAutoSeekSetting = TRUE;

 

//    outDebt = 0;

    skipFract = 0;

 

    tempo = 1.0f;

    setParameters(44100, DEFAULT_SEQUENCE_MS, DEFAULT_SEEKWINDOW_MS, DEFAULT_OVERLAP_MS);

    setTempo(1.0f);

    clear();

}

一些參數的初始化。

先看看在原始碼TDStretch.cpp中實現的類成員函數setParameters()

// Sets routine control parameters. These control are certain time constants

// defining how the sound is stretched to the desired duration.

//

// 'sampleRate' = sample rate of the sound

// 'sequenceMS' = one processing sequence length in milliseconds (default = 82 ms)

// 'seekwindowMS' = seeking window length for scanning the best overlapping

//      position (default = 28 ms)

// 'overlapMS' = overlapping length (default = 12 ms)

 

void TDStretch::setParameters(int aSampleRate, int aSequenceMS,

                              int aSeekWindowMS, int aOverlapMS)

{

    // accept only positive parameter values - if zero or negative, use old values instead

    if (aSampleRate > 0)   this->sampleRate = aSampleRate;

    if (aOverlapMS > 0)    this->overlapMs = aOverlapMS;

    if (aSequenceMS > 0)

    {

        this->sequenceMs = aSequenceMS;

        bAutoSeqSetting = FALSE;

    }

    else if (aSequenceMS == 0)

    {

        // if zero, use automatic setting

        bAutoSeqSetting = TRUE;

    }

    if (aSeekWindowMS > 0)

    {

        this->seekWindowMs = aSeekWindowMS;

        bAutoSeekSetting = FALSE;

    }

    else if (aSeekWindowMS == 0)

    {

        // if zero, use automatic setting

        bAutoSeekSetting = TRUE;

    }

    calcSeqParameters();

    calculateOverlapLength(overlapMs);

    // set tempo to recalculate 'sampleReq'

    setTempo(tempo);

}

其中主要參數的計算通過以下三個類成員函數來完成：

calcSeqParameters();

calculateOverlapLength(overlapMs);// set tempo to calculate 'sampleReq'

setTempo(tempo);

通過代碼中類成員函數的實現，我們可以知道calcSeqParameters()用來計算seekWindowLength，還有seekLength，都是通過一個簡單的換算公式Length = (sampleRate * sequenceMs) / 1000;毫秒換算到多少個Sample。

/// Calculates processing sequence length according to tempo setting

void TDStretch::calcSeqParameters()

{

    // Adjust tempo param according to tempo, so that variating processing sequence length is used

    // at varius tempo settings, between the given low...top limits

    #define AUTOSEQ_TEMPO_LOW   0.5     // auto setting low tempo range (-50%)

    #define AUTOSEQ_TEMPO_TOP   2.0     // auto setting top tempo range (+100%)

 

    // sequence-ms setting values at above low & top tempo

    #define AUTOSEQ_AT_MIN      125.0

    #define AUTOSEQ_AT_MAX      50.0

    #define AUTOSEQ_K           ((AUTOSEQ_AT_MAX - AUTOSEQ_AT_MIN) / (AUTOSEQ_TEMPO_TOP - AUTOSEQ_TEMPO_LOW))

    #define AUTOSEQ_C           (AUTOSEQ_AT_MIN - (AUTOSEQ_K) * (AUTOSEQ_TEMPO_LOW))

 

    // seek-window-ms setting values at above low & top tempo

    #define AUTOSEEK_AT_MIN     25.0

    #define AUTOSEEK_AT_MAX     15.0

    #define AUTOSEEK_K          ((AUTOSEEK_AT_MAX - AUTOSEEK_AT_MIN) / (AUTOSEQ_TEMPO_TOP - AUTOSEQ_TEMPO_LOW))

    #define AUTOSEEK_C          (AUTOSEEK_AT_MIN - (AUTOSEEK_K) * (AUTOSEQ_TEMPO_LOW))

 

    #define CHECK_LIMITS(x, mi, ma) (((x) < (mi)) ? (mi) : (((x) > (ma)) ? (ma) : (x)))

 

    double seq, seek;

   

    if (bAutoSeqSetting)

    {

        seq = AUTOSEQ_C + AUTOSEQ_K * tempo;

        seq = CHECK_LIMITS(seq, AUTOSEQ_AT_MAX, AUTOSEQ_AT_MIN);

        sequenceMs = (int)(seq + 0.5);

    }

 

    if (bAutoSeekSetting)

    {

        seek = AUTOSEEK_C + AUTOSEEK_K * tempo;

        seek = CHECK_LIMITS(seek, AUTOSEEK_AT_MAX, AUTOSEEK_AT_MIN);

        seekWindowMs = (int)(seek + 0.5);

    }

 

    // Update seek window lengths

    seekWindowLength = (sampleRate * sequenceMs) / 1000;

    if (seekWindowLength < 2 * overlapLength)

    {

        seekWindowLength = 2 * overlapLength;

    }

    seekLength = (sampleRate * seekWindowMs) / 1000;

}

類成員函數calculateOverlapLength()計算重疊的長度，

/// Calculates overlapInMsec period length in samples.

void TDStretch::calculateOverlapLength(int overlapInMsec)

{

    int newOvl;

 

    assert(overlapInMsec >= 0);

    newOvl = (sampleRate * overlapInMsec) / 1000;

    if (newOvl < 16) newOvl = 16;

 

    // must be divisible by 8

    newOvl -= newOvl % 8;

 

    acceptNewOverlapLength(newOvl);

}

類成員函數acceptNewOverlapLength()分配重疊部分需要佔用的記憶體空間.

/// Set new overlap length parameter & reallocate RefMidBuffer if necessary.

void TDStretch::acceptNewOverlapLength(int newOverlapLength)

{

    int prevOvl;

 

    assert(newOverlapLength >= 0);

    prevOvl = overlapLength;

    overlapLength = newOverlapLength;

 

    if (overlapLength > prevOvl)

    {

        delete[] pMidBuffer;

        delete[] pRefMidBufferUnaligned;

 

        pMidBuffer = new SAMPLETYPE[overlapLength * 2];

        clearMidBuffer();

 

        pRefMidBufferUnaligned = new SAMPLETYPE[2 * overlapLength + 16 / sizeof(SAMPLETYPE)];

        // ensure that 'pRefMidBuffer' is aligned to 16 byte boundary for efficiency

        pRefMidBuffer = (SAMPLETYPE *)((((ulong)pRefMidBufferUnaligned) + 15) & (ulong)-16);

    }

}

類成員函數setTempo()重新設定了音訊伸縮.

// Sets new target tempo. Normal tempo = 'SCALE', smaller values represent slower

// tempo, larger faster tempo.

void TDStretch::setTempo(float newTempo)

{

    int intskip;

 

    tempo = newTempo;

 

    // Calculate new sequence duration

    calcSeqParameters();

 

    // Calculate ideal skip length (according to tempo value)

    nominalSkip = tempo * (seekWindowLength - overlapLength);

    intskip = (int)(nominalSkip + 0.5f);

 

    // Calculate how many samples are needed in the 'inputBuffer' to

    // process another batch of samples

    //sampleReq = max(intskip + overlapLength, seekWindowLength) + seekLength / 2;

    sampleReq = max(intskip + overlapLength, seekWindowLength) + seekLength;

}

先記下Stretch用到的參數,現在我們來看看這些參數的實際物理意義。

音訊伸縮一般採用Sola的演算法來實現。如所示：

 

演算法大致如下：

從原始聲音資料的開頭處取出一定大小的資料，假如取7個sample，放在一個新的Buffer，如所示，然後在未經處理資料再往後面的資料中取9個sample，與前面的7個sample做一個疊加，疊加的範圍我們假設為2，那麼(7-2) /9 =0.555，這就意味著聲音的期間和原來相比減少了約44.5%，同時注意到一點，時間的間隔(採樣頻率)並沒有改變，也就是說聲音的頻率(音調)沒有發生改變。至於為什麼要疊加一部分，就是為了抑制這種由不連續的抽取聲音訊號造成的資料丟失所引發的噪音或者聲音過度不自然。這個圖對照上面TDStretch三個類成員函數，就理解了那些函數初始化的定義。同時變調不變調的處理過程更為清晰。就和SoundTouch類成員函數putSamples條件判斷中的一致，無非就是先伸縮後重採樣，或者先重採樣再伸縮的問題。

Sola的具體流程，TDStretch類成員函數processSamples十分清晰的表達，先拷貝一個序列到開頭，接著找到最佳的疊加位置，通過計算歸一化互相關係數來比較得到，主要實現是通過類成員函數seekBestOverlapPosition(const SAMPLETYPE *refPos)判斷是單聲道和雙聲道，分別調用不同的 TDStretch::seekBestOverlapPositionXXXX(const SAMPLETYPE *refPos)；有浮點和定點兩個版本，同樣以單聲道浮點版本為例：

int TDStretch::seekBestOverlapPositionMono(const SAMPLETYPE *refPos)

{

    int bestOffs;

    double bestCorr, corr;

    int tempOffset;

    const SAMPLETYPE *compare;

    // Slopes the amplitude of the 'midBuffer' samples

    precalcCorrReferenceMono();

    bestCorr = FLT_MIN;

    bestOffs = 0;

    // Scans for the best correlation value by testing each possible position

    // over the permitted range.

    for (tempOffset = 0; tempOffset < seekLength; tempOffset ++)

    {

        compare = refPos + tempOffset;

        // Calculates correlation value for the mixing position corresponding

        // to 'tempOffset'

        corr = (double)calcCrossCorrMono(pRefMidBuffer, compare);

        // heuristic rule to slightly favour values close to mid of the range

        double tmp = (double)(2 * tempOffset - seekLength) / seekLength;

        corr = ((corr + 0.1) * (1.0 - 0.25 * tmp * tmp));

        // Checks for the highest correlation value

        if (corr > bestCorr)

        {

            bestCorr = corr;

            bestOffs = tempOffset;

        }

    }

    // clear cross correlation routine state if necessary (is so e.g. in MMX routines).

    clearCrossCorrState();

    return bestOffs;

}

類成員函數seekBestOverlapPositionMono調用了類成員函數calcCrossCorrMono()

double TDStretch::calcCrossCorrMono(const float *mixingPos, const float *compare) const

{

    double corr;

    double norm;

    int i;

    corr = norm = 0;

    for (i = 1; i < overlapLength; i ++)

    {

        corr += mixingPos[i] * compare[i];

        norm += mixingPos[i] * mixingPos[i];

    }

 

    if (norm < 1e-9) norm = 1.0;    // to avoid div by zero

    return corr / sqrt(norm);

}

根據互相關係數的計算公式corr = x(n)*h(-n);*為卷積。和我們的形式有點不一樣。下次再慢慢分析。最後把後面一個序列拷貝到疊加的位置，疊加部分的幅值通過TDStretch類成員函數overlap來計算，具體代碼如下，通過判斷聲道調用一個單聲道或者雙聲道的類成員函數來處理。以單聲道為例，主要考慮到比較好理解。其實雙聲道也差不多。就是注意處理資料迴圈的增量，和在迴圈處理中每次多一個右聲道或者左聲道的資料處理。

// Overlaps samples in 'midBuffer' with the samples in 'pInputBuffer' at position

// of 'ovlPos'.

inline void TDStretch::overlap(SAMPLETYPE *pOutput, const SAMPLETYPE *pInput, uint ovlPos) const

{

    if (channels == 2)

    {

        // stereo sound

        overlapStereo(pOutput, pInput + 2 * ovlPos);

    } else {

        // mono sound.

        overlapMono(pOutput, pInput + ovlPos);

    }

}

類成員函數overlapMono的具體實現如下：

// Overlaps samples in 'midBuffer' with the samples in 'pInput'

void TDStretch::overlapMono(SAMPLETYPE *pOutput, const SAMPLETYPE *pInput) const

{

    int i, itemp;

    for (i = 0; i < overlapLength ; i ++)

    {

        itemp = overlapLength - i;

        pOutput[i] = (pInput[i] * i + pMidBuffer[i] * itemp ) / overlapLength;    // >> overlapDividerBits;

    }

}

pMidBuffer與pInput重疊，重疊長度為 overlapLength。

留意到核心的演算法僅僅是一行代碼pOutput[i] = (pInput[i] * i + pMidBuffer[i] * itemp ) / overlapLength;設a = i;b = itemp;k = overlapLength;x = pInput[i],y = pMidBuffer[i], z = pOutPut[i]把這行代碼用下面兩行虛擬碼替代：x,y分別作為系統的兩個輸入，z作為輸出。

a  +  b  =  k;

ax  +  by  =  kz;

很眼熟，但是一會半刻又說不上來是什麼。暫時記下吧。以後再搞明白這個演算法叫什麼。

 

本文來自CSDN部落格，轉載請標明出處：http://blog.csdn.net/suhetao/archive/2010/09/04/5863477.aspx