Native Audio Plugin SDK
Thisdocument?describes?the?built-in?native?audio?plugin?interface?of?unity?5.0.?Wewill?do?this?by?looking?at?some?specific?example?plugins?that?grow?incomplexity?as?we?move?along.?This?way,?we?start?out?with?very?basic?conceptsand?introduce?more?complex?use-cases?near?the?end?of?the?document.
本文檔介紹unity?5.0中內(nèi)建的音頻插件接口闸溃，通過一些的插件例程并不斷提高難度的方式進行講解虐块。這樣眼姐，我們會從基本概念的開始而到文檔結(jié)尾部分會涉及到復(fù)雜的用例。
Download
Firstthing?you?need?to?do?is?to?download?the?newest?audio?plugin?SDK?from?here.
下載
首先要做的是：從這兒(https://bitbucket.org/Unity-Technologies/nativeaudioplugins)下載最新的插件SDK。
Overview
概覽
Thenative?audio?plugin?system?consists?of?two?parts:
內(nèi)建的音頻插件系統(tǒng)包括兩個部分：
The?native?DSP?(Digital?????Signal?Processing)?plugin?which?has?to?be?implemented?as?a?.dll?(Windows)?????or?.dylib?(OSX)?in?C?or?C++.?Unlike?scripts?and?because?of?the?high?????demands?on?performance?this?has?to?be?compiled?for?any?platform?that?you?????want?to?support,?possibly?with?platform-specific?optimizations.
The?GUI?which?is?developed?????in?C#.?Note?that?the?GUI?is?optional,?so?you?always?start?out?plugin?????development?by?creating?the?basic?native?DSP?plugin,?and?let?Unity?show?a?????default?slider-based?UI?for?the?parameter?descriptions?that?the?native?????plugin?exposes.?We?recommend?this?approach?to?bootstrap?any?project.
使用C或C++語言編寫.dll文件或.dylib(OSX)文件實現(xiàn)原生的DSP(數(shù)字信號處理)插件村生。和腳本不同雄可，DSP對性能要求很高罕偎，這就得在希望支持的平臺上進行編譯躏将，甚至還包括對特定平臺的優(yōu)化硬毕。
使用C#開發(fā)的GUI界面呻引。GUI是可選的，而原生DSP插件開發(fā)是必不可少的吐咳，此外逻悠，Unity會為原生插件中的公開參數(shù)生成缺省的滑桿條控制元践，因此，我們建議以DSP作為工程開始童谒。
Notethat?you?can?initially?prototype?the?C#?GUI?as?a?.cs?file?that?you?just?dropinto?the?Assets/Editor?folder?(just?like?any?other?editor?script).?Later?on?youcan?move?this?into?a?proper?MonoDevelop?project?as?your?code?starts?to?grow?andneed?better?modularization?and?better?IDE?support.?This?enables?you?to?compileit?into?a?.dll,?making?it?easier?for?the?user?to?drop?into?the?project?and?alsoin?order?to?protect?your?code.
注意单旁，可以將C#的GUI原型文件保存為.cs文件放到Assets/Editor文件夾下（就像其他編輯器腳本一樣），后面隨著代碼增長饥伊，需要模塊化和IDE支持時你可以使用MonoDevelop項目管理它象浑，這樣，你就能將其編譯成一個.dll文件琅豆，讓用戶在需要時將其導(dǎo)入工程中就可以使用愉豺，同時還能起到保護代碼的作用。
Alsonote?that?both?the?native?DSP?and?GUI?DLLs?can?contain?multiple?plugins?andthat?the?binding?happens?only?through?the?names?of?the?effects?in?the?pluginsregardless?of?what?the?DLL?file?is?called.
另外茫因，原生的DSP和GUI?DLL中都可以包含多個插件蚪拦，調(diào)用時的動態(tài)綁定僅通過插件中的效果名稱進行而不必在意調(diào)用了哪個DLL。
What?are?all?these?files?
SDK中文件用途节腐？
Thenative?side?of?the?plugin?SDK?actually?only?consists?of?one?file(AudioPluginInterface.h),?but?to?make?it?easy?to?have?multiple?plugin?effectswithin?the?same?DLL?we?have?added?supporting?code?to?handle?the?effectdefinition?and?parameter?registration?in?a?simple?unified?way(AudioPluginUtil.h?and?AudioPluginUtil.cpp).?Note?that?the?NativePluginDemoproject?contains?a?number?of?example?plugins?to?get?you?started?and?show?avariety?of?different?plugin?types?that?are?useful?in?a?game?context.?We?placethis?code?in?the?public?domain,?so?feel?free?to?use?this?code?as?a?startingpoint?for?your?own?creations.
原生插件的SDK只有一個文件（AudioPluginInterface.h）外盯，但為了在同一DLL中包含多個插件效果，我們用統(tǒng)一的方式（AudioPluginUtil.h和AudioPluginUtil.cpp）通過添加支持代碼來解決效果定義和參數(shù)注冊問題翼雀。注意NativePluginDemo例子中包含了一些示例插件以便開始學(xué)習(xí)并展示了在一個游戲場景下不同類型插件的用途饱苟。我們已將這些代碼公開，你可以在自己的游戲中隨便使用這些代碼狼渊。
Developmentof?a?plugin?starts?with?defining?which?parameters?your?plugin?should?have.?Youdon’t?need?to?have?a?detailed?master?plan?of?all?the?parameters?that?the?pluginwill?have?laid?out?before?you?start,?but?it?helps?to?roughly?have?an?idea?ofhow?you?want?the?user?experience?to?be?and?what?components?you?will?need.
開發(fā)插件從定義插件需要的參數(shù)開始箱熬。雖然不需要你一開始就有一個面面俱到的參數(shù)表，但有關(guān)于用戶體驗和希望使用組件的大致認識是不無裨益的狈邑。
Theexample?plugins?that?we?provide?have?a?bunch?of?utility?functions?that?make?iteasy?Let’s?take?a?look?at?the“Ring?Modulator”example?plugin.?This?simpleplugin?multiplies?the?incoming?signal?by?a?sine?wave,?which?gives?a?niceradio-noise?/?broken?reception?like?effect,?especially?if?multiple?ringmodulation?effects?with?different?frequencies?are?chained.
提供的示例插件包括一系列功能函數(shù)以方便使用城须，在“Ring?Modulator”例子插件中，將輸入信號與sine波形復(fù)合米苹，產(chǎn)生一種radio-noise?/?broken?reception（收音機噪聲/接收中斷）似的效果糕伐，特別是將多個不同頻率的ring?modulation效果串聯(lián)起來后。
Thebasic?scheme?for?dealing?with?parameters?in?the?example?plugins?is?to?definethem?as?enum-values?that?we?use?as?indices?into?an?array?of?floats?for?bothconvenience?and?brevity.
例子插件中處理參數(shù)的基本方式是將它定義為枚舉值蘸嘶，為了方便和簡潔我們就用浮點數(shù)組來索引這些枚舉值良瞧。
enum?Param
{
P_FREQ,
P_MIX,
P_NUM
};
intInternalRegisterEffectDefinition(UnityAudioEffectDefinition&?definition)
{
intnumparams?=?P_NUM;
definition.paramdefs?=?new?UnityAudioParameterDefinition?[numparams];
RegisterParameter(definition,?"Frequency",?"Hz",
0.0f,kMaxSampleRate,?1000.0f,
1.0f,3.0f,
P_FREQ);
RegisterParameter(definition,?"Mix?amount",?"%",
0.0f,1.0f,?0.5f,
100.0f,?1.0f,
P_MIX);
returnnumparams;
}
Thenumbers?in?the?RegisterParameter?calls?are?the?minimum,?maximum?and?defaultvalues?followed?by?a?scaling?factor?used?for?display?only,?i.e.?in?the?case?ofa?percentage-value?the?actual?value?goes?from?0?to?1?and?is?scaled?by?100?whendisplayed.?There?is?no?custom?GUI?code?for?this,?but?as?mentioned?earlier,Unity?will?generate?a?default?GUI?from?these?basic?parameter?definitions.?Notethat?no?checks?are?performed?for?undefined?parameters,?so?the?AudioPluginUtilsystem?expects?that?all?declared?enum?values?(except?P_NUM)?are?matched?up?witha?corresponding?parameter?definition.
在RegisterParameter調(diào)用時使用的參數(shù)包括一個最小值、最大值和缺省值以及一個僅用于顯示的縮放因子训唱，這是一個在0到1之間的百分比褥蚯，在顯示時會放大100倍。這個例子中沒有自定義的GUI况增，像之前提到的赞庶，Unity會為這些基本的參數(shù)定義提供一個缺省的界面。注意，未定義的參數(shù)不會進行檢查歧强，因此在AudioPluginUtil中的每個參數(shù)都應(yīng)與聲明的枚舉值（除P_NUM外）一一對應(yīng)澜薄。
Behindthe?scenes?the?RegisterParameter?function?fills?out?an?entry?in?theUnityAudioParameterDefinition?array?of?the?UnityAudioEffectDefinition?structurethat?is?associated?with?that?plugin?(see“AudioEffectPluginInterface.h”).?Therest?that?needs?to?be?set?up?in?UnityAudioEffectDefinition?is?the?callbacks?tothe?functions?that?handle?instantiating?the?plugin?(CreateCallback),setting/getting?parameters?(SetFloatParameterCallback/UnityAudioEffect_GetFloatParameterCallback),doing?the?actual?processing?(UnityAudioEffect_ProcessCallback)?and?eventuallydestroying?the?plugin?instance?when?done?(UnityAudioEffect_ReleaseCallback).
在場景之后，RegisterParameter函數(shù)填充了UnityAudioParameterDefinition數(shù)組中的一個條目誊锭，該條目為UnityAudioEffectDefinition類型的結(jié)構(gòu)體表悬，該結(jié)構(gòu)體與插件相關(guān)（參見“AudioEffectPluginInterface.h”）。在UnityAudioEffectDefinition中剩下要設(shè)置是回調(diào)函數(shù)：處理插件初始化的函數(shù)（CreateCallback）丧靡，包括：設(shè)置/獲取參數(shù)（SetFloatParameterCallback/UnityAudioEffect_GetFloatParameterCallback）；進行音頻處理的函數(shù)(UnityAudioEffect_ProcessCallback)籽暇，和最終銷毀插件實例的回調(diào)函數(shù)(UnityAudioEffect_ReleaseCallback)温治。
Tomake?it?easy?to?have?multiple?plugins?in?the?same?DLL,?each?plugin?is?residingin?its?own?namespace,?and?a?specific?naming?convention?for?the?callbackfunctions?is?used?such?that?the?DEFINE_EFFECT?and?DECLARE_EFFECT?macros?canfill?out?the?UnityAudioEffectDefinition?structure.?Underneath?the?hood?all?theeffects?definitions?are?stored?in?an?array?to?which?a?pointer?is?returned?bythe?only?entry?point?of?the?library?UnityGetAudioEffectDefinitions.
為在一個DLL中實現(xiàn)多個插件，每個插件都必須處于自身的命名空間中戒悠，此外熬荆，還使用了一個特定的回調(diào)函數(shù)名稱轉(zhuǎn)換機制，這樣就可以用DEFINE_EFFECT和DECLARE_EFFECT宏填寫UnityAudioEffectDefinition結(jié)構(gòu)體了绸狐。為了封裝效果卤恳，所有效果定義都存儲在一個數(shù)組中，并在動態(tài)庫UnityGetAudioEffectDefinitions的唯一訪問點提供返回指向該數(shù)組的指針寒矿。
Thisis?useful?to?know?in?case?you?want?to?develop?bridge?plugins?that?map?fromother?plugin?formats?such?as?VST?or?AudioUnits?to?or?from?the?Unity?audioplugin?interface,?in?which?case?you?need?to?develop?a?more?dynamic?way?to?setup?the?parameter?descriptions?at?load?time.
如果你想開發(fā)如：從VST或AudioUnits向Unity音頻插件接口雙向映射的bridge插件的話突琳，你就需要建立一種更加動態(tài)化的方法去設(shè)置加載時的參數(shù)描述。
Instantiating?the?plugin
實例化插件
Thenext?thing?is?the?data?for?the?instance?of?the?plugin.?In?the?example?plugins,we?put?all?this?into?the?EffectData?structure.?The?allocation?of?this?musthappen?in?the?corresponding?CreateCallback?which?is?called?for?each?instance?ofthe?plugin?in?the?mixer.?In?this?simple?example?there’s?only?one?sine-wave?thatis?multiplied?to?all?channels,?other?more?advanced?plugins?need?allocateadditional?data?per?input?channel.
下一個要處理的是插件實例的數(shù)據(jù)符相。在例子插件中拆融，我把它們?nèi)糠胚MEffectData結(jié)構(gòu)體中，分配必須在混合器為每一個插件實例調(diào)用的相應(yīng)的CreateCallback函數(shù)中進行啊终。在這個簡單的例子中镜豹，只將sin波形與所有的信道相疊加。而在更加復(fù)雜的插件中需要為每個輸入信道單獨分配附加的數(shù)據(jù)蓝牲。
struct?EffectData
{
struct?Data
{
floatp[P_NUM];?//?Parameters??//參數(shù)
floats;????????//?Sine?output?of?oscillator??//?sine波形輸出
floatc;????????//?Cosine?output?of?oscillator??//?cosine波形輸出
};
union
{
Datadata;
unsignedchar?pad[(sizeof(Data)?+?15)?&?~15];
};
};
Waita?minute!?What’s?all?that?union?and?padding?stuff?about??Well,?unless?youdevelop?your?plugin?for?the?PlayStation?3?you?can?ignore?this?and?just?focus?onthe?members?inside?the“Data”structure,?but?in?case?you?want?to?port?yourplugin?to?this?platform?you?need?to?be?aware?that?on?PS3?the?signal?processingis?happening?on?the?SPUs?and?in?order?to?transfer?data?back?and?forth?betweenthe?main?CPU?and?the?SPU?you?need?to?make?sure?that?the?data?is?aligned?to?a?16byte?boundary,?and?thus?the“pad”array?makes?sure?that?the?total?size?of?theEffectData?structure?will?be?a?multiple?of?16?even?though?the?actual?Datastructure?where?our?plugin?data?lives?may?not?be?divisible?by?16.?Thisrestriction?makes?the?code?a?little?funkier?to?look?at,?but?in?the?end?thebenefit?is?that?there’s?only?one?piece?of?shared?code?to?maintain?on?allplatforms?and?that?it’s?easy?to?port?your?code?to?the?PS3?if?you?follow?the?wayit’s?done?in?the?examples.
在此還是要稍微解釋以下union和padding的作用的趟脂？當(dāng)然，如果你不是在為PlayStation3平臺開發(fā)插件例衍，你可以忽略這兩個參數(shù)昔期，而集中注意力于Data結(jié)構(gòu)體中的成員變量。但如果你希望插件應(yīng)用于PS3平臺的肄渗，你就需要知道镇眷，在PS3上信號處理是由SPU進行的，為了在CPU和SU之間來回傳輸數(shù)據(jù)翎嫡，你得保證數(shù)據(jù)是16位邊界對齊的欠动。這樣，padding數(shù)組就是為了保證EffectData結(jié)構(gòu)體大小正好是16的整數(shù)倍，即使真實數(shù)據(jù)的大小不一定能被16整除具伍。雖然這種實現(xiàn)方式可能使代碼看起來有點難翅雏，但卻可以獲得在所有平臺上一致的代碼。所以人芽，如果你按照例子編碼望几，那么恭喜啦，你的代碼就容易向PS3移植萤厅。
UNITY_AUDIODSP_RESULT?UNITY_AUDIODSP_CALLBACKCreateCallback(
UnityAudioEffectState*?state)
{
EffectData*effectdata?=?new?EffectData;
memset(effectdata,?0,?sizeof(EffectData));
effectdata->data.c?=?1.0f;
state->effectdata=?effectdata;
InitParametersFromDefinitions(
InternalRegisterEffectDefinition,?effectdata->data.p);
returnUNITY_AUDIODSP_OK;
}
TheUnityAudioEffectState?contains?various?data?from?the?host?such?as?the?samplingrate,?the?total?number?of?samples?processed?(for?timing),?or?whether?the?pluginis?bypassed,?and?is?passed?to?all?callback?functions.
UnityAudioEffectState包含了從調(diào)用程序輸入的數(shù)據(jù)橄抹，包括：采樣率、處理樣本數(shù)（為計時）惕味，是否忽略插件楼誓、并將這些參數(shù)傳遞給所有的回調(diào)函數(shù)。
Andobviously?to?free?the?plugin?instance?there?is?a?corresponding?function?too:
很明顯名挥，也需要負責(zé)釋放插件實例的函數(shù)
UNITY_AUDIODSP_RESULT?UNITY_AUDIODSP_CALLBACKReleaseCallback(
UnityAudioEffectState*?state)
{
EffectData::Data*?data?=&state->GetEffectData()->data;
delete?data;
returnUNITY_AUDIODSP_OK;
}
Themain?processing?of?audio?happens?in?the?ProcessCallback:
主要的音頻處理在ProcessCallback中進行：
UNITY_AUDIODSP_RESULT?UNITY_AUDIODSP_CALLBACKProcessCallback(
UnityAudioEffectState*?state,
float*inbuffer,?float*?outbuffer,
unsigned?intlength,
intinchannels,?int?outchannels)
{
EffectData::Data*?data?=&state->GetEffectData()->data;
float?w?=2.0f?*?sinf(kPI?*?data->p[P_FREQ]?/?state->samplerate);
for(unsignedint?n?=?0;?n?<?length;?n++)
{
for(inti?=?0;?i?<?outchannels;?i++)
{
outbuffer[n?*?outchannels?+?i]?=
inbuffer[n?*?outchannels?+?i]?*
(1.0f?-?data->p[P_MIX]?+?data->p[P_MIX]?*?data->s);
}
data->s?+=?data->c?*?w;?//?cheap?way?to?calculate?a?sine-wave?//計算sine波形的低代價方法
data->c?-=?data->s?*?w;
}
returnUNITY_AUDIODSP_OK;
}
We’vestripped?the?PS3-specific?parts?from?this?function?in?this?listing.?TheGetEffectData?function?at?the?top?is?just?a?helper?function?casting?the?effectdatafield?of?the?state?variable?to?the?EffectData::Data?in?the?structure?wedeclared?above.
在列出的函數(shù)中我們略過了為PS3處理的部分疟羹，最前面的GetEffectData函數(shù)用于將狀態(tài)變量effectdata轉(zhuǎn)換成我們上面聲明的結(jié)構(gòu)體EffectData::Data類型。
Othersimple?plugins?included?are?the?NoiseBox?plugin,?which?adds?and?multiplies?theinput?signal?by?white?noise?at?variable?frequencies,?or?the?Lofinator?plugin,which?does?simple?downsampling?and?quantization?of?the?signal.?All?of?these?maybe?used?in?combination?and?with?game-driven?animated?parameters?to?simulateanything?from?mobile?phones?to?bad?radio?reception?on?walkies,?brokenloudspeakers?etc.
其他的簡單插件包括NoiseBox插件禀倔，實現(xiàn)為輸入信號添加或復(fù)合各種頻率的隨機白噪聲的功能榄融；Lofinator插件，對信號進行簡單的降頻采樣和量化救湖。這些插件可以組合使用或者基于游戲驅(qū)動的動畫參數(shù)來模擬從移動電話響聲到未校準(zhǔn)通訊頻率的步話機愧杯，壞了的擴音機，等等效果捎谨。
TheStereoWidener,?which?decomposes?a?stereo?input?signal?into?mono?and?sidecomponents?with?variable?delay?and?then?recombines?these?to?increase?theperceived?stereo?effect.
StereoWidener將輸入的立體聲信號分解到單聲道和具有不同延遲的音源方向組件民效，接著重新組合這些信號以增強立體聲效果感受。
A?bunch?of?simple?plugins?withoutcustom?GUIs?to?get?started?with.
大量可以用于起步學(xué)習(xí)的簡單且沒有自定義GUI界面的示例插件涛救。
Which?plugin?to?load?on?which?platform?
為不同平臺加載不同插件
Nativeaudio?plugins?use?the?same?scheme?as?other?native?or?managed?plugins?in?thatthey?must?be?associated?with?their?respective?platforms?via?the?plugin?importerinspector.?You?can?read?more?about?the?subfolders?in?which?to?place?plugins?here.?The?platform?association?is?necessary?so?that?thesystem?knows?which?plugins?to?include?on?a?each?build?target?in?the?standalonebuilds,?and?with?the?introduction?of?64-bit?support?this?even?has?to?bespecified?within?a?platform.?OSX?plugins?are?special?in?this?regard?since?theUniversal?Binary?format?allows?them?to?contain?both?32?and?64?bit?variants?inthe?same?bundle.
原生音頻插件和其他原生及受管插件使用方式一樣：它們必須通過引入插件的觀察器建立與不同平臺的關(guān)聯(lián)畏邢。你可以閱讀此處以了解這些組件需要被設(shè)置在哪些子文件夾中。平臺相關(guān)性是必須要考慮的检吆，這樣系統(tǒng)才能針對不同的平臺構(gòu)建獨立應(yīng)用時選擇加載對應(yīng)的插件舒萎，對于64位平臺更是如此。不過蹭沛，OSX的插件就比較爽臂寝，通用的二進制格式使得它能在一個包中同時提供32位和64位版本。
Nativeplugins?in?Unity?that?are?called?from?managed?code?get?loaded?via?the[DllImport]?attribute?referencing?the?function?to?be?imported?from?the?nativeDLL.?However,?in?the?case?of?native?audio?plugins?things?are?different.?Thespecial?problem?that?arises?here?is?that?the?audio?plugins?need?to?be?loadedbefore?Unity?starts?creating?any?mixer?assets?that?may?need?effects?from?theplugins.?In?the?editor?this?is?no?problem,?because?we?can?just?reload?andrebuild?the?mixers?that?depend?on?plugins,?but?in?standalone?builds?the?pluginsmust?be?loaded?before?we?create?the?mixer?assets.?To?solve?this,?the?currentconvention?is?to?prefix?the?DLL?of?the?plugin“audioplugin”(case?insensitive)so?that?the?system?can?detect?this?and?add?it?to?a?list?of?plugins?that?willautomatically?be?loaded?at?start.?Remember?that?it’s?only?the?definitionsinside?the?plugin?that?define?the?names?of?the?effects?that?show?up?insideUnity’s?mixer,?so?the?DLL?can?be?called?anything,?but?it?needs?to?start?withthe?string“audioplugin”to?be?detected?as?such.
Unity中的原生插件在受管代碼中調(diào)用摊灭，從本地的DLL文件通過[DllImport]屬性引用導(dǎo)入函數(shù)咆贬。然而原生音頻插件卻不相同，問題的原因在于帚呼，音頻插件必須在Unity開始創(chuàng)建需要使用其效果的混音器資源創(chuàng)建之前先行加載掏缎，在編輯器環(huán)境中皱蹦，這沒有什么問題，因為我們可以通過重新加載并重構(gòu)建依賴插件的混音器解決此問題眷蜈，但在獨立發(fā)布的游戲中沪哺，必須在混音器創(chuàng)建前完成插件加載。為了解決這個問題酌儒，慣常的做法是在插件DLL中加入audioplugin前綴（大小寫敏感）辜妓，這樣系統(tǒng)就可以檢測到音頻插件，并將其添加到啟動時自動加載的插件列表中忌怎。注意籍滴，只有在插件內(nèi)部的定義才會決定顯示在Unity混音器中的名稱和效果，因此DLL文件可以隨意命名榴啸，但是它需要以字符串a(chǎn)udioplugin開始才能被檢測和加載异逐。
Forplatforms?such?as?IOS?the?plugin?code?needs?to?be?statically?linked?into?theUnity?binary?produced?by?the?generated?XCode?project?and?there?-?just?likeplugin?rendering?devices?-?the?plugin?registration?has?to?be?added?explicitlyto?the?startup?code?of?the?app.
對于IOS這類的平臺，插件代碼需要靜態(tài)鏈接到Unity二進制文件中（通過生成的XCode工程——就像渲染設(shè)備插件一樣——插件注冊必須以顯式的方式添加到app的啟動代碼中）插掂。
OnOSX?one?bundle?can?contain?both?the?32-?and?64?bit?version?of?the?plugin.?Youcan?also?split?them?to?save?size.
OSX中，一個包可以包含32位和64位版本的插件腥例，你也可以分開以減少體積辅甥。
Plugins?with?custom?GUIs
帶自定義GUI的插件
Nowlet’s?look?at?something?a?little?more?advanced:?Effects?for?equalization?andmultiband?compression.?Such?plugins?have?a?much?higher?number?of?parametersthan?the?simple?plugins?presented?in?the?previous?section?and?also?there?issome?physical?coupling?between?parameters?that?require?a?better?way?tovisualize?the?parameters?than?just?a?bunch?of?simple?sliders.?Consider?anequalizer?for?instance:?Each?band?has?3?different?filters?that?collectivelycontribute?to?the?final?equalization?curve?and?each?of?these?filters?has?the?3parameters?frequency,?Q-factor?and?gain?which?are?physically?linked?and?definethe?shape?of?each?filter.?So?it?helps?the?user?a?lot,?if?an?equalizer?pluginhas?a?nice?big?display?showing?the?resulting?curve,?the?individual?filter?contributionsand?can?be?operated?in?such?a?way?that?multiple?parameters?can?be?setsimultaneously?by?simple?dragging?operations?on?the?control?instead?of?changingsliders?one?at?a?time.
現(xiàn)在學(xué)習(xí)些更高級的內(nèi)容：均衡和多波段壓縮效果。這些插件比前面說的簡單插件需要的參數(shù)多得多燎竖，同時璃弄，參數(shù)間存在的物理相關(guān)性使得僅用一系列滑桿條不足以展示它們。以均衡器為例：每個波段都有三個不同的濾波器共同控制均衡曲線而且這三個濾波器每個都有：頻率构回、Q-因子和增益三個參數(shù)夏块，這三個參數(shù)物理相關(guān)確定了濾波器的形狀。因此纤掸，為均衡器插件提供一個大的顯示結(jié)果曲線的界面脐供，其中每個濾波器的作用都可以在此操作，各個參數(shù)也可以通過簡單的拖拽操作在此設(shè)置而不是每次修改滑桿條進行設(shè)置是很有必要的借跪。
CustomGUI?of?the?Equalizer?plugin.?Drag?the?three?bands?to?change?the?gains?and?frequenciesof?the?filter?curve.?Hold?shift?down?while?dragging?to?change?the?shape?of?eachband.
均衡器插件的自定義的GUI政己，支持以拖放三個波段的方式修改濾波曲線的增益和頻率，在拖放時按下shift鍵來修改每個波段的形狀掏愁。
Soonce?again,?the?definition,?initialization,?deinitialization?and?parameterhandling?follows?the?exact?same?enum-based?method?that?the?simple?plugins?use,and?even?the?ProcessCallback?code?is?rather?short.?Well,?time?to?stop?lookingat?the?native?code?and?open?the?AudioPluginDemoGUI.sln?project?in?MonoDevelop.Here?you?will?find?the?associated?C#?classes?for?the?GUI?code.?The?way?it?worksis?simple:?Once?Unity?has?loaded?the?native?plugin?DLLs?and?registered?thecontained?audio?plugins,?it?will?start?looking?for?corresponding?GUIs?thatmatch?the?names?of?the?registered?plugins.?This?happens?through?the?Nameproperty?of?the?EqualizerCustomGUI?class?which,?like?all?custom?plugin?GUIs,must?inherit?from?IAudioEffectPluginGUI.?There’s?only?one?important?functioninside?this?class?which?is?the?bool?OnGUI(IAudioEffectPlugin?plugin)?function.Via?the?IAudioEffectPlugin?plugin?argument?this?function?gets?a?handle?to?thenative?plugin?that?it?can?use?to?read?and?write?the?parameters?that?the?nativeplugin?has?defined.?So?to?read?a?parameter?it?calls:
plugin.GetFloatParameter("MasterGain",?outmasterGain);
whichreturns?true?if?the?parameter?was?found,?and?to?set?it,?it?calls:
plugin.SetFloatParameter("MasterGain",masterGain);
whichalso?returns?true?if?the?parameter?exists.?And?that’s?basically?the?mostimportant?binding?between?GUI?and?native?code.?You?can?also?use?the?function
plugin.GetFloatParameterInfo("NAME",?outminVal,?out?maxVal,?out?defVal);
帶有自定義的GUI的音頻插件的定義歇由、初始化、反初始化和參數(shù)控制與簡單插件中使用基于枚舉值的方法是完全一致的果港，ProcessCallback的代碼甚至可以更短沦泌，現(xiàn)在可以關(guān)閉原生代碼學(xué)習(xí)GUI開發(fā)了，在MonoDevelop中打開AudioPluginDemoGUI.sln工程辛掠，在此你可以看到與GUI代碼相關(guān)的C#類谢谦，其工作方式簡單：一旦Unity完成原生插件的DLL的加載并注冊其中的音頻插件，它就會通過EqualizerCustomGUI類的名稱屬性來尋找和注冊的插件名稱對應(yīng)的GUI，和所有的帶自定義GUI音頻插件一樣他宛，必須繼承自IAudioEffectPluginGUI接口船侧。在此類中只有一個重要的函數(shù)：bool?OnGUI(IAudioEffectPlugin?plugin)，通過函數(shù)參數(shù)IAudioEffectPluginplugin為函數(shù)提供了一個原生插件的句柄厅各，可以用來讀寫原生插件定義中的相關(guān)參數(shù)镜撩，其中，方法plugin.GetFloatParameter("MasterGain",out?masterGain);用于讀取一個參數(shù)設(shè)置队塘，若該函數(shù)返回真袁梗，則參數(shù)被發(fā)現(xiàn)；為了設(shè)置參數(shù)需要調(diào)用函數(shù)：plugin.SetFloatParameter("MasterGain",?masterGain);
和讀取函數(shù)一樣憔古，該函數(shù)在參數(shù)存在的情況下返回真值遮怜。基本上鸿市，這些是GUI和原生代碼之間最重要的關(guān)聯(lián)函數(shù)锯梁。也可以用函數(shù)plugin.GetFloatParameterInfo("NAME",?out?minVal,?outmaxVal,?out?defVal);
來查詢名稱為“NAME”的參數(shù)的最小值、最大值和缺省值以避免在原生庫和UI代碼中出現(xiàn)重復(fù)定義焰情。注意陌凳，如果OnGUI函數(shù)返回真，inspector會在自定義GUI下顯示缺省UI滑桿條控制内舟。這對開始進行GUI開發(fā)是大有裨益的合敦，因為這樣你可以在開發(fā)自定義GUI時訪問到所有參數(shù)，進而檢查在參數(shù)修改時插件的行為是否正確验游。
toquery?parameter“NAME”for?it’s?minimum,?maximum?and?default?values?to?avoidduplicate?definitions?of?these?in?the?native?and?UI?code.?Note?that?if?yourOnGUI?function?return?true,?the?Inspector?will?show?the?default?UI?slidersbelow?the?custom?GUI.?This?is?again?useful?to?bootstrap?your?GUI?development?asyou?have?all?the?parameters?available?while?developing?your?custom?GUI?and?havean?easy?way?to?check?that?the?right?actions?performed?on?it?result?in?theexpected?parameter?changes.
在此不詳談均衡器和多波段插件的DSP庫處理過程充岛，感興趣的話，這些過濾器資料來自于Robert?Bristow?Johnson超牛的音頻EQ?Cookbook耕蝉，其中曲線繪制使用了Unity提供的一些內(nèi)部API函數(shù)來繪制頻率反饋的抗鋸齒曲線崔梗。
Wewon’t?discuss?the?details?about?the?DSP?processing?that?is?going?on?in?theEqualizer?and?Multiband?plugins?here,?for?those?interested,?the?filters?aretaken?from?Robert?Bristow?Johnson’s?excellent?Audio?EQ?Cookbook?and?to?plot?thecurves?Unity?provides?some?internal?API?functions?to?draw?antialiased?curvesfor?the?frequency?response.
還有一點要提一下：均衡器和多波段插件都包含將輸入和輸出譜疊加顯示的代碼實現(xiàn)插件的可視化，這提供了有趣的一點赔硫，GUI更新頻率（幀率）相比音頻處理代碼更新率要低炒俱，此外，GUI代碼不能訪問音頻流爪膊，那么如何讀取數(shù)據(jù)权悟？為此，原生代碼中提供了一個特定的函數(shù)實現(xiàn)該功能：
Onemore?thing?to?mention?though?is?that?both?the?Equalizer?and?Multiband?pluginsdo?also?provide?code?to?overlay?the?input?and?output?spectra?to?visualize?theeffect?of?the?plugins,?which?brings?up?an?interesting?point:?The?GUI?code?runsat?much?lower?update?rate?(the?frame?rate)?than?the?audio?processing?anddoesn’t?have?access?to?the?audio?streams,?so?how?do?we?read?this?data??Forthis,?there?is?a?special?function?for?this?in?the?native?code:
UNITY_AUDIODSP_RESULT?UNITY_AUDIODSP_CALLBACKGetFloatParameterCallback(
UnityAudioEffectState*?state,
int?index,
float*value,
char*valuestr)
{
EffectData::Data*?data?=&state->GetEffectData()->data;
if(index>=?P_NUM)
returnUNITY_AUDIODSP_ERR_UNSUPPORTED;
if(value?!=NULL)
*value?=data->p[index];
if(valuestr!=?NULL)
valuestr[0]?=?0;
returnUNITY_AUDIODSP_OK;
}
Itsimply?enables?reading?an?array?of?floating-point?data?from?the?native?plugin.Whatever?that?data?is,?the?plugin?system?doesn’t?care?about,?as?long?as?therequest?doesn’t?massively?slow?down?the?UI?or?the?native?code.?For?theEqualizer?and?Multiband?code?there?is?a?utility?class?called?FFTAnalyzer?whichmakes?it?easy?to?feed?in?input?and?output?data?from?the?plugin?and?get?aspectrum?back.?This?spectrum?data?is?then?resampled?by?GetFloatBufferCallbackand?handed?to?the?C#?UI?code.?The?reason?that?the?data?needs?to?be?resampled?isthat?the?FFTAnalyzer?runs?the?analysis?at?a?fixed?frequency?resolution?whileGetFloatBufferCallback?just?returns?the?number?of?samples?requested,?which?isdetermined?by?the?width?of?the?view?that?is?displaying?the?data.?For?a?verysimple?plugin?that?has?a?minimal?amount?of?DSP?code?you?might?also?take?a?lookat?the?CorrelationMeter?plugin,?which?simply?plots?the?amplitude?of?the?leftchannel?against?the?amplitude?of?the?right?channel?in?order?to?show“howstereo”the?signal?is.
該函數(shù)允許從原生插件中讀取浮點數(shù)組推盛，而不管該數(shù)據(jù)的意義峦阁。另一方面，只要該處理不顯著降低UI和原生代碼的效率插件系統(tǒng)也不會在意耘成。在均衡器和多波段代碼中有一個FFTAnalyzer功能類榔昔，實現(xiàn)了從插件獲取輸入并計算頻譜并輸出數(shù)據(jù)的功能驹闰。頻譜數(shù)據(jù)通過GetFloatBufferCallback函數(shù)重采樣再交由C#的UI代碼處理。重采樣的原因在于FFTAnalyzer按照固定的頻率分辨率執(zhí)行而GetFloatBufferCallback僅需采樣返回顯示視圖大小需要的數(shù)據(jù)撒会。一個非常簡單且包含最小的DSP代碼的插件是CorrelationMeter嘹朗，用于繪制左通道幅值相對于右通道幅值以展示立體聲的形態(tài)。
Left:Custom?GUI?of?the?CorrelationMeter?plugin.
左：CorrelationMeter插件的自定義GUI
Right:Equalizer?GUI?with?overlaid?spectrum?analysis?(green?curve?is?source,?red?isprocessed).
右：帶頻譜分析的均衡器GUI（綠色曲線是源音頻诵肛，紅色為處理后）
Atthis?point?we?would?also?like?to?point?out?that?both?the?Equalizer?and?Multibandeffects?are?kept?intentionally?simple?and?unoptimized,?but?we?think?they?serveas?good?examples?of?more?complex?UIs?that?are?supported?by?the?plugin?system.There’s?obviously?a?lot?of?work?still?in?doing?the?relevant?platform-specificoptimizations,?tons?of?parameter?tweaking?to?make?it?fell?really?right?andrespond?in?the?most?musical?way?etc.?etc…We?might?also?implement?some?of?theseeffects?as?built-in?plugins?in?Unity?at?some?point?simply?for?the?convenienceof?increasing?Unity’s?standard?repertoire?of?plugins,?but?we?sincerely?hopethat?the?reader?will?also?take?up?the?challenge?to?make?some?really?awesomeplugins–and?who?knows,?they?might?at?some?point?end?up?as?built-in?plugins.;-)
在此需要指出屹培，我們故意讓均衡器和多波段效果是保持簡單，且未進行優(yōu)化怔檩。但它們都是具有更復(fù)雜UI界面的音頻插件的好例子褪秀，針對專門平臺的優(yōu)化還有還有大量的工作，需要成噸的參數(shù)調(diào)節(jié)才能獲得正確的感受和最音樂化效果的反饋薛训。我們也可以實現(xiàn)一些這類效果的插件作為Unity的內(nèi)建插件以增加Unity標(biāo)準(zhǔn)插件庫媒吗，但是我們真心希望讀者們接受挑戰(zhàn)來制作一些真正奧薩母的插件——說不好它們會成為內(nèi)置插件呢？;-)
Convolutionreverb?example?plugin.?The?impulse?response?is?decaying?random?noise,?definedby?the?parameters.?This?is?only?for?demonstration?purposes,?as?a?productionplugin?should?allow?the?user?to?load?arbitrary?recorded?impulses,?theunderlying?convolution?algorithm?remains?the?same?nevertheless.
Convolutionreverb示例插件乙埃。沖擊響應(yīng)是由參數(shù)定義的不斷衰減的隨機噪聲闸英。在此只是為了演示，如果是產(chǎn)品介袜，應(yīng)當(dāng)為用戶提供加載任意的沖擊記錄支持自阱，然而底層的環(huán)繞算法仍然是一樣的。
Exampleof?a?loudness?monitoring?tool?measuring?levels?at?3?different?time?scales.?Alsojust?for?demonstration?purposes,?but?a?good?place?to?start?building?a?monitortool?that?conforms?to?modern?loudness?standardizations.?The?curve?renderingcode?is?built?into?Unity.
音量監(jiān)控工具在三個不同的時間尺度測量音量水平米酬。一樣只是為了演示目的，但這是個建立符合現(xiàn)代音量標(biāo)準(zhǔn)的監(jiān)控工具好開始趋箩，其中赃额，曲線渲染使用了Unity的內(nèi)置代碼。
Synchronizing?to?the?DSP?clock
與DSP時鐘同步
Timefor?some?fun?exercises.?Why?not?use?the?plugin?system?to?generate?sound?insteadof?just?processing?it??Let’s?try?to?do?some?simple?bassline?and?drumsynthesizers?that?should?be?familiar?to?people?who?listen?to?acid?trance–somesimple?clones?of?some?of?the?main?synths?that?defined?this?genre.?Take?a?lookat?Plugin_TeeBee.cpp?and?Plugin_TeeDee.cpp.?These?simple?synths?just?generatepatterns?with?random?notes?and?have?some?parameters?for?tweaking?the?filters,envelopes?and?so?forth?in?the?synthesis?engine.?Again,?we?won’t?discuss?thosedetails?here,?but?just?point?out?that?the?state->dsptick?parameter?is?readin?the?ProcessCallback?in?order?to?determine?the?position?in?the“song”.?Thiscounter?is?a?global?sample?position,?so?we?just?divide?it?by?the?length?of?eachnote?specified?in?samples?and?fire?a?note?event?to?the?synthesis?enginewhenever?this?division?has?a?zero?remainder.?This?way,?all?plugin?effects?stayin?sync?to?the?same?sample-based?clock,?and?ifyou?would?for?instance?play?a?prerecorded?piece?of?music?with?a?known?tempothrough?such?an?effect,?you?could?use?the?timing?info?to?applytempo-synchronized?filter?effects?or?delays?on?the?music.
是時候做些有趣的工作了叫确，與其只是處理音頻跳芳，也可以使用插件系統(tǒng)產(chǎn)生聲音哦。咱們從一個簡單的單音節(jié)鼓點合成器開始竹勉，聽acid?trance的人應(yīng)該很熟悉這種風(fēng)格飞盆，通過簡單的復(fù)制定義了這種風(fēng)格的基本合成器來實現(xiàn)這種效果。代碼在Plugin_TeeBee.cpp和Plugin_TeeDee.cpp中次乓。該合成器使用隨機音符生成鼓點并具有一些用于調(diào)整濾波器吓歇、封包等的參數(shù)。在此我們還是不涉及具體實現(xiàn)票腰，但是要指出的是state->dsptick參數(shù)會被傳入ProcessCallback函數(shù)中以確定“歌曲”的位置城看，這個計數(shù)器代表了一個全局的采樣位置，我們用計數(shù)器除以采樣中指定的每個音符的長度杏慰，每當(dāng)整除時测柠，就發(fā)出一個音節(jié)炼鞠，以此實現(xiàn)插件效果與采樣時鐘的同步。如果你打算用此插件播放一個節(jié)拍已知的預(yù)錄制的音樂轰胁，你可以用計時信息設(shè)置節(jié)拍-同步濾波器效果或者音樂播放的延遲谒主。
Simple?bassline?and?drum?synthesizers?to?demonstratetempo-synchronized?effects.
簡單單音節(jié)鼓點合成器演示節(jié)奏同步效果
Spatialization
空間化
Thenative?audio?plugin?SDK?is?the?foundation?of?the?Spatialization?SDK?whichallows?developing?custom?spatialization?effects?that?are?instantiazed?per?audiosource.?More?information?about?this?can?be?found?here.
原生音頻插件SDK是空間化SDK的基礎(chǔ)，空間化SDK通過實例化每個聲音源組件以實現(xiàn)自定義空間效果赃阀。更多內(nèi)容可以在此查看霎肯。
Outlook
前瞻
Thisis?just?the?start?of?an?effort?to?open?up?parts?of?the?sound?system?to?highperformance?native?code.?We?have?plans?to?integrate?this?in?other?parts?ofUnity?as?well?to?make?the?effects?usable?outside?of?the?mixer?as?well?asextending?the?SDK?to?support?other?parameter?types?than?floats?with?support?forbetter?default?GUIs?as?well?as?storage?of?binary?data.
這些僅是將部分聲音系統(tǒng)放到高性能原生代碼實現(xiàn)的開始，我們已有計劃將這個部分與Unity的其他部分進行集成凹耙，以支持在混音器外使用插件效果姿现，同時擴展SDK以支持float外的其他參數(shù)類型，提供更好的缺省GUI界面和二進制數(shù)據(jù)存儲支持肖抱。
Havea?lot?of?fun?creating?your?own?plugins.?Hope?to?see?them?on?the?asset?store.;-)
創(chuàng)建你自己的插件很有趣备典，希望能在Unity的Asset商店見到它們。;-)
“Disclaimer”
為什么不提供更多插件
Whilethere?are?many?similarities?in?the?design,?Unity’s?native?audio?SDK?is?notbuilt?on?top?of?other?plugin?SDKs?like?Steinberg?VST?or?Apple?AudioUnits.?Itshould?be?mentioned?that?it?would?be?easy?for?the?interested?reader?toimplement?basic?wrappers?for?these?using?this?SDK?that?allow?using?such?pluginsto?be?used?in?Unity.?It?is?not?something?the?dev-team?of?Unity?is?planning?todo.?Proper?hosting?of?any?plugin?quickly?gets?very?complex,?and?dealing?withall?the?intricacies?of?expected?invocation?orders?and?handling?custom?GUIwindows?that?are?based?on?native?code?quickly?grows?by?leaps?and?bounds?whichmakes?it?less?useful?as?example?code.
雖然設(shè)計上存在相似性意述，但是Unity的原生音頻SDK不像Steinberg?VST或AppleAudioUnits一樣在其他插件SDK的基礎(chǔ)上實現(xiàn)提佣。順便一提，感興趣的讀者使用此SDK輕松實現(xiàn)對它們進行基本封裝并在Unity中使用荤崇。這不是開發(fā)團隊計劃要做的事拌屏，合理容納任意插件將使得Unity變得十分復(fù)雜，而管理錯綜復(fù)雜的期望調(diào)用順序和基于原生代碼的自定義GUI窗口的處理需求會急速增長术荤，而這些都使得示例代碼的更有意義倚喂。
Whilewe?do?understand?that?it?could?potentially?be?quite?useful?to?load?your?VST?orAU?plugin?or?even?effects?for?just?mocking?up?/?testing?sound?design,?bear?inmind?that?using?VST/AU?also?limits?you?to?a?few?specific?platforms.?Thepotential?of?writing?audio?plugins?based?on?the?Unity?SDK?is?that?it?extends?toall?platforms?that?support?software-mixing?and?dynamically?loaded?native?code.That?said,?there?are?valid?use?cases?for?mocking?up?early?sound?design?withyour?favourite?tools?before?deciding?to?devote?time?to?develop?custom?plugins(or?simply?to?be?able?to?use?metering?plugins?in?the?editor?that?don’t?alterthe?sound?in?any?way),?so?if?anyone?wants?to?make?a?nice?solution?for?that,please?do.
我們當(dāng)然知道提供VST或AU插件，甚至只是用于模仿和測試聲音設(shè)計的效果會為用戶帶來益處瓣戚，但請記住端圈，使用VST/AU也會將你限制在特定的平臺上，編寫開發(fā)基于Unity?SDK的音頻插件的優(yōu)勢就在于它可以在所有平臺實現(xiàn)軟件混音和原生代碼的動態(tài)加載支持子库。也就是說它可以支持以下的用例：用自己熟悉的工具進行早期音效設(shè)計舱权，進而決定是否花時間進行自定義插件開發(fā)（或者簡單使用編輯器中的metering插件，而無需對聲音進行任何修改）仑嗅。因此宴倍，如果你想得到優(yōu)秀的解決方案，就請使用它吧
a
