"Translator" Kinect v2 program Design (c++-) Audiobeam

Kinect v2,MicrophoneArray can be used forthe direction of the horizontal sound sourceSpeculateAudiobeam）and theSpeech Recognition（Speech Recognition）. This section is an introduction to how to obtainAudiobeam. The previous section describes how to use theKinect SDKV2preview version, fromKinectV2The preview version of the color camera and depth sensor to get the data in the method. this sectionWilldescribes getting Audiobeam from the Kinect microphone array（the direction of the horizontal sound sourceSpeculatethe method. Microphone Array
described in the first section，In addition to the Kinect is equipped withColor Camera，outside the depth sensor，also havemicrophone Array.   microphone array consisting of 4 microphone，can be carried out in the direction of the horizontal sound sourceSpeculateAudiobeam）and speech recognition（Speech Recognition）and so on. In this section, we will describe how to get audiobeam.
Figure 1 Kinect v2 preview version of microphone Array Figure 2 Sample program for Kinect SDK v2 Preview (audiobasics) Sample ProgramUsing the Kinect SDK v2get Audiobeam and display the results in the example program on the console. Features of audio（Audiobeam，Speech Recognition）The data acquisition processIn"sensor"~"source" This piece with the previousImage（Color，Depth，Bodyindex，Body) of the acquisition processThe same. But after that, please note thataudio feature-specific section, here is the main introduction andthe same data acquisition process as Image. The full contents of this sample program，It's all public on the following GitHub. Https://github.com/UnaNancyOwen/Kinect2Sample Figure 3 The process of data acquisition in the Kinect SDK V2 Preview (re-sent) "sensor"Get "sensor"

 
// Sensor IKinectSensor* pSensor; // ……1 HRESULT hResult = S_OK;
hResult = GetDefaultKinectSensor（ &pSensor ）; //……2 if（ FAILED（ hResult ） ）{
  std::cerr << "Error : GetDefaultKinectSensor" << std::endl; return -1;
}
hResult = pSensor->Open（）; //……3 if（ FAILED（ hResult ） ）{
  std::cerr << "Error : IKinectSensor::Open（）" << std::endl; return -1;
}

Listing 1.1 corresponds to part of Figure 1"sensor" (re-sent)

1 Handle the sensor interface of the Kinect v2 preview. 2 Get the defaultSensor. 3 OpenSensor. "source"Obtained "source" from "sensor".

 
 
// Source
IAudioSource* pAudioSource;  //……1
hResult = pSensor->get_AudioSource（ &pAudioSource ）;  //……2
if（ FAILED（ hResult ） ）{
  std::cerr << "Error : IKinectSensor::get_AudioSource（）" << std::endl;
  return -1;
}

List 1.2 equivalent to part of Figure 1"source"

1 The source interface for the audio feature. 2 obtained from sensorSource. "audiobeamlist"~"openaudiobeam"Get "audiobeamlist" from "source" and open the specified "audiobeam" from the list.

 
// Get Audio Beam List
IAudioBeamList* pAudioBeamList;   //……1
hResult = pAudioSource->get_AudioBeams（ &pAudioBeamList ）;  //……2
if（ FAILED（ hResult ） ）{
  std::cerr << "Error : IAudioSource::get_AudioBeams（）" << std::endl;
  return -1;
}
// Open Audio Beam
IAudioBeam* pAudioBeam;  //……3
hResult = pAudioBeamList->OpenAudioBeam（ 0， &pAudioBeam ）;  //……4
if（ FAILED（ hResult ） ）{
  std::cerr << "Error : IAudioBeamList::OpenAudioBeam（）" << std::endl;
  return -1;
}

List 1.3 get microphone Array and audiobeam

1 Audiobeamlist interface. 2 get Audiobeam from source and get the list of microphone array. 3 Audiobeam interface.   4 Get Audiobeam from list, open microphone Array. 0 represents the first found default microphone Array. "get Angle and confidence"from"stream" reading audio data，get the direction of the sound source andSpeculatethe Trust value.

 
while（ 1 ）{
  // Get Angle and Confidence
  FLOAT angle = 0.0f;
  FLOAT confidence = 0.0f;
  pAudioBeam->get_BeamAngle（ &angle ）; // radian [-0.872665f， 0.872665f]  ……1
  pAudioBeam->get_BeamAngleConfidence（ &confidence ）; // confidence [0.0f， 1.0f]  ……2
  // Show Result
  // Convert from radian to degree : degree = radian * 180 / Pi
  if（ confidence > 0.5f ）{
    std::system（ "cls" ）;
    std::cout << "Angle : " << angle * 180.0f / M_PI << "， Confidence : " << confidence << std::endl;  //……3
  }
  // Input Key （ Exit ESC key ）
  if（ GetKeyState（ VK_ESCAPE ） < 0 ）{
    break;
  }
}

Listing 1.5 Obtaining the source direction and the inferred trust value

1 Get the direction of the audio source. The angular unit is radian (radians).   2 Gain a speculative trust value in the direction of the sound source. Take valueRangeIt's 0.0f~1.0f, the bigger the number.indicates a higher level of confidence. 3 PutRadian（radians）Switch to DgreeDegreesoutput on the console. Displays only the current value if the comment is "std::system (" CLS ");   , then the previous value will remain on the cconsole. The direction of the audio source that can be obtained isKinectV2the center is heading horizontally toward the left and rightRange of +/-50°.  
FIG. 4 Detection range of audio source direction（ +/-50°)
Because the angular unit of the obtained source direction is "radian (radians)" To use equation 1 to convert to "degree (degrees)". "Equation 1 Adian (radians) converted to (degrees)"

• Radian (radian) → angle (degree): degree=radianx180÷π
• Angle (degree) → radians (radian): radian=degreexπ÷180

Run ResultsRunning the sample program, like Figure 5, will show the sound source angle (Angle) and the inferred Trust value (Confidence）. Ifmicrophone The response of an array is overly sensitive or insensitive, you can passadjust the operating system's recording device properties to improve(in the Recording device [Control Panel]-[Sound]-[Recording], the mic arranges the properties of the Xbox NUI sensor"to adjust the rating).
Figure 5 Running resultsThe angle of the source direction and the inferred trust value are displayed on the console. SummaryThis section describes how to use theKinect SDKV2Preview version madeAudiobeam. AndKinect SDKV1 phasethan，frommicrophone array to get audio data easier to handle.

"Translator" Kinect v2 program Design (c++-) Audiobeam