HoloLens Development Notes-Unity's locatable camera using cameras

Enabling the capability for Photo Video camera capability

In order to use the camera, we must enable webcam capability.

1. Open the player settings in unity
2. Expand the Windows Store tab page
3. Check webcam capabilities in the "publishing Settings > Capabilities" section

Only one camera operation can be performed at the same time. To identify which mode the current camera is in (photo or video), you can check through the UnityEngine.VR.WSA.WebCam.Mode API.

Photo Capture capturing photos

namespaces : UnityEngine.VR.WSA.WebCam

type : photocapture

The Photocapture class allows us to take still photos using the camera. The normal usage pattern is as follows:

1. Create a Photocapture object
2. Use the settings we want to create a Cameraparameters object
3. Call the Startphotomodeasync () method to start the camera mode
4. Take the photos you want
   • (optional) Further processing of captured images
5. Turn off photo mode and release resources

Common Set up for photocapture common practice for using photocapture

For any photo-taking method, the start step is like this:

Create a Photocapture object

NULL ;    void Start ()   {       Photocapture.createasync (false, onphotocapturecreated);   }

Storage objects, configuring shooting parameters and starting the camera mode

voidonphotocapturecreated (photocapture captureobject) {photocaptureobject=Captureobject; Resolution cameraresolution= PhotoCapture.SupportedResolutions.OrderByDescending (RES) = Res.width *res.height).       First (); Cameraparameters C=Newcameraparameters (); C.hologramopacity=0.0f; C.cameraresolutionwidth=Cameraresolution.width; C.cameraresolutionheight=Cameraresolution.height; C.pixelformat=Capturepixelformat.bgra32; Captureobject.startphotomodeasync (c,false, onphotomodestarted); }

The same cleanup code is required to close the camera at the end

void Onstoppedphotomode (photocapture.photocaptureresult result)   {       photocaptureobject.dispose ();        NULL ;   }

After completing these steps, you can pick which method to use to capture the photo.

Capture a photo to a file capturing photos to files

The simplest way to do this is to capture a photo directly to a file. Photos can be stored as PNG or JPG files.

If we have successfully started the camera mode, we need to take pictures and store the photos on disk, as follows:

Private voidonphotomodestarted (photocapture.photocaptureresult result) {if(result.success) {stringfilename =string. Format (@"capturedimage{0}_n.jpg", Time.time); stringFilePath =System.IO.Path.Combine (application.persistentdatapath, filename);       Photocaptureobject.takephotoasync (FilePath, photocapturefileoutputformat.jpg, Oncapturedphototodisk); }       Else{debug.logerror ("Unable to start photo mode!"); }   }

When you have finished capturing photos to a file, you need to exit Photo mode and clean up resources

void Oncapturedphototodisk (photocapture.photocaptureresult result)   {       if  (result.success)       {           Debug.Log ("Saved Photo to disk! " );           Photocaptureobject.stopphotomodeasync (Onstoppedphotomode);       }        Else        {           Debug.Log ("Failed to save Photo to disk");       }   

Capture a Photo to a texture2d capturing files to texture2d objects

We can save captured photos as Texture2d objects, and save them to a file like. The steps are as follows:

In the Onphotomodestarted () method, captures a frame of image into memory.

Private void onphotomodestarted (photocapture.photocaptureresult result)   {       if  (result.success)       {           Photocaptureobject.takephotoasync (oncapturedphototomemory );       }        Else        {           debug.logerror ("Unable to start photo mode! " );       }   }

We need to assign the resulting results to the Texture2d object and then clean up the camera resources

voidoncapturedphototomemory (photocapture.photocaptureresult result, Photocaptureframe photocaptureframe) {if(result.success) {//Create a Texture2d object with the correct resolutionResolution cameraresolution = PhotoCapture.SupportedResolutions.OrderByDescending (res) = Res.width *res.height).           First (); Texture2d targettexture=Newtexture2d (Cameraresolution.width, cameraresolution.height); //copy image data to a Texture2d objectphotocaptureframe.uploadimagedatatotexture (targettexture); //further use of Texture2d objects, such as those assigned to the material god horse       }       //Cleaning the cameraPhotocaptureobject.stopphotomodeasync (Onstoppedphotomode); }

capture a photo and Interact with the raw bytes captures photos and interacts with raw data

In order to manipulate image raw data in memory, the steps required are similar to capturing images to texture2d, except that the Oncapturedphototomemory () method allows you to obtain image raw data and manipulate them.

In this example, we will create a list<color> for further processing or to apply the Texture2d object directly through the Setpixels () method.

voidoncapturedphototomemory (photocapture.photocaptureresult result, Photocaptureframe photocaptureframe) {if(result.success) {List<byte> imagebufferlist =Newlist<byte>(); //Copy the original Imfmediabuffer data to the empty listPhotocaptureframe.copyrawimagedataintobuffer (imagebufferlist); //This example captures a photo using the BGRA32 format.           intStride =4; floatDenominator =1.0f/255.0f; List<Color> Colorarray =NewList<color>();  for(inti = Imagebufferlist.count-1; I >=0; I-=Stride) {               floatA = (int) (Imagebufferlist[i-0]) *denominator; floatR = (int) (Imagebufferlist[i-1]) *denominator; floatg = (int) (Imagebufferlist[i-2]) *denominator; floatB = (int) (Imagebufferlist[i-3]) *denominator; Colorarray.add (NewColor (R, G, B, a)); }           //the list can then be used for further processing.} photocaptureobject.stopphotomodeasync (Onstoppedphotomode); }

Video Capture Capturing videos

namespaces : UnityEngine.VR.WSA.WebCam

type : videocapture

The use of capturing video is similar to capturing a photo, except that you must specify a frame rate (FPS) and you can only store the video directly on disk in MP4 format. The steps are as follows:

1. Create a Videocapture object
2. Use the settings we want to create a Cameraparameters object
3. Call the Startvideomodeasync () method to start video capture mode
4. Start recording video
5. Stop Recording video
6. Stop video capture mode and release camera resources

Creating and configuring Videocapture objects

voidStart () {Videocapture.createasync (false, onvideocapturecreated); }voidonvideocapturecreated (videocapture videocapture) {if(Videocapture! =NULL) {m_videocapture=videocapture; Resolution cameraresolution= VideoCapture.SupportedResolutions.OrderByDescending (RES) = Res.width *res.height).           First (); floatCameraframerate = Videocapture.getsupportedframeratesforresolution (cameraresolution). OrderByDescending (fps) =fps).           First (); Cameraparameters cameraparameters=Newcameraparameters (); Cameraparameters.hologramopacity=0.0f; Cameraparameters.framerate=cameraframerate; Cameraparameters.cameraresolutionwidth=Cameraresolution.width; Cameraparameters.cameraresolutionheight=Cameraresolution.height; Cameraparameters.pixelformat=Capturepixelformat.bgra32; M_videocapture.startvideomodeasync (Cameraparameters, Videocapture.audiostate .       None, Onstartedvideocapturemode); }       Else{debug.logerror ("Failed to create Videocapture instance!"); }   }

Once the Videocapture object is configured, we start recording the video

void Onstartedvideocapturemode (videocapture.videocaptureresult result)   {       if  (result.success)       {           stringstring. Format ("myvideo_{0}.mp4", Time.time);            string filepath = System.IO.Path.Combine (application.persistentdatapath, filename);           M_videocapture.startrecordingasync (filepath, onstartedrecordingvideo);       }   }

After you start recording, we need to update the UI or behavior to ensure that video capture can be stopped. Here we only output log.

void Onstartedrecordingvideo (videocapture.videocaptureresult result)   {       Debug.Log ("Started recording video! " );        // We will stop the video capture in some way, such as a timer or a tap gesture, etc.   }

Finally we need to stop the video capture, which can be implemented by Chinese timers or other input methods.

   void Stoprecordingvideo ()   {       M_videocapture.stoprecordingasync (onstoppedrecordingvideo);   }

Once the video capture is stopped, you need to exit video capture mode and release the camera resources in a timely manner.

void Onstoppedrecordingvideo (videocapture.videocaptureresult result)   {       Debug.Log ("Stopped recording video! " );       M_videocapture.stopvideomodeasync (Onstoppedvideocapturemode);   }    void Onstoppedvideocapturemode (videocapture.videocaptureresult result)   {       m_videocapture.dispose ();        NULL ;   }

Troubleshooting Problem Diagnosis

• Get no resolution

  • Make sure you have webcam capability enabled in your project
• Holographic images cannot be captured in a picture or video
  • Future updates will support capturing holographic images

HoloLens Development Notes-Unity's locatable camera using cameras