3D image acquisition and display (SurfaceView adaptive Camera, recording video, and extracting frames ),

3D image acquisition and display (SurfaceView adaptive Camera, recording video, and extracting frames ),

Recently, we have made a 3D image collection and presentation.

Main features: Custom Camera (google has abandoned Camera and we recommend using Camera2. In the future, I will use Camera2 to replace Camera), and record a video around an object at 360 degrees, then a certain number of frames are extracted from the video and saved as images. Finally, the first image is displayed on an Activity page. You can slide or click to switch to the next image to create a 3D image. The main purpose of this project is to collect 3D image materials, then upload them to the server for processing, and finally display them on the user client or webpage side through OpenGL ES.

Technical points:

1. When SurfaceView displays Camera, if it is not in accordance with the size ratio supported by camera, the preview will show stretching.

MCamera. getParameters (). getSupportedPreviewSizes ();
MCamera. getParameters (). getSupportedPictureSizes ();

You can view the preview box size and image size supported by the camera through debug. To adapt, you need to obtain the desired range size through round robin. Generally, the full screen size is supported, that is, the pixel size of the mobile phone.
If you want to set the preview box to a square, in principle, it will not work (I have not found the corresponding method, please kindly advise), then we can first use the full screen photo box size, then, a square is formed in the hidden area of the stack,
When you get an image, you can do it based on the hidden area to get any effect you want. (For example)

There are also some minor issues in preview. For example, Camera uses landscape capture by default. You need mCamera. setDisplayOrientation (90). At the same time, you need to rotate the image 90 degrees before you can preview the image.

2. Some problems may occur during video recording. The image will be stretched, which is inconsistent with the preview. In this case, obtain the parameters supported by the device.

CamcorderProfile mProfile = null;
If (CamcorderProfile. hasProfile (CamcorderProfile. QUALITY_1080P )){
MProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_1080P );

} Else if (CamcorderProfile. hasProfile (CamcorderProfile. QUALITY_720P )){
MProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_720P );
} Else {
MProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_480P );
}
You need to obtain the information based on your actual situation and then set the parameters:

MMediaRecorder. setProfile (mProfile); // if you do not know why, setting this directly does not work, set the following:

// MMediaRecorder. setProfile (mProfile); // if you do not know why, setting this directly does not work, set the following:
MMediaRecorder. setOutputFormat (mProfile. fileFormat); // video output format
MMediaRecorder. setVideoFrameRate (mProfile. fileFormat); // sets the recorded video frame rate.
MMediaRecorder. setVideoSize (mHeight, mWidth); // sets the Resolution:
MMediaRecorder. setVideoEncodingBitRate (mProfile. videoBitRate); // set the frame frequency, and then it is clear
MMediaRecorder. setVideoEncoder (mProfile. videoCodec); // video recording format

3. extract frames from a video. Each frame is returned in the form of bitmap. The frame extraction method provided by the Java interface,

MediaMetadataRetriever class
GetFrameAtTime (long timeUs)
Frame extraction through time, and the time of each frame may not be even, which leads to repeated frames, which requires further research (further research ).
The specific method is:

/*** Get the video key frame */public void getFrameFromVideo (String filePath, String dirName) {mFile = new File (filePath); retriever = new MediaMetadataRetriever (); retriever. setDataSource (filePath); fileLength = retriever. extractMetadata (MediaMetadataRetriever. METADATA_KEY_DURATION); long lengthLong = Long. parseLong (fileLength) * 1000/(Constants. num_Frame-1); String prefixName = Utils. randomcaploud (5); // image name prefix Int k = 0; for (long I = 0; I <Long. parseLong (fileLength) * 1000 + lengthLong; I = I + lengthLong) {k ++; Bitmap bitmap = retriever. getFrameAtTime (I); if (bitmap! = Null) {Matrix matrix = new Matrix (); matrix. reset (); matrix. setRotate (90); // The image is a horizontal disk by default. if (bitmap. getWidth ()> bitmap. getHeight () {bitmap = Bitmap. createBitmap (bitmap, (bitmap. getWidth ()-bitmap. getHeight ()/2, 0, bitmap. getHeight (), bitmap. getHeight (), matrix, true);} else {bitmap = Bitmap. createBitmap (bitmap, 0, (bitmap. getHeight ()-bitmap. getWidth ()/2, bitmap. getWidth (), bitmap. getWidth (), matrix, true);} BitmapUtils. saveBitmap (bitmap, dirName, prefixName + k + ". jpg ") ;}if (k <50) {sendMessageForProgress (dirName, k, true) ;}else {sendMessageForProgress (dirName, k, false );}} // Delete the video file mFile. delete ();}

　　

Finally, let's preview and record the video code. RecordVideoActivity

Public class RecordVideoActivity extends AppCompatActivity implements View. onClickListener, SurfaceHolder. callback, MediaRecorder. onErrorListener {private ProgressBar mProgressbar; private SurfaceView mSurfaceView; private MediaRecorder mMediaRecorder; // class of video recording private SurfaceHolder mSurfaceHolder; private Camera mCamera; private Timer mTimer; // timer private boolean isOpenCamera = true; // whether to enable the camera at the beginning Header private final static int mRecordMaxTime = 20; // maximum time of one shot private OnRecordFinishListener mOnRecordFinishListener; // callback interface for recording completion private int mTimeCount; // time count private File mVecordFile = null; // file private int mWidth = 0; // video resolution width private int mHeight = 0; // video resolution height private boolean isStarting = false; @ Override protected void onCreate (Bundle savedInstanceState) {requestWindowFeature (Window. FEATURE _ NO_TITLE); // remove the title bar getWindow (). setFlags (WindowManager. layoutParams. FLAG_FULLSCREEN, WindowManager. layoutParams. FLAG_FULLSCREEN); // sets full screen super. onCreate (savedInstanceState); setContentView (R. layout. activity_record_video); initView ();} private void initView () {mProgressbar = (ProgressBar) findViewById (R. id. progressBar); mProgressbar. setMax (mRecordMaxTime); findViewById (R. id. btn_start ). setOnClickLis Tener (this); mSurfaceView = (SurfaceView) findViewById (R. id. surfaceView); mSurfaceHolder = mSurfaceView. getHolder (); // Get holder mSurfaceHolder. addCallback (this); // Add the holder to the callback interface mSurfaceHolder. setKeepScreenOn (true);}/*** initialize Camera */private void initCamera (int width, int height) {if (mCamera! = Null) {freeCameraResource ();} try {mCamera = Camera. open (); if (mCamera = null) return; mCamera. setDisplayOrientation (90); // The default camera is landscape. You need to adjust the angle to vertical mCamera. setPreviewDisplay (mSurfaceHolder); Camera. parameters parameters = mCamera. getParameters (); // obtain the Camera Parameter mWidth = width; mHeight = height; parameters. setPreviewSize (height, width); // sets the preview image size parameters. set ("orientation", "portrait"); List <String> FocusModes = parameters. getSupportedFocusModes (); if (focusModes. contains ("continuous-video") {parameters. setFocusMode (Camera. parameters. FOCUS_MODE_CONTINUOUS_VIDEO);} mCamera. getParameters (). getSupportedPreviewSizes (); mCamera. getParameters (). getSupportedPictureSizes (); mCamera. setParameters (parameters); // sets the Camera Parameter mCamera. startPreview (); // start to preview mCamera. unlock (); // unlock and grant video recording permission} catch (Exception e) {E. printStackTrace (); freeCameraResource () ;}@ Override protected void onDestroy () {super. onDestroy (); stop () ;}/ *** release camera Resources */private void freeCameraResource () {if (mCamera! = Null) {mCamera. setPreviewCallback (null); mCamera. stopPreview (); mCamera. lock (); mCamera. release (); mCamera = null ;}/ *** before recording, initialize */private void initRecord () {try {mMediaRecorder = new MediaRecorder (); mMediaRecorder. reset (); if (mCamera! = Null) mMediaRecorder. setCamera (mCamera); mMediaRecorder. setOnErrorListener (this); mMediaRecorder. setVideoSource (MediaRecorder. videoSource. CAMERA); // video source CamcorderProfile mProfile = null; if (CamcorderProfile. hasProfile (CamcorderProfile. QUALITY_1080P) {mProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_1080P);} else if (CamcorderProfile. hasProfile (CamcorderProfile. QUALITY_720P) {mProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_720P);} else {mProfile = CamcorderProfile. get (CamcorderProfile. QUALITY_480P);} // mMediaRecorder. setProfile (mProfile); // if you do not know why, this setting does not take effect. set the following message to mMediaRecorder. setOutputFormat (mProfile. fileFormat); // The video output format is mMediaRecorder. setVideoFrameRate (mProfile. fileFormat); // sets the recorded video frame rate (mMediaRecorder. setVideoSize (mHeight, mWidth); // sets the resolution: mMediaRecorder. setV IdeoEncodingBitRate (mProfile. videoBitRate); // set the frame frequency, and then the mMediaRecorder is clear. setVideoEncoder (mProfile. videoCodec); // video recording format //} mMediaRecorder. setOutputFile (mVecordFile. getAbsolutePath (); mMediaRecorder. prepare (); mMediaRecorder. start ();} catch (Exception e) {e. printStackTrace () ;}/ *** start recording video */public void startRecord (final OnRecordFinishListener onRecordFinishListener) {isStarting = true; t His. mOnRecordFinishListener = onRecordFinishListener; createRecordDir (); try {initRecord (); mTimeCount = 0; // The time counter value is mTimer = new Timer (); mTimer. schedule (new TimerTask () {@ Override public void run () {// TODO Auto-generated method stub mTimeCount ++; mProgressbar. setProgress (mTimeCount); // set the progress bar if (mTimeCount = mRecordMaxTime) {// when the specified time is reached, stop shooting stop (); if (mOnRecordFinishListener! = Null) mOnRecordFinishListener. onRecordFinish (); }}, 0, 1000);} catch (Exception e) {e. printStackTrace () ;}/ *** stop shooting */public void stop () {stopRecord (); releaseRecord (); freeCameraResource ();} /*** stop recording */public void stopRecord () {mProgressbar. setProgress (0); if (mTimer! = Null) mTimer. cancel (); if (mMediaRecorder! = Null) {// mMediaRecorder is not collapsed after the setting. setOnErrorListener (null); mMediaRecorder. setPreviewDisplay (null); try {mMediaRecorder. stop ();} catch (IllegalStateException e) {e. printStackTrace ();} catch (RuntimeException e) {e. printStackTrace ();} catch (Exception e) {e. printStackTrace () ;}}/ *** release resource */private void releaseRecord () {if (mMediaRecorder! = Null) {mMediaRecorder. setOnErrorListener (null); try {mMediaRecorder. release ();} catch (IllegalStateException e) {e. printStackTrace ();} catch (Exception e) {e. printStackTrace () ;}} mMediaRecorder = null;}/*** create a directory and File */private void createRecordDir () {File FileDir = new File (Environment. getExternalStorageDirectory () + File. separator + "RecordVideo/"); if (! FileDir. exists () {FileDir. mkdirs ();} // create a File try {mVecordFile = new File (FileDir. getAbsolutePath () + "/test.mp4"); Log. d ("Path:", mVecordFile. getAbsolutePath ();} catch (Exception e) {e. printStackTrace () ;}} OnRecordFinishListener recordFinishListener = new OnRecordFinishListener () {@ Override public void onRecordFinish () {Intent intent = new Intent (RecordVideoActivity. this, MainActivity. clas S); intent. putExtra ("filePath", mVecordFile. getAbsolutePath (); setResult (Activity. RESULT_ OK, intent); finish () ;};@ Override public void onBackPressed () {if (! IsStarting) {finish () ;}@ Override public void onClick (View v) {switch (v. getId () {case R. id. btn_start: if (! IsStarting) startRecord (iterator); break ;}@ Override public void surfaceCreated (SurfaceHolder holder) {}@ Override public void surfaceChanged (SurfaceHolder holder, int format, int width, int height) {initCamera (width, height) ;}@ Override public void surfaceDestroyed (SurfaceHolder holder) {freeCameraResource () ;}@ Override public void onError (MediaRecorder mr, int what, int extra) {Try {if (mr! = Null) mr. reset ();} catch (IllegalStateException e) {e. printStackTrace ();} catch (Exception e) {e. printStackTrace () ;}/ *** callback interface for recording completion */public interface OnRecordFinishListener {void onRecordFinish ();}}

Final source code: https://github.com/xiaoxiaoqingyi/3DShow