[Turn] Augmented reality colar Mix analysis

The original: An analysis of augmented reality Colar Mix

It was interesting to see how Colar Mix's augmented reality application was implemented on the forum. I will explain to you from the angle I understand. We must be familiar with the realization principle of augmented reality, and use the algorithm to locate environment or material. This is the technical basis for the need to overlay virtual information into real-world environments or objects without this step. Then we try to use many ways to enhance the connection between the material and the superimposed information, which is part of the interaction. For example, the first frame of the video is the same as the material, adding shadows to the object, drilling out of the material, and so on. The other is to get information from the environment and materials to interact with the virtual information, such as identifying gestures. Colar Mix is the extraction of information from the material. Let's take a look at how to achieve this effect.

Basic knowledge texture coordinates and mapping

Steal a picture to explain briefly, 3D model in the production of the map will need to expand the model to the plane, on the plane of the model each vertex corresponding coordinates is the texture coordinates. A texture map is a picture of a painting that finishes with color on the plane. So the question comes, will the model unfold to the plane after the appearance is not bound to be very tangled? The answer is that texture coordinates can be adjusted, and there are many different ways to expand in various 3D modeling software. So the Colar Mix is likely to be a material image.

Camera-target-patch

I don't know if it's called a camera material matte, but the goal is to use the camera capture screen as a texture. This method used in AR is still very common, such as the Taggar in the water ripple effect, "Tokyo News" in the flip effect and so on. In the Vuforia turned over, before there is an article dedicated to the realization of this technology, the article is now broken link can not see, so only through the online bits and pieces of information to get.

Making Models

I do the model here using Blender, this is a free software, foreign use more people.

About the basic operation of Blender you can look at Youtube to see the video. One is the editing interface of objects, and one is the editing interface of texture coordinates. For the sake of simplicity my side only unfolded three faces and changed to the right way. Finally, the model is exported as an FBX file.

In order not to mistake the position of the vertex, I made a map to do the test. There should be a more convenient way, but I did not do 3D design, so I can only use the stupid way.

Material production

Since our cube is simply too simple (the fill image in Colar is complex enough), if it is directly used as a material map, it must not be recognized, so I added a picture for it.

Unity Configuration

To import our FBX model into Unity project, we can change its Material to diffuse, and then change the map to our material map to see if it is correct. In addition to our model we have to add a transparent plane to mark where the material map is, and we set it to the same position size as the material chart. The final configuration is as follows.

Code

First, let's revise the code for Vuforia's online cameraimageaccess.

========

Here's the code, the cat's copy.

========

Where Mplane uses the Quad in the configuration. The purpose of this script is to get the picture captured by the camera and pass it as a texture to our shader. The texture we need is not the image captured by the entire camera, so we need to use a shader to calculate the true texture coordinates to get the correct texture.

========

This is shader "Custom/cameratargetpatch" .

========

Create a material and apply the shader to the above. Finally, both the script and the material are used to run the code on our Cube to see if the effect is great.

[Turn] Augmented reality colar Mix analysis