OpenCV deep image denoising for single Kinect multi-frame stationary scenes

Boss Kinect to the task of the noise has been more than half a month, in addition to looking at a few days outside the document on the soy sauce, as if every day is very busy, but do not know what is busy. These days for the sake of the work, we randomly gathered a few pieces of code, get a result, also know not, first deal with, then plan.

The idea of the program is very simple, the static scene of a continuous sampling of several frames, and then all points in the time domain to take the median value, the invalid point after the median value in the spatial domain to take the nearest neighbor, reluctantly filled the black hole. Because the code is longer, it now has a few key pieces:

#include <cv.h>
#include 
 
This is the definition of the header file, pretending to write a class, inside the constructor, in addition to allocating memory for denoisedimage other than 0, the destructor needs to release the memory of the denoisedimage and frameset arrays. Numofframes was originally designed as the number of frames in an image in frameset, and the result was a fixed-length array due to laziness.

void Kinectdenoising::setimageroi (bool isupdate) {if (!isupdate) {Imageroi = Cvrect (22,44,591,434);
		} else {iplimage* image8u = Cvcreateimage (Cvsize (width,height), ipl_depth_8u,1);

		iplimage* bitimage = Cvcreateimage (Cvsize (width,height), ipl_depth_8u,1);
		Cvthreshold can only handle images of 8UC1 or 32fc1 Cvconvertscale (frameset[0],image8u,255.0/4096.0);

        Cvthreshold (image8u,bitimage,0,1,cv_thresh_binary); The Mats rowreduced and colreduced has the to IS CV_32SC1 type//For function cvreduce () seems not to suitable f
	    The or 16U type and//8U type doesn ' t has enough for the result.
	    cvmat* rowreduced = Cvcreatemat (1,BITIMAGE-&GT;WIDTH,CV_32FC1); Bitimage->width represents number of cols, while bitimage->height stands for rows cvmat* colreduced = Cvcrea

	    Temat (BITIMAGE-&GT;HEIGHT,1,CV_32FC1);
	    Cvreduce (bitimage,rowreduced,0,cv_reduce_sum);

		Cvreduce (bitimage,colreduced,1,cv_reduce_sum); Compute imageroi.x
	    for (int i=0;i<rowreduced->cols;i++) {Float temp = Cv_mat_elem (*rowreduced,float,0,i);
			    if (TEMP&GT;BITIMAGE-&GT;HEIGHT/3) {imageroi.x = i;
		    Break }}//Computer imageroi.width for (int i=rowreduced->cols;i>0;i--) {Float temp = cv_mat_elem (*
			ROWREDUCED,FLOAT,0,I-1);
		     	if (TEMP&GT;BITIMAGE-&GT;HEIGHT/3) {imageroi.width = i-imageroi.x;
			Break }}//Compute IMAGEROI.Y for (int i=0;i<colreduced->rows;i++) {Float temp = Cv_mat_elem (
    		*colreduced,float,i,0);
    			if (TEMP&GT;BITIMAGE-&GT;HEIGHT/3) {imageroi.y = i;
	    	Break }}//Compute imageroi.height for (int i=colreduced->rows;i>0;i--) {Float temp = Cv_mat_ele
    		M (*colreduced,float,i-1,0);
	    		if (TEMP&GT;BITIMAGE-&GT;HEIGHT/3) {imageroi.height = I-IMAGEROI.Y;
	    	Break
		}}//Set Memory free cvreleaseimage (&bitimage);Cvreleaseimage (&image8u);
		Cvreleasemat (&rowreduced);
	Cvreleasemat (&colreduced); }
}
 
This is the filtering range of the computed depth image. Because of the inconsistent view of the depth image and the color image, the effective pixels are reduced when the depth image is mapped to the color image, and the typical phenomenon is that there is a black area around the depth image. This function is used to remove the surrounding black box. The method of projecting with OpenCV. Because the Cvreduce () function is to accumulate and calculate, in order not to overflow the data, the target array should use a 32-bit floating point type (this function only supports 8-bit unsigned char and 32-bit float type).

void Kinectdenoising::medianfiltering ()
{
	//Set result image zero
	Cvsetzero (denoisedimage);

	unsigned short data[nframes];
	int total;
	for (int i=imageroi.y;i<imageroi.y+imageroi.height;i++)
	{
		unsigned short* denoisedimagedata = (unsigned short*) (denoisedimage->imagedata+denoisedimage->widthstep*i);
		for (int j=imageroi.x;j<imageroi.x+imageroi.width;j++)
		{Total
			= 0;
			for (int k=0;k<nframes;k++)
			{
				insertsort (Data,total,cv_image_elem (frameset[k],unsigned short,i,j));
			}
			if (Total! = 0)
			{
				Denoisedimagedata[j] = DATA[TOTAL/2];}}}
 
Median filtering, statistically valid points are sorted and then median is taken. The Insertsort () function is used to insert values into the order of small to large, and in view of the length of the relationship, it is not posted.

void Kinectdenoising::nearestfiltering () {Cvpoint topleft,downright;
	iplimage* tempimage = Cvcloneimage (denoisedimage); for (int i=imageroi.y;i<imageroi.y+imageroi.height;i++) {unsigned short* data = (unsigned short*) (denoisedimage-&gt
		; imagedata+denoisedimage->widthstep*i); for (int j=imageroi.x;j<imageroi.x+imageroi.width;j++) {for (int k=1;data[j]==0;k++) {topleft = Cvpoint (j-k    , i-k);
				J is the row number I is the number of columns downright = Cvpoint (j+k,i+k);
					for (int m=topleft.x; (m<=downright.x) && (data[j]==0); m++) {if (m<0) continue;
					if (m>=width) break;
			    		if (topleft.y>=0) {unsigned short temp = Cv_image_elem (tempimage,unsigned short,topleft.y,m);
							if (Temp > 0) {data[j] = temp;
		    			Break }} if (Downright.y < height) {unsigned short temp = Cv_image_elem (tempimage,unsigned short,downri
			    		GHT.Y,M);
							if (Temp > 0) {data[j] = temp; BrEak }}} for (int m=topleft.y; (m<downright.y) && (data[j]==0); m++) {if (m<0) continu
					E
					if (m>=height) break;
						if (topleft.x>0) {unsigned short temp = Cv_image_elem (tempimage,unsigned short,m,topleft.x);
							if (Temp > 0) {data[j] = temp;
		    			Break }} if (Downright.x<width) {unsigned short temp = Cv_image_elem (tempimage,unsigned short,m,downri
						Ght.x);
							if (Temp > 0) {data[j] = temp;
		    			Break
}}}}}} cvreleaseimage (&tempimage); }
 
Finally, median filtering, starting from the inner layer, expands from one layer to the outside until a valid value is found.


Operation Result:

SOURCE Image:



Result Image:


Note: This program was originally performed on a 8-bit image. A 16-bit unsigned short depth image is obtained first, and then converted to a 8-bit unsigned char by the Cvconvertscale () function, resulting in a de-noising, which will unsigned Char-type data in MATLAB, found in the unsigned short data in the 0 value of the pixel inexplicably in the unsigned char type has a small value (such as 1, 2, 3, 4, 5 What, that is, not 0). It's strange that I don't know what's going on in OpenCV. It seems that the source data is reliable, so it changed to 16-bit unsigned short type, the result is a good situation.

http://blog.csdn.net/chenli2010/article/details/7006573