Image semantic Segmentation of the JSON file and the original, using the PLT to draw the image mask

1. Weak supervision

As the company recently prepared to open a new project, using deep learning to train a model that can be automatically labeled, but the model required training set is more troublesome, to first use ffmpeg from the video to intercept a video, in the use of OPENCV to draw frames to get pictures, so I can only be the semantic segmentation of the JSON file and the original image, The mask of the synthetic image.

2. Environment installation

Operating system: Windows 7

Python Environment: 3.6.4

Required libraries: Numpy,matplotlib,pil,opencv-python

Software: FFmpeg

3. Capturing video

Intercept a video segment that has always had the same person appear.

#-*-coding:utf-8-*-ImportOSdefcut (filename, start, end):assertos.path.exists (filename) isTrue,"The soruse file is not exists."Start1= Start.replace (":","") End1= End.replace (":","")    #Print (Start1 + "" + end1)Videoname ="{}{}-{}.mp4". Format (Filename.rsplit (".", 1) [0],start1,end1) cmd="ffmpeg-i {}-vcodec copy-acodec copy-ss {}-to {} { }-y". Format (filename,start,end,videoname) result=os.popen (cmd)returnresultif __name__=="__main__": File= Input ("videos that need to be intercepted:") Start= Input ("start Time (HH:MM:SS):") End= Input ("End Time (HH:MM:SS):")    Print(Cut (file, start, end))

Output Result:

This is the original video on the left, and this is the intercepted video on the right.

4, Video pumping frame

From the video, extract one picture at every 40 frames.

ImportCv2defget_video_pic (name,zhen): Path= Name.rsplit (".", 1) [0] Cap=Cv2. Videocapture (name) forIinchRange (1,int (Cap.get (7) ), Zhen): Cap.set (1, i) Rval, frame=Cap.read ()ifRval:picname="{}{}.jpg". Format (PATH,STR (i)) Cv2.imwrite (Picname, frame) cap.release ()if __name__=="__main__": Video= R"C:/users/yuanpeng.xie/desktop/test/yongcun-3.30-3.36.mp4"Frame= 40get_video_pic (Video,int (frame))Print(" Over")

Output Result:

The video looks like a total of 131 frames, and every 40 frames is the following four pictures

5. Image is segmented and labeled

Find your own tool to mark, save as a JSON file

Output Result:

JSON file part content

6. Use the PLT to draw the mask with the JSON file and the original image.

ImportNumPy as NPImportMatplotlib.pyplot as PltImportmatplotlib.image as IMG fromPILImportImageImportOSImportJSONdefPicturetomask (D_object, sourcepicture):" "get the width and height of the original" "im=Image.open (sourcepicture) Size=list (im.size) Width=size[0] Height= Size[1]        " "converts the width and height of a picture's pixels to inches in width and height" "DPI= 80#resolutionYcwidth = width/dpi#width (inch) = pixel width/ResolutionYcheight = height/dpi#height (inches) = pixel Height/ResolutionColor= ["g","R","b","y","Skyblue","k","m","C"] Fig, ax= Plt.subplots (figsize=(ycwidth,ycheight)) forRegioninchD_object:" "convert x-axis coordinate points and y-coordinate points into numpy arrays, add and then transpose to multiple rows of two columns" "x=Np.array (d_object[region][0]) y= Np.array (d_object[region][1]) *-1XY=Np.vstack ([x, Y]). T" "#设置画框的背景图片为原图 fig = plt.figure (figsize= (ycwidth,ycheight), dpi=dpi) bgimg = Img.imread (sourcepict Ure) Fig.figimage (bgimg)" "        " "Map the coordinates in the numpy to the PLT" "Plt.plot (xy[:,0],xy[:,1],color=color[int (region)]) Plt.fill_between (xy[:,0],xy[:,1],facecolor=color[int (region)])#fills the split area with colorplt.xticks ([0,width]) plt.yticks ([0,-height]) Plt.axis ("off")    #Save PicturePath = Sourcepicture.rsplit (".", 1) [0]Print(sourcepicture)Print(Path) plt.savefig ( path+"-mask.png", format='PNG', bbox_inches='Tight', Transparent=true, dpi=100)#bbox_inches= ' tight ' picture border blank tight, background transparent    #plt.show ()defGetjson (filepath):" "get JSON file contents from folder, return dictionary" "Files=Os.listdir (filepath) forFileinchFiles:ifFile.split (".") [1] = ="JSON": Jsonfile= FilePath +file BreakJsonstr= Open (Jsonfile,"R", encoding="UTF8"). Read () D_json=json.loads (JSONSTR)#print (D_json)    returnD_jsondefGetPath ():" "Enter picture folder path" "filepath= Input ("Picture folder path:")    ifFilepath.endswith! ="/" orFilepath.endswith! ="\\": filepath= FilePath +"/"    returnfilepathdefMain (): filepath=GetPath () D_json=Getjson (filepath) forKeyinchD_json:data=d_json.get (key) Picturename= data["filename"] D_object= {}         forRegioninchdata["Regions"]: L_object=[] x= data["Regions"][region]["shape_attributes"]["all_points_x"] y= data["Regions"][region]["shape_attributes"]["all_points_y"] L_object.append (x) l_object.append (y) d_object[region]=L_object sourcepicture= FilePath +picturename picturetomask (D_object, sourcepicture)if __name__=="__main__": Main ()

Output Result:

Image Mask

7. Minor problems

There is a small problem, that is, the size of the mask will be larger than the original size, because the save will be the whole figure is saved, equal to the mask more than a border, and then turn over the data, the size of the mask and figure into the same.

Plt.axes ([0,0,1,1])

And then save the picture.

By removing the bbox_inches= ' tight ' from the code, you can save the mask in the same size as the original Sing Woo.

　　

Image semantic Segmentation of the JSON file and the original, using the PLT to draw the image mask