Advanced Android App Development-scene text recognition

As the graduate program needs to complete a mobile terminal-based scene character recognition system, although the graduation is still early, but for the sake of interest, the recent time to complete the system. The basic architecture is as follows:

Client: Android application to capture the scene picture, roughly underline the area of interest text, through the socket communications upload server-side recognition;

Server-side: Python Server for socket communication monitoring, connected after the call word recognition engine (EXE executable), the recognition results returned;

The following is a sample diagram of the system operation:

1. Client

Contains two activity,: mainactivity main interface, such as left 1, choose to call the system inside the camera after shooting services such as left 2; take the picture completed into the kernelactivity such as left 3, roughly underline the area of interest to upload the server side, to obtain recognition results such as Left 4.

Client taking photos and establishing network communication need to be aware of declaring permissions in the Androidmanifestxml file

    <!--authorized to use the camera--    <uses-permission android:name= "Android.permission.CAMERA"/>    <!-- Create and delete files in SD card permissions--    <uses-permission android:name= "Android.permission.MOUNT_UNMOUNT_FILESYSTEMS"/>    <!--write data to the SD card--    <uses-permission android:name= "Android.permission.WRITE_EXTERNAL_STORAGE "/>    <!--authorized access to the Internet--    

Client Android programming when connecting to the server to communicate with the socket, there are a few issues to be aware of during the programming process, otherwise it is easy to cause errors:

1.1 Network connection cannot be established directly in main thread

To prevent thread blocking, Android version 4.0 does not allow the network connection to be established directly on the main thread (socket communication requires a network connection). Therefore, when the socket communication is required, the new thread should be used for the network connection, using the example below

New Thread () {///main thread cannot connect directly to the network, need to create new child thread @overridepublic void Run () {//todo}}.start ();

1.2 Client socket send and receive data stream inappropriate use of the close () method

The client needs to accept and send the data at the same time, so there will be two objects out and in

Start socket communication Socket SOCKET = new Socket ("210.77.27.123", 9058);//Here IP according to its own server-side settings//write output (upload) data dataoutputstream out = new DataOutputStream (Socket.getoutputstream ());//write input (GET) data bufferedreader in = new BufferedReader (New InputStreamReader ( Socket.getinputstream (), "gb2312")); Set the character encoding format to read Chinese correctly

You might want to close an object with Out.close () if you think that out is no longer needed after you send the data to the server side. If so, the exception "socket closed" will be thrown when you use the in object to receive data! The reason is that the close () method of the in or out object causes the socket to be closed after it is used. Therefore, if you need to close, consider using the Shutdowninput () and Shutdownoutput () methods, or do not process and wait for the last Socket.close ().

1.3 Child threads cannot update UI

When obtaining server-side recognition results, it is necessary to update the client UI with this content (TextView in the lower-left corner), which is not feasible if you try to update the UI directly using the Text.settext () method in the sub-thread of socket communication mentioned above. Therefore, in order to update the UI you need to use the handler object to update the UI within its handlemessage () method.

The server-side results are stored in a message and sent to a predefined handler object in a child thread

String response = In.readline (); Message msg = Handler.obtainmessage ();//The UI cannot be updated in a child thread, with handlermsg.obj = Response;handler.sendmessage (msg);

immediately after handler calls its Handlemessage () method update ui,handlemessage () is defined as follows

Handler = new Handler () {@Overridepublic void handlemessage (Message msg) {String message = (string) msg.obj;if (Message! = Nu ll)    text.settext (message), Else    text.settext ("Sorry, not recognized"); Super.handlemessage (msg);//This sentence must have! Otherwise, the UI cannot be updated}};

2. Server-side

The server-side primarily deploys Python scripts for socket snooping, and once the connection is established, the developed scene text recognition engine (STREngine.exe) is called and the results are returned to the client. The Python server script contents are as follows

###################################### # # STC Recognition Server # # ##################################### #import socketimport oss = socket.socket () host = Socket.geth Ostname () port = 9058s.bind ((host, Port)) S.listen (5) while true:c, addr = s.accept () print ' Got connection from ', ad        Dr #get uploaded params params = C.recv (+) LST = Params.split (") If Len (LST) < 4:c.close () Continue print ' x=%d, y=%d, w=%d, h=%d '% (int (lst[0]), int (lst[1]), int (lst[2]), int (lst[3]) #get uploaded image Si ze info size = Int (C.RECV (4)) #save uploaded image f = open ('. \\upload\\tmp.jpg ', ' WB ') for I in range (siz E): F.write (C.RECV (1024x768)) F.close () print ' Image received, size%dkb '% size #execute strengine on server and send back the result of result = Os.popen (' strengine./upload/tmp.jpg ' + params). Read () print ' recognition result: ', result C.send (result + ' \ n ') print ' Close connection with ', addr, ' \ n ' c.close () 

The internal code of the word recognition engine is inconvenient to share, but the executable file is shared with interested friends.

Server side There is a very important place, do not pay attention may bring you endless trouble, I spent two nights only to find this very small problem, here to share in order to avoid wasting other people's time:

When returning the recognition results to the client on the server side, be sure to end the string with a newline character ' \ n ' after it! If not added, the client In.readline () method will block until the end of the row is found, and the client UI cannot be updated with the returned results. This is a very annoying problem because if you return without ' \ n ' on the server, it is no problem to take advantage of the Android virtual machine on the local computer, and the UI is updated normally on the VM, but once the machine is switched on it will not respond (do not update the UI).

Summarize:

In fact, the client manually draw interest text upload, this area location in the ImageView and the actual image is to be carefully converted, here is not detailed, there is a need for friends directly read the code bar. Or strongly suggest that you analyze the derivation, and find out the relationship, it will deepen the understanding of ImageView and bitmap greatly help.

In addition, because not to do the product, the efficiency of the consideration did not care. Now is the mobile phone photos after the entire picture upload, although the compression, but a picture still has hundreds of KB size, which is too wasted on traffic. The solution is also simple, as long as the text area will be extracted separately to upload (but four weeks need a certain degree of expansion), the size should be reduced by dozens of times times.

Client and server source code (including the recognition Engine executable program) has been shared to CSDN, there is a problem please feel free to communicate at any time.