Source: http://www.onjava.com/pub/a/onjava/2003/10/08/multithreaded_xml.html
XML plays an important role in B2B (enterprise-to-enterprise) applications. In these applications, the simple API for XML (SAX) or document. nbspobject model (DOM) parser is used to parse XML files. (These two Resolvers are Java APIs, which can be found in the appendix below) parsing in a single-threaded application is simple and clear. However, in multi-threaded applications, this is very complicated and challenging. For example, to create an application server, because applications often create a special thread for parsing XML, the parsed data is used to serve many concurrent threads. This article Article Describes the XML parsing implementation of a concurrent application. Design Method
Based on the concept of concurrent Production and Consumption Design, a special thread serves as a producer to parse XML. A group of threads are used as consumers and as production threads for parsing XML data. They store data in a shared data structure for consumption threads to obtain data during future processing, to maximize the ability to generate data while minimizing memory usage, this design uses a special queue to store and locate parsed data for the producer and consumer respectively. Smart queuing)
The smartqueue queue class provides the queue function for production and consumption threads. Its main responsibility is to maintain the queue to prevent (thread) overload and interruption. In other words, smartqueue maintains a queue with a fixed length to ensure resource application efficiency. When appropriate, the queue suspends the production thread until a consumption thread moves from the queue. The following smartqueueCodeThe snippet shows the implementation of this policy.
Public synchronized void put (Object Data ){
// Check to see if the length is 2
While (list. Size ()> = 2 ){
Try {
System. Out. println ("waiting to put data ");
Wait ();
}
Catch (exception ex ){
}
} List. Add (data );
Policyall ();
} Public synchronized object take (){
// Wait until there is data to get
// Come out if the end of file signaled
While (list. Size () <= 0 & (EOF! = True )){
Try {
System. Out. println ("waiting to consume data ");
Wait ();
} Catch (exception ex ){
}
} Object OBJ = NULL; If (list. Size ()> 0 ){
OBJ = List. Remove (0 );
} Else {
System. Out. println ("woke up because end of document. quot ;);
} Policyall ();
Return OBJ;
} |
XML Parsing
This design uses the sax API to parse XML files for the following reasons:
This API is fast and efficient to read XML data. It does not construct any internal XML data description. Correspondingly, it simply transmits the data to the application when encountering XML elements.Program. The sax API is very suitable for the production-consumption mode.
The class of the XML parsing controller (xmlparserhandler) inherits from the sax and implements the callback method to receive XML data from the parser. When the parsing controller class receives XML data from the parser, put the data into hashtable. At the end of each document, the parsing controller puts the data into the smartqueue queue. This controller enters a waiting state. If there is space in the smartqueue queue, the put method will be called once the consumption thread removes one from the smartqueue queue. After parsing the entire XML document, parse the controller (
Xmlparserhandler) notifies the consumption thread to stop searching for more documents. Let's take a look at the call back method. It stores data in the smartqueue queue and notifies the waiting consumption thread. The startelement method shows a new hashtable for each element in each XML file.
Public void startelement (string namespaceuri, string localname,
string QNAME, attributes ATTS)
throws saxexception {
system. out. println (
"startelement local names ............. "+
localname +" "+ QNAME);
If (QNAME. inclusignorecase (elemmark) {
Doc = new hashtable ();
}< br>
ELEM = QNAME;
} |
The End Element (endelement) method is used to add parsed data to the smartqueue queue. As mentioned above, the smartqueue queue suspends this thread until there is no space to store data.
Public void endelement (string namespaceuri, string localname,
String QNAME)
Throws saxexception {
String S = sbdata. tostring (); System. Out. println ("element" + ELEM + "character" + S ); If (Doc! = NULL) & (s! = NULL )&! (S. Trim (). Equals ("")))
Doc. Put (ELEM, S ); Sbdata = new stringbuffer ();
System. Out. println ("endelement ending element ......" + QNAME ); If (QNAME. inclusignorecase (elemmark )){
System. Out. println (
"Endelement ending element ......" + localname ); Smartqueue. Put (DOC );
Doc = NULL;
}
} |
Finally, the calling method of the end document element (enddocument. nbsp) notifies the consumption thread to reach the end of the XML document. This means that the consumption thread does not have to wait for other data to complete their work.
Public void enddocument.) throws saxexception {
Smartqueue. End ();
System. Out. println ("end document .............");
} |
Consumption thread
The consumption thread is moved from the smartqueue queue, except the project once the production thread puts the project into the smartqueue queue. If the smartqueue queue is empty, each consumption thread enters the waiting state. The consumption thread runs until the production line reaches the end of the document element and there is no project in the smartqueue queue. Here is an example of a consumption thread, which keeps retrieving data from the smartqueue queue until there is no data in the queue or reaches the end of the document element.
Public void run (){
While (! Queue. isempty () |! Queue. onend ()){
Hashtable val = (hashtable) queue. Take (); System. Out. println ("obtained by" + this. getname () + "" + val ); // Try {
// System. Out. println ("simulate lengthy
Processing ...........");
// Thread. Sleep (2000 );
//}
// Catch (exception ex ){}
}
} |
Advantages
This design has the following advantages:
Parsing and data consumption can be performed concurrently. Large XML files can be parsed with only a small amount of memory. Design Extension
Smartqueue queues execute a fixed-length queue policy to maintain memory efficiency. You can change the t take and put methods to execute a different policy. As mentioned above, the XML parsing controller (xmlparserhandler) generates a hashtable for XML elements and values. However, this class can be customized to create an object specified by the application. Example Program
The source.zip file contains a testproducerconsumerforxml class that can run the XML file as a parameter. Run the program according to the following instructions,
Unzip the source.zip file.
Run the program testproducerconsumerforxml with order. xml.
For example C: \ testarea> JAVA-classpath \
C: \ testarea prodcons. testproducerconsumerforxml \
C: \ testarea \ prodcons \ order. xml
This article describes how to parse XML document elements using programs, some concepts about production and consumption patterns, and applications of threads. |
