Process large XML files

See "effective methods for modifying large XML files.

We all know that loading an XML file in xmldocument loads all the content at once. When a large XML file is encountered, the problem is very high. I used to load a 3 MB file, the time consumed during debugging is obvious. Xmlreader is good. It is said that xmlreader only uses dozens of K (?) Memory, read in sequence, but read-only.

Msdn this article describes how to use xmlreader and xmlwriter to process large XML files. I wrote the following code to test it. In the middle, I saved some xmlnametable and xmlvalidatingreader used in this article.

Xmlreader and xmlwriter are used for processing.
Private void button#click (Object sender, system. eventargs E)
{
System. Diagnostics. Debug. writeline (system. datetime. Now. tostring () + "" + datetime. Now. millisecond. tostring ());
File. Copy ("resourcelist. xml", "Temp. xml", true); // to update the file, copy a copy first.
Xmltextreader reader = new xmltextreader ("Temp. xml ");
Reader. whitespacehandling = whitespacehandling. Significant;
Reader. movetocontent ();

Streamwriter Sw = new streamwriter ("resourcelist. xml", false, system. Text. encoding. utf8); // create a writer
Xmlwriter XW = new xmltextwriter (SW );
XW. writestartelement (reader. prefix, reader. localname, reader. namespaceuri );
Reader. Read ();
Do
{
If (reader. Name = "resource ")
XW. writestartelement ("resource ");
Else if (reader. Name = "ctno ")
XW. writeelementstring ("ctno", reader. readstring ());
Else if (reader. Name = "itmnum ")
XW. writeelementstring ("itmnum", reader. readstring ());
Else if (reader. Name = "restyp ")
XW. writeelementstring ("restyp", reader. readstring ());
Else if (reader. Name = "resnam ")
{
If (reader. readstring (). Equals ("i99000000001") // compare and replace the content
XW. writeelementstring ("resnam", "hello ");
Else
XW. writeelementstring ("resnam", reader. readstring ());
}
Reader. Read ();
If (reader. Name = "resource ")
{
If (reader. nodetype = xmlnodetype. endelement)
{
XW. writeendelement ();
Reader. Read ();
}
}
} While (reader. nodetype = xmlnodetype. element );

XW. Close ();
Reader. Close ();
System. Diagnostics. Debug. writeline (system. datetime. Now. tostring () + "" + datetime. Now. millisecond. tostring ());
}

The following code uses xmldocument to process this file:
Private void button2_click (Object sender, system. eventargs E)
{
System. Diagnostics. Debug. writeline (system. datetime. Now. tostring () + "" + datetime. Now. millisecond. tostring ());
Xmldocument domtemp = new xmldocument ();
Domtemp. Load ("resourcelist. xml ");
Xmlnode node = domtemp. selectsinglenode ("Descendant: Resource [resnam = 'i990000000000001']");
If (node! = NULL)
{
Node. selectsinglenode ("resnam"). innertext = "hello ";
}
Else
{
Node = domtemp. selectsinglenode ("Descendant: Resource [resnam = 'hello']");
If (node! = NULL)
Node. selectsinglenode ("resnam"). innertext = "i99000000001 ";
}
Domtemp. Save ("resourcelist. xml ");
System. Diagnostics. Debug. writeline (system. datetime. Now. tostring () + "" + datetime. Now. millisecond. tostring ());
}

My test uses. NET Framework 1.1, and the XML file size is 808kb.
Execution result:
Memory consumption: Because I do not use CLR profiler very much and will not analyze it, it will certainly be different.
Execution time: it takes 370 ms to use xmlreader + xmlwriter, and 391 ms to use xmldocument