A memory error that cannot capture ADO. NET Dataset

Dataset is a new structure used by ADO. NET to store data in the memory. In some respects, Dataset and ADO have similar Recordset objects. However, Dataset can save the entire schema (including table, Link, keyword, and real data) in the memory, at this point, Dataset is more powerful than Recordset. Therefore, you can query and modify Dataset without worrying about affecting the database that is using it.

When considering that Dataset stores all its data in the memory, some people may worry about problems if the memory is exhausted. It is easy to make a reasonable guess, but the actual situation may be more complicated than you guess. To demonstrate how. NET works in this situation, let me first talk about how to create a project that keeps loading data to a Dataset until the memory is exhausted. Note that we do not recommend this process. It just proves to you how easy it is to exhaust the memory.

Create a TooMuchData Project

It is easy to create a project that constantly loads data to a Dataset. Open Visual Studio. NET and create a new VB. NET Window application. Add a button control to the form and double-click it to open the code window. Enter the following code in the code window:

Private Sub button#click (ByVal sender As System. object, _ ByVal e As System. eventArgs) Handles Button1.ClickDim sConnString As String = "Server = localhost; Database = pubs; uid = sa; pwd =; "Dim sSQL As String =" SELECT * FROM authors "Dim daProduct As SqlDataAdapter = New SqlDataAdapter (sSQL, sConnString) Dim myDS As New DataSet () Do While TruedaProduct. fill (myDS, "authors") If myDS. tables ("authors "). rows. count Mod 100 = 0 ThenDebug. writeLine (myDS. tables ("authors "). rows. count. toString () End IfLoopEnd Sub

The third line of the Code establishes a string-to-variable sConnString connection. If you want to recreate this project, you may need to modify this line of code, unless you run it on the server and have a "sa/no password" user ID and the corresponding password.

The fourth line of the Code creates a simple SQL query. Select * is generally not recommended, but there is no problem here, because our goal is to catch as many records as possible. The primary key field of the selected table is not a problem, because the Dataset we created is weakly type and the primary key field of the DataTable used to load data is not set.

The following two lines of code create the DataAdapter and Dataset objects. Then, enter an endless loop, call the Fill method of DataAdapter in the endless loop, and add the record to the able called "authors. The cycle includes an If statement to show whether the current value of the number of rows can be fully divided by 100. This is not unnecessary, but it can have two functions: first, you can know that the program is still running; second, you can know how many records are added to the Dataset.

Run this program
After you create this program, you may want to make some modifications before running it. As Dataset grows, it consumes more and more memory. Once it consumes all available memory, Windows begins to swap the memory to the memory swap file. On many machines, the file exchange is quite large, so this program may have to run for several hours. For example, I used a laptop with a-MHz PIII CPU and 5-12 MB RAM when testing this program. Set the page switch file to the minimum value to shorten the program running time-this is the trick to complete the test as soon as possible. Even so, the program runs for dozens of minutes or hours before it fills up all available memory.

Now you can run the program and click the button control to start the process. It establishes a connection to the server, reads data from the table, and transmits the data to the able object in the Dataset. The same record is also saved to the Dataset until the memory is eventually exhausted. You can use the task manager to observe the memory usage. You can even see the increase or decrease of the available memory with the data exchange between the memory and the disk. In my machine, Dataset grew to 1400000 records before the machine ran out of memory.

Exhausted memory
What will happen when you run out of memory depends on whether you are running the program or the compile version of the program in the VS. NET environment. If you are running the compiled version of the program without error handling, you will not find any errors during the program running. It only stops running when the memory is exhausted. If you run the code in the VS. NET environment without error handling, the program stops running and the following error message appears in the debugging window:

Fatal out of memory error.
The program '[2340] TooMuchData.exe' has exited with code 0 (0x0 ).

Therefore, you may try to add an error handler to detect System. OutOfMemoryException. For example, you may use a Try... Catch statement to check for exceptions. A common method is as follows:

Do While TrueTrydaProduct. fill (myDS, "authors") If myDS. tables ("authors "). rows. count Mod 100 = 0 ThenDebug. writeLine (myDS. tables ("authors "). rows. count. toString () End IfCatch ex As predictionmessagebox. show ("Error:" & ex. message) End TryLoop

Unfortunately, this method does not work at all. When you set a breakpoint on the MessageBox. Show statement, it should appear theoretically when you run it on this statement, but this will never arrive. When the memory is eventually exhausted, the debugging window displays the following information:

Fatal out of memory error.
An unhandled exception of type 'System. outofmemoryexception' occurred in System. data. dll

If MessageBox is used, this message may be better, but on the contrary, VS. NET (or Framework) generates and displays the preceding message, and tracing this specific error is not an effective solution.

If you add error handling in this program and compile and run it, you will get another result. This time, you will find MessageBox reported that the program encountered an error that could not be handled, type: System. OutOfMemoryException, located in system. data. dll. However, this MessageBox comes from the Framework rather than the MessageBox you write in code.

Not as easy as it sounds
You can create a Dataset that can consume all the available memory, but the process is not simple. It requires a lot of records, especially a lot of time. Dataset may take several hours to fill up the memory. Almost no application can run for a long time in this case, which leads to a problem. Of course, the configurations of each machine are different, such as the processor speed, memory capacity, and swap file size. However, this result is not welcome.

No matter how annoying the results are, errors should be captured. Unfortunately, the System. OutOfMemoryException error cannot be caught in this case, which means it is difficult for us to handle this specific error in the application.