C # Stream Chapter (v)--MemoryStream

MemoryStream

Directory:

1 Brief Introduction MemoryStream

2 The difference between MemoryStream and FileStream

3 through some of the source of deep understanding of the next MemoryStream

4 analysis of Memorysteam most common outofmemory anomalies

5 Structure of the MemoryStream

6 Properties of the MemoryStream

7 Methods of MemoryStream

8 MemoryStream Simple example: using MemoryStream in XmlWriter

9 MemoryStream Simple example: Customizing a httphandler that handles pictures

10 Summary of this chapter

A brief introduction to MemoryStream

MemoryStream is a memory stream that provides read and write operations for system memory, since MemoryStream is made up of unsigned byte arrays, it can be said that MemoryStream performance can be

Is good, so it serves as an intermediary for some other streams of data exchange, while reducing the need for temporary buffers and temporary files in the application, in fact MemoryStream

is as important as FileStream, and in many cases we have to use it to improve performance

The difference between MemoryStream and FileStream

As mentioned in the previous article, FileStream mainly on the file of a series of operations, belonging to the higher level of operation, but MemoryStream is very different, it is more inclined to the underlying memory operation, so

can achieve faster speed and performance, but also their fundamental difference, many times, the operation of files need MemoryStream to actually read and write, and finally put into the corresponding FileStream,

In addition to this, it is necessary to use MemoryStream to improve read and write speeds in operations such as XmlWriter.

Deep understanding of the next MemoryStream through some source code

Due to the length of the relationship, this article can not be described in detail its source code, also please everyone haihan, here I will briefly introduce the write () method of the source code

  public override void Write (byte[] buffer, int offset, int count) {if (!_isopen) __error.streamisclosed ();            if (!_writable) __error.writenotsupported (); if (buffer==null) throw new ArgumentNullException ("Buffer", environment.getresourcestring ("Argumentnull_buf            Fer ")); if (offset < 0) throw new ArgumentOutOfRangeException ("offset", environment.getresourcestring ("Argumento            Utofrange_neednonnegnum ")); if (count < 0) throw new ArgumentOutOfRangeException ("Count", environment.getresourcestring ("Argumentout            Ofrange_neednonnegnum ")); if (buffer. Length-offset < count) throw new ArgumentException (Environment.getresourcestring ("Argument_invalidoffl                En "));            int i = _position + count; Check for overflow if (i < 0) throw new IOException (environment.getresourcestring ("IO").            Io_streamtoolong ")); if (i;                _length) {bool Mustzero = _position > _length;                    if (i > _capacity) {bool Allocatednewarray = ensurecapacity (i);                if (Allocatednewarray) Mustzero = false;                } if (Mustzero) array.clear (_buffer, _length, I-_length);            _length = i;                } if (count <= 8) {int byteCount = count;            while (--bytecount >= 0) _buffer[_position + byteCount] = Buffer[offset + ByteCount];            } else Buffer.internalblockcopy (Buffer, offset, _buffer, _position, Count);            _position = i;        Return }

About the source code of MemoryStream Everyone can learn, here the main analysis under the MemoryStream the most critical write () method, top-down, the first series of judgments we are easy to see and understand,

In the future, you should be aware of the possible anomalies, you will get the length of this data int i=_position+count, next to determine whether the length of the data exceeds the length of the stream,

If you exceed the check in the flow's disposable capacity (bytes), (Note the Ensurecapacity method, the method will automatically expand the stream's capacity, but only if you use the MemoryStream

The second constructor, that is, with the parameter is Capaciy) if the flow exceeds the disposable capacity of the tail will be deleted (more than the part of the data cleared), then you will certainly ask, why to Judge Count<=8,

In fact, 8 this number is critical in the flow, the personal think that Microsoft for performance needs to write: When the byte is less than 8 read one after the other, when the byte is greater than eight when the block copy of the way, in this range to decrement the loop

Writes data to the buffer _buffer in the stream, this buffer _buffe is a private byte array type of MemoryStream, and the stream is placed inside that buffer buffer by reading the external byte data, if the stream

is longer than 8, the Buffer.internalbloackcopy method is used to replicate the array, unlike Array.copy, which uses memory displacement rather than index displacement, so there is a significant performance improvement. In fact

The prototype of this method belongs to C + +.

Analysis of the most common outofmemory anomalies in Memorysteam

Let's look at the following very simple test code

         The test byte array assumes that the array capacity is 256M            byte[] testbytes=new byte[256*1024*1024];            MemoryStream ms = new MemoryStream ();            using (ms)            {for                (int i = 0; i <; i++)                {                    Try                    {                        Ms. Write (testbytes, 0, testbytes.length);                    }                    Catch                    {                        Console.WriteLine ("The memory stream has already used {0}m capacity memory, the maximum capacity of this memory stream is {1}m, the overflow capacity is {2}m",                             GC.) Gettotalmemory (FALSE)/(1024x768),//memorystream has consumed the amount of MS in memory                            . Capacity/(1024x768),//memorystream maximum usable capacity                            Ms. Length/(1024x768 *);//memorystream the current stream (capacity) break                        ;            }}} Console.ReadLine ();

Since we set a 256M byte (a bit scary), look at the state of the overflow

From the output results, MemoryStream default maximum available capacity is 512M when the exception is its maximum capacity, smart you will certainly ask: if you use 2 MemoryStream and even multiple memory

How is it distributed? Good, or code to look at the output, it can be seen that the average memory of 2 MemoryStream but the maximum capacity or 512M

But the question is, suppose we need to operate a larger file, what should we do? There are 2 ways to do this, one is the fragment processing described earlier, and we divide the byte array into equal parts.

One way to do this is to increase the maximum usable capacity (bytes) of MemoryStream as much as possible, and we can take advantage of its overloaded version when declaring the MemoryStream constructor:

MemoryStream (int capacity)

How to use which method is better? In fact, I think the specific project analysis, the former segmented processing can solve the problem of large data volume operation, but sacrificing performance and time (multi-threaded temporary

Not considered), which gives a performance advantage but the maximum allowable capacity is int. MAX, so you can't give a definite answer, everyone is doing the project according to the requirements of their own, the most critical

To get the best bit of performance and overhead.

There is also a more disgusting overflow way, often let everyone crazy, is not timed overflow, that is, MemoryStream processing files may only 40M or more can occur outofmemory

The exception, about this problem, finally in a foreigner's article has been explained, good luck, Chen Yanming eldest brother in his blog just translated under, remove my translation work ^ ^, because this involves

Windows memory mechanism, including memory pages, process of virtual address space, etc., so complex, so we look at him before this article, I first and everyone briefly introduce the next page and the virtual address of the process

Memory Pages: Memory pages divided into: file pages and calculation pages
In-memory file pages are file buffers, file-type memory pages, memory pages that hold file data (also called permanent pages), which can be used to reduce access to the disk when reading and writing files, if its size

Setting too small causes the system to frequently access the disk, increasing disk I/O, and setting it too large to waste memory resources. In-memory computing pages, also known as computational memory pages, are primarily used to store program code and pro-

The data used when

Virtual address of the process: each process is given its very private virtual address space. For 32-bit processes, the address space is 4G because a 32-bit pointer can have from 0x00000000 to 0xFFFFFFFF

Any value between the two. This range allows the pointer to have one from 4,294,967,296 values, covering the 4G range of a process. For 64-bit processes, the address space is 16EB because a 64-bit pointer can point to

One of 18,446,744,073,709,551,616 values that covers the 16eb range of a process. This is a very broad range.

These concepts are from the Windows core programming this book, in fact, this book is very important to our programmers, for the memory operation, I am also small white, it seems that this book is not to buy ....

The structure of MemoryStream

MemoryStream ()

MemoryStream allow construction without parameters

MemoryStream (byte[] byte)

A byte array is a byte array that contains certain data, which is important for beginners or programmers who do not use many of them to ignore this construct, resulting in the discovery of MemoryStream in the subsequent reading or writing of data.

Without byte data, it can lead to a very depressing feeling, attention on the line, and sometimes may not need this, because many methods return value is already MemoryStream

MemoryStream (int capacity)

This is the most serious, why do you say so? I also mentioned in this article about OutOfMemory anomalies, and if you want to increase the throughput (bytes) of the MemoryStream, you can only increase it by this method

A certain throughput, up to int only. Max, this method is also a feasible solution to OutOfMemory

MemoryStream (byte[] byte, bool writeable)

The writeable parameter defines whether the stream is writable

MemoryStream (byte[] byte, int index, int count)

The index parameter defines a byte array from which indexes are indexed,

The count parameter is the number of data fetched

MemoryStream (byte[] byte,int index, int count, BOOL writeable, BOOL publiclyvisible)

The publiclyvisible parameter represents true to enable the GetBuffer method, which returns an array of unsigned bytes, the stream is created from the array; otherwise false, (you must think this is hard to understand, don't worry about the following methods

I will elaborate on this stuff)

Properties of the MemoryStream

The properties of Memory are roughly similar to their parent classes, which have been discussed in detail in my article, so I'll simply list their properties:

Its unique properties are:

Capacity: This previous article has actually mentioned that it represents the amount of disposable capacity (bytes) of the stream, a very important attribute

Methods of MemoryStream

For the overridden method is not repeated here, you can refer to the first article I wrote

Here are the unique methods of MemoryStream

Virtual byte[] GetBuffer ()

This method should be used with caution, because this method returns an array of unsigned bytes, that is, even if I enter only a few characters such as "Hellowworld" we only want to return 11 data on the line,

However, this method will return the entire buffer data, including those that have been allocated but not actually used, if you want to enable this method, you must use the last construct above

Set the Publiclyvisible property to True, which is the function of the above constructor.

virtual void WriteTo (Stream stream)

The purpose of this method is actually to discuss the performance problem at the beginning of this article, it has been pointed out that MemoryStream often plays the role of intermediate flow,

So read and write writes the memory stream to another stream after processing

Simple example using MemoryStream in XmlWriter

        <summary>//demo using MemoryStream///</summary> public static void U in XmlWriter            Sememorystreaminxmlwriter () {MemoryStream ms = new MemoryStream ();                Using (MS) {//define a XmlWriter using (XmlWriter writer = xmlwriter.create (ms)) {//write to XML header writer.                    WriteStartDocument (TRUE); Writes an element of writer.                    WriteStartElement ("Content"); Add a test property writer to this element.                    WriteStartAttribute ("test"); Sets the value of the test property writer.                    WriteValue ("Counterclockwise Wind"); Release the buffer, which may not be released here, but in the actual project you might want to consider the partial release of the boost writer for performance.                    Flush (); Console.WriteLine ("Memory usage at this time is: {2}kb, the MemoryStream has already used a capacity of {0}byte, the default capacity is {1}byte", Math.Round (double) m S.length, 4), Ms. CAPACITY,GC.                 Gettotalmemory (FALSE)/1024);   Console.WriteLine ("The location of the MemoryStream before repositioning is {0}", Ms.                    Position); Moves the current position in the stream 7-bit backwards, which is equivalent to a space Ms.                    Seek (7, seekorigin.current); Console.WriteLine ("MemoryStream location after repositioning is {0}", Ms.                    Position); If you set the position of the stream to the location shown below, the XML file is scrambled//ms.                    Position = 0; Writer.                    WriteStartElement ("Content2"); Writer.                    WriteStartAttribute ("Testinner"); Writer.                    WriteValue ("Counterclockwise wind inner"); Writer.                    WriteEndElement (); Writer.                    WriteEndElement (); Release writer again.                    Flush (); Console.WriteLine ("Memory usage at this time is: {2}kb, the MemoryStream has already used a capacity of {0}byte, the default capacity is {1}byte", Math.Round (double) m S.length, 4), Ms. Capacity, GC.                    Gettotalmemory (FALSE)/1024); Set up a FileStream file creation destination is d:\test.xml FileStream fs = new FileStream (@ "D:\test.xml", Filemode.openorcrea      TE);              The using (FS) {//memory stream is injected into FileStream Ms.                        WriteTo (FS); if (Ms. CanWrite)//release buffer fs.                    Flush (); }                }            }        }

Output Result:

Simple example: Customizing a httphandler that handles pictures

Sometimes in the project we have to take pictures for certain operation, such as watermark, download, etc., for convenience and management we can customize a httphander to be responsible for these tasks.

Background:

  public class Imagehandler:ihttphandler {#region IHttpHandler members public bool IsReusable {        get {return true;} }///<summary>///IHttpHandler must be implemented after the implementation of the method//</summary>//<param name= "cont Ext ">httpcontext context </param> public void ProcessRequest (HttpContext context) {context.            Response.Clear (); Get the picture name var ImageName = context. request["ImageName"] = = null? "Anti-clockwise Wind": Context. request["ImageName"].            ToString (); Get the image ID, here is just demo, the actual project is not doing so var id = context. request["Id"] = = null? "The same": Context. request["Id"].            ToString (); Get the picture address var Stringfilepath = context. Server.MapPath (String.            Format ("~/image/{0}{1}.jpg", ImageName, id));            Declares a FileStream used to temporarily put the picture into the stream FileStream stream = new FileStream (Stringfilepath, FileMode.Open);          using (Stream)  {//The picture is placed in a byte array via the Getimagefromstream method byte[] Imagebytes = this.                Getimagefromstream (Stream,context); The context determines the file type to write to the client short time context.                Response.ContentType = "Image/jpeg"; The context writes the data in the imagebytes to the previous context.                Response.BinaryWrite (imagebytes); Stream.            Close (); }}////<summary>///Send the picture information in the stream into a byte array and return the array///</summary>//<param N Ame= "Stream" > File stream </param>//<param name= "context" > Contexts </param>///<returns></ Returns> Private byte[] Getimagefromstream (FileStream Stream, HttpContext context) {//via Strea            M gets the image image image = Image.fromstream (stream);            Add watermark image = Setwaterimage (image, context);            Get a MS object MemoryStream ms = new MemoryStream ();          Using (MS) {//Save picture to memory stream      Image.                Save (MS, Imageformat.jpeg); byte[] imagebytes = new Byte[ms.                Length]; Ms.                Position = 0; Read to imagebytes MS via memory stream.                Read (imagebytes, 0, imagebytes.length); Ms.                Close ();            return imagebytes return imagebytes; }}///<summary>//For the image watermark, this method does not care, just demo, so no transparency///next time add it//&LT;/SUMMARY&G        T <param name= "image" > Images to be watermarked </param>//<param name= "context" > Context </param>/// lt;returns></returns> Private Image Setwaterimage (image Image,httpcontext context) {Gr            Aphics graphics = graphics.fromimage (image); Image Waterimage = image.fromfile (context.            Server.MapPath ("~/image/counter-clockwise Wind 01.jpg"); In the lower right corner of the larger image, draw a watermark map on the line graphics. DrawImage (Waterimage, new Point {X = image). Size.width-waterimage.size.width,                   Y = image.            Size.height-waterimage.size.height});        return image; } #endregion}

Don't forget to configure it in Web. config, don't forget the verb and path properties, or you'll get an error.

    
So the front desk will be able to use
<asp:content id= "bodycontent" runat= "Server" contentplaceholderid= "maincontent" >    
Output results
This chapter summarizes
This chapter mainly introduces some concepts of memorystream, anomalies, structure, including how to use, how to solve some anomalies, etc., thank you have been support and encouragement, the text in the case of errors also please everyone haihan, late at night to write the text is not easy,
Also please a lot of attention, the next chapter will introduce BufferedStream, please look forward to!
C # Stream Chapter (v)--MemoryStream