Unpublished features of the. NET Framework [String]

This article conducts in-depth analysis and research on the String class of. NET Framework. It provides detailed information about the implementation of the String class, and describes the efficiency of various usage methods of the String.
Many of the information provided here cannot be found in MSDN or other books, because Microsoft has not published them.
Background:

String
Is a primitive type of. NET. CLR and JIT perform special processing and Optimization for some special classes. String is one of them. Others include: Other primitive types,
StringBuilder, Array, Type, Enum, Delegate, and some Reflection classes, such as MethodInfo.

In. NET
In 1.0, all objects allocated to the heap contain two things: an object header (4 bytes) and a pointer to a method table (4 bytes ). The object header provides five bits, one of which is
The GC retains the object and identifies whether it is a "reachable object" ("reachable object" is a term in the garbage collection algorithm, in short, it refers to the object being used by the application ). The remaining 27
An index, called syncindex, points to a table. This index can be used for multiple purposes: first, use "lock"
Keyword is used for thread synchronization. In addition, when the Object. GetHashCode () method is called, it is used as the default hash code (the inheritance class is not overwritten ).
Object. GetHashCode ). Although this index does not provide the best distribution feature for hash code, another requirement for hash code-a pair with the same value
It can meet the same hash code returned. Syncindex remains unchanged throughout the lifecycle of an object.
The actual memory usage of all objects can be calculated based on the object header. The formula is as follows (from the Rotor package sscli20020326 \ sscli \ clr \ src \ vm \ object. h ):
MT-> GetBaseSize () + (OBJECTTYPEREF-> GetSizeField () * MT-> GetComponentSize ())
For most objects, their sizes are fixed. The String class and the Array class (including the inheritance class of the Array) are only two variable-length objects. That is to say, after they are created, the object length can change.
String is a bit similar to OLE BSTRs-an array of Unicode characters starting with length data and ending with null characters. The following lists the three fields maintained internally by String:
 [NonSerialized] private int m_arrayLength;
[NonSerialized] private int m_stringLength;
[NonSerialized] private char m_firstChar;
Their meanings are shown in the following table:

M_arrayLength	This is the actual length (in characters) allocated to the string ). When you create a string, m_arrayLength is the same as the logical length of the string (m_stringLength. However, if a string is returned using StringBuilder, the actual length may be larger than the logical length.
M_stringLength	This is the logical Length of a String, which can be obtained through the String. Length attribute. Some high positions of m_stringLength are used as identifiers to optimize performance. So the maximum length of String is much smaller than UInt32.Max (32-bit operating system ). Some of these identifiers are used to indicate whether the String is a simple character (for example Plain ASCII), so that complex UNICODE algorithms are not used in sorting and comparison.
M_firstChar	This is the first character of the string. If it is a null string, it is a null character.

String always ends with an empty character, which enhances its interoperability with unmanaged code and traditional Win32API.
The total memory occupied by the String is 16 bytes + 2 bytes * number of characters + 2 bytes (the last null character ). As described in table 1, if StringBuilder is used to create a String, the actually allocated memory may be larger than String. Length.
Very efficient StringBuilder

StringBuilder is closely associated with String. Although StringBuilder is placed in the System. Text namespace, it is not a common class.
StringBuilder is specially processed by the JIT compiler during runtime. It is not easy to write a class that is as efficient as it is.
StringBuilder internally maintains a string variable --- m_StringValue, and allows direct modification to it. By default, the m_arrayLength field of m_StringValue is 16. The following are two internal fields maintained by StringBuilder:
Internal int m_MaxCapacity = 0;
Internal String m_StringValue = null;
Their meanings are shown in table 2:

M_MaxCapacity	The maximum capacity of StringBuilder, which specifies the maximum number of characters that can be placed in m_StringValue. The default value is Int32.MaxValue. You can specify a smaller capacity. m_MaxCapacity cannot be changed once it is specified.
M_StringValue	A string maintained by StringBuilder (Jeffrey Richter in Applied Microsoft. NET Framework In Programming, StringBuilder maintains a character array, which is easier to understand as a character array. However, from the source code of the Rotor package It is maintained as a string.

Let's take a look at the code of an Append method in StringBuilder (from the Rotor package sscli20020326 \ sscli \ clr \ src \ bcl \ system \ text \ stringbuilder. cs ):
Public StringBuilder Append (String value ){
// If the value being added is null, eat the null and return.
If (value = null ){
Return this;
}
Int tid;
// Hand inlining of GetThreadSafeString
String currentString = m_StringValue;
Tid = InternalGetCurrentThread ();
If (m_currentThread! = Tid)
CurrentString = String. GetStringForStringBuilder (currentString, currentString. Capacity );

Int currentLength = currentString. Length;
Int requiredLength = currentLength + value. Length;
If (NeedsAllocation (currentString, requiredLength )){
String newString = GetNewString (currentString, requiredLength );
NewString. AppendInPlace (value, currentLength );
ReplaceString (tid, newString );
} Else {
CurrentString. AppendInPlace (value, currentLength );
ReplaceString (tid, currentString );
}
Return this;
}
Private bool NeedsAllocation (String currentString, int requiredLength ){
// <= Accounts for the terminating 0 which we require on strings.
Return (currentString. ArrayLength <= requiredLength );
}
Private String GetNewString (String currentString, int requiredLength ){
Int newCapacity;
RequiredLength ++; // Include the terminating null.
If (requiredLength> m_MaxCapacity ){
Throw new ArgumentOutOfRangeException (Environment. GetResourceString ("ArgumentOutOfRange_NegativeCapacity "),
"RequiredLength ");
}
NewCapacity = (currentString. Capacity) * 2; // To force a predicatable growth of 160,320 etc. for testing purposes
If (newCapacity <requiredLength ){
NewCapacity = requiredLength;
}
If (newCapacity> m_MaxCapacity ){
NewCapacity = m_MaxCapacity;
}
If (newCapacity <= 0 ){
Throw new ArgumentOutOfRangeException (Environment. GetResourceString ("ArgumentOutOfRange_NegativeCapacity "));
}
Return String. GetStringForStringBuilder (currentString, newCapacity );
}
We can see from the code above that if the length of the constructed string exceeds the capacity of m_StringValue (m_arrayLength) when adding a new character, a new string will be created. The new string capacity is generally twice the original capacity (if it does not exceed m_MaxCapacity ).

Let's take a look at the source code of StringBuilder. ToString () (from the Rotor package sscli20020326 \ sscli \ clr \ src \ bcl \ system \ text \ stringbuilder. cs ):
Public override String ToString (){
String currentString = m_StringValue;
Int currentThread = m_currentThread;
If (currentThread! = 0 & currentThread! = InternalGetCurrentThread ()){
Return String. InternalCopy (currentString );
}

If (2 * currentString. Length) <currentString. ArrayLength ){
Return String. InternalCopy (currentString );
}

CurrentString. ClearPostNullChar ();
M_currentThread = 0;
Return currentString;
}
// Used by StringBuilder to avoid data upload uption
Internal static String InternalCopy (String str ){
Int length = str. Length;
String result = FastAllocateString (length );
FillStringEx (result, 0, str, length); // The underlying's String can changed length is StringBuilder
Return result;
}

When a string is returned using StringBuilder. ToString (), the actual string is returned (that is, the internal dimension of StringBuilder is returned by this method ).
Protected string field
(M_StringValue) instead of creating a new string ). If the capacity (ArrayLength) of StringBuilder exceeds twice the actual number of characters
StringBuilder. ToString () returns the concise version of a string. After StringBuilder is called.
After the ToString () method, modifying StringBuilder again will generate a copy action, which will create a new string; then the new string will be modified.
To the string that has been returned.
In addition to the memory used by the string, StringBuilder overwrites 16 bytes. However, the same StringBuilder object can be used multiple times to generate multiple strings, which leads to only one additional overhead.
We can see that using StringBuilder is very efficient.
Other performance skills:
1> when "+" is used to connect strings, such as "a" + "B" + "c", the compiler will call the Concat (a, B, c) method, in this way, a large number of additional string copies can be eliminated (here is a specific discussion ).
2> using StringBuilder is faster than string connection
Minimize garbage collection
Garbage Collector

Using StringBuilder to create strings can significantly reduce memory allocation. It should be noted that many tests show that it takes only a fraction of a second for a full garbage collection-almost none
Time detected by the algorithm. Therefore, it is undesirable to avoid garbage collection without analyzing the program. On the other hand, frequent garbage collection may damage the performance. Running the. NET program sometimes
It is difficult to determine whether this is caused by the JIT compiler, garbage collector, or other factors when there is uninterpretable stagnation. Previous programs (such as Windows
Shell, Word, and IE.
. NET uses three generations to recycle memory. This approach is based on the assumption that the more memory is allocated, the more frequently it is recycled. On the contrary, the less frequently it is recycled. The 0th generation is the youngest generation. After a 0th generation of garbage collection is completed, survivors will be moved to the 1st generation. Similarly, after the 1st generation of garbage collection is completed, survivors will be moved into 2 generations. Generally, garbage collection only occurs in generation 0.
According to Microsoft, the time required to perform a 0-generation garbage collection is equivalent to a page error-0-10 milliseconds; the 1-generation garbage collection takes 10-30 milliseconds, and the 2-generation garbage collection needs to look at the working environment. In addition, my own analysis shows that the number of garbage collection times in the 0 generation is 10-times higher than that in the 1 and 2 generations.

Applied Microsoft. NET Framework written by Jeffrey Richter
As mentioned in Programming: During CLR initialization, different thresholds will be selected for three generations, namely, 0 generations of 256Kb, 1 generation of 2 Mb, and 2 generations of 10 Mb. However
The CLI of the Rotor package is another result (the following code comes from the Rotor package sscli20020326 \ sscli \ clr \ src \ vm \ gcsmp. cpp): 0 generation 800Kb, 1 generation 1 Mb. Of course, these are not made public, and no written explanation will be made if the change is made. These initial values will be automatically adjusted based on the actual program memory allocation. If the 0-generation memory is rarely recycled (many are moved to the 1-generation memory), the 0-generation threshold will increase.
Void gc_heap: init_dynamic_data ()
{
Dynamic_data * dd = dynamic_data_of (0 );
Dd-> current_size = 0;
Dd-> promoted_size = 0;
Dd-> collection_count = 0;
Dd-> desired_allocation = 800*1024;
Dd-> new_allocation = dd-> desired_allocation;

Dd = dynamic_data_of (1 );
Dd-> current_size = 0;
Dd-> promoted_size = 0;
Dd-> collection_count = 0;
Dd-> desired_allocation = 1024*1024;
Dd-> new_allocation = dd-> desired_allocation;

// Dynamic data for large objects
Dd = dynamic_data_of (max_generation + 1 );
Dd-> current_size = 0;
Dd-> promoted_size = 0;
Dd-> collection_count = 0;
Dd-> desired_allocation = 1024*1024;
Dd-> new_allocation = dd-> desired_allocation;
}
Unsatisfactory String class methods
Many methods provided by the String class often generate unnecessary memory allocations, which increase the garbage collection frequency.

For example, the ToUpper () method or the ToLower method will generate a new string regardless of whether the string has changed. A more efficient method should be to perform operations on the original string.
Modify and return the original string. Similarly, the Substring () method returns a new string, even though it returns an entire string or a null string.

The. NET class library has a large number of hidden memory allocations, which are difficult to avoid. For example, when the value type (such as int and float) is formatted as a string (
String. Format or Console. WriteLine), a new hidden String is created. In this case, you can write your own code to format it.
You can control your code to prevent it from generating new strings. Obviously, writing such a piece of code is possible, but it is more difficult at the same time.
Other parts of the. NET class library expose similar
Low efficiency. In Widows
Forms class library, the Text attribute of Control almost always returns a new string. This may be understandable because attributes cannot be stored, so it must call Windows
The API function GetWindowText () to obtain the control value.
GDI + is the worst misuse of the garbage collector, because each call to MeasureText or DrawText creates a new string.
A better String method is GetHashCode (), which generates an integer for every character of the String. The generated values are evenly distributed to the int storage range.
Alternative to the String class
To reduce the pressure on the garbage collector, you can use the following solution to replace the String class:
1) stack-based character array
Char * array = stackalloc char [arraysize];
2) fixed-size String Structure
[StructLayout (LayoutKind. Explicit, Size = 514)]
Public unsafe struct Str255
{
[FieldOffset (0)] char start;
[FieldOffset (512)] byte length;

# Region Properties
Public int Length
{
Get {return length ;}
Set
{
Length = Convert. ToByte (value );
Fixed (char * p = & start) p [length] = '\ 0 ';
}
}

Public char this [int index]
{
Get
{
If (index <0 | index> = length)
Throw (new IndexOutOfRangeException ());
Fixed (char * p = & start)
Return p [index];
}
Set
{
If (index <0 | index> = length)
Throw (new IndexOutOfRangeException ());
Fixed (char * p = & start) p [index] = value;
}
}
# Endregion
}
Str255 is the value type assigned by a stack. It operates on strings without causing GC pressure. It can process 255 characters, including one Length byte.
References pointing to this structure can be directly passed to a Windows API call, because the structure starts with the first character of C-string. Of course, you can write some other methods to edit the string, but note when writing: ensure that the string ends with a null character (to be compatible with Windows ).

To allow other CLR methods to use it, you need to write a conversion program to convert this structure into a. NET string (like StringBuilder. ToString () method 1
Sample ). If you use it to create a. NET string, it is superior to StringBuilder in that Str255 requires less memory allocation.
3) "In-situ" string Modification
Public static unsafe void ToUpper (string str)
{
Fixed (char * pfixed = str)
For (char * p = pfixed; * p! = 0; p ++)
* P = char. ToUpper (* p );
}

The above example shows how to use the unsafe pointer to modify an immutable string. This method is very efficient. An example is to compare it with str. ToUpper,
Str. ToUpper () returns a new string, regardless of whether the content of the string has actually changed. The above Code only modifies the original string.
The fixed keyword is used to fix the string on the heap. This avoids moving it during garbage collection. It also allows the string address to be converted into a pointer pointing to the starting position of the string.
Since the string can actually be changed, why should we emphasize that the. NET string is "fixed? There are two reasons: first, there is no thread synchronization problem in the "fixed" string operation. Second, the "fixed" string allows multiple references to point to the same string object, this reduces the number of strings in the system and memory overhead.
By modifying the internal field m_arrayLength of the string object, the length of the string can also be changed. However, this is very dangerous. The future implementation of CLR or non-Windows systems may change the implicit implementation of strings.
When using this method to modify strings, note that the high characters of m_stringLength contain some signs and they cannot be changed. Some of the symbols are related to the content of the string, for example, whether the character string contains only seven ASCII characters.
Eliminate range check
To index an array or string, you must perform a range check. According to Microsoft, the compiler has made some special Optimizations to Improve the Performance of traversing arrays or strings. Let's first compare the following three methods of traversing strings to see which one is faster.
1)
Int hash = 0;
For (int I = 0; I <s. Length; I ++)
{
Hash + = s [I];
}
2)
Int hash = 0;
Int length = s. length;
For (int I = 0; I <length; I ++)
{
Hash + = s [I];
}
3)
Foreach (char ch in s)
{
Hash + = s [I];
}
Surprisingly, in the current JIT compiler, the first example is the fastest, and the third is the slowest. In the next version of the JIT compiler, the third example will have the same speed as the first example.
Is
What is the first example faster than the second one? This is because the compiler knows for (int I = 0; I <s. length; I ++)
This mode (only for strings and arrays ). The length of a string is fixed (its length is fixed), and the compiler will store its length, so that no method is called every time (because JIT Encoding
The interpreter can automatically embed non-virtual methods that only contain simple flow control and IL commands that do not exceed 32 bytes. Here, the compiler embeds the reference of string length ).

In addition, the compiler also removes the range check for s [I] for each loop, Because I has been restricted between 0 and string length in the for condition. In the second example, a word is replaced by an integer.
Character string length, the compiler will not consider it as for (int I = 0; I <s. length; I ++)
Mode (I is not assumed to be between 0 and the string length), so a range check is executed for each loop. This is why the second example is slower than the first one.
The following method is faster than the first example (I did not confirm it ).
Fixed (char * pfixed = s)
{
For (char * p = pfixed; * p; p ++)
Hash + = * p ++;
}
Efficient string switch statements
C # supports switch statements on strings. It adopts a very effective mechanism.
First, let's look at the following code:
Using System;
Public class abc
{
Static void Main (string [] args)
{
If (args. Length> 0)
{
Switch (args [0])
{
Case "":
Console. WriteLine ("");
Break;
Default:
Break;
}
}
}
}
This is the code IL
. Method private hidebysig static void Main (string [] args) cel managed
{
. Entrypoint
// Code size 52 (0x34)
. Maxstack 2
. Locals init (string V_0)
IL_0000: ldarg.0
IL_0001: ldlen
IL_0002: conv. i4
IL_0003: ldc. i4.0
IL_0004: ble. s IL_0033
IL_0006: ldstr ""
IL_000b: leave. s IL_000d
IL_000d: ldarg.0
IL_000e: ldc. i4.0
IL_000f: ldelem. ref
IL_0010: dup
IL_0011: stloc.0
IL_0012: brfalse. s IL_0031
IL_0014: ldloc.0
IL_0015: call string [mscorlib] System. String: IsInterned (string)
IL_001a: stloc.0
IL_001b: ldloc.0
IL_001c: ldstr ""
IL_0021: beq. s IL_0025
IL_0023: br. s IL_0031
IL_0025: ldstr ""
IL_002a: call void [mscorlib] System. Console: WriteLine (string)
IL_002f: br. s IL_0033
IL_0031: br. s IL_0033
IL_0033: ret
} // End of method abc: Main

CLR automatically maintains an intern pool)
It contains a single instance of each unique String constant declared in the program, and the String
. Now let's take a look at the above IL code. First, IL calls the IsInterned method and passes the specified string args [0] In the switch statement. If
IsInterned returns null, indicating that args [0] does not match any character string of case. Instead, the default code is executed. If IsInterned
Args [0] is found in the "detention pool", which returns a reference to the string object in the hash table, and then compares the reference with the address of the string specified by each case statement. Compare URLs
All characters in the string are much faster, and the code can quickly determine which case statement should be executed.
The above only refers to the case where a small number of cases exist.
If the number is large, the compiler will generate a hash table. The load factor of this hash table is 0.5, and the initial capacity is twice the number of cases (in fact, the hash table ratio is about 1/3, because the hash table will
Maintain the ratio of elements to buckets at 0.72. 0.72 is the optimal balance between speed and memory, which is derived from Microsoft's performance tests ). The string of the case statement will be added to the hash table.
, There is no difference between other comparison steps.