Ignore case-insensitive. Net dirty word filtering algorithms

Duplicate content is no longer repeated. For more information, see http://www.cnblogs.com/xingd/archive/2008/02/01/4251800.html.

In addition to case-insensitive, the performance of other aspects has also been improved, mainly through the following class, stringsegement, to achieve case-insensitive comparison and gethashcode, while avoiding substring calls.

Public   Class Stringsegment
{
Private   String Original;
Private   Int Offset =   0 ;
Private   Int Length =   0 ;

Public stringsegment ( string S)
{< br> This . original = S;
This . length = original. length;
}

Public VoidSlice (IntOffset,IntLength)
{
This. Offset=Offset;
This. Length=Length;
}

Public   Override   Bool Equals ( Object OBJ)
{
Stringsegment SG = OBJ As Stringsegment;
Return SG ! =   Null   && SG. Length =   This . Length &&   String . Compare ( This . Original, This . Offset, SG. Original, SG. offset, This . Length, True ) =   0 ;
}

Public Override IntGethashcode ()
{
//Call Char. tolower and calculate hash code
}
}

The implementation of gethashcode fully references the implementation of string, so it won't be repeated. It takes a long time.

In addition, the data for feature judgment is planned to be 64 k x 4 bytes, that is, 4 bytes feature data for each char.AlgorithmImprovement: as long as you re-plan in these 4 bytes, the current definition is as follows:

[Structlayout (layoutkind. explicit, size =   32 )]
Internal   Struct Fastcheckflag
{
[Fieldoffset ( 0 )]
Public   Byte Occur; // 1st ~ 8th char occurrence
[Fieldoffset ( 8 )]
Public   Byte Length; // 2 ~ 9 word length begin with this char
[Fieldoffset ( 16 )]
Public   Byte Rlength; // 2 ~ 9 word length end with this char (reverse length );
[Fieldoffset ( 24 )]
Public   Bool Single; // Single char bad words
[Fieldoffset ( 25 )]
Public   Bool Last; // Last occurrence
[Fieldoffset ( 26 )]
Public   Byte Occurparity; // 9th ~ Occurrence parity flag
[Fieldoffset ( 28 )]
Public   Byte Lengthparity; // 10 ~ Length parity flag
[Fieldoffset ( 30 )]
Public   Byte Rlengthparity; // 10 ~ Rlength parity flag
}

In addition, when initializing data, take the case of characters and process them at the same time, for example:

If (Word. Length =   1 )
{
Fastcheck [ Char . Tolower (word [ 0 ])]. Single =   True ;
Fastcheck [ Char . Toupper (word [ 0 ])]. Single =   True ;
}

CompleteCodeI will not post it. After actual tests, the efficiency is still very good. Encourage readers to follow my articlesArticleWrite the specific code and implement the replacement. If you have any questions, please leave a comment. I will reply one by one, but the code is free.

Index:
The first article I saw contains dirty code and test code. It has two implementations: string and RegEx. Http://www.cnblogs.com/goody9807/archive/2006/09/12/502094.html
My first article, case sensitive, two bitarray optimizations, mainly introduces ideas. Http://www.cnblogs.com/xingd/archive/2007/09/26/906013.html
The implementation I have compiled is case sensitive. The first article is the version we are using. Http://www.cnblogs.com/xingd/archive/2008/01/23/1050443.html
The first improvement, case sensitive, extends to 8 characters for judgment. Http://www.cnblogs.com/xingd/archive/2008/01/31/1060425.html
The second improvement is case-sensitive, with the length judgment and terminator added. Http://www.cnblogs.com/xingd/archive/2008/02/01/1061800.html
The final version is case-insensitive and removes the price of substring.