PHP array for millions of data to exclude duplicate data implementation code _php tips

If you get a UID list, the number is more than million lines, the format is as follows:

Copy Code code as follows:

10001000
10001001
10001002
......
10001000
......
10001111

In fact, using the characteristics of PHP arrays, good row weight, we first look at the definition of the PHP array: PHP in the array is actually an ordered map. A mapping is a type that associates values to the keys. This type is optimized in many ways, so you can think of it as a real array, or a list (vector), a hash list (an implementation of a map), a dictionary, a collection, a stack, a queue, and more possibilities. The value of an array element can also be another array. The tree structure and multidimensional array are also allowed.

In the PHP array, the key (keys) is also called an index, which is unique, and we are able to use this feature for row weight, sample code as follows:

Copy Code code as follows:

<?php
Defines an array that is used to store the results after the weight is placed
$result = Array ();
Read UID list file
$fp = fopen (' test.txt ', ' R ');

while (!feof ($FP))
{
$uid = fgets ($FP);
$uid = Trim ($uid);
$uid = Trim ($uid, "\ r");
$uid = Trim ($uid, "\ n");

if ($uid = = ")
{
Continue
}
Use UID as key to see if the value exists
if (Empty ($result [$uid]))
{
$result [$uid] = 1;
}
}

Fclose ($FP);

Save the results to a file
$content = ';
foreach ($result as $k => $v)
{
$content. = $k. " \ n ";
}
$fp = fopen (' Result.txt ', ' W ');
Fwrite ($fp, $content);
Fclose ($FP);
?>

More than 20 lines of code, can be more than millions of data to row weight, efficiency is good, very practical. Mobile phone number, email, you can also use this way to row weight.

Also, this method can also be used for two files to row heavy work, if you have two UID list files, the format and the UID list above the same, the sample program is as follows:

Copy Code code as follows:

<?php
Defines an array that is used to store the results after the weight is placed
$result = Array ();
Read the first UID list file and put it in the $result_1
$fp = fopen (' test_1.txt ', ' R ');
while (!feof ($FP))
{
$uid = fgets ($FP);
$uid = Trim ($uid);
$uid = Trim ($uid, "\ r");
$uid = Trim ($uid, "\ n");
if ($uid = = ")
{
Continue
}
Writes the $result with the UID key, overwriting if there is a duplicate
$result [$uid] = 1;
}
Fclose ($FP);
Reads the second UID list file and makes the row-weight operation
$fp = fopen (' test_2.txt ', ' R ');
while (!feof ($FP))
{
$uid = fgets ($FP);
$uid = Trim ($uid);
$uid = Trim ($uid, "\ r");
$uid = Trim ($uid, "\ n");
if ($uid = = ")
{
Continue
}
Use UID as key to see if the value exists
if (Empty ($result [$uid]))
{
$result [$uid] = 1;
}
}
Fclose ($FP);
$result saved in the row after the result, you can output to the file, the code omitted
?>

To think about it, it is easy to see that using this feature of the array can also solve more problems in our work.