Introduction to the fastest way to insert strings in Python Programming

In MapReduce distributed computing, mapper inputs come from multiple different data sources. In common, the first column of each record is the id column of the key, and CER needs to be different based on the data source,. Since the data to the CER stage can no longer distinguish the files from, the general method is to map a TAG for the data record. For ease of use, I used to hit the TAG to the second column of data (the first column is the id column as the key of reduce/join), so there is such a mapper function:

Def mapper1 (line ):
L = line. split ('t', 1)
Return "% st % s" % (l [0], 'tag', l [1])

The input is given as follows:

S = "3001 VALUE"

The result of mapper1 (s) is:

S = "3001 tag value"

This is a straightforward function that the subconscious thought of, but today I suddenly become confused and fall into a strange circle of "Is this the fastest. So I thought, what other methods are there? Oh, the formatted expression can be expressed using the string + operation:

Def mapper2 (line ):
L = line. split ('t', 1)
Return l [0] + 'T' + 'tag' + 'T' + l [1]

The above deliberately writes 'T' separately, because the TAG is generally passed in as a variable. Also, join is faster than +, so we can also do this:

Def mapper3 (line ):
L = line. split ('t', 1)
L. insert (1, 'tag ')
Return 'T'. join (l)

Split may consume extra space, so use find:

Def mapper4 (line ):
Pos = line. find ('t ')
Return "% st % s" % (line [0: pos], 'tag', line [pos + 1:])

A little abnormal. The first number is an integer. Replace it with an integer output:

Def mapper5 (line ):
Pos = line. find ('t ')
Pid = long (line [0: pos])
Return "% dt % st % s" % (pid, 'tag', line [pos + 1:])

In another way, split can be replaced with partition:

Def mapper6 (line ):
(H, s, t) = line. partition ('t ')
Return "% st % s" % (h, 'tag', t)

Or simply ticky a little bit, replace the first found tab with replace:

Def mapper7 (line ):
Return line. replace ('t', 'T' + 'tag' + 'T', 1)

Wow, let's take a look. There are quite a few options available in the past, and I believe this will not list all methods. Here, there are several limited algorithms. Which of the following is the fastest? What is the fastest speed than the slowest one?

First paste the timing method:

For I in range (1, 8 ):
F = 'er er % d (s) '% I
Su = "from _ main _ import mapper % d, s" % I
Print f, ':', timeit. Timer (f, setup = su). timeit ()

The answer is as follows:

Mapper1 (s): 1.32489800453
Mapper2 (s): 1.2933549881
Mapper3 (s): 1.65229916573
Mapper4 (s): 1.22059297562
Mapper5 (s): 2.60358095169
Mapper6 (s): 0.956777095795
Mapper7 (s): 0.726199865341

The final winner is mapper7 (the replace method of tricky). The slowest is mapper5 (the id-to-number method of egg pain). The slowest time consumption is about 3.6 times that of mapper7. The earliest thought of mapper1 method ranks 5th among the 7 methods! The time consumed is 1.8 times the fastest method. Considering that mapper is simple enough, it makes a little sense to overhead nearly doubled.