Explore the sharing of variables among Python multi-process programming threads

1. Questions:

A group of students posted the following code, asked why the list last print is a null value?

From multiprocessing import Process, managerimport OS Manager = Manager () vip_list = [] #vip_list = Manager.list () def TESTF UNC (cc):  vip_list.append (cc)  print ' Process ID: ', os.getpid () if __name__ = = ' __main__ ':  threads = []   For LL in range:    t = Process (Target=testfunc, args= (ll,))    T.daemon = True    threads.append (t) for   I in Range (len (threads)):    Threads[i].start () for   J in range (len (threads)):    threads[j].join ()   print " ------------------------"  print" Process ID: ', os.getpid ()  print vip_list

In fact, if you understand the multi-threaded model of Python, GIL problem, and then understand the multi-threading, multi-process principle, the above questions are not difficult to answer, but if you do not know it is OK, run the above code you know what the problem is.

Python aa.pyprocess id:632process id:635process id:637process id:633process id:636process id:634process Id:639proce SS Id:638process id:641process id:640------------------------process id:619[]

Turn on the 6th line comment and you'll see the following result:

Process id:32074process id:32073process id:32072process id:32078process id:32076process id:32071process Id:32077pro Cess id:32079process id:32075process id:32080------------------------process id:32066[3, 2, 1, 7, 5, 0, 6, 8, 4, 9]

2. Python multi-process shared variables in several ways:
(1) Shared Memory:
Data can is stored in a shared memory map using the Value or Array. For example, the following code

Http://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes

From multiprocessing import Process, Value, Array def f (N, a):  N.value = 3.1415927 for  i in range (Len (a)):    a[i ] =-a[i] if __name__ = = ' __main__ ':  num = Value (' d ', 0.0)  arr = Array (' I ', Range (Ten))   p = Process (Target=f, a rgs= (num, arr))  P.start ()  p.join ()   print num.value  print arr[:]

Results:

3.1415927[0,-1,-2,-3,-4,-5,-6,-7,-8,-9]

(2) Server process:

A manager object returned by manager () controls a server process which holds Python objects and allows other processes to manipulate them using proxies.
A Manager returned by manager () would support types list, dict, Namespace, Lock, Rlock, Semaphore, Boundedsemaphore, Condit Ion, Event, Queue, Value and Array.
The code is shown at the beginning of the example.

Http://docs.python.org/2/library/multiprocessing.html#managers
3, multi-process problems far more than this: the synchronization of data

Look at the simple code: a simple counter:

From multiprocessing import Process, managerimport OS Manager = Manager () sum = Manager. Value (' tmp ', 0) def testFunc (cc):  sum.value + = cc if __name__ = = ' __main__ ':  threads = [] for   ll in range (100) :    t = Process (Target=testfunc, args= (1,))    T.daemon = True    threads.append (t) for   I in range (Len (threads ):    Threads[i].start () for   J in range (len (threads)):    threads[j].join ()   print "------------------ ------"  print" Process ID: ', os.getpid ()  print Sum.value

Results:

------------------------Process id:1737897

Perhaps you would ask: WTF? In fact, this problem in the multi-threaded era exists, but in the multi-process era and a replay of the Cup: lock!

From multiprocessing import Process, manager, Lockimport OS lock = Lock () Manager = Manager () sum = Manager. Value (' tmp ', 0)  def testFunc (CC, lock): With  Lock:    sum.value + = cc  if __name__ = = ' __main__ ':  threads = [] for   ll in range:    t = Process (Target=testfunc, args= (1, lock))    T.daemon = True    threads.a Ppend (t) for   I in range (len (threads)):    Threads[i].start () for   J in range (len (threads)):    threads[j ].join ()   print "------------------------"  print ' Process ID: ', os.getpid ()  print Sum.value

How does this piece of code perform? Run to see, or increase the number of cycles to try ...
4, the final proposal:

Note that usually sharing data between processes is the best choice, because of all the synchronization issues; An approach involving actors exchanging messages are usually seen as a better choice. See also Python Documentation:as mentioned above, when doing concurrent programming it's usually best to avoid using Sha Red state as far as possible. Particularly true when using multiple processes. However, if you really does need to use some GKFX data then multiprocessing provides a couple of ways for doing so.

5, Refer:

Http://stackoverflow.com/questions/14124588/python-multiprocessing-shared-memory

http://eli.thegreenplace.net/2012/01/04/shared-counter-with-pythons-multiprocessing/

Http://docs.python.org/2/library/multiprocessing.html#multiprocessing.sharedctypes.synchronized

