use Joblib. Parallel parallel processing of data is the presence of Typeerror:can ' t pickle Instancementhod objects Error
The first is the implementation of a function call:
Import joblib
import NumPy as NP
def load_data (K, T): Return
k+t
jobs = 8
Ts = Np.repeat (Np.array ([1] , jobs, axis=0)
Ks = Np.repeat (Np.array ([1]), jobs, axis=0) with
joblib. Parallel (N_jobs=jobs) as Parallel:
outpout = Parallel (joblib.delayed (Load_data) (k, s)
for K, s in Zip (Ks, Ts))
Print (output)
This code is no problem output [5, 5, 5, 5, 5, 5, 5, 5]
However, if you are working in a class, as follows:
Import joblib
import NumPy as NP
class Testparallel:
def load_data (self, K, T): Return
k+t
del Run ( Self):
jobs = 8
Ts = Np.repeat (Np.array ([1]), jobs, axis=0)
Ks = Np.repeat (Np.array ([1]), jobs, axis=0) with
joblib. Parallel (N_jobs=jobs) as Parallel:
outpout = Parallel (joblib.delayed (Load_data) (k, s)
for K, s in Zip (Ks, Ts))
This will appear: Typeerror:can ' t pickle Instancementhod objects error. The bug appears in the way it was found, and finally, in a blog post, the first approach solves the problem of using joblib and multiprocessing to implement parallel processing based on pickling and explains why.
In order to pickle a object,object must be a global variable, this allows Unpickle to pass (Ps:pickle access to structured data). If you are in a class, the output parameters are of the following types:
[(' Self ', 0], (' Self ', 1), (' Self ', 2), (' Self ', 3), (' Self ', 4), (' Self ', 5), (' Self ', 6), (' Self ', 7), (' Self ', 8), (' SE LF ', 9)]
It is therefore necessary to define the function outside the class.