Python:joblib. Parallel method in a class ' Typeerror:can ' t pickle instancementhod objects ' __python

Source: Internet
Author: User
use Joblib. Parallel parallel processing of data is the presence of Typeerror:can ' t pickle Instancementhod objects Error

The first is the implementation of a function call:

Import joblib
import NumPy as NP

def load_data (K, T): Return
    k+t

jobs = 8

Ts = Np.repeat (Np.array ([1] ,  jobs, axis=0)
Ks = Np.repeat (Np.array ([1]),  jobs, axis=0) with

joblib. Parallel (N_jobs=jobs) as Parallel:
    outpout = Parallel (joblib.delayed (Load_data) (k, s)
                        for K, s in Zip (Ks, Ts))
Print (output)

This code is no problem output [5, 5, 5, 5, 5, 5, 5, 5]

However, if you are working in a class, as follows:

Import joblib
import NumPy as NP

class Testparallel:
    def load_data (self, K, T): Return
        k+t

    del Run ( Self):
        jobs = 8

        Ts = Np.repeat (Np.array ([1]),  jobs, axis=0)
        Ks = Np.repeat (Np.array ([1]),  jobs, axis=0) with

        joblib. Parallel (N_jobs=jobs) as Parallel:
            outpout = Parallel (joblib.delayed (Load_data) (k, s)
                                for K, s in Zip (Ks, Ts))

This will appear: Typeerror:can ' t pickle Instancementhod objects error. The bug appears in the way it was found, and finally, in a blog post, the first approach solves the problem of using joblib and multiprocessing to implement parallel processing based on pickling and explains why.

In order to pickle a object,object must be a global variable, this allows Unpickle to pass (Ps:pickle access to structured data). If you are in a class, the output parameters are of the following types:

[(' Self ', 0], (' Self ', 1), (' Self ', 2), (' Self ', 3), (' Self ', 4), (' Self ', 5), (' Self ', 6), (' Self ', 7), (' Self ', 8), (' SE LF ', 9)]

It is therefore necessary to define the function outside the class.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.