Word2vec Multithreading (TensorFlow)

Workers = []

For _ in Xrange (opts.concurrent_steps):

t = Threading. Thread (Target=self._train_thread_body)

T.start ()

Workers.append (t)

??

??

the word2vec.py uses multithreading

It is generally considered that Python multithreading is a single thread due to Python's design GPL memory is not ready to secure

but this is because it calls the C + + code Internally, so it can be multithreaded .

??

while Word2vec 's skipgramoperator internal class design solves the problem of multi-threaded access violation is the lock

Mutex mu_;

Random::P hiloxrandom philox_ guarded_by (mu_);

Random::simplephilox rng_ guarded_by (mu_);

Int32 Current_epoch_ guarded_by (mu_) =-1;

Int64 total_words_processed_ guarded_by (mu_) = 0;

Int32 Example_pos_ guarded_by (mu_);

Int32 Label_pos_ guarded_by (mu_);

Int32 label_limit_ guarded_by (mu_)

??

It is thought that the operator operation is a single-threaded parallel execution due to the lock

The batch calculations that follow are parallel

def _train_thread_body (self):

Initial_epoch, = Self._session.run ([Self._epoch])

While True:

_, Epoch = Self._session.run ([Self._train, Self._epoch])

If epoch! = Initial_epoch:

BreaK

??

(words, counts, Words_per_epoch, Self._epoch, self._words, examples,

Labels) = Word2vec.skipgram (Filename=opts.train_data,

Batch_size=opts.batch_size,

Window_size=opts.window_size,

Min_count=opts.min_count,

Subsample=opts.subsample

??

??

??

The threading lock only affects? PythonCode. If your thread is waiting for disk I/O or if it is calling C functions (e.g. via math Library) You can ignore the GIL.

You are able to use the async pattern to get around threading limits. Can You supply more information on what the your program actually does?

I have issues with the technical accuracy of the video linked. David Beazley has the do many well respected talks on the GIL at various pycons. You can find them on pyvideo.org.

??

From

??

??

Word2vec Multithreading (TensorFlow)