Tf.name_scope () and Tf.variable_scope ()

Tf.name_scope () and Tf.variable_scope () are two scopes, typically paired with the functions tf.variable () and tf.get_variable () that create/Invoke variables.

tf.name_scopeAnd variable_scope also a role as a context manager, the following manager: It means that what you do under this manager will be managed by this manager.

I. Use of Name_scope and Variable_scope:
Name_scope and Variable_scope are mainly due to the need for variable sharing.

Variable sharing mainly involves two functions: tf.variable () and tf.get_variable (); That is, you must use the tf.get_variable () function in the scope of the Tf.variable_scope. Here use Tf.get_variable () instead of TF. Variable () is because the former has a variable checking mechanism that detects if the existing variable is set to a shared variable, and if the existing variable is not set to a shared variable, TensorFlow runs to the second variable with the same name, and then the error is made.

Two ways to create variables. If you use TF. Variable () Creates a new variable every time. But most of the time we want to reuse some variables, so we use the get_variable (), it will go to search for variable names, it is directly used, no more new. The Name field. Since the variable name is used, the concept of the Name field is involved. It's hard to tell the names of variables by different domains, after all, so we can give all the variables a different name. This is why there is a concept of scope. Name_scope acts on the action, Variable_scope can affect the variables under the domain by setting the reuse flag and initializing it, because to achieve the effect of variable sharing, it is necessary to use tf.get_ at the scope of the Tf.variable_scope () Variable () generates and extracts variables in this way. Not like TF. Variable () generates a new variable every time, tf.get_variable () if it encounters a variable that already has a name, it simply extracts the same name variable, if the variable with no name exists.

Example:

With Tf.variable_scope ('V1', reuse=True): A1= Tf.get_variable (name='A1', Shape=[1], Initializer=tf.constant_initializer (1)) A2= TF. Variable (Tf.random_normal (shape=[2,3], mean=0, stddev=1), name='A2') with Tf.variable_scope ('V2', reuse=True): A3= Tf.get_variable (name='A1', Shape=[1],initializer=tf.constant_initializer (1)) A4= TF. Variable (Tf.random_normal (shape=[2,3], mean=0, stddev=1), name='A2') with TF. Session () as Sess:sess.run (Tf.initialize_all_variables ())Print(A1.name)Print(A2.name)Print(A3.name)Print(A4.name)

Output:

v1/a1:0v1_14/a2:0v2/a1:0v2_2/a2:0  does not resuse this parameter in Tf.name_scope () and cannot do this.

Two. TensorFlow the name scope and variable scope difference
There are two types of scopes in TF
Named domain (name scope), created by Tf.name_scope or tf.op_scope;
Variable field (variable scope), created by Tf.variable_scope or tf.variable_op_scope;
Both scopes, for the use of TF. Variable () Creates a variable with the same effect, preceded by the variable name, plus the domain name.

For variables created by tf.get_variable (), only the variable scope name is prepended to the variable name, and name scope is not prefixed. such as print (V1.name) # var1:0

Example:

With Tf.name_scope ("my_name_scope"):    = tf.get_variable ("  Var1", [1], dtype=tf.float32)     = tf. Variable (1, name="var2", dtype=tf.float32)    = tf.add (v1, v2)     Print (v1.name)     Print (v2.name)      Print (A.name)

Output:

var1:0my_name_scope/var2:0my_name_scope/add:0

Summary: Name_scope will not be prefixed with the tf.get_variable variable, but will be used as a TF. The variable prefix.

With Tf.variable_scope ("my_variable_scope"):    = tf.get_variable ("  var1", [1], dtype=tf.float32)    = tf. Variable (1, name="var2", dtype=tf.float32)    = tf.add (v1, v2)     Print (v1.name)      Print (v2.name)     Print

Output:

my_variable_scope/var1:0my_variable_scope/var2:0

my_variable_scope/add:0

Summary: In the scope of Variable_scope, tf.get_variable () and TF. Variable () all add the scope_name prefix. Therefore, in the scope of Tf.variable_scope, the variable can be shared by using the variable that has already been created by Get_variable ().

Original link: 80099822

Tf.name_scope () and Tf.variable_scope () (EXT)