Use of python Tqdm module and pythontqdm Module

Use of python Tqdm module and pythontqdm Module

Tqdm is a fast and scalable Python progress bar. You can add a progress prompt in the Python long loop. You only need to encapsulate any iterator tqdm (iterator ).

My system is a Windows environment. First install python and then pip.

Pip installation:

Create a get-pip.py file in the python root directory with the following content:

Https://bootstrap.pypa.io/get-pip.py

Then, enter python in the CMD window:

Output:

python -m pip install -U pip 

Because Tqdm requires pip version 9.0, You need to manually install pip9.0
Http://pypi.python.org/pypi/pip

Download installer 9.0

Decompress the package and enter python setup. py install in the CMD window.

Then you can install Tqdm,

pip install tqdm 

Install the latest development version

pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm 

Finally, let's see how to use it? Https://pypi.python.org/pypi/tqdm

Basic usage:

from tqdm import tqdm for i in tqdm(range(10000)):    sleep(0.01) 

Of course, besides tqdm and trange, they are used in the same way.

for i in trange(100):     sleep(0.1) 

As long as the list is passed in, you can:

pbar = tqdm(["a", "b", "c", "d"]) for char in pbar:   pbar.set_description("Processing %s" % char) 

You can also manually control updates.

with tqdm(total=100) as pbar:   for i in range(10):     pbar.update(10) 

You can also do this:

pbar = tqdm(total=100) for i in range(10):   pbar.update(10) pbar.close() 

Tqdm usage in Shell

Count the number of rows of all python scripts:

$ time find . -name '*.py' -exec cat \{} \; | wc -l 857365  real  0m3.458s user  0m0.274s sys   0m3.325s  $ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 857366it [00:03, 246471.31it/s] 857365  real  0m3.585s user  0m0.862s sys   0m3.358s 

Parameters used:

$ find . -name '*.py' -exec cat \{} \; |   tqdm --unit loc --unit_scale --total 857366 >> /dev/null 100%|███████████████████████████████████| 857K/857K [00:04<00:00, 246Kloc/s] 

Back up a directory:

$ 7z a -bd -r backup.7z docs/ | grep Compressing |   tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 100%|███████████████████████████████▉| 8014/8014 [01:37<00:00, 82.29files/s] 

By looking at the Demo code, we can find that the core functions are tqdm and trange. To analyze the tqdm functions at the code level, the first is init. py.

__all__ = ['tqdm', 'tqdm_gui', 'trange', 'tgrange', 'tqdm_pandas',       'tqdm_notebook', 'tnrange', 'main', 'TqdmKeyError', 'TqdmTypeError',       '__version__'] 

Trace _ tqdm. py to see the declaration of the tqdm class. The first is initialization.

def __init__(self, iterable=None, desc=None, total=None, leave=True,          file=sys.stderr, ncols=None, mininterval=0.1,          maxinterval=10.0, miniters=None, ascii=None, disable=False,          unit='it', unit_scale=False, dynamic_ncols=False,          smoothing=0.3, bar_format=None, initial=0, position=None,          gui=False, **kwargs): 

Parameters iterable: iterable, optional Iterable to decorate with a progressbar. progress bar that can be iterated. Leave blank to manually manage the updates. left blank to manually manage updates ?? Desc: str, optional Prefix for the progressbar. progress bar description total: int, optional The number of expected iterations. if unspecified, len (iterable) is used if possible. as a last resort, only basic progress statistics are displayed (no ETA, no progressbar ). if gui is True and this parameter needs subsequent updating, specify an initial arbitrary large positive integer, e.g. int (9e9 ). the expected number of iterations. The default value is None. Possible iterations. If the gui is set to True, subsequent updates are required. You must specify a positive integer with an initial Random value, for example, int (9e9) leave: bool, optional If [default: True], keeps all traces of the progressbar upon termination of iteration. keep the trace of the progress bar. Simply put, the final form of the progress bar will be retained. The default value is True file: io. textIOWrapper or io. stringIO, optional Specifies where to output the progress messages [default: sys. stderr]. uses file. write (str) and file. flush () methods. output ncols: int, Optional The width of the entire output message. if specified, dynamically resizes the progressbar to stay within this bound. if unspecified, attempts to use environment width. the fallback is a meter width of 10 and no limit for the counter and statistics. if 0, will not print any meter (only stats ). the width of the entire output message. If specified, the dynamic adjustment progress stays at this boundary. If not specified, try to use the width of the environment. If it is 0, nothing is printed (statistics only ). Mininterval: float, optional Minimum progress update interval, in seconds [default: 0.1]. Minimum progress update interval, in seconds (default: 0.1 ). Maxinterval: float, optional Maximum progress update interval, in seconds [default: 10.0]. Maximum progress update interval, in seconds (default: 10 ). Miniters: int, optional Minimum progress update interval, in iterations. if specified, will set mininterval to 0. minimum progress update cycle ascii: bool, optional If unspecified or False, use unicode (smooth blocks) to fill the meter. the fallback is to use ASCII characters 1-9 #. if this parameter is not set, the default value is unicode encoding: disable: bool, optional Whether to disable the entire progressbar wrapper [default: False]. whether to disable the entire progress bar packaging (if True, progress Not Displayed) unit: str, optional String that will be used to define the unit of each iteration [default: it]. will be used to define the strings of each unit ??? Unit_scale: bool, optional If set, the number of iterations will be discounted CED/scaled automatically and a metric prefix following the International System of Units standard will be added (kilo, mega, etc .) [default: False]. if this parameter is set, the number of iterations is automatically prefixed by 10, or. The default value is false dynamic_ncols: bool, optional If set, constantly alters ncols to the environment (allowing for window resizes) [default: False]. constantly changing the ncols environment, allowing Adjust the window size smoothing: float, optional Exponential moving average smoothing factor for speed estimates (ignored in GUI mode ). ranges from 0 (average speed) to 1 (current/instantaneous speed) [default: 0.3]. bar_format: str, optional Specify a custom bar string formatting. may impact performance. if unspecified, will use '{l_bar} {bar} {r_bar}', where l_bar is '{desc} {percentage: 3.0f }%|' and r_bar is' | {N_fmt}/{total_fmt} [{elapsed_str} <{remaining_str}, {rate_fmt}] 'possible vars: bar, n, n_fmt, total, total_fmt, percentage, rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. customize bar string formatting... The format of {l_bar} {bar} {r_bar} is used by default. The format is The same as initial: int, optional The initial counter value. useful when restarting a progress bar [default: 0]. initial counter value. the default value is 0 position: int, optional Specify the line offset to print this bar (starting from 0) Automatic if unspecified. useful to manage multiple bars at once (eg, from threads ). specify the offset. This function can be used in multiple entries: gui: bool, optional WARNING: internal parameter-do not use. use Tqdm_gui (...) Instead. If set, will attempt to use matplotlib animations for a graphical output [default: False]. Internal parameter... Returns out: decorated iterator. Return as an iterator

In fact, you don't need to analyze more code. Let's take a look at several examples: (examples on the official website)

7zx. py compression progress bar

# -*- coding: utf-8 -*- """Usage:  7zx.py [--help | options] <zipfiles>... Options:  -h, --help   Print this help and exit  -v, --version Print version and exit  -c, --compressed    Use compressed (instead of uncompressed) file sizes  -s, --silent  Do not print one row per zip file  -y, --yes   Assume yes to all queries (for extraction)  -D=<level>, --debug=<level>          Print various types of debugging information. Choices:              CRITICAL|FATAL              ERROR              WARN(ING)              [default: INFO]              DEBUG              NOTSET  -d, --debug-trace   Print lots of debugging information (-D NOTSET) """ from __future__ import print_function from docopt import docopt import logging as log import subprocess import re from tqdm import tqdm import pty import os import io __author__ = "Casper da Costa-Luis <casper.dcl@physics.org>" __licence__ = "MPLv2.0" __version__ = "0.2.0" __license__ = __licence__   RE_SCN = re.compile("([0-9]+)\s+([0-9]+)\s+(.*)$", flags=re.M)   def main():   args = docopt(__doc__, version=__version__)   if args.pop('--debug-trace', False):     args['--debug'] = "NOTSET"   log.basicConfig(level=getattr(log, args['--debug'], log.INFO),           format='%(levelname)s: %(message)s')   log.debug(args)    # Get compressed sizes   zips = {}   for fn in args['<zipfiles>']:     info = subprocess.check_output(["7z", "l", fn]).strip()     finfo = RE_SCN.findall(info)      # builtin test: last line should be total sizes     log.debug(finfo)     totals = map(int, finfo[-1][:2])     # log.debug(totals)     for s in range(2):       assert(sum(map(int, (inf[s] for inf in finfo[:-1]))) == totals[s])     fcomp = dict((n, int(c if args['--compressed'] else u))            for (u, c, n) in finfo[:-1])     # log.debug(fcomp)     # zips : {'zipname' : {'filename' : int(size)}}     zips[fn] = fcomp    # Extract   cmd7zx = ["7z", "x", "-bd"]   if args['--yes']:     cmd7zx += ["-y"]   log.info("Extracting from {:d} file(s)".format(len(zips)))   with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()),        unit="B", unit_scale=True) as tall:     for fn, fcomp in zips.items():       md, sd = pty.openpty()       ex = subprocess.Popen(cmd7zx + [fn],                  bufsize=1,                  stdout=md, # subprocess.PIPE,                  stderr=subprocess.STDOUT)       os.close(sd)       with io.open(md, mode="rU", buffering=1) as m:         with tqdm(total=sum(fcomp.values()), disable=len(zips) < 2,              leave=False, unit="B", unit_scale=True) as t:           while True:             try:               l_raw = m.readline()             except IOError:               break             l = l_raw.strip()             if l.startswith("Extracting"):               exname = l.lstrip("Extracting").lstrip()               s = fcomp.get(exname, 0) # 0 is likely folders               t.update(s)               tall.update(s)             elif l:               if not any(l.startswith(i) for i in                     ("7-Zip ",                     "p7zip Version ",                     "Everything is Ok",                     "Folders: ",                     "Files: ",                     "Size: ",                     "Compressed: ")):                 if l.startswith("Processing archive: "):                   if not args['--silent']:                     t.write(t.format_interval(                       t.start_t - tall.start_t) + ' ' +                       l.lstrip("Processing archive: "))                 else:                   t.write(l)       ex.wait()   main.__doc__ = __doc__   if __name__ == "__main__":   main() 

Tqdm_wget.py

"""An example of wrapping manual tqdm updates for urllib reporthook. # urllib.urlretrieve documentation > If present, the hook function will be called once > on establishment of the network connection and once after each block read > thereafter. The hook will be passed three arguments; a count of blocks > transferred so far, a block size in bytes, and the total size of the file. Usage:   tqdm_wget.py [options] Options: -h, --help   Print this help message and exit -u URL, --url URL : string, optional   The url to fetch.   [default: http://www.doc.ic.ac.uk/~cod11/matryoshka.zip] -o FILE, --output FILE : string, optional   The local file path in which to save the url [default: /dev/null]. """  import urllib from tqdm import tqdm from docopt import docopt   def my_hook(t):   """   Wraps tqdm instance. Don't forget to close() or __exit__()   the tqdm instance once you're done with it (easiest using `with` syntax).   Example   -------   >>> with tqdm(...) as t:   ...   reporthook = my_hook(t)   ...   urllib.urlretrieve(..., reporthook=reporthook)   """   last_b = [0]    def inner(b=1, bsize=1, tsize=None):     """     b : int, optional       Number of blocks just transferred [default: 1].     bsize : int, optional       Size of each block (in tqdm units) [default: 1].     tsize : int, optional       Total size (in tqdm units). If [default: None] remains unchanged.     """     if tsize is not None:       t.total = tsize     t.update((b - last_b[0]) * bsize)     last_b[0] = b   return inner   opts = docopt(__doc__)  eg_link = opts['--url'] eg_file = eg_link.replace('/', ' ').split()[-1] with tqdm(unit='B', unit_scale=True, leave=True, miniters=1,      desc=eg_file) as t: # all optional kwargs   urllib.urlretrieve(eg_link, filename=opts['--output'],             reporthook=my_hook(t), data=None) 

Examples. py

""" # Simple tqdm examples and profiling # Benchmark for i in _range(int(1e8)):   pass # Basic demo import tqdm for i in tqdm.trange(int(1e8)):   pass # Some decorations import tqdm for i in tqdm.trange(int(1e8), miniters=int(1e6), ascii=True,            desc="cool", dynamic_ncols=True):   pass # Nested bars from tqdm import trange for i in trange(10):   for j in trange(int(1e7), leave=False, unit_scale=True):     pass # Experimental GUI demo import tqdm for i in tqdm.tgrange(int(1e8)):   pass # Comparison to https://code.google.com/p/python-progressbar/ try:   from progressbar.progressbar import ProgressBar except ImportError:   pass else:   for i in ProgressBar()(_range(int(1e8))):     pass # Dynamic miniters benchmark from tqdm import trange for i in trange(int(1e8), miniters=None, mininterval=0.1, smoothing=0):   pass # Fixed miniters benchmark from tqdm import trange for i in trange(int(1e8), miniters=4500000, mininterval=0.1, smoothing=0):   pass """  from time import sleep from timeit import timeit import re  # Simple demo from tqdm import trange for i in trange(16, leave=True):   sleep(0.1)  # Profiling/overhead tests stmts = filter(None, re.split(r'\n\s*#.*?\n', __doc__)) for s in stmts:   print(s.replace('import tqdm\n', ''))   print(timeit(stmt='try:\n\t_range = xrange'            '\nexcept:\n\t_range = range\n' + s, number=1),      'seconds') 

Pandas_progress_apply.py

import pandas as pd import numpy as np from tqdm import tqdm  df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))  # Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` # (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.) tqdm.pandas(desc="my bar!")  # Now you can use `progress_apply` instead of `apply` # and `progress_map` instead of `map` df.progress_apply(lambda x: x**2) # can also groupby: # df.groupby(0).progress_apply(lambda x: x**2)   # -- Source code for `tqdm_pandas` (really simple!) # def tqdm_pandas(t): #  from pandas.core.frame import DataFrame #  def inner(df, func, *args, **kwargs): #    t.total = groups.size // len(groups) #    def wrapper(*args, **kwargs): #      t.update(1) #      return func(*args, **kwargs) #    result = df.apply(wrapper, *args, **kwargs) #    t.close() #    return result #  DataFrame.progress_apply = inner 

Referencing tqdm is not mandatory as a dependency:

Include_no_requirements.py

# How to import tqdm without enforcing it as a dependency try:   from tqdm import tqdm except ImportError:   def tqdm(*args, **kwargs):     if args:       return args[0]     return kwargs.get('iterable', None) 

Refer:
Https://github.com/tqdm/tqdm/tree/master/examples
Https://pypi.python.org/pypi/tqdm
Https://github.com/tqdm/tqdm

The above is all the content of this article. I hope it will be helpful for your learning and support for helping customers.