How to use the "idle Time" of deep learning hardware to dig mine

Without a GPU, deep learning is not possible. But when you do not optimize anything, how to make all the teraflops are fully utilized.

With the recent spike in bitcoin prices, you can consider using these unused resources to make a profit. It's not hard, all you have to do is set up a wallet, choose what to dig, build a miner's software and run it. Google searches for "how to start digging on the GPU", and there are many articles detailing how to dig mine.

How to make it more convenient to dig mine.

If I suddenly want to put all the horsepower into the new deep learning problem, how to make mine easy, automatic and without interference. The ideal solution is: Some background thing constantly check the utilization of the GPU, and when no one uses it, start the miner. However, when TensorFlow or Pytorch or other tools want to compress some numbers, the monitor must stop digging as soon as possible in order to release the compute kernel to a useful task.

Although the problem is simple, I haven't found anything like it, so I wrote a GPU monitor on my own. It's versatile, not only for digging, but you can also try to do something else with it.

Necessary Conditions

My project is called Gpu_mon, the source code can find here: Https://github.com/Shmuma/gpu_mon. It's written in Python 3 and doesn't depend on anything but the standard library, but it should run on a Linux system, so if you use Windows,gpu_mon on the deep learning box it won't work.

The overall logic is exactly the same as described above: Gpu_mon periodically checks the GPU, and if no one uses it, it runs the program you specified in the config file. If a process opens a GPU device, the running mining will be interrupted to free up resources. So, after you've set everything up and started the display, all you have to do is use your GPU box as usual, and the overlap between the miners and the deep learning takes only a few seconds.

To obtain a list of processes that access the GPU device (assumed/Dev/nvidia *), use the Fuser command-line tool. In distribution based on Debian, such as Ubuntu or Debian, it is provided by the PSMISC package. If I am not mistaken, it will be included in the basic system installation, so there is no need to install anything. If your system does not provide fuser, please install it.

Configuration

The entire project is configured in a separate configuration file that has a ini-file format and is expected to be located in the home directory of the 〜/. config/gpu_mon.conf file. The sample configuration file, as shown below, can also be found in the project source file.

A configuration file can contain four parts:

1. Default global configuration [defaults]. There is only one option, which specifies how often the GPU device checks for use. By default, a check is performed every 10 seconds and data is collected on all the GPU in the system.

2. The GPU configuration can be specified by one or more names with Gpu-prefix parts. You can describe the installed GPU card group by specifying its ID (an integer in the/Dev/nvidiax device file). For each GPU group, you can provide a list of programs that do not preempt the GPU's miners process. This is useful for tools such as NVIDIA-SMI, which can access GPU devices, but should not cause mining to stop.

3. The miner Process configuration section can be described by one or more parts of the name with the process-prefix. For each section, you can specify the command line of the miner to run, starting the miner's directory name, and hoping that the miners will be limited by the name of the GPU identity and log file.

4. The TTY Monitoring section allows you to enable optional pseudo terminal monitoring to preempt the miners. This feature is disabled by default.

Here's a sample of the configuration file I used on box with 2 GPU cards.

[ Defaults]; How frequently perform GPU open and TTY checks interval_seconds=10; Configuration of GPUs to monitor for external program access. It could be several such sections [Gpu-all]; List of comma-separated GPU indices or all to handle all available GPUs gpus=all; comma-separated List of programs which can access GPU and should be ignored ignore_programs=nvidia-smi; Program which'll is started on the GPU during idle time [process-0] dir=/tmp Cmd=/var/bin/miner; List of GPU indices or all to handle all available GPUs. If not all, cuda_visible_devices'll be set gpus=0 log=/var/log/miner-0.log [process-1] dir=/tmp Cmd=/var/bin/miner; List of GPU indices or all to handle all available GPUs. If not all, cuda_visible_devices'll be set Gpus=1 log=/var/log/miner-1.log; Configuration of TTY monitoring [TTY] Enabled=false 

This configuration allows fine-grained control over GPU use by assigning individual processes to each card in the system. So if I run a deep learning process that takes up only the first GPU (by exporting Cuda_visible_device = 0), the miner process that starts on the second GPU will continue to work. However, if my optimization turns on two GPU cards, two miners will be replaced and resources will be released.

As I said, everything is transparent. So you shouldn't spend a lot of time starting and stopping miners, just think about how many GPU to allocate for your depth learning when you run TF or Pytorch.

Here are some additional information that I can provide.

Auto Start Gpu_mon

To make the Gpu_mon completely free, we need to make sure that it starts in the background when the system starts. There are many ways to start a process, but my favorite solution is to use Supervisord to check the running process and restart the process in the event of a crash. To start Gpu_mon, you first need to set up Supervisord. If it is not installed, you can put the configuration file into the/etc/supervisor/conf.d/gpu_mon.conf. Here's the configuration I used:

[Program:gpu_mon]
Command=/usr/bin/python3 <path_to_cloned_gpu_mon>/gpu_mon/gpu_mon.py
user=<your USERNAME>
environment=home=<your_home_path>,user=<your_username>
autostart=true
autorestart=true

After the installation is complete, you will need to reboot the Supervisord and check that Gpu_mon is started by running the following command: Supervisorctl status Gpu_mon, which should return the following:

root@gpu:/etc/supervisor/conf.d# supervisorctl status Gpu_mon
gpu_mon RUNNING pid 1526, uptime, 18:14:27

Multi-user access GPU

If Gpu_mon is running as a user, but the depth learning software can be run as another user or by multiple users, Gpu_mon may not be able to preempt the miners. This occurs because of the security restrictions of the fuser command, which does not show other user processes opening device files. If you are experiencing this, but still want to use Gpu_mon, you have two options:

* Run Gpu_mon as root. I do not recommend this approach because it is always a bad idea to start something as root.

* Add the SUID bit to the fuser binary file. In Ubuntu distribution, this can be done by running the chmod + s/bin/fuser command, and it effectively allows Fuser to view all users accessing the file, just like the suid bit, binary is also started with file owner credentials. But you still need root privileges to do this.

how to dig mine. Digging what.

Thanks to the boom in cryptographic money now, there are many options available. My personal favorite is based on equihash currencies, such as Zcash or Komodo, which can be dug with a miner. I use the modified version EWBF miner, 10% faster than the original version.

As I said, Gpu_mon doesn't dig itself, it's just a GPU access tracker. So you can run any cuda you want to optimize the miners.

can you make money.

can make money. But the complexity of the cryptographic currency is growing, and don't expect to dig millions of dollars with a single 1080 card. But on the whole, the mining proceeds less the mining electricity bill, finally can be profitable. So go dig it. There is a website that can tell you the amount of mining income, given the complexity, the rate of exchange, and the strength of your calculations. The URL is https://whattomine.com/.