Using Python to set tmpfs to accelerate the project tutorial

This article describes how to use Python to set tmpfs to accelerate a project. The following describes how to use a Python script to store tmpfs programs stored in the memory to a local hard disk, it takes a lot of time for anyone who needs to perform all tests on my current project. Since there are 26 GB of idle memory, why not let it take advantage of the waste heat? Tmpfs can accelerate the test execution efficiency by saving the file system in large memory.

However, tmpfs only saves the results in the memory, so you must write your own scripts to write the results back to the disk for retained. In addition, these scripts must be well written and executed; otherwise, part or whole of the work will be lost.

A common method is to work directly in the tmpfs folder and back up the work results to a folder on the disk. When your machine starts, you recover the tmpfs folder from that backup folder. After startup, use cron to synchronize the tmpfs folder and disk folder.

I found this setting a little complicated and error-prone. At startup, or for crom, I never really believe it. Now I use a much simpler setting, and I don't need cron at all.

Running a single test on my machine, using the IDE and publishing it on the web server, usually has a reasonable performance, and it takes a lot of time to run all the tests.

I found that the best thing is to set up a workspace on the disk to synchronize with tmpfs running all tests under the/dev/shmand directory. This makes my settings more or less unchanged and eliminates the possibility of loose work, just because I am too spam in correct settings.

The results of performance improvement are reasonable:

$ nosetests && run_tests.py........................................................................................................................................................................................................................................................----------------------------------------------------------------------Ran 248 tests in 107.070s OK........................................................................................................................................................................................................................................................----------------------------------------------------------------------Ran 248 tests in 19.423s OK

It is now 5 times faster than the original one.

Setting using python is very simple:

#!/bin/bash -e WORK=src/pyLOG=$(pwd)/test.logTARGET=$(hg root)SHADOW=/dev/shm/shadow/$TARGET date > $LOGmkdir -p $SHADOW cd $SHADOWrsync --update --delete --exclude=".*" --exclude=ENV --archive $TARGET ./.. if [ ! -d ENV ]then  virtualenv ENVfi. ENV/bin/activate cd $WORKpython setup.py develop >> $LOGnosetests $* | tee -a $LOGexit ${PIPESTATUS[0]}

I just need to synchronize to the/dev/shm directory again, set the test environment (virtualenv and python setup. PY) and run the test (nosetests ).

You can still run a single test using the command line in the tmpfs folder. You can also kick this away from your IDE, but you will lose your testing and debugging capabilities. As I said before, I don't need this now.

I hope this work ing on tmpfs can help you build a faster development environment without having to suffer from all the script troubles.