Automated deployment based on fabric and HG

Automating deployment

Fabric is an easy-to-use automated deployment tool that is less powerful than Puppet,saltstack, but wins in Python and is free to install the server.

Of course, you have to say that Docker is better. I agree, however, I often use FreeBSD, and there are some 32-bit low-distribution systems that are not suitable for Docker. let alone virtual machines.

The purpose of automated deployment is primarily to simplify the hassle of manual deployment, including initial installation deployment and code-Modified update deployment. The initial deployment is primarily the installation of the underlying environment, initializing the database, and so on. Updating the deployment is more cumbersome and requires modifying the underlying environment configuration, changing the database structure, and so on. In contrast, code publishing and updating are the simplest, with a version control tool.

I am more accustomed to use HG to do code version management, of course, here to replace Hg to Git can also, but I just like Hg, you bite me ah.

Fabric

The website of fabric is fabfile.org, the basic usage can see the official document, here is only a few common usage to explain briefly.

Installation is simple, as long as the local use PIP install fabric, remote as long as there is SSH service, do not need to install additional things, this is more convenient than puppet and Saltstack.

It's also very simple to use, and the most important thing is that the deployment script is Python, which is more my taste than the puppet Ruby.

The default script file name is: fabfile.py, of course you can use a different name, but it is inconvenient to use, like make to use the default makefile file name is convenient.

The command to execute the default script is: Fab < function name >[<[parameter name:] value;]

The function name is any function defined in fabfile.py (when using the @task adorner you can only use a function that has already been decorated), or you can take a parameter. For a specific host or user, you can also add a role adorner to the function.

Examples of running local commands are as follows:

from fabric.api import localdef deploy_1():    local(“hg commit”)    local(“hg push”)fab deploy_1

Examples of running remote commands are as follows:

from fabric.api import rundef deploy_2():    run(“hg pull”)    run(“hg update”)fab deploy_2 -H hostname_or_ip

Toggle Current directory:

with lcd(“path”) # 本地目录with cd(“path”) # 远端

Error handling: Any operation with a return value of 0 will cause an exception unless ...

with settings(warn_only=True)

and use the. Failed property of the command to determine the result of the execution

Env is used to save related configuration environment, such as SSH key file: Key_filename, and host Name list: hosts, role definitions: roledefs, etc.

Use of roles:

env.roledefs={‘role1’:[“[email protected]:port1”, “[email protected]:port2”],    ‘role2’:[“[email protected]:port3”]}@roles(“role1”)def deploy_3():    pass

Deployment with fabric and HG

Take a simple Python web app as an example.

A complete manual initial deployment roughly includes the following (assuming that the server-side system has the necessary software installed, such as HG, Python, virtualenv, database, webserver, where database and webserver have been configured, Separate virtualenv have been created), etc.:

• Install the necessary dependency package Pip Install-r in the VIRTUALENV environment requirements.txt
• Release the code to be deployed via HG [local] HG push Ssh://[email Protected]/path; [Remote] HG update
• Initializing the database
• Start (via Gunicorn, Supervisord, etc.)

The following are some of the things that are involved in deploying the code after it has been modified:

• Update dependent Packages
• Update code
• Update database structure
• Restart Service

Considering the possible need to deploy separately to the test environment and the formal environment, you need to consider the following issues:

• The test environment and the formal environment involve different configurations (such as databases that are not connected, configuration of ports, or even static files pointing to different paths)
• Must be a test pass in a test environment before it can be updated in a formal environment
• Formal environments have stricter authority management (development departments cannot deploy directly to formal environments, or even access to formal environment configuration information)

Thus, we need at least two code warehouses: one is the development code base (REPO_DEV), including the test configuration, and the other is the formal configuration warehouse (Repo_prod).

So, the basic fabfile.py deploy_dev function is roughly the following:

local("hg push repo_dev")with cd("/target"):    run("hg pull repo_dev")    run("hg update")    run("workon venv") # 切换到指定的virtualenv    run("pip install -r requirements.txt")    # 初始化数据库或更新数据库结构    sudo("supervisorctl restart xxx") # gunicorn不能用supervisor重启，因为停止需要等待一段时间，建议用kill信号进行软重启

The remote server specified by the Repo_dev can be configured for the test host testhost by the role configuration, and the test section can be deployed to the formal machine prodhost after testing on the test host.

with cd("/prodconf"):    run("hg pull repo_prod")with cd("/prod"):    run("hg pull testhost") # 从测试主机上更新代码    run("hg update rev") # 注意，这里要更新测试过的指定版本    run("cp /prodconf/config .") # 使用正式配置替换测试配置    run("workon venv")    run("pip install -r requirements.txt")    # 更新数据库结构    sudo("supervisorctl restart xxx")

This deployment function uses another user role, so long as the test department has permission to control the role, the development department will fail if it inadvertently runs the deployment function.

Automated deployment based on fabric and HG