芝麻HTTP：Ansible擴充，芝麻ansible擴充

芝麻HTTP：Ansible擴充，芝麻ansible擴充
Ansible簡介

 

Ansible是由Python開發的一個營運工具，因為工作需要接觸到Ansible，經常會整合一些東西到Ansible，所以對Ansible的瞭解越來越多。

那Ansible到底是什麼呢？在我的理解中，原來需要登入到伺服器上，然後執行一堆命令才能完成一些操作。而Ansible就是來代替我們去執行那些命令。並且可以通過Ansible控制多台機器，在機器上進行任務的編排和執行，在Ansible中稱為playbook。

那Ansible是如何做到的呢？簡單點說，就是Ansible將我們要執行的命令產生一個指令碼，然後通過sftp將指令碼上傳到要執行命令的伺服器上，然後在通過ssh協議，執行這個指令碼並將執行結果返回。

那Ansible具體是怎麼做到的呢？下面從模組和外掛程式來看一下Ansible是如何完成一個模組的執行

PS：下面的分析都是在對Ansible有一些具體使用經驗之後，通過閱讀原始碼進一步得出的執行結論，所以希望在看本文時，是建立在對Ansible有一定瞭解的基礎上，最起碼對於Ansible的一些概念有瞭解，例如inventory，module，playbooks等

 

Ansible模組

 

模組是Ansible執行的最小單位，可以是由Python編寫，也可以是Shell編寫，也可以是由其他語言編寫。模組中定義了具體的操作步驟以及實際使用過程中所需要的參數

執行的指令碼就是根據模組產生一個可執行檔指令碼。

那Ansible是怎麼樣將這個指令碼上傳到伺服器上，然後執行擷取結果的呢？

 

Ansible外掛程式 connection外掛程式

串連外掛程式，根據指定的ssh參數串連指定的伺服器，並切提供實際執行命令的介面

shell外掛程式

命令外掛程式，根據sh類型，來產生用於connection時要執行的命令

strategy外掛程式

執行策略外掛程式，預設情況下是線性外掛程式，就是一個任務接著一個任務的向下執行，此外掛程式將任務丟到執行器去執行。

action外掛程式

動作外掛程式，實質就是任務模組的所有動作，如果ansible的模組沒有特別編寫的action外掛程式，預設情況下是normal或者async（這兩個根據模組是否async來選擇），normal和async中定義的就是模組的執行步驟。例如，本地建立臨時檔案，上傳臨時檔案，執行指令碼，刪除指令碼等等，如果想在所有的模組中增加一些特殊步驟，可以通過增加action外掛程式的方式來擴充。

 

Ansible執行模組流程

 

在task_queue_manager.py中找到run中

擴充Ansible執行個體

 

執行節點Python環境擴充

實際需求中，我們擴充的一些Ansible模組需要使用三方庫，但每個節點中安裝這些庫有些不易於管理。ansible執行模組的實質就是在節點的python環境下執行產生的指令碼，所以我們採取的方案是，指定節點上的Python環境，將區域網路內一個python環境作為nfs共用。通過擴充Action外掛程式，增加節點上掛載nfs，待執行結束後再將節點上的nfs卸載。具體實施步驟如下：

擴充代碼：

重寫ActionBase的execute_module方法

# execute_modulefrom __future__ import (absolute_import, division, print_function)__metaclass__ = typeimport jsonimport pipesfrom ansible.compat.six import text_type, iteritemsfrom ansible import constants as Cfrom ansible.errors import AnsibleErrorfrom ansible.release import __version__try:    from __main__ import displayexcept ImportError:    from ansible.utils.display import Display    display = Display()class MagicStackBase(object):    def _mount_nfs(self, ansible_nfs_src, ansible_nfs_dest):        cmd = ['mount',ansible_nfs_src, ansible_nfs_dest]        cmd = [pipes.quote(c) for c in cmd]        cmd = ' '.join(cmd)        result = self._low_level_execute_command(cmd=cmd, sudoable=True)        return result    def _umount_nfs(self, ansible_nfs_dest):        cmd = ['umount', ansible_nfs_dest]        cmd = [pipes.quote(c) for c in cmd]        cmd = ' '.join(cmd)        result = self._low_level_execute_command(cmd=cmd, sudoable=True)        return result    def _execute_module(self, module_name=None, module_args=None, tmp=None, task_vars=None, persist_files=False, delete_remote_tmp=True):        '''        Transfer and run a module along with its arguments.        '''        # display.v(task_vars)        if task_vars is None:            task_vars = dict()        # if a module name was not specified for this execution, use        # the action from the task        if module_name is None:            module_name = self._task.action        if module_args is None:            module_args = self._task.args        # set check mode in the module arguments, if required        if self._play_context.check_mode:            if not self._supports_check_mode:                raise AnsibleError("check mode is not supported for this operation")            module_args['_ansible_check_mode'] = True        else:            module_args['_ansible_check_mode'] = False        # Get the connection user for permission checks        remote_user = task_vars.get('ansible_ssh_user') or self._play_context.remote_user        # set no log in the module arguments, if required        module_args['_ansible_no_log'] = self._play_context.no_log or C.DEFAULT_NO_TARGET_SYSLOG        # set debug in the module arguments, if required        module_args['_ansible_debug'] = C.DEFAULT_DEBUG        # let module know we are in diff mode        module_args['_ansible_diff'] = self._play_context.diff        # let module know our verbosity        module_args['_ansible_verbosity'] = display.verbosity        # give the module information about the ansible version        module_args['_ansible_version'] = __version__        # set the syslog facility to be used in the module        module_args['_ansible_syslog_facility'] = task_vars.get('ansible_syslog_facility', C.DEFAULT_SYSLOG_FACILITY)        # let module know about filesystems that selinux treats specially        module_args['_ansible_selinux_special_fs'] = C.DEFAULT_SELINUX_SPECIAL_FS        (module_style, shebang, module_data) = self._configure_module(module_name=module_name, module_args=module_args, task_vars=task_vars)        if not shebang:            raise AnsibleError("module (%s) is missing interpreter line" % module_name)        # get nfs info for mount python packages        ansible_nfs_src = task_vars.get("ansible_nfs_src", None)        ansible_nfs_dest = task_vars.get("ansible_nfs_dest", None)        # a remote tmp path may be necessary and not already created        remote_module_path = None        args_file_path = None        if not tmp and self._late_needs_tmp_path(tmp, module_style):            tmp = self._make_tmp_path(remote_user)        if tmp:            remote_module_filename = self._connection._shell.get_remote_filename(module_name)            remote_module_path = self._connection._shell.join_path(tmp, remote_module_filename)            if module_style in ['old', 'non_native_want_json']:                # we'll also need a temp file to hold our module arguments                args_file_path = self._connection._shell.join_path(tmp, 'args')        if remote_module_path or module_style != 'new':            display.debug("transferring module to remote")            self._transfer_data(remote_module_path, module_data)            if module_style == 'old':                # we need to dump the module args to a k=v string in a file on                # the remote system, which can be read and parsed by the module                args_data = ""                for k,v in iteritems(module_args):                    args_data += '%s=%s ' % (k, pipes.quote(text_type(v)))                self._transfer_data(args_file_path, args_data)            elif module_style == 'non_native_want_json':                self._transfer_data(args_file_path, json.dumps(module_args))            display.debug("done transferring module to remote")        environment_string = self._compute_environment_string()        remote_files = None        if args_file_path:            remote_files = tmp, remote_module_path, args_file_path        elif remote_module_path:            remote_files = tmp, remote_module_path        # Fix permissions of the tmp path and tmp files.  This should be        # called after all files have been transferred.        if remote_files:            self._fixup_perms2(remote_files, remote_user)        # mount nfs        if ansible_nfs_src and ansible_nfs_dest:            result = self._mount_nfs(ansible_nfs_src, ansible_nfs_dest)            if result['rc'] != 0:                raise AnsibleError("mount nfs failed!!! {0}".format(result['stderr']))        cmd = ""        in_data = None        if self._connection.has_pipelining and self._play_context.pipelining and not C.DEFAULT_KEEP_REMOTE_FILES and module_style == 'new':            in_data = module_data        else:            if remote_module_path:                cmd = remote_module_path        rm_tmp = None        if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp:            if not self._play_context.become or self._play_context.become_user == 'root':                # not sudoing or sudoing to root, so can cleanup files in the same step                rm_tmp = tmp        cmd = self._connection._shell.build_module_command(environment_string, shebang, cmd, arg_path=args_file_path, rm_tmp=rm_tmp)        cmd = cmd.strip()        sudoable = True        if module_name == "accelerate":            # always run the accelerate module as the user            # specified in the play, not the sudo_user            sudoable = False        res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)        # umount nfs        if ansible_nfs_src and ansible_nfs_dest:            result = self._umount_nfs(ansible_nfs_dest)            if result['rc'] != 0:                raise AnsibleError("umount nfs failed!!! {0}".format(result['stderr']))        if tmp and "tmp" in tmp and not C.DEFAULT_KEEP_REMOTE_FILES and not persist_files and delete_remote_tmp:            if self._play_context.become and self._play_context.become_user != 'root':                # not sudoing to root, so maybe can't delete files as that other user                # have to clean up temp files as original user in a second step                tmp_rm_cmd = self._connection._shell.remove(tmp, recurse=True)                tmp_rm_res = self._low_level_execute_command(tmp_rm_cmd, sudoable=False)                tmp_rm_data = self._parse_returned_data(tmp_rm_res)                if tmp_rm_data.get('rc', 0) != 0:                    display.warning('Error deleting remote temporary files (rc: {0}, stderr: {1})'.format(tmp_rm_res.get('rc'), tmp_rm_res.get('stderr', 'No error string available.')))        # parse the main result        data = self._parse_returned_data(res)        # pre-split stdout into lines, if stdout is in the data and there        # isn't already a stdout_lines value there        if 'stdout' in data and 'stdout_lines' not in data:            data['stdout_lines'] = data.get('stdout', u'').splitlines()        display.debug("done with _execute_module (%s, %s)" % (module_name, module_args))        return data

整合到normal.py和async.py中，記住要將這兩個外掛程式在ansible.cfg中進行配置

from __future__ import (absolute_import, division, print_function)__metaclass__ = type from ansible.plugins.action import ActionBasefrom ansible.utils.vars import merge_hash from common.ansible_plugins import MagicStackBase  class ActionModule(MagicStackBase, ActionBase):     def run(self, tmp=None, task_vars=None):        if task_vars is None:            task_vars = dict()         results = super(ActionModule, self).run(tmp, task_vars)        # remove as modules might hide due to nolog        del results['invocation']['module_args']        results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))        # Remove special fields from the result, which can only be set        # internally by the executor engine. We do this only here in        # the 'normal' action, as other action plugins may set this.        #        # We don't want modules to determine that running the module fires        # notify handlers.  That's for the playbook to decide.        for field in ('_ansible_notify',):            if field in results:                results.pop(field)         return results

• 配置ansible.cfg，將擴充的外掛程式指定為ansible需要的action外掛程式
• 重寫外掛程式方法，重點是execute_module
• 執行命令中需要指定Python環境，將需要的參數添加進去nfs掛載和卸載的參數

  ansible 51 -m mysql_db -a "state=dump name=all target=/tmp/test.sql" -i hosts -u root -v -e "ansible_nfs_src=172.16.30.170:/web/proxy_env/lib64/python2.7/site-packages ansible_nfs_dest=/root/.pyenv/versions/2.7.10/lib/python2.7/site-packages ansible_python_interpreter=/root/.pyenv/versions/2.7.10/bin/python"