Ironfort---A bastion machine based on Django and WebSocket

There are many webssh, and there are many Django-based Web services, and there are more SSH accesses in Python using Paramiko. But by combining the three with gevent, it is rare to implement a bastion machine accessed through a browser. This article will briefly introduce the Ironfort fortress machine I developed, which is detailed in the video tutorials of my official website liujiangblog.com.

I. Overview of the Fortress machine

Baidu Encyclopedia: Fortress Machine, in a specific network environment, in order to ensure that the network and data from external and internal users from the intrusion and destruction, and the use of various technical means in real time to collect and monitor the network environment of each component of the system status, security incidents, network activities, so that centralized alarm, timely processing and audit accountability.

For a medium-sized company, when the number of users and staff, the company has a large number of servers, its server account management difficulty will increase sharply, refer to the following image:

There are inevitably many problems, such as:

• User, host, account number too much, heavy workload, management confusion;
• Each person's authority and the user account can not be systematically managed, the level is unclear;
• The user directly master the Host account password;
• Passwords may be cross-used;
• The employee may also be able to use the company's account number;
• Internal staff can skip the firewall, directly using the account in the computer room access;
• Internal personnel before leaving a trojan or dark door, a period of time after the outbreak;
• Access to personnel records, past operations without logs and audits, lack of post-mortem means;
• Other risks

At the beginning of the operation, the Company may take Excel forms and other tools, the use of manual management, by the ' rule of man ' and the moral level of restraint, but when the company's volume gradually increased, this approach will inevitably be eliminated, so there is the concept of Fortress machine, as shown:

This architecture brings the following benefits:

• Users can not directly access the remote host, but need to jump through the fortress machine;
• Users no longer master the remote Host account password, only access to the Fort machine account;
• Restrict user's ability to change password after login to remote host, do not allow modification;
• The user of the fortress machine, the user of the remote host, the user password, the user privilege and so on are all unified centralized management, the substantial savings labor cost;
• All operations performed by the user after logging into the fortress machine will be recorded and used for later behavioral audits;
• Because there is no remote Host account password, even into the room can not directly connected to the host;
• can also achieve batch command execution, file distribution and other ancillary functions;
• Other benefits.

The core concept of a bastion machine is that users no longer have access to the account password, and the user's behavior is recorded for auditing. Bastion machine is mainly aimed at the internal network and internal personnel, for people with strong mobility, large volume, high industry risk of enterprise demand is particularly strong, such as the financial industry.

The fortress machine already has the commercial product, most takes the hardware server as the carrier carries on the sale, the price hundreds of thousands of varies. There are also open source solutions, but some of these scenarios are not browser-based, the interface is not friendly, logging is difficult, some based on tornado, and only simple command execution function, and the company is using Django; more is inconsistent with the company's needs, need two times development, maintenance and upgrade difficulties , and so on.

' To give people to fish than to give people to fishing ', the development of their own core skills, can be fast, convenient, flexible for the specific needs of the company to develop customized, both for the company to save the purchase of hardware funds, but also conducive to maintenance upgrades.

Second, the Ironfort Fortress machine system structure

The architecture of the Ironfort Fortress machine is as follows:

A complete communication process is as follows:

1. The user sends a WebSocket request to the bastion machine by using a browser that supports HTML5, on the basis of HTTP;
2. Use Gevent on a bastion machine to receive websocket requests and forward them to Django;
3. After Django receives the request, call Paramiko to establish an SSH channel with the remote host;
4. After the remote host executes the user's command, the data is returned to Django via SSH;
5. Django returns to the user's browser via gevent in the form of websocket;
6. The user's browser simulates a Linux terminal using the Term.js plugin, showing the results returned by the remote host.

This is the core mechanism, so let's look at the development process.

Iii. introduction of development 1. Project creation

The bastion machine itself is usually arranged on a Linux host, which provides services in the form of HTTP, rather than ubuntu16.04.

First you need to set up a virtual environment, and install Python3.6 and Django2.0, no longer repeat.

Use django-admin startproject and python manage.py startapp app_name create separate projects and apps.

At this point, you can try to run the Django service, and if you see the following page, it means OK.

Django2.0 's welcome interface is a little more beautiful than before, with links, with the times.

2. ORM Model

Any Web project must first design the ORM model, that is, the table structure of the database, in the case of in-depth analysis of the project requirements.

Six models were designed in Ironfort, namely:

• Remote host
• Remote Host User
• Users with remote host bindings
• Bastion Machine User
• Bastion Machine User Group
• Log

Here's what to remind you:

• Each remote host account can be bound to multiple remote hosts, the two are actually many-to-many relationships;
• Bastion machine users can not directly bind the remote host;
• Bastion machine user binding is actually a host + Host account object;
• Consider whether the account is activated or is durable by the enabled attribute;
• Consider the Unique_together property of some fields;

With regard to the design of the model, everyone has the needs and ideas of each person, which has many pits and places to be noted, confined to space, unable to unfold the discussion. Detailed explanations are available in the video tutorials on my personal website liujiangblog.com.

The model is designed so that you can register the Django Admin background at the same time. Then Makemigrations, migrate, and Createsuperuser, after restarting the server, you can create test cases in admin, as shown in:

3. URLs and routes

The design of the URL is not complex, there are not too many complex pages, the following are some of the URLs used in the project:

fromimport adminfromimport path, re_pathfromimport= [    path('admin/', admin.site.urls),    path('', views.login),    path('login/', views.login),    path('logout/', views.logout),    path('index/', views.index),    path('log/', views.get_log),    path('host/<int:user_bind_host_id>/', views.connect),]

Django2.0 's URL syntax moves closer to frames such as flask, but it can still use regular mode. For the difference between 2.0 and previous versions, you can check out one of the blogs I've written about Django 2.0 new features preemptive!. In fact, is not a heavy user, the basic feeling does not change, how to use or how to use. The biggest difference is in URL authoring, and support for Python2 and 3.

4. Front-End frame Adminlte

In order to make the user interface beautiful, I use the bootstrap-based open source framework Adminlte.

Adminlte is hosted on GitHub and can be downloaded from the following address:

Https://github.com/almasaeed2010/AdminLTE/releases

Adminlte comes with jquery and Bootstrap3, no additional downloads.

Adminlte comes with a variety of color matching skin, can be adjusted in real-time as needed.

The Adminlte is mobile-adaptive and does not need to be considered separately.

Adminlte comes with a large number of plugins, such as DataTables, which can be loaded as needed.

But within the source package of the Adminlte, Missing font-awesome-4.6.3 and ionicons-2.0.1 These two icon plug-ins, it is loaded through the form of CDN, if the network is not very good, loading may be difficult or slow, preferably in the form of local static files, please customize the download and introduction of the project.

We do not need to adminlte so many features, just need its basic framework. In its source package, the index file is clipped and static file import processing, forming a basic base.html for expansion, on the basis of which we can expand the index and log page.

5. Bastion Machine User Login page

Bastion Machine User Login page does not need to use adminlte, preferably a single simple page, the less content the better.

The user login process view is straightforward, using the Django built-in auth authentication system.

User authentication and logon sessions can be completed using Django's own authenticate and login methods.

Now that you have a login, you must log out. In order to restrict access to the bastion system by users who are not logged in, all related views must first use the adorner for login verification.

In general, bastion machines do not need to provide user-facing registration pages. Fortress Machine User Registration is Super Administrator control, in the background!

6. Host Account page

That's the fort. When the user logs into the system, the default page displayed is index. This will list the remote host accounts that the current bastion machine user can use in the form of a table. The view is simple:

@login_required(login_url='/login/')def index(request):    # ...通过ORM的API查询可使用的帐号    return render(request, 'fort/index.html', locals())

The front-end page of the Host account, index based on Base.html, uses the DataTable plugin to provide advanced features such as search, sort, and paging, which are displayed as follows:

7. Open the WebSocket channel in the browser

Baidu Encyclopedia: WebSocket Protocol is a new network protocol based on TCP. It implements browser-to-server full-duplex (Full-duplex) communication-allowing the server to proactively send information to the client.

This article is not intended to become a websocket of popular science, interested in in-depth study can see the Essence of Blog Park WebSocket Protocol: 5 minutes from beginner to proficient

Simply put, there are the following points:

• HTTP itself is a stateless connection and does not support real-time communication;
• WebSocket based on HTML5, requires browser support;
• By adding the upgrade attribute to the HTTP header, request that the communication protocol be upgraded to WebSocket;
• Once upgraded to WebSocket communication, full-duplex real-time communication between the browser and the remote server can be achieved.

For WebSocket's use of tutorials, you can refer to Ruan Yi Feng Expert's blog WebSocket tutorial

Its specific API as shown (image from the rookie tutorial):

To simply create and use a websocket, follow the following routines:

• Using the new WebSocket(url, [protocol] ); create WS object
• Using WS, Call the OnOpen, OnMessage, OnError, and OnClose methods to process data in the communication process
• Using WS, call the Send method to send data to the back-end server
• Using WS, call the Close method to close the WebSocket connection.

We hide a Host Account ID field in the Host Account form, get the value of the field through the JS code, and then start websocket communication, passing this ID as one of the parameters used to construct the URL used by the websocket communication.

I am using the Term.js plugin in the browser to simulate the Linux terminal. This is an open source on GitHub browser simulation Linux Terminal JS Plugin, the address is: https://github.com/chjj/term.js . Its official documents are relatively simple, and interested students can delve into their source code, or use Xterm as an alternative.

The final effect is as follows:

Because the backend is not finished at this point, it is not connected to any host.

8. Create a WebSocket server

Django itself is a synchronous web framework and does not support WebSocket. So you use its runserver, is unable to receive and process WebSocket request. To solve this problem, you can use Gevent, the third-party asynchronous network framework for Python.

Gevent is based on Greelet libraries with WSGI server and its extended library Gevent-websocket supports websocket communication.

Please pip install gevent gevent-websocket install these two libraries first.

Create a script in the root directory of the Ironfort project start_ironfort.py , and this is our service startup script.

from gevent import monkeymonkey.patch_all()from gevent.pywsgi import WSGIServerfrom geventwebsocket.handler import WebSocketHandlerfrom ironfort.wsgi import applicationprint('ironfort is running ......')ws_server = WSGIServer(    (host, port),    application,    log=None,    handler_class=WebSocketHandler)try:    ws_server.serve_forever()except KeyboardInterrupt:    print('服务器关闭......')    pass

The core point is to use the Gevent wsgiserver server instead of the Django Runserver, using Geventwebsocket's websockethandler to handle the WebSocket communication requests sent by the browser, And forward it to the Django application.

We know that the Django Communication portal exists in from ironfort.wsgi import application this method. With Gevent's help, we let Django have the ability to receive websocket communication requests.

Run python start_ironfort can start a new server, in the browser to verify that the normal access.

9. Create a view in Django to handle websocket requests

We have already written the relevant URL in the root route in front of us and posted it here:

path('host/<int:user_bind_host_id>/', views.connect),

In this way, the ws://ip:port/host/15/ URL request in the form will be forwarded to the Connect view for processing, which passes the parameters of the ' 15 ' Host Account ID. The specific connect view has the following local code:

@login_required(Login_url='/login/')def Connect(Request, USER_BIND_HOST_ID):# Exit If the current request is not a WebSocket request    # ... Omitted    # Get Remote_user_bind_hostBridge=Wsshbridge (Request.environ.get (' Wsgi.websocket '), Request.user)Try: Bridge.Open(host_ip=Remote_user_bind_host.host.ip, Port=Remote_user_bind_host.host.port, username=Remote_user_bind_host.remote_user.remote_user_name, password=Remote_user_bind_host.remote_user.password)except Exception  asE:message= ' try to connect{0}An error occurred during the process:\ n {1}'.format(Remote_user_bind_host.remote_user.remote_user_name, E)Print(message) Add_log (request.user, message, Log_type=' 2 ')returnHttpResponse ("Error! Unable to establish SSH connection! ") Bridge.shell () Request.environ.get (' Wsgi.websocket '). Close ()Print(' user disconnected ... ')returnHttpResponse ("OK")

Description

• Get the remote account number corresponding to the ID;
• Call the Wsshbridge () method, pass in the WebSocket object and the current user, create a bridging class for websocket and SSH communication, which we'll cover in a moment.
• Call the Open method to start SSH communication;
• Invoke the Shell method to start the terminal environment;
• Call the Close method after the communication has ended to close the channel.

So what is the Wsshbridge class here?

Wsshbridge Bridge Communication Class

Wsshbridge:

import geventfrom gevent.socket import wait_read, wait_writeimport paramikoimport jsonclass WSSHBridge:    """    桥接websocket和SSH的核心类    """    def __init__(self, websocket, user):        self.user = user        self._websocket = websocket        self._tasks = []        #...    def open(self, host_ip, port=22, username=None, password=None):        """        建立SSH连接        """        pass    def _forward_inbound(self, channel):        """        正向数据转发，websocket ->  ssh        """        pass    def _forward_outbound(self, channel):        """        反向数据转发，ssh -> websocket        """        pass    def _bridge(self, channel):        """        桥接websocket和ssh        """        pass    def close(self):        """        结束桥接会话        """        pass    def shell(self):        """        启动一个shell通信界面        """       pass

The module needs to be pip install paramiko installed first.

The Wsshbridge class, in essence, is bridging the WebSocket channel and Paramiko open SSH channel for data bidirectional forwarding.

The Open method invokes the relevant API of Paramiko, passing in the host IP, port, user name and password, opening the SSH channel, _forward_inbound and _forward_outbound methods to implement the forward and reverse forwarding of the data respectively.

The key to the core is the _bridge method:

self._tasks = [            gevent.spawn(self._forward_inbound, channel),            gevent.spawn(self._forward_outbound, channel),        ]        gevent.joinall(self._tasks)

Use the Gevent Spawn method to create two collaborative tasks, and then call the Joinall method to wait for their task to end. This realizes the data in the WebSocket channel and the SSH channel between the pitch, one to receive the communication mechanism.

Once this is done, restart the server and we can show the entire communication process.

First, the connection is successful:

Next is the interactive command like Python:

And then the top this dynamic command results back:

Finally, this editing environment is vim:

As you can see, we support the color output:

11. Logging and behavioral audits

Regarding the user operation, in the data from WebSocket to the SSH sends the process, may save the user through the front end Linux emulator terminal to strike all key record, and very regular with the return enter separates, very easy to distinguish.

We just need to create a log model, write a method to save the log, and then save the log where it's needed.

The Log Presentation page is very similar to the Host Account page, which is also processed using the DataTable plugin, as shown in the final effect:

At this point, the core function of Fortress machine based on WEBSSH has been developed. Confined to the space, it is not possible to drip, the branches do not leak all the narrative, I here is just a way to stimulate the process.

Iv. Summary

Remote host creation, Host Account management, bastion machine user and user group management, this series of work, currently I still put in the admin backstage. Later, you can migrate it to the Bastion machine page to manage together. If Ironfort is used in production environment, adding batch command execution, file distribution function, system deployment on line, Linux operation and so on, will need a lot of extra work and security mechanism, which left everyone to study.

In addition, I am in the official homepage Liu Jiang's blog and the tutorial, also has a about the CMDB host management system's tutorial, unifies it to the Fortress Machine Project, a Operation maintenance management platform Core bottom function basically has, on this foundation expands, the promising!

Finally, personal technology and limited capacity, there must be shortcomings, please light spray, more valuable advice, thank you!

Ironfort---A bastion machine based on Django and WebSocket