Zato: An ESB and backend application server based on Python

Overview

Zato is an open source ESB and application server written in Python. By design, it is used to build back-end applications (that is, APIs only) and to integrate systems in an SOA.

View Zato's project documentation Click here to view its GitHub page click here.

The target users of Zato are developers who use other dynamic languages such as Python or Ruby and PHP, or technical teams that consider trying dynamic languages at work, either because they see dynamic languages elsewhere, or because they are willing to try out a non front-end system written in one of the dynamic languages.

The platform is lightweight but complete, encompassing all perspectives of architects, programmers, or system administrators, and provides out-of-the-box support for many features, including HTTP, SON, SOAP, SQL, AMQP, JMS WebSphere MQ, ZeroMQ, Redis NoSQL, FTP, browser-based GUI, CLI, API, security, statistics, job scheduling, load balancing, and hot deployment. In addition, it provides a wide range of guidelines and reference style documentation.

Its original version was released on May 18, 2013, and the latest 1.1 edition was released in early June.

Architecture

A Zato environment is a collection of one or more clusters. Each cluster is comprised of multiple servers that share the same SQL and Redis databases. The front-end of these servers is the cluster-specific HA HTTP load balancer.

All servers are always active and the same group of services is always running. To achieve Active-standby settings, the load balancer can take any server offline as needed.

A load balancer is an embedded Haproxy instance that administrators can remotely control by calling SSL XML-RPC from the command line or through the GUI, either with or without a client certificate. Users can assign weights to the server and use other features provided by Haproxy itself, such as connection ACLs or rate limits.

The server is built on gunicorn/gevent projects. The project is a consortium that uses Libevent to select the best asynchronous event notification library supported by each platform, such as the Epoll on the Linux platform.

In order to make full use of all the Cpu,zato that a single computer can provide, a certain number of worker processes are generated beforehand, each process uses the selected asynchronous network connection library to process the connection, and all processes listen for the same socket. Load balancers are used to distribute loads between different computers and provide ha.

One server in the cluster assumes the role of starting the AMQP/JMS WebSphere MQ/ZEROMQ Resource Scheduler and connectors. If this server is unexpectedly down, the ping mechanism, which is always active, can ensure that another server takes over the role.

Applications can be integrated using a variety of protocols, including HTTP (special support for JSON/SOAP and pure XML), FTP, AMQP, JMS WebSphere MQ (for seamless interoperability with existing MQ Java applications), Redis, and SQL. Where the HTTP protocol is the only way to synchronize calls to the Zato service. In this case, the requesting application will enter a blocking state waiting for a response.

Programmers can use any Python library. If Zato itself has not yet provided a feature, it can be implemented using other technologies, such as XML-RPC or SMTP, which is only a matter of importing Python's built-in packages.

Cluster Management uses a browser-based GUI and CLI. The former is primarily used to manage clusters that are running, while the latter is used to install Zato components, such as servers, in the operating system.

The cluster configuration information is stored in the Redis and SQL operational databases. Redis stores fast-changing and frequently updated data, such as statistics or user run-time information, while SQL ODB storage can be simply mapped to relational structure data.

Although the main use of the GUI for environmental configuration, you can also export/import the configuration information of the cluster in JSON format, and the export results can be stored in an external configuration version library, so that it can be versioned, markup, or differences between versions.

A built-in scheduler with a GUI can be used for one-time jobs or for looping jobs (or cron syntax).

Servers and services are only shared state by Redis and SQL oDB. There is no custom protocol or data format to keep the server state consistent.

Zato uses more than 160 self-contained management services to manage itself, each of which can be obtained by invoking the public API Json/soap the command line or on HTTP. The GUI and CLI tools themselves are the clients of these services.

Zato has created a convenient client for Python applications, so applications written in Python can only use Python objects when communicating with Zato exposed services.

Service

The Zato service is a Python class that implements a single, specific method. It can receive input and produce output, or it can receive input only or produce output only.

Services can be installed either statically or in hot deployment from the GUI or from the command line. The installation process automatically compiles the service into bytecode.

Services can use any data format, but Zato provides more support for JSON, SOAP, and normal XML. If you use either of these, serialization and deserialization are done in the background, and developers can use pure Python objects with only the point number, without having to create bean/models/stubs/classes based on schemas like XSD-although that means there will be no code completion.

The same service can be exposed on HTTP, AMQP, JMS WebSphere MQ, ZEROMQ, and scheduler without any need to modify any code or reboot the server. In particular, only services exposed on HTTP can be invoked synchronously.

Simpleio (SIO) is a declarative syntax used to represent simple requests and responses. Using this syntax, the service can be exposed through JSON or XML/SOAP without modifying the code. Sio cannot handle complex documents, and it does not accept arbitrary nested structures. Any document of any structure can be used for zato, but sometimes not for Sio.

The following is an example of a basic service that uses the Yahoo Yql/json and Google ' s XML APIs to capture the total market capital of a company.

The service receives a stock code (such as GOOG or RHT), emits two HTTP requests, and then cleans up the responses and combines them into a common format. Depending on the format of the request, the combined results are returned in JSON or XML format.

# anyjson
from anyjson import loads

# bunch
from bunch import bunchify

# decimal
from decimal import Decimal

# lxml
from lxml.objectify import fromstring

# Zato
from zato.server.service import Service

class GetMarketCap (Service):
"" "Returns the total market capital of a company based on its stock code, in USD 1 billion.

"" "
class SimpleIO:
response_elem = 'market_cap'
input_required = ('symbol',)
output_required = ('provider', 'value')

def handle (self):

# ...............................

# Yahoo
# ...............................

# Get the connection to Y! By name
yahoo = self.outgoing.plain_http.get ('Yahoo! YQL')

# Create URL parameters for YQL queries.
q = 'select * from yahoo.finance.quotes where symbol = "{}"'. format (
self.request.input.symbol)
url_params = ('q': q, 'format': 'json',

'env': 'http: //datatables.org/alltables.env'}

# Invoke Y! And create a bunch instance from the JSON response so you can use dots
# Reference these elements.
yahoo_response = bunchify (loads (yahoo.conn.get (self.cid, url_params) .text))

# Clean up the response of Y!-If there is a business response, remove the last character.
# Assume that the response is always in billions.
if yahoo_response.query.results.quote:
value1 = yahoo_response.query.results.quote.MarketCapitalization
value1 = Decimal (value1 [:-1]) if value1 else 'n / a'
else:
value1 = 'n / a'

# A new response entry is appended to the entry list. Depending on how the service is invoked,
#Zato will serialize it to JSON or XML.

item1 = {'provider': 'Yahoo!', 'value': str (value1)}
self.response.payload.append (item1)

# ...............................

# Google
# ...............................

# Get a connection to Google by name
google = self.outgoing.plain_http.get ('Google Finance')

# Create URL parameters for calling Google
url_params = {'stock': self.request.input.symbol}

# Call Google and create an Objectify instance from the XML response so that you can
# Reference these elements with dots.
google_response = fromstring (google.conn.get (self.cid,

url_params) .text)

# Clean up Google's response-if there is a business response, convert millions to billions.
if hasattr (google_response.finance, 'market_cap'):
value2 = Decimal (google_response.finance.market_cap.get ('data')) / 1000
else:
value2 = 'n / a'

# In addition, a pure Python dictionary (hashmap) is attached to the response object, and Zato
# Complete serialization
item2 = {'provider': 'Google', 'value': str (value2)}
self.response.payload.append (item2) 

This is a very simple integration example, and not all scenarios allow the use of SIO, but no matter how complex the service is, one thing to emphasize is that you should not avoid coding with python. Besides being executable, it looks very much like pseudocode in many ways--there are more examples to illustrate this.