"Dry" front high energy! How to ensure high performance for Python applications

Python, as a kind of "simple, elegant, clear" object-oriented language, with its simple and clear grammatical structure, rich class library, excellent ease of use in the Internet era has been widely used. Whether it's a foreign Facebook, Youtube, Instagram, Pinterest, or a home-grown, web app developed in Python is ubiquitous.

With the advent of the cloud computing era and the popularity of the Python-based open source project OpenStack, more and more enterprises are introducing the concept of cloud services and trying to use cloud computing services to build new high-availability architectures. In the same way, the design and development of enterprise applications has shifted-developers need to build native cloud computing applications to reduce operational costs and increase flexibility more effectively. The use of cloud platforms and cloud services to combine Python for application development has become an effective approach.

Python is really a good language, easy to use fast, the standard library and PyPI third-party libraries have rich and useful resources, can quickly solve the problem of developers, without having to reinvent the wheel, these features make Python in the past few years gradually popular. In contrast, C is limited to lower-level syntax, long development cycles, and is typically used to develop software with high performance requirements. Java is biased toward enterprise development, and slow JVM startup speeds cause Java to be unsuitable for developing system administration scripts. Python, as a generalist, is widely used in web development, scientific computing, data analysis, Cloud Computing (OpenStack), operational platforms, and automated operations (Saltstack).

Python has many advantages, but with the migration of the business to the cloud and the mobile Internet, the complexity of the real-world online environment, the huge traffic pressure, and the high availability of it architecture, the performance bottleneck of the Python application will be created. As a pythoner, are you often bothered by these questions:

• Is code execution really fast?

• Where is the code performance bottleneck?

• Not much memory consumption?

• Is there a memory leak?

Perspective Treasure Python Monitoring implementation principle

At the Pyconchina 2015 conference, Mr. Bruce Liu, the leading application performance management service provider in China, brought the theme of "Python Application performance management" to Liu Guoqiang for the vast pythoner. Together with you to explore how cloud smart Perspective bao is to ensure that Python applications in the production environment of high performance.

For the complex IT architecture, cloud wisdom uses backbone distributed monitoring node monitoring to achieve unified system scheduling monitoring tasks, all monitoring points are executed synchronously, relying on reliable backbone monitoring points to perform monitoring tasks, technical elimination of network jitter and noise caused by interference, stable and reliable data can be used to evaluate the SLA.

The smart probe deployed in the application system automatically installs the corresponding probe program according to the language of the application system, and draws the application topology for the system. The perspective Treasure Python Probe Pythonagent Adds the context manager to the framework's RequestHandler , and implements the context management protocol through the Smartpythonagent module __ The enter__ () and __exit__ () methods control the start and end of the tracer process. The tracer process uses the Python Sys.settrace () library method to open and close the trace process of the code.

The output module generates the data we want for the Code trace information process that is returned by the tracer process. The tree is the structure of the code execution process with the method name as the node-spanning trees, can be directly through the tree to restore the code execution of the method call process. Map records execution information for each node in the tree, such as memory consumption, execution time, and so on. When the output module finishes processing the data, it sends the data to the Smartagent Sendproxy, and finally Sendproxy sends the data to the Perspective Treasure server. Pythonagent's tracer process can track "call", ' line ', ' return ', ' exception ', ' C_call ', ' C_return ', or ' c_exception ' events.

Based on a variety of Web lightweight application frameworks, Python has the broadest support for Web applications, while Smartagent supports the mainstream Django, Tornado, CherryPy, Flask, pylons, bottle and other application frameworks, Ensure the accuracy and efficiency of Python code performance data fetching. With the other application service monitoring deployed on the service side, including Apache, Nginx, Tomcat, Weblogic, MySQL, Memcache, Redis, Oracle, MongoDB, PostgreSQL, etc. Development and operations personnel are able to discover the potential problems of the application system for the first time and pinpoint the real reason for the slow application execution.

How to monitor Python apps with perspective treasure

users can access the Perspective Treasure Product Website: http://toushibao.com/ , apply for a free trial account, enter application management, download Smart agent and install, install complete access to "system → plug-in management", find the host installed by the Smart Agent, follow the instructions below to install, configure and open the relevant code plug-in.

1. Install Python app Discovery plugin

Installs the app Discovery plug-in based on the Web container type, which automatically discovers all application instances within the container and generates an application topology diagram.

• If you are using a Python Server directly, you can skip this step directly.

• If the Web container is Apache

Please install and configure the Apache type App Discovery plugin Apacheapp, and then turn on the plugin when the configuration is complete.

• If the Web container is Nginx

Please first select the Install nginx type Application Discovery plugin Nginxapp in the configuration interface of the plugin. After installation you also need to configure the Nginx plug-in, unlike Apache, you need to manually compile on the host, please refer to the "./plugins/nginx_path/readme" document in the installation directory.

2. Install and open the Python Code monitoring plugin Pythoncode

Select the Pythoncode plugin to install and turn on, smartagent will automatically download the plugin to the smart_agent/plugins of the installation directory.

It is installed by default using System Python. Modify the Pythoncommand value in pythonagent.sh to install to other Python environments in the system.

3. Restart the web Server

After installing the Python code monitoring plugin, you may need to manually restart the Web server (Apache\httpd\nginx\python server and other software).

4. View Python app data

After the configuration is complete, you can view the data in the Apps module in approximately two minutes.

Perspective Bao Official website: www.toushibao.com

"Dry" front high energy! How to ensure high performance for Python applications