Background processes and background tasks

When building a high-performance, scalable WEB application, you can use a background task to decompose some time-consuming operations from the processing of HTTP requests into a background process to execute asynchronously. Most WEB requests should be done in 500ms, and if you find that your app has more than one request that can take up to two seconds to complete, then you should consider using a background task to put these time-consuming requests into a background process to execute.

For example: Send mail, read the slow API interface, upload data to the cloud storage, these tasks should be placed in the background task to execute, the front-end Web process should add a background task immediately after the reply to the client, and then the client can poll check the update to confirm whether the task is completed.

We take a Web RSS reader as an example, this kind of application will have a user to add a new feed URL function, after the completion of the user will be transferred to a display this new feed content page. The simplest way to do this is to read a third-party Feed directly in the process of the HTTP request and return it. However, reading the content of an RSS feed is a time-consuming operation, depending on the response speed of the feed, and in a few seconds, if the service providing the feed is problematic, it can even result in a few 10 seconds (depending on the set time-out). These can cause the application's page to die or the request to be killed when the processing time of the WEB process goes online, causing the request to fail and the user experience to be very poor.

The workaround is to use a background task that puts the background task into the background process and returns immediately after the WEB process has finished adding the task. The background process does not process the HTTP request, but simply takes the task from the task queue and executes it. The following is an improved Feed add process.

Background process

The app can specify a background process to run by applying the Procfile file in the root directory of the code.

Each line of the Procfile file declares a background process that needs to be run in the following format:

Name:command

name is the background process name , and command is the one that starts the background process.

The following time a Python app launches the configuration information for a background task process:

Worker:python vendor/rq/cli.py Worker

After the code is submitted, you can see the running status of all the background processes on the app/background processes page, which will run only one instance by default as soon as the process is created, and you can increase the number of instances as needed.

How to implement a background task

On Sina cloud You can easily implement background tasks through redis/ database + background process + Third party task Queue Library . Some common implementations are in the following table.

language	Implementation Scenarios
Php	Using Php-resque to implement background tasks
Python	Using RQ for background tasks

Background processes and background tasks