Value of batch processing technology to Web developers

Value of batch processing technology to Web developers

Author: Justin James
Translation: purpleendurer
Category: Application Development and website development
Tags: website, batch processing, network application, cloud computing, programming language, software development, software/network development

One of the greatest transformations in the Internet era is the migration of applications to real-time data processing. Many developers have never written anything in the batch processing mode. Thanks to the changes in teaching and experience in the business world. Even so, batch processing technology should still be used in the developer's Development Kit.

What is batch processing?
Batch Processing is designed to perform iterative input on the accumulation since the last run, and is also designed to run periodically. For example, the bank will record the entire transaction data, and then send it to the customer account after batch processing at night. This is opposite to real-time processing. Computing in real-time processing directly responds to input.
The first time I learned about batch processing was in high school. At that time, we were learning to use COBOL for batch processing. Today, there are still a large number of COBOL applications in many batch processing solutions, including the end-of-day processing of banks ), insurance companies and other similar Big Data Environments. However, it is unreasonable to restrict batch processing to host-type programs. For example, a service that contains a periodic bill processing part or a periodic report system requires a batch processing system.

 

　　Advantages of Batch Processing

Batch Processing has some advantages over real-time processing, including the following:

▲Transactions can be processed faster because they do not have to be processed in the foreground. Instead, you only need to provide enough data to find the required data for batch processing tasks.
▲If your company is less busy for a period of time every day, your batch processing can take full advantage of idle infrastructure during this period of time, this reduces the requirements for processing peak loads.
▲Batch processing is suitable for joining in a virtualized and cloud environment. In this environment, resources can be scaled or load transferred as needed, which can save a lot of money for hardware and energy.
▲When an application occupies a large amount of resources because the system needs to interact with the user, the processing part is separated from the system for batch processing; here we can think of cloud and virtualized environments again.

Why is it out of favor?
If the batch processing function is so large, why does it disappear slowly? The development of web is closely related, but it is also an important factor for enterprises to face. The main reasons are as follows:

 

▲Web applications cannot be started by themselves. They need a trigger to start them, which makes the idea of scheduling batch processing hard to imagine.
▲Web applications are expected to be deployed only through file copying, which breaks away the idea of operating system-level scheduling.
▲Web applications should execute small and individual tasks and return results within a few seconds. It is not easy to trigger any form of batch processing using user operations.
▲The company is basically engaged in global business, with no "overnight" or few downtime.
▲Users expect immediate results or transactions to be reflected.

 
　　Methods

Although I haven't used COBOL programming for a long time, I have written applications that use regular batch processing. I tried two methods when I switched my work to a web application.
The first method is that some Web applications execute batch processing and arrange a task to retrieve the page according to its request. Although this method can work and make other parts of your code library easy and leisurely, I think there are too many sports parts in a movie scenario, especially when it becomes a long-term running task.

Another method I have tried is to write a completely independent application for system scheduling. I prefer this method, but it can only work when the batch processing task can access the logic of the main application (usually through a shared library.
There are also other options. For example, the database has the ability to schedule tasks. If your processing can fully include the database, this may be a great choice. Some development frameworks and systems (such as the outsystems agile platform I recently used many) have built in the batch task concept, saving a lot of trouble. One possibility is that your system runs continuously (as a daemon or service) and uses an application-level timer to trigger processing as needed.
However, when you start, make sure that your method is scheduled on demand and can run well after deployment. (End)
(Http://blogs.techrepublic.com.com/programming-and-development? P = 2727)