Although the sparrow is small, it is very dirty: It looks at the software development process from a small function

From Monday to Wednesday, my leaders made a small task.

Although the functions are small, the entire process from conception to implementation to stable operation is almost included in software development.

This process perfectly demonstratesWrite a good piece of codeLet me share with you how much things you need to consider.

 

Simple requirements:

Analyze the content of various log files (such as Oracle error logs) and forward the qualified lines to syslog.

 

Requirement Analysis:

It is relatively simple to read logs and filter the content. The key point of this requirement is that timeliness and performance must be considered.

 

Generally, logs are constantly increasing,

Timeliness, requiring the program to analyze new logs as soon as possible.

Performance, requiring the program to perform incremental log analysis to avoid unnecessary I/O overhead.

 

Develop solutions:

Based on the functional and non-functional requirements obtained from the requirement analysis, we agree that a method is required to obtain new content in the file in real time.

On Unix/Linux, the tail command can implement this function.

 

We found a serious problem after investigating the tail command:

Tail cannot process log switching.

 

This problem rejects the idea of using Unix commands to write shell, so we decided to write a tail-like tool.

You can use either C or Java in the implementation scheme, based on the following considerations:

1. logs on Unix/Linux account for more than 80%

2. The target host may not have a Java environment

3. The resource consumption on the target host must be minimized.

 

Therefore, we decided to use C for implementation. Because the project team mainly focused on Java, the leaders went into battle.

 

Technical rehearsals:

The technical rehearsal aims to verify the feasibility of the scheme. If the scheme cannot be implemented due to technical reasons, you need to modify the scheme.

 

Technical rehearsals for this small feature mainly include:

1. Analyze the tail source code to understand how tail obtains file increments in real time.

2. Investigate how syslog messages are sent in C.

 

Implementation:

 

Basic functions:

Leaders initially thought that this function was very simple, with no more than 30 lines of code, and the basic function was indeed very simple.

 

Function enhancements:

Multiple files can be opened,

Supports log switching,

Regular Expressions are supported to filter rows,

You can use command line parameters to specify the severity and facility of syslog output.

 

Test:

Compile the program and mount it to the actual environment for testing. Some problems will soon be found,

For example:

Insufficient consideration for parameter exceptions

Two file switching modes are available and must be compatible (one is inode change and the other is unchanged)

Check whether the target file exists and whether it is a common text file (the binary file cannot be processed, or the directory can be opened as a file)

 

Program correction:

Solve problems encountered during the test,

Added file type determination, compatible file switching mode, and usage output when parameters are incorrect.

 

In addition, the program further considers the release of resources under abnormal circumstances (for example, the process is killed.

 

Retest:

On the HP-UX platform, the function test basically did not find the problem,

However, it is found that regular expression Parsing is faulty on the Linux platform and useless rows cannot be filtered out.

 

Modify again:

This amendment mainly addresses cross-platform compatibility issues.

 

Acceptance Test and Stability Test:

Deployed on multiple platforms and run for a long time.

 

Package and release:

Pre-compile the binary code of common platforms,

Write related installation and usage instructions.

 

----

 

Software development, regardless of the function size, actually requires these processes,

It's just for small functions, and each phase is quickly switched (sometimes it's just an idea in my mind ),

But be clear:

Sometimes it may be more time-consuming to save things. For example, when you do not know the demand analysis, you can start to do it, which will only cause a lot of rework.

In other cases, if you are stuck in a place for too long and cannot find out clearly, you may need to first try to push forward, and many problems will be solved ".

 

There is no way to go in the mountains and waters,

This is a process that is often experienced in the software development process.