Analysis of external interface data audit problem in application system and experience analysis

The BPM application system based on PAAs platform was launched at the end of last year, this system to-do task in the process of audit with the portal interface, there is a problem, always feedback "add to do failure."

Note: The portal system is a stable system on-line in 2012, which provides the interface specification for the application system. It is now provided by OPS personnel in the System maintenance support (configuration level).

Since January 25, 2016, received the portal to do audit problem feedback, the painful process began, because the portal is a stable system, its developers have already withdrawn, maintenance personnel support is limited, and we are just on-line new system, theoretical problems in my new online system.

1. Audit problem analysis and process overview

Audit problem analysis Process lasted 3 months, we invested 5 people (design, testing, development, coordination and communication and other related staff), more than 2 people months of work, I as a project manager, input is not less than 1 people months of work.

The whole process is roughly divided into four stages:

1.1. The first stage, code analysis and problem analysis positioning phase;

By modifying the interface audit view, gradually locate the problem point, and remove the system hidden trouble and bugs. For example, there is a bug in the return operation through auditing.
This phase of the audit is still to help the new online system to troubleshoot the bug phase.

1.2. Second stage, code transformation phase;

According to the specification recommendation, the interface that transforms the business data into the audit view is changed into the interface view of the increment operation log type.
The problem remains.

1.3. The third stage, the vacant stage;

This phase is in the early March to early April, is the most painful phase, the customer daily feedback audit issues, there is no real progress on the audit, but it is gratifying to find some problems, such as:

Application, database, time inconsistency between different databases.

In addition, the Portal maintenance personnel feedback the interface data, and according to the data content by item and combination, almost every day to do verification tests.

Finally, it falls to the "_id" content.

1.4. Phase IV, steady

Helpless, I found once involved in the Portal project (note: I was only responsible for the development of two process services), the development of data, anti-compiling audit tools, finally find the problem point, it is so simple.

The original design to do audit data structure as follows:

The Portal Maintenance personnel feedback interface data is as follows:

As shown in the table above, the portal-side data "_id" is composed of AppID (that is, Pendingnode) +pengcode+apptoken.
If Apptoken is set to "123456", then _id will eventually be "123456", and if Apptoken is set to "", then _id is composed of Appid+pengcode, which conforms to the design table structure of the first period.

The code snippet for the Anti-compilation audit tool is as follows:

    booleanfalse;    this.pendingDB.getCollection("pending");    String pendingId = pendingNote + pendingCode.trim();    new BasicDBObject("_id", pendingId);    query = query.append("pendingStatus""0");

Through the code and previous business tests, audit the existence of new to-DOS rules are: take the professional system to do the push log in the pending state (Pendingstatus) to do (0) of the Pendingnode+pendingcode, and the portal data, if not exist is recorded as " Add pending failure. "

2. Solution

Call the service parameters of the push portal, Apptoken string configuration into empty.

UCLOUD_ASSIGN_APP_ID= 此处隐去UCLOUD_AUTH_TOKEN=

The corresponding application-side calling code is as follows:

    /**     * 实现推送，更新表中数据     *      * @param 准备推送的数据     * @param 状态     * @return 根据类的实例调用该类中定义的非静态方法，实现其他实现类与主类的代码分离     */    publicaddUcloudPanding(PendingEntity pendingEntity, String appID, String adppToken, String eipPendingRestUrl) {        TaskAppService taskAppService = ProxyFactory.create(TaskAppService.class, eipPendingRestUrl);        PendingEntity[] pe = { pendingEntity };        Map<String, String> map = taskAppService.addPending(appID, adppToken, pe);        return map.get(pe[0].getPendingCode());    }

3. Empirical Analysis 3.1. Time consistency of cluster environment

In the audit problems, the portal maintenance personnel repeatedly prompt the time is prone to problems, we follow to do task creation time, write log time, the portal to receive the creation time of the order of carding, found the audit log time inconsistencies, timezone problems.

Time consistency, time zone issues are business rules that large, complex systems must pay attention to and be careful about, and find the following problems.

(1), due to the use of the system platform Cordys products are foreign products, time is the standard time, with Beijing, 8 time zones, that is, less than 8 hours.

(2), Oracle time conversion characters, minutes and months of confusion, such as the original code to use mm to express minutes, but the SQL is not case-sensitive, mm and mm are considered to be the same format code, is converted to the month.

Compare the SQL code as follows:

--原sql语句--修改后 to_char(t.create_datetime + interval ‘8‘ hour ,‘yyyymmddHH24MIss‘) as pendingDate,

(3), the system is composed of Oracle database (minicomputer), MySQL (PC server), MongoDB, OpenText Cordys, a total of 9 server devices, where the minicomputer server time is 10 minutes slower.

(4), the system use time source is not unified, for example has the Oracle database, has the application service Cordys and so on.

The issue of time is still the eternal topic, especially the large-scale cluster system, although the application server in this system has already deployed the cluster time synchronization service, but the Oracle server can not be integrated together, need more attention in future maintenance.

3.2 Two systems business expression inconsistency problem

In the Portal to do task management, the task is only to be done, has been done, but in the professional system, the task management is more complex, not only to do, has been done, there are return, forwarding and so on.

In the case of "forwarding", the corresponding relationship with the portal to-do task is as follows:

Professional system to-do tasks in the forwarding process, the first to push this to-do update for the status of the service, and then push a new to-do, this new to do as a forwarding target.

3.3. Interface Specification Interpretation

The Portal builder provides detailed interface development specifications for the developers of subsequent access systems, such as the new to-do description fragment as follows:

And the development process is provided, as shown in:

It is theoretically possible to obtain a log of the push portal in this operation through business operations, but it should be made as an incremental log as recommended by the specification.

However, for this simple incremental log, the interface developers have caused a lot of misunderstanding, "why the same to-do task, why there are many records?" ”。

It seems that the text plus graphic description is not necessarily clear, the direct communication between developers how important, but no developers can communicate with you.

It would be nice if there was a code sample.

        Try{Pendingentity PE =NewPendingentity ();//Cloud Portal interface needs to push information---get the desired value in Taskxml            //TODO IDPe.setpendingcode (Getnodevalue (TASKXML,"Pendingcode"));//TODO titlePe.setpendingtitle (Getnodevalue (TASKXML,"Pendingtitle")); ......//Implement push Cloud PortalC_syncportal c_syncportal =NewC_syncportal ();//new log C_syncportal.adducloudtasklog (pe,ucloud_assign_app_id, "a");s = c_syncportal.adducloudpanding (PE, ucloud_assign_app_id, Ucloud_auth_token, Ucloud_server_rest_ip_add_task); Loggermanager.logasync (Loggermanager.debug, Loggermanager.log_type_soap,"UNICOM_BPM","Ucloud","Taskappservice.addpending","","", Node.writetostring (TASKXML,true), S,"Call taskappservice.addpending OK"); }Catch(Exception e) {Logger.error (">>>>>>>>>>>>>>>>>>>>>>>>>> >>> Ucloud_task_sync_error:ucloud push to do failure: "+ WriteException (e)); Loggermanager.logasync (Loggermanager.error, Loggermanager.log_type_soap,"UNICOM_BPM","Ucloud","Taskappservice.addpending","","", Node.writetostring (TASKXML,true), WriteException (E),"Call taskappservice.addpending KO"); }

3.4. Interface Specification and Circle information

As mentioned above, we have been involved in the portal two period and have undertaken the construction of the Process Services Section, as shown in the unified to-do design at that time:

The developer, based on the design and understanding at the time, initialized the value for Apptoken when the portal was pushed to do so when the system was developed.

The nightmare has since begun.

Now, looking back to the online front, the portal Access application content, application content and feedback content, have not mentioned Apptoken.

The information of the development personnel in the circle is not reliable, must be the interface specification, the application single content prevail. This information was delayed for 2 months and cost 2 people a month.

Communication, communication is necessary, help information is feasible, need to pre-judgment and rapid identification. In the case of asymmetric information, it is very deadly.

The spec is too important!

3.5. Inconsistent Application Service configuration

Due to the existence of multiple servers in the system, there have been multiple configuration parameter inconsistencies before and after the line, such as the modification of the Apptoken parameter, the development of the deployment staff to modify the 143 server configuration is unsuccessful.

And I check, first from 143 to check, in the inspection, 143 has not been modified to complete, this omission, good hanging miss.

Cluster configuration unification, inspection, should cooperate well, careful, careful.

Summarize:

System Designers in the formulation, implementation of development and interface norms, must be strictly prohibited, cautious, and pay attention to the source of the norms.

As a project manager, after the majority of developers withdraw, personally organize, personally develop the analysis of audit issues, analysis, pre-judgment system in the existing problems, a great help.

Postscript:

Today, the portal audit passed smoothly, small celebration under!

However, the maintenance personnel feedback Portal side to do not disappear, even in accordance with my previous write "Access portal unified to do operations experience summary-update to do as done" method, also not good to make!

I immediately reply: Pengdingcode "123456" after the test, these two days estimated to deal with a lot of updated work.

The system operation and maintenance personnel feedback: Sure enough plus 123456 to be made. Give me a crazy icon, too.

There is no more concise way, can only deal with such sequelae.

Reference:

"External System interface Audit Experience" Sho Yongwei 2016.2

"Office process competency platform based on operational architecture" Sho Yongwei 2015.6

Analysis of external interface data audit problem in application system and experience analysis