Accelerate Java application development speed 3-unit/integration testing + CI, application development ci

Accelerate Java application development speed 3-unit/integration testing + CI, application development ci

You may be familiar with the following scenarios:

• If you have developed an SSH web project, it may be slow to start the server. Some projects may take more than one minute or more minutes, and the wait time is generally wasted;
• Some functions were complicated during project development, and they thought the idea was very clear at the time. However, after a while, I forgot that bugs frequently occurred during function improvement, reducing the development speed;
• During the maintenance project, I do not know whether the modification is correct or not, and whether there is a hidden danger; the maintenance speed is reduced;
• If you develop an interface that many people use, such as a typical user system, you must ensure downward compatibility during upgrades;
• When collaboration between teams, sometimes only the interface is defined, and the other party has not yet implemented, how to conduct synchronous development?

As long as you are a developer, you may have encountered the above problems. If you have a unit/integration test, we can solve these problems well. (Note: If there are no special cases, unit test/integration test is not deliberately emphasized, that is, unit test/integration test is mentioned)

 

I will introduce the test in the following aspects:

1. Why test?

2. How to perform the test?

3. What are the benefits of testing?

4. Does everything need to be tested?

 

1. Why test?

What is the purpose of the test? My understanding is:

• Speed up Problem Discovery to problem resolution;
• After a program is modified, it can verify whether the guarantee is correct. (regression test)
• If it is an open-source software, we can learn how to use it through unit tests. For example, I have learned how to use cglib through cglib unit tests;

Therefore, if you encounter the above problems, you need to write a test. The write test may be for yourself (1, 2), or to help others (3 ).

 

2. How to perform the test?

Many friends don't know how to perform the test. In fact, the test is very simple. Don't think about it as complicated. Just test whether each function point is correct based on your own ideas.

2.1 test process

Unit test process

 

Integration test process

 

Integration test process

 

 

It can be seen that the only difference between unit test and integration test is that one call depends on the system and one does not call it. Because unit test is a test with the minimum granularity, for example, testing a class in Java, system dependencies are not tested, while system dependencies are tested for integration testing.

 

Test procedure:

Environment: It is also called a fixture or firmware, which indicates the data to be prepared/cleaned when calling the tested system;

Tested System: The class to be tested in Java, such as UserService;

Dependent System: Part of the system to be tested, such as UserDao;

Test Cases: Contains the test method class. There are many test methods in it to test the tested system.

 

Next, let's take a closer look at what work has been done in each part.

 

2.2 Environment

Environment, also known as fixture, indicates the data to be prepared/cleaned when calling the tested system; ensures that the environment is clean during testing, such as not affected by the previous database data; ensure that each test is performed in a clean/fresh environment. The so-called clean environment indicates that the current test is not affected by the data inserted, deleted, or modified by the previous test. In junit, you can use:

• @ Before (setUp) Installation fixture or preparation environment: Execute Before each test method in the test case. For example, create a fresh tested system and install the Mock dependent system during unit test;
• @ After (tearDown) uninstall the jigs or clean up the environment: Execute After each test method in the test case. For example, roll back the transaction during the database test, delete the data, and close the file;
• @ BeforeClass: executed before the entire test case;
• @ AfterClass: run the command after the entire test case;

Use the above method instead of directly installing/uninstalling it in the test method. It is because no matter whether there is any exception, @ After/@ AfterClass will be executed, so as to avoid exceptions that may cause environment exceptions.

 

If you use spring test to test database-related systems, you can use @ TransactionConfiguration to support default transaction rollback, which will not affect the existing system. For details, refer to Chapter 4 testing 13th overview 13.1 unit test- springspring3 and Chapter 4 integration test- springspringspring3.

 

Make sure that the environment is clean/fresh during testing.To ensure that the results of each test are the same.

 

2.3 tested system and dependent System

Tested system: Java is the tested Java class.

Dependent system: other classes that the Java class is tested to depend on.

 

In a unit test, the dependent system is typically simulated (Mock) to give it a false implementation; A typical example is to inject a Mock DAO layer when testing the service layer, which has the following advantages:

• The test speed is accelerated. Because the real tested system will not be called, the speed is extremely fast;
• The test has not yet completed the function; especially in multi-team collaboration, you can develop the system only when the interface is defined;

 

If you inject a real dependent system directly during integration testing, the benefits are:

• Complete joint debugging;
• Discover your own problems;
• You may also find problems with your use and APIs;

Although the unit test is good, it is an isolated test, that is, the tested system will not be called to complete the test. Because it is not a real joint debugging, there may be some potential problems, therefore, integration testing is required. (So it is not very deliberate to perform unit or integration testing, and some systems may only perform integration testing)

 

However, the integration test speed is relatively slow. Generally, it is submitted to the CI for execution without affecting the current development progress.

 

2.4 Verification

Purpose: ensure that the actual results are consistent with our expected results. To put it bluntly, it is what we think.

 

Generally, assertions are used for verification, such:

Assert. assertEquals (expectedResult, actualResult); // verify that the expected and actual results are equal

 

There are two main types of verification:

• Result Verification
• Behavior Verification

Result verification: Verify that the results returned by the tested system are correct, for example:

Java code

1. @ Test
2. Public void testCount (){
3. String ql = "select count (o) from User o ";
4. Long expectedCount = repositoryHelper. count (ql) + 1;
6. User user = createUser ();
7. RepositoryHelper. getEntityManager (). persist (user );
9. Long acutalCount = repositoryHelper. count (ql );
10. Assert. assertEquals (expectedCount, acutalCount );
12. }

Total number of data returned for verification = Total number of data inserted before + 1; that is, result verification. Here we use a relative (delta) test, that is, we do not care about the number of records in the database, but only care about the actual and expected differences.

 

Behavior verification: verifies whether the tested system has called an API that depends on the system. This is simple only when we use Mock for testing, such as when a user registers:

1. Add points

2. Send system messages

3 ,......

In this case, we cannot pass the result verification to verify whether these methods are called. Then we can use the Mock technology to verify whether these APIs are called. For example, the jmock testing framework supports behavior verification. Integration testing is difficult to perform behavior verification. If testing requires a reserved spyware interface.

 

3. What are the benefits of testing?

The purpose of code writing is to correctly complete a function. How can we ensure correctness? Test! Therefore, we also need to test the unit test technology, but this test is manually verified. Obvious disadvantages:

• It is not automatic. You need to compare the expected results with the actual results each time, especially when the data volume/logic is complex;
• It is not regression. After the previous test is completed, you have to repeat it again next time;

To solve this problem, we use unit test technology such:

• Test automation: Verify the expected results and actual results to the computer;
• Test the regression. You can repeat the test to verify whether the modified logic is correct;

That is, the benefits of testing have been extracted from the above:

• Shorten the time from Problem Discovery to problem resolution;
• Repeat the test to ensure that the modified Code is correct;
• For open-source projects, users can refer to how to use them;
• Because unit tests are fast, the development speed is improved;

4. Does everything need to be tested?

Certainly not. Everything is relative; which does not need to be tested:

• Features you are very familiar;

• Some simple CRUD;

• You don't think you need to test it. For example, if you are confident about something, there is no need to waste time testing;

Which tests are required:

• Complex business logic/system core functions, such as the order system, must have sufficient unit test guarantee, which is the core of an e-commerce system; there are also user systems, points systems, and so on;
• Code at the framework level, tool level, and general level needs to be tested, that is, the Code provided to a third party, because the code may be relied on by many systems and should be correct; it also ensures backward compatibility with future version upgrades;
• What you think needs to be tested, such as something you are not sure about, or write a test to shorten the restart time for a web project;

Testing is not time-free, and meaningless testing is a waste of time. The most typical example is to test an addition, deletion, modification, and query in some books. The actual project has no significance. Therefore, you should test the code that is difficult to control. Do not be a test mad. Test everything.

 

For some tests, refer to the Code in my es -- JavaEE quick development scaffolding. I have gained many benefits through testing.

 

Now we have completed the test, but if we use integration testing, the test may be executed slowly and it may take 5 minutes to run the test again, what should we do?

• Run the integration test once every day before work, fix the problem, and leave work;

• CI: continuous integration is handed over to the continuous integration server. After the test is completed automatically, the test report is sent to the developer's email address;

 

------------------------------------ Split line ----------------------------------

 

 

Next, let's introduce CI.

1. why CI?

2. How does CI work?

3. Introduction to travis-ci

 

1. why CI?

As mentioned above, we may encounter the following problems in separate tests:

• If you write a test, you have to run all the tests to check whether the entire system is correct. Therefore, each wait time is very long;
• If other members of the team have changed the function and submitted the function, how can they quickly get feedback on whether the current system code is correct or failed;

What should we do? Automated continuous integration (CI )! The core of CI is to do this. Automated continuous integration testing.

 

After using CI, if Maven is used, you can create multiple profiles:

• Some slow tests are ignored during local tests;
• All tests are performed on the CI server;

 

2. How does CI work?

A typical continuous integration process:

 

:

 
 

 

The continuous Integration Server is actually a timer that automatically downloads the latest code, compilation, testing, integration, and generate reports to developers.

 

Common CI servers include:

• Apache Continuum
• Hudson
• CruiseControl
• Jenkins CI
• TeamCity
• Travis CI

 

I used TeamCity community edition in, which is enough to meet common requirements. Currently, I use github to host projects and use Travis CI for Distributed continuous integration, which is free of charge. It seems to be good at the moment.

 

3. Introduction to travis-ci

The ES-JavaEE project development scaffolding I have developed is to use travis ci for continuous integration. For details, refer to Getting started for integration with Github. The Supported languages are as follows:

• C
• C ++
• Clojure
• Erlang
• Go
• Groovy
• Haskell
• Java
• JavaScript (with Node. js)
• Objective-C
• Perl
• PHP
• Python
• Ruby
• Scala

Supported databases:

• MySQL
• PostgreSQL
• MongoDB
• CouchDB
• Redis
• Riak
• RabbitMQ
• Memcached
• Cassandra
• Neo4J
• ElasticSearch
• Kestrel
• SQLite3

For more information, see the introduction on its official website.

 

 

For Java developers, the supported JDK types include OpenJDK and OracleJDK. If OpenJDK is used, the following configuration is required when the ascii2native plug-in is used in Maven:

Java code

1. <Plugin>
2. <GroupId> org. codehaus. mojo </groupId>
3. <ArtifactId> native2ascii-maven-plugin </artifactId>
4. <Version> 1.0-alpha-1 </version>
5. <Executions>
6. <Execution>
7. <Phase> generate-resources </phase>
8. <Goals>
9. <Goal> native2ascii </goal>
10. </Goals>
11. <Configuration>
12. <Encoding> UTF-8 </encoding>
13. <Src> src/main/messages </src>
14. & Lt; dest & gt; target/$ {project. artifactId}/WEB-INF/classes & lt;/dest & gt;
15. <Shortdes> messages. properties </shortdes>
16. </Configuration>
17. </Execution>
18. </Executions>
19. <! -- Native2ascii tools. jar -->
20. <Dependencies>
21. <Dependency>
22. <GroupId> com. sun </groupId>
23. <ArtifactId> tools </artifactId>
24. <Version> 1.7.0 </version>
25. <Scope> system </scope>
26. <SystemPath >$ {java. home}/../lib/tools. jar </systemPath>
27. </Dependency>
28. </Dependencies>
29. </Plugin>

If mysql is used, the port must be 3306.

If you want to test the port, this is not allowed.

 

 

The following is a configuration in my project. travis. yml, put it under the root of the project:

-----------------------------------

Language: java language

 

Env: Environment

-DB = mysql

 

Jdk:

-Openjdk: Use openjdk

 

Mysql:

Database: es database Name

Username: root User Name

Password: the password is empty.

Encoding: utf8 encoded as utf8

 

Install: script executed during installation

-Mvn install-Dmaven. test. skip = true mvn install and skip the test

 

Before_script: Test executed before script

-Cd web

-Mvn db: create the mvn command for creating a database (the maven-db-plugin plug-in is used here)

-Mvn db: the mvn command used to create a schema script

-Mvn db: mvn command for data Installation

-Cd ..

 

Script: the pace of execution during testing

-Cd common

-Mvn test: test the common submodule

-Cd ..

-Cd web

-Mvn test-Pit: test the web sub-module and specify the it profile for testing (that is, the configuration of the integration test. For details, refer to profile/it in pom. xml)

 

Events: triggered

Email: where will the test report be sent after the test is completed?

Zhangkaitao0503@gmail.com

-----------------------------------

 

 

Continuous integration cannot fix code errors, but, like unit testing, shortens the time for Problem Discovery and resolution, which improves development efficiency, reduces project risks, and improves project stability. In addition, especially for team collaboration, you can find out whether other people's code has an impact on your own code. 

 

 

By now, we can use unit test + CI to Accelerate the development speed of developers. Make good use of unit testing and CI, and do not do these things purely for unit testing and CI.

 

This article does not introduce TDD, and TDD is not so beautiful. I think we can use some of TDD's ideas, but we must not be superstitious about TDD. Sometimes, especially for developing enterprise applications, writing a function before testing may be more efficient, and TDD is not required in most cases. In addition, I have not been able to get too many TDD benefits in actual projects, but I have gained the benefits of testing.

 

This article does not introduce the test coverage rate. I don't think we should blindly pursue the coverage rate. Sometimes some coverage rates do not have any significance. Therefore, do not slow down the project development progress for coverage.

 

 

Just as a post on stackoverflow titled How deep are your unit tests? Kent Beck's answer:

I wrote that my boss pays for my code instead of testing. So, my value for this is that the less test, the better, the less confident you are about your code quality.

You can go to coolshell's "How detailed is unit test ?" To get some experience.

Detailed steps of java software development

The software development steps should be the same, regardless of JAVA or C.

Requirement Validation-Outline Design-detailed design-code-unit test-integration test-system test-Maintenance
Requirement confirmation: Requirement Specification
Outline Design: system use case diagram and use case scenario
Detailed Design: system design report and database design report
Test: Test Case Report

The main process is

Are you posted for the exam ??

Java web Development Process

Let me talk about it. What I mentioned here refers to the situation where the business needs are clear:
I. Development Framework Technology Selection: spring, struts, hibernate, ajax and so on.
Ii. Database Table Structure Design
3. UI design and implementation (front-end)
Iv. backend business code implementation and interface transfer code implementation (backend and frontend and backend connections)
5. improved and refined functions (such as interface details and program efficiency optimization)
Vi. Integration Testing