This is a creation in Article, where the information may have evolved or changed.
"Editor's note" China-UK Life introduces DevOps in mobile application development and operations, greatly improving development efficiency and enabling continuous delivery capabilities. Continuous delivery allows mobile apps to go live from month to week or even to day. Automate the process of building, testing, and deploying software by building automated deployment lines that are used on enterprise clouds (Docker, Git, Jenkins etc). Transform from IaaS to PAAs with virtualization container management on architecture-level management tools as the IT architecture transforms to the cloud architecture. containerized management of mobile application systems and Docker-based container management on enterprise cloud platforms.
The following is a key share of how it applies and promotes container technology for its own business characteristics. It is based on the characteristics of different user groups, targeted selection of container advantages for advocacy, so that the application and promotion of container technology more smoothly.
Characteristics of life insurance industry and Sino-British business
Insurance company business Complex, while Sino-British life is a Sino-foreign joint ventures, foreign shareholders on management, especially information technology management has many institutional requirements, complex and cumbersome approval, to promote the application of new technologies have many shackles. Financial enterprises in the use of IT technology is relatively cautious, coupled with a wide range of regulations and departments, the application of new technologies and promotion, often difficult. How to persuade users and management, break the traditional concept, and reasonable control of the risk are it in the application of new technology must be fully considered.
- The system architecture is complex : The application system is not an isolated independent system but a running business system composed of various services in front and back. A standard Internet service will involve the front-desk display, customer identification, constant data query, all kinds of standard business processes, payment channels, even including online underwriting, security, claims, image processing, all kinds of Notification Services and a variety of service quality tracking certification.
Business Rules complex : Life insurance business logic is generally complex, due to the complexity and characterization of life insurance products are more and more high, product categories are generally in hundreds of kinds, general business rules thousands of, but also contains more complex medical health rules, and long-term insurance to the class of security rules, not to mention the rules of claims.
Many management systems: Insurance companies have a large number of internal management system, involving regulatory, compliance, risk control, information security and daily business rules and information technology and development operations related to various systems. such as Sino-British joint ventures and include shareholders of various types of information security and data security management requirements. In order to meet the requirements of various management systems, the flexibility of system design is greatly restricted.
frequent demand changes : The rules are not fixed, but are constantly increasing. As services, products and various business activities increase, the requirements for system functions and business rules are increasing. And most of these rules are not substitution relationships, but superimposed, all need to be implemented through the system support.- User Requirements Fast online : With the rapid development of the Internet, the external environment is rapidly changing, the demand of business units changes rapidly with the change of the external environment, frequent changes and demands for fast delivery have become the norm.
- Infrastructure Burden : A large number of complex business systems make great adjustments to the infrastructure, whether it is server, network, storage or information security, the burden of operations is increasing.
Why do we use containers
Actual business needs
3 months must be complete 2 mobile application development and on-line, and to ensure adequate performance and scalability of the needs of efficiency:
- The infrastructure needs to be resilient enough to scale and performance on the basis of security.
- The finalization of the overall business requirements is expected to be completed in 1.5 months after the launch, leaving it less than 2 months of construction time.
- Involving a number of business team collaboration, the development team contains 4 teams in Guangzhou, Beijing, each with a division of labor but also need to be able to coordinate the rapid progress, and timely see the development of the phased effect.
Control of costs
- The project input cost is fixed, the long-term investment may have the very big fluctuation, the initial basic investment must control.
- The purchase time of the entire project base investment must be in line with the project development cycle.
- Operation and maintenance personnel are insufficient, short-term inability to add operations personnel, must maximize the use of existing personnel.
Container technology has been concerned since 2015, with the rapid development of container technology in 2016 from the promotion of the use of the point of view are relatively mature. But as a new technology, how to make a relatively traditional, but also have a lot of rules and regulations of the financial industry companies to use, there are still many difficulties. The specific use of the population and management of the understanding of the container has become the key to promote container technology Shinjuku. The promotion and comprehension of the key population is much more difficult than the use of technology itself, but it is not possible to avoid the biggest obstacle that may exist. To be successful, you must be able to turn obstacles into momentum.
It can be seen from the characteristics of life insurance business that the complexity of the system architecture and business rules urgently needs standardization to simplify the workload of development and operation. Therefore, the advantages of the container on the CI/CD become an important hand to promote the container technology to the actual executives of the development and Operations department.
The advantages of automation, isolation and centralized management are more evident in management. This point in the project implementation and on-line, gradually reflect the advantages of the period. The implementation of the project to the management of the relevant knowledge and advantages of the crowd, as a further comprehensive promotion of container technology is an important point.
The container scheme based on public cloud does not necessarily reduce cost overall, but it has a great advantage in cost control flexibility. At the same time, with the development of container technology, the stable version of the container is fully able to withstand the actual application. In particular, the commercial technical support of the current container allows users who do not have too much experience in container development to quickly use container technology. There are several characteristics in controlling cost. From the actual use of the results, equipment procurement arrival to installation time from the traditional monthly calculation, to now basically ignore the purchase time. Deployment on the underlying operating system installation is negligible on the cloud, and environment deployment is basically minute-level based on the configuration of the underlying image. This has been met by our use requirements.
How do we practice it?
In order to maximize the benefits of container technology, the project itself is a new project that avoids excessive historical baggage and other related issues. In the implementation of the project, the adoption of public cloud deployment speeds up the deployment of the environment, and the use of containers is thorough in this project. At the same time, based on data security considerations, all data in the project is non-landed, and Redis is used for loading. At the beginning of the design to avoid the security of too many traditional rules and regulations, to avoid wasting too much time on interpretation and explanation. At the same time do not stress the test automation, as far as possible to maintain the original test procedures and methods. There is no change in the basic sense of using the user level. Avoid excessive communication and interpretation.
Automated builds, code management, and versioning are both automated and can be rolled back at any time. Let operations personnel fully feel the advantages of container technology. At the same time, the utilization of the server is greatly improved, which basically reduces the server requirements by nearly half. Taking into account future disaster recovery and system expansion requirements, this public cloud deployment only uses the public cloud's basic services. All applications in the project operate independently under the container in the form of a service, allowing for adequate isolation. Reduce reliance on public cloud services and ensure that the financial industry's business systems are disaster-tolerant. If any fault can be in the enterprise in a timely manner to restore the exact same production operating environment.
In the entire project practice, a mature continuous integration scheme is adopted and all the containers of the public cloud are deployed. To minimize the technical challenges, the entire project in addition to the use of elk, due to the poor performance of the environment configuration needs to be adjusted, the overall smooth implementation, did not encounter too many technical difficulties. However, there are some special problems in container use, so it is very important for financial enterprise it to find an experienced partner to drive the container. Do not recommend the preparation is not sufficient, hasty horse.
What we have learned
The key to the application and promotion of container technology is the actual use of personnel, so it is important to understand the container technology within it. The involvement of the IT team must be the actual participation, development, operations and testing of personnel must be involved. The promotion of this kind of project is first the knowledge of it internal managers, only the managers themselves recognize the advantages and urgency of container technology, it is possible to truly promote. Pure technical staff's preferences, it is difficult to make container technology to be truly effective promotion.
Practical application must be gradual, not too much pursuit of advancement and perfection. We must start with our own abilities. In practical applications it is best to find a mature partner, must talk about container technology is a new technology, for financial and it is more focused on not changing new technology, but the combination of business and technology applications. Therefore, in order to reduce detours, mature partners are very important to protect.
Operation and maintenance automation needs the cooperation of the process system, the tool can not replace the personnel's own management requirements. While automation can improve the efficiency of the underlying configuration, the real increase in efficiency is the shift in the concept of development and operations personnel. So this process must be constantly summarized and revised to look for problems and awareness. Do not think that on the Docker once and for all, in fact, real development and operations will encounter more new problems, so must face up to these problems.
Because of the rapid development of container technology and related CI/CD technology, it is very important for the owner to grasp the technical scheme in the application promotion. Try to avoid using too many technologies or versions of the same project that have never been tried, and you must control the scope of new technology attempts to control the overall risk of implementation. This is actually a tradeoff between the pursuit of innovation and stability, in the implementation of the responsible person to avoid too much influence by external factors.
At the same time, we must consider the workload of operations personnel, container technology is more to the operation of the operation of personnel changes, it is necessary to master more new knowledge and try more new technology. Therefore, in the project implementation of the actual operation of the workload is increased, this must be taken into account, otherwise it will outweigh the gains.
Well, the above is my early Chinese and English in the container aspects of the practice of some experience sharing, thank you.
Q&a
Q: Hello, there are a lot of container-based solutions, I would like to know how you set up the monitoring system after deployment, if the monitoring container, may be because the container life cycle is not fixed, resulting in some of the monitoring data will disappear with the container, become useless data, if through the host monitoring, So how do you achieve, through self-developed, or third-party open source programs, how to achieve the accurate collection and integration of resources, application data?
A: Your question is also the most worried about some problems in practice, because the container technology changes quickly, the current monitoring problems we use the software provided by third-party vendors to monitor. Insurance company it itself to the container-related technology, more currently in the groping and familiar stage. We use containers as a flexible tool, so there are no Docker-brought tools for background management and monitoring.
Q: In the process of Docker, what is the biggest problem encountered, and finally how to solve it? Thank you!
A: The biggest difficulty is the understanding of the container, especially the management and operation of the development of practical operators how to understand the container. Actual containers in the second half of 2015 we would like to use, but in view of the actual promotion difficulties, we have been looking for the right opportunity, until there is a suitable opportunity to really start the application and promotion. I have also mentioned the principle of prudence in sharing, because once the first step of the promotion department is smooth, the subsequent difficulties will be great. So our first step did not give ourselves too high a goal.
Q: How do containers automate management and container configuration files when they are scaled up? Thank you!
A: The size of the problem more to see their own situation, the current more on the introduction of the container on the scale of the advantages. But as I began to mention, the actual size of the business scenario is growing as the business grows. Our current consideration is that as the scale grows, the use of containers and the accumulation of experience will grow with them. And for life insurance companies like us, more business solutions are used. This is a more sustainable and reliable long-term support, after all, the technology changes quickly, we pay more attention to flexibility and the advantages we selected to play. Automation management and container preparation files, as long as the project department is a lot, then the problem is not big. If it is generalized (and this is our next step) then the development and operational capabilities of the people themselves are the key to automating the solution, not just the software.
Q: Hello, can you introduce the experience of micro-service technical transformation, especially the data splitting aspect, how to break the table between the database dependencies, such as in the monomer application many functions are directly through the database related query implementation, if the micro-service transformation, how to deal with similar related queries? In addition, can you introduce the technical solution of distributed transaction management?
A: Data section I will answer the paragraph. Data splitting itself more is it to business understanding problem, China and Britain it in this respect is relatively good at, after all, is their own business. First of all, I think there is no universal solution, which must be considered according to the actual situation. Let me introduce you to the situation in China and Britain.
At the beginning, we also introduced that because of the complexity of business logic, we have a unified interface layer and a business logic package layer on the basis of the service. The key is that this time we will use a large number of relational data in cache + routing query through Redis for the foreground query. To push forward the data we understand, this logic and implementation is actually more complex, but the result is basically do not rely on the background of the constant query. But there are many limitations of the scene, and this is the actual implementation of the best effort to solve.
There are many problems associated with queries, especially performance. So I also mentioned that this technology solution is difficult to solve, and must be combined with business logic to consider how to push forward data. With regard to distributed transaction management, we currently do not recommend the use of Distributed transaction management, which will only lead to more complex logical relationships. Most of us consider Redis's cluster deployment, but transaction processing is centralized. As far as the complex problem is concerned, we try to avoid it, and there are many challenges in our project, and we do not wish to have more destabilizing factors in this respect.
Q: Database data moved forward to Redis, and additions and deletions are also through Redis?
A: Not the database forward, we are considering information security and compliance requirements, the actual data is not landed. Redis is cache data and does not write file systems. Used only as a query, to delete background data, or to invoke the service remotely. According to business analysis, most of our applications are queries, so the query speed is guaranteed first.
Q: If the background data is updated in the way of service invocation, how does the centralized management of the transaction be done?
A: Since the core business systems of life insurance companies are centralized systems, transaction processing begins with the division of business modules and logic. So inside itself is the transaction logic very tightly controlled. Here first of all the business logic has strict pre-and post-logical order, what can do what is not to do what is strictly divided. At the DB level, our DB itself is a centralized db, not a distributed one, so the DB itself controls the transaction and there is no uncontrollable situation. But if you call too many, there will be serious lock problem, this is our more headache. At the same time the first business interaction frequency is not so high, because unlike e-commerce, this problem is not prominent.
Q: After the scale up, there is no test performance of the scene, such as a certain configuration of the physical equipment, how many containers can be loaded, performance-related indicators? Thank you!
A: We do not have practical experience in this field at the moment. But in the implementation of the size of the problem, so the actual choice is the public cloud, the direct use of cloud-based services. Also on performance and physical compounding. This is, at least for the moment, the main problem we are experiencing is memory not CPU. Reasonable allocation of memory is a process of accumulation of experience, I also mentioned in the share, we met because of the unreasonable elk performance is very poor, but the application environment itself is not a problem. Because we do not have much experience on the scale, so we can not give you more useful suggestions, this Internet enterprise should be more experienced.
Q: Hello, do you have any Windows applications in your company that use container Unified deployment management for these applications?
A: We have Windows apps, at least for now I think it's not appropriate to use Docker to deploy Windows apps uniformly, and it's a good choice if we have this in-house vm,vbox.
Q: Did you mention that you don't overemphasize test automation and try not to change the test process, and do you think about automating automated builds and unit testing? After all, these are the more labor-intensive parts.
A: Automatic construction I think it is more realistic that unit testing is considered. But our test case is too complex to see the short-term implementation is unrealistic. And performance is also a problem, if we want to do next we will consider a few more specific scenarios. For example, after the release of the product regression test, this is possible, but not universal application.
Q: After the container transformation, compared to the original system what aspects of the specific improvement?
A: Performance itself I don't think it has anything to do with containers, although this performance has improved a lot, but more because of the use of public cloud and Redis cache. The biggest improvement after container retrofit I think is now the efficiency of deployment and update. The operational efficiency has been greatly improved, after all, most of the application environment updates are automated. The essential change of this.
Q: Will the application of the updated automation process of Sino-British life be introduced, and does the application and configuration separation be transformed?
A: We're a new project, so we don't have to change. The automation section is the standard continuous integration Git+jenkins Automation pipeline. Git makes a few branches of the environment and deploys it directly to Docker. This is a set of. Docker-related compounding is currently managed in Csphere, and we don't really have much to worry about this part. There is not much change for development. It's just an automated build that makes sense.
Q: Is it recommended that enterprise best practices be containerized from the start of a new development or refactoring application?
A: This is the safest method, but I don't think the focus is new or refactoring. But is not to bring the actual benefit, I say is the business effect. For the user actually do not understand and care about container technology, want to promote a user off, a cross-transport and development coordination of the clearance.
The above content is organized according to the July 25, 2016 night group sharing content. Share people
Zhang Qi, general manager of Information Technology Department of Sino-British life. Graduated from Central University of Finance and economics, with more than 15 of experience in life insurance. Long-term engaged in life insurance business and information technology related work. Dockone Weekly will organize the technology to share, welcome interested students add: Liyingjiesz, into group participation, you want to listen to the topic or want to share the topic can give us a message.
