Aliyun Solution Architect, about distributed architecture Cloud Platform solution (with graphic)

Technical Framework

The distributed architecture cloud Platform is fully analyzing the development trend of it technology, adhere to the centralized, standardized, integrated, reliable and scalable design principles, with value creation as the mission, with a standardized, integrated, intelligent cloud Platform as the support, the realization of transparent information sharing, business agile coordination, timely control, The science of decision making is the design goal to choose the traditional mature Java EE, SOA, application integration and BI information technology and a new generation of cloud computing, large data, mobile application information technology combination of technology route.

Distributed architecture cloud Platform design the enterprise IT infrastructure of intensive and cloud architecture dynamic configuration;

Shared, centralized data storage management of enterprise data resources services;

Modular, platform, flexible integration of enterprise application support services;

standardization, service, integration of Intelligent enterprise Business application services;

One-stop, multi-terminal Service Enterprise Information display Interactive services and other technical layers, each layer also includes a number of mature and stable technical components, the technical layer, bottom-up, layer support, the technical components of loose coupling, interconnection, scientific and efficient, easy to expand, reduce the information islands, enhance the standardization and intensification of the system, Optimize the user experience of the system, improve the work efficiency.

Design principles of distributed architecture cloud Platform Technology

The principle of advanced nature

In the overall design and implementation, relying on cloud computing, large data areas of well-known open source projects (such as Hadoop, Spark, OpenStack, etc.). Due to the industry's widely recognized fact standards, we can fully borrow the resources of the global ecological Circle, promote the software and hardware layered decoupling, and continuously improve the compatibility. Compatible with a variety of heterogeneous physical devices to avoid vendor binding. Data level, support for a variety of data sources, including structured/unstructured types of data processing, data itself, data calculation also support open sharing. Priority to adopt advanced mature technology components, set up a stable and efficient large data cloud computing management platform, and on the platform to achieve large-scale data collection and analysis of related business applications. Platform design to meet the current business functions, taking into account the trend of future development.

Reliability Principles

Reliability includes three levels of overall reliability, data reliability and single device reliability. Through the distributed computing and storage architecture of large data cloud computing platform, the system can improve the reliability and reduce the reliability of single equipment. The platform design ensures the stability and efficiency of the platform of the cluster system based on Hadoop and virtualization, Provides tuning for Hadoop and virtualization-related technical components for existing underlying hardware devices. And for the whole cluster of supporting monitoring system and cluster maintenance and management related programs, application design to adopt a clear application of layered architecture, on the one hand, the upper layer of data application and the underlying data dependence separation, To realize the decoupling on the application architecture, on the other hand, it can improve the analysis efficiency of the upper data and reduce the running cost. The relevant fault-tolerant technology and fault-handling technology are used to ensure the security and reliability of data application, and to ensure the availability of data analysis platform to meet the requirements.

Security confidentiality

The use of a unified user authentication, unified user, rights Management and control, password control and other security and confidentiality measures. To ensure the security of information, the information on the internal network to establish security requirements of the firewall, intrusion detection, digital certificates, anti-virus, data encryption technology, can strictly and effectively prevent the intrusion of foreign illegal users, can avoid network attacks, to prevent the occurrence of official secrets, to prevent the damage caused by illegal intrusion.

Scalability

The application development platform adopts modular construction and expansion mode. Support small-scale start-up, linear expansion to meet different scenarios, different investment plans and scale requirements; With the expansion of data scale and the improvement of application, the data platform can be flexibly and conveniently used for cluster expansion without affecting the normal use of current users.

Open

Cloud computing platform is completely independent research and development in the mature landing program, mainly applies the open source technology. Distributed Key Technologies

Micro Service

The system function is divided into the smallest service unit, complete the single function, each service is deployed independently, and the service is formed through mutual invocation to form the complete business logic. Main Features:

-high cohesion, low coupling

-Opening and closing principles

-High efficiency

-Elastic calculation

Distributed transactions

Distributed transaction is realized by message mechanism and distributed lock, and the integrity of business logic is ensured in the micro-service architecture. Main Features:

-Message Queuing

-Atomic operation

-Roll back mechanism

Calling Across machines

Tasks are assigned to more nodes to run, and calls across machines replace the calls within the original single node, in-process. Main Features:

-Multi-node

-Synchronous + Asynchronous

Scalability and fault tolerance

The lateral extension replaces the vertical expansion, which makes the scalability better and the overall fault tolerance greatly improved. Main Features:

-Consistent Hash

-Multiple replica platform critical components

Enterprise Service Bus

Using Dubbo+zookeeper technology as an enterprise service bus to manage all micro services, the service bus has the following characteristics:

-Automatic discovery and registration services, Plug and Play.

-Provides a load-balancing strategy for micro-services and requires additional load-balancing software.

-Statistics and monitoring of service invocation and recording of response time. Provide statistical data for program tuning and scaling.

Message Queuing

Platform Message Queuing employs Kafka technology, Kafka is a highly-throughput distributed subscription message system that handles all the action flow data in a consumer-scale web site. Mainly used for:

-Message communication between services to achieve complete business logic.

-Provide queue services for large concurrent business to avoid service crashes under large concurrency.

Distributed File System

The platform uses HDFS and Fastdfs Distributed file systems.

HDFs mainly solve the storage of large files (such as log files, video files, etc.) and hbase large data storage. Mainly divided into Namenode and datanode,namenode storage file meta information, Datanode storage data blocks. Multiple blocks of data read from the name node to a file at the time of the client invocation and merged into a single file after being fetched from multiple servers. Fastdfs is a lightweight Distributed file solution, the main solution to store a large amount of small files, such as upload pictures, upload files, resource files, and so mass of small files, these files are not suitable for HDFS storage, so use Fastdfs storage.

Cloud Server IaaS

The platform adopts OpenStack series technology to support virtualization technology such as Xen/kvm/hyper-v/esx. Provides flexible computing services for distributed and large data.

You can add my Java Architecture Group: 671017482, which has a large Java Internet Hot technology video free to share with everyone.