Why does an application need a new data center stack?

Why does an application need a new data center stack?

In the past few decades, the Internet has greatly changed people's living conditions. We use a variety of apps to search for information, purchase commodities, and communicate with people in all regions of the world without barriers. The speed and ease of operation are unprecedented. The screen on the dashboard of a private car can recommend movies for us. Smartphones can make and publish videos. We can also buy thermostats that can adjust the room temperature before we arrive at the office.

Although these advances are magical to the outside world, only the developers know the level of development technology that safeguards these programs. However, as the times develop, everyone who deals with business should understand this, which is becoming more and more important.

The reason is simple: Smart companies all understand that the competition in the next decade requires us to have new ideas on information technology. From mobile applications to refrigerators, they all need to be more personalized and intelligent, with more emphasis on feedback and interaction.

Companies need the right tools to facilitate machine learning, sensor data acquisition, and the ability to process unexpectedly high user traffic.

The trend is irreversible. If an application cannot bring users the desired experience, they will find another one.

 

Rebuilding application Construction Methods

Over the past few years, the use of applications and web pages has been expanding, and the number of users has reached millions or even hundreds of millions. In this trend of large traffic and big data, the old application architecture began to collapse. Recently, the trend of using massive data has stimulated the interest of a brand new technology group in the market. They specialize in this terminal.

Processing methods include opening up a new storage space, processor, and database framework, in order to easily expand the scale of cross-cluster servers, simplify the processing process, and increase the speed of information exchange between various parts. Google, LinkedIn, Facebook, Yahoo, and Twitter are repeating this process over and over again.

One of the modes is applied to databases. Almost everyone starts from relational databases and finally develops a new plan. Some companies have spent millions of dollars (still paying) at the expense of personal time to find ways to break through the limitations of MySQL.

Another mode is clearly presented in the big data processing architecture. This special technology may vary in different fields, but all large network companies use a specific architecture to process real-time, quasi-real-time, and batch data. They all want to deliver a quick and personalized experience to users and ensure that internal data analysts and data scientists can meet the requirements of the job.

 

Encounter Data Center Application Stack

The results of all IT innovations are impressive. Google, Facebook, and Amazon provide services to billions of users (millions of users can be simultaneously served) and store massive amounts of data. But they have almost never crashed. Because a large amount of infrastructure is invested.

These companies have developed many well-known technologies, including MapReduce, Hadoop, Cassandra, and Kafka. In addition, they have developed a new set of tools (some from startups, some from laboratories, and some from open-source projects) to help applications run better and on a larger scale, add new features when necessary. These new tools include Spark, Storm, and Elasticsearch.

The new architecture that matches the new technology also emerges, trying to solve the problems brought about by R & D of applications that can run stably in a large-scale environment.

One architecture is the microservice concept. It regards an application as a service set that can provide services for a variety of applications, rather than a monolithic entity that uses its own dedicated components. In addition, microservices reduce the dependency between components and improve the scalability of personal services, which can be achieved without rebuilding the entire application.

Another large architecture is the trend of assembly. It is built in a developer-friendly way like Docker or a low-level way like the Linux control group. Assembly allows you to easily insert applications to distributed servers and change the focus point based on the running position. Further, developers can focus on where applications need to run.

In general, this new distributed service integration and architecture technology collectively referred to as "Data Center Application Heap ". Service integration or similar technical concepts are used to build applications that can serve millions of users on multiple platforms and utilize the quantity, types, and speed of today's data.

In fact, these technologies are rapidly gaining popularity. Many of them have become the technical leader of startups, in order to transmit everything from the next large consumer application to the next Salesforce.com.

These technologies are surpassing the pure network path, and they are entering the world's top 500 companies, even medium-sized enterprises that are not tied to IT innovators. They know everything about data center services (limited to the company level) and are eager to join this exciting innovation ..

"Big Data", "real-time", and "Iot" are not just buzzwords. In the 21st century, the success of a company depends on them in many ways.

 

Operating system of data center applications

Although data center application technologies are becoming more and more important, the problem is that (IT vendors, open-source supporters, or professional Facebook engineers won't tell you) building is very difficult. Deployment, management, and large-scale Hadoop; deployment, management, and large-scale Cassandra; deployment, management, and large-scale Kubernetes. You must clear and repeat each architecture or service you want.

In a sense, companies may not really want to write programs, build data pipelines, and validate the elasticity of the architecture.

Large companies, such as Google and Microsoft, use Borg and Autopilot systems to solve (or to a large extent) their own problems. The system automatically manages resource allocation to achieve high practicability of servers and applications that serve millions of users. Algorithms, rather than developers or Microsoft architects, determine the program trend and the number of machines.

Of course, they are great systems, but they are also proprietary to the company. Google also officially recognized the existence of Borg. Microsoft has hardly publicly discussed Autopilot, let alone sales.