Narrow waist for Network and Data

In the computer network, there is a concept called "narrow waist".

The emergence of this concept solves a key problem: on any given network device, imagine a stack, the underlying hardware layer, and the top-level software layer. There may be various network hardware in the middle, and similarly, there are various software and applications. A method is needed to ensure that the software can still connect to the network regardless of the hardware; Regardless of the software, network hardware knows how to handle network requests.

In the network, the role of the narrow waist is played by Internet Protocol (IP), which is the common interface of the lower level networking protocol and higher level application and transport protocol designed for LAN. (This is a good explanation.) and (in a broad over-simplification), this public interface becomes the universal language of the computer, making the network interconnected, devices communicating, and this "network" can evolve into today's rich and varied internet.

In our view, this equates to the fact that SQL has become the "narrow waist" of data analysis.

We live in a time when data is becoming "the most precious resource in the World" ("
"Economist", May 2017). We saw Cambrian's professional database (OLAP, time series, documents, charts, etc.), data processing tools (Hadoop,spark,flink), the data Bus (KAFKA,RABBITMQ) and other Red Sea. There are more applications that need to rely on this data infrastructure, whether it is a third-party data visualization tool (Tableau,grafana,powerbi,superset), a web framework (Rails,django), or a custom data-driven application.

Like the web, we have a complex stack of underlying infrastructure and top-of-the-line applications. Usually, we end up writing a lot of glue code to make this stack work. But the glue code can be fragile: it needs to be maintained and fit.

What we need is a public interface that allows the various parts of the stack to communicate with each other. The industry has been standardized. It allows communication barriers between different tiers to be minimized.

This is the power of SQL. Like IP, SQL is also a public interface.

But in fact, SQL is more complex than IP. Because the data also needs to be analyzed by humans. And the goal that the SQL creator initially set for it is readability.

Is SQL Perfect? No, but this is the language that most people in the community already know. Although engineers are already developing a more harmonious language interface, where will the systems eventually connect? or SQL.

So there's a layer on the top of the stack. That layer is us.

SQL regression

SQL has come back. Not only is it annoying to write glue code using NoSQL tools. Not only because training everyone to learn countless new language costs is huge, not just because of the importance of uniform standards.

And because the world is full of data. It surrounds us and binds us. First, we rely on our human senses and the sensory nervous system to deal with it. Now our software and hardware systems are getting smarter and can help us. As we collect more and more data, we can better understand the world, the complexity of the system, storage, processing, analysis and visualization needs will only continue to grow.

We live in a fragile world and a world of 1 million different interfaces. Perhaps we can continue to embrace SQL. Everything follows the law of Conservation of energy.

Narrow waist for Network and Data