Memory Data Management (version 2nd)

Memory Data Management (version 2nd)
Basic Information
Original Title: In-Memory Data Management: Technology and Applications, Second Edition
Original Press: Springer; 2nd ed. 2012 edition
Author: (de) Haso Alexander cayr
Translator: SAP
Press: Tsinghua University Press
ISBN: 9787302292562
Mounting time:
Published on: February 1, August 2012
Start: 16
Page number: 291
Version: 1-1
Category: Computer> database storage and management

More about memory data management (version 2nd)
Introduction
Books
Computer books
Suppose you live in a big city in the United States. Now, imagine that every time you need a cup of water, you don't need to go to the kitchen, but you need to drive to the airport first, then fly to Germany by plane, and then pick up a cup of water there. From the perspective of modern CPUs, Accessing memory data is like getting a cup of water from the kitchen. Accessing the data on a computer's hard disk is like flying to Germany to fetch a cup of water. In the past, the cost of primary storage was high, so it was necessary to fly to Germany to fetch water. However, in the past few years, the cost of primary storage per MB has been significantly reduced, making it a cost-effective and more convenient choice to get a glass of water from the kitchen.
This huge difference in access time has a significant impact on enterprise-level applications. In the past, things that have not been taken into consideration due to a long time have become possible, which helps enterprises transform previous jobs dominated by speculation and speculation into practical insights.
The memory computing revolution described in this book not only refers to storing data in the memory for faster access. We also elaborated on the integration of the other two major trends in the IT industry: (a) emergence of multi-core CPUs and the necessity of software parallelism (B) congestion caused by DRAM access latency, software is required to cleverly balance CPU and memory activity; the above two points must be used to truly tap into potential performance advantages. Another important aspect of our memory data management vision is the change in data storage methods in the underlying database. As described in the following sections, this is particularly relevant to the enterprise-level applications we are concerned about. The role of memory data management is to associate all these connections.
Combination of memory data management and column Storage
Our experience shows that many enterprise applications use databases in similar ways. During execution, they process a large number of data rows, but the most important thing is that only a small portion of the data columns in the table may be useful for a specific query. The column storage model described in this book allows you to read only the required columns, while the rest of the table can be ignored. The traditional row priority model is different. All columns in the table must be accessed, and even unnecessary columns in the result are within the access range.
The column storage model also stores the elements in a given column. In the row-first model, data in a given column is stored together with other data in the row. Therefore, aggregation operations common for enterprise-level applications are much faster in the column-store model than in the row-first model.
Implementation of parallelism on multiple cores and multiple computers
The speed of a single CPU kernel will not be faster, but the number of CPU cores is expected to double every 18 months. Therefore, using parallel processing capabilities of multi-core CPUs is crucial for future software development. As described earlier, the memory column store stores data in a given column in the memory, so that one or more kernels can be allocated to process a single column. This is called vertical fragment.
In addition, tables can be divided into data row sets and then distributed to different processors. This process is called horizontal partitioning. Horizontal sharding is particularly important when the amount of data continues to increase, and has achieved some success in the parallel processing of data warehouse applications. These two methods can be applied not only on a single multi-core computer, but also on multiple computers in a cluster or data center.
Use compression technology to improve performance and save space
Data compression technology utilizes data redundancy and relevant knowledge in the data field. Compression is especially suitable for column storage in enterprise data management scenarios, because all data in the same column (a) has the same data type (B), in many cases, there are very few unique values, for example, in the country column or Status column. There are two reasons for using compression technology in column store: space saving and performance improvement.
The effective use of space is particularly important for memory data management, because it is relatively more expensive than the disk even though the primary storage cost has been greatly reduced. Because the column is compressed, the information density related to space consumption is improved. In this way, more information can be loaded each time, thus improving the performance. Compared with row-store, it requires less loading operations. In Row Storage, unrelated columns are also loaded but not used for queries.
Conclusion
Memory Data management is not only a technology, but also an idea of software development. We must consider the basic hardware factors, such as the access time to the primary storage or disk and possible parallelism through multi-core CPU. Considering the new hardware environment, we must write software that can directly make full use of it. From a positive perspective, for enterprise-level application developers, this book lays the technical foundation for the development of the database layer for all these problems. However, from a negative perspective, the database itself cannot solve all problems. Developers must understand the underlying software and hardware knowledge to take full advantage of the potential performance advantages. This book aims to help build an understanding of this.

Directory
Part 1 Turning Point of enterprise-level applications
Chapter 2 availability, applicability, and feasibility-Impact of memory computing technology
1.1 real-time information-obtain any information anytime, anywhere
1.1.1 fast-paced response
1.1.2 real-time analysis and dynamic computing
1.2 impact of the latest hardware trends
1.2.1 enterprise-level application database management system
1.2.2 primary storage is a new disk
1.2.3 from maximizing CPU speed to multi-core processor
1.2.4 bandwidth between the added CPU and the primary memory
1.3 reduce costs through memory data management
1.3.1 total cost of ownership
1.3.2 cost factors in Enterprise Systems
1.3.3 memory computing performance improves cost reduction
1.4 conclusion
Chapter 1 Why are enterprise applications so complicated and messy
2.1 current enterprise applications
2.2 enterprise application examples
2.3 enterprise application architecture
2.4 data processing in enterprise applications
2.5 Data Access Modes in enterprise applications
2.6 conclusion
Chapter 2 sanssoucidb-future blueprint for the enterprise Memory Database System
3.1 Focus on multi-core and primary storage
3.2 Memory Database System Design
3.3 data organization and access in sanssoucidb
3.4 Conclusion
Part 2 sanssoucidb: provides a single data source through memory computing technology
Chapter 2 technical basis of sanssoucidb
4.1 understand the Memory Hierarchy
4.1.1 introduction to primary storage
4.1.2 organizational structure of the primary storage hierarchy
4.1.3 trend of memory hierarchies
4.1.4 view memory from the programmer's perspective
4.2 Use multi-core and cross-server Concurrent Data Processing
4.2.1 add resources to increase capacity
4.2.2 Parallel System Architecture
4.2.3 enterprise-level application database parallelization
4.2.4 parallel data processing in sanssoucidb
4.3 increase speed and reduce memory consumption through compression
4.3.1 lightweight compression
4.3.2 heavyweight Compression
4.3.3 data-related optimization
4.3.4 compression-aware query execution
4.35 Compression Analysis of real data
44-column priority, row priority, and hybrid mode -- Optimize Data Layout
4.4.1 vertical partitioning
4.4.2 find the optimal layout
4.4.3 challenges faced by hybrid Databases
4.4.4 application scenarios
4.5 impact of Virtualization
4.5.1 Virtualization of analytic workloads
4.5.2 data model and benchmark test environment
4.5.3 virtual execution and local execution
4.5.4 reduce response time using parallel virtual machines
4.6 Summary of technical concepts
4.7. Conclusion
Chapter 2 data organization and access in sanssoucidb
5.1 SQL used to access memory data
5.1.1 SQL role
5.1.2 lifecycle of a query
5.1.3 Stored Procedure
5.1.4 data organization and Indexing
5.1.5 Any attribute can be indexed
5.2 improve performance with data aging
5.2.1 active and passive data
5.2.2 Implementation Considerations of the aging process
5.2.3 use case for horizontal partitioning of sales leads.
5.3 efficient retrieval of Business Objects
531 retrieve business data from the database
532 object data guide
5.4 efficient execution of business functions
5.4.1 differentiate between business functions and application functions
5.4.2 compare business functions
5.5 process data changes in read-optimized Databases
5.5.1 impact on sanssoucidb
5.5.2 merge process
5.5.3 improve performance through single-column merge
5.6 add or delete only to maintain the integrity of historical records
5.6.1 "insert only" Implementation Policy
5.6.2 use the insert-only operation to minimize the lock
5.6.3 impact on enterprise-level applications
5.6.4 feasibility of the "insert only" Method
5.7 support transaction data analysis
5.7.1 dynamic aggregation
5.7.2 analysis query without star mode
5.8 no downtime Extended Data Layout
5.8.1 reorganization in row-store
5.8.2 dynamic attachment in column store
5.9 use advanced log Technology to Improve Service Recovery
5.9.1 restore in column store
5.9.2 differential log records of the Priority Database
5.9.3 High Availability
5.10 importance of optimizing and scheduling hybrid Loads
5.10.1 scheduling Overview
5.10.2 hybrid load features
5.10.3 short-running and long-running Task Scheduling
5.11 conclusion
Part 3 changes brought about by memory computing technology
Chapter 2 Application Development
6.1 optimize sanssoucidb Application Development
6.11 programming mode of Memory Applications
6.12 application architecture
6.13 move the business logic to the database
6.14 Best Practices
6.15 create a view image
6.2 innovative enterprise applications
6.21 Brand New Analysis Application
6.22 Operation Processing helps simplify daily business
6.23 innovative user interfaces at your fingertips
6.24 Merge Analysis and text search
6.25 Basic Search type
6.26 Enterprise Search
6.3 conclusion
7th chapter: The real business intelligence system to be presented
7.1 operation data analysis
7.1.1 past business intelligence
7.1.2 today's business intelligence
7.1.3 disadvantages of separating analysis from daily operations
7.1.4 dedicated database designed for the Analysis System
7.1.5 analysis and Query Language
7.1.6 drivers for business intelligence changes
7.1.7 future business intelligence
7.2 change. How to evaluate the database later
7.2 enterprise computing Benchmark Test
7.2.2 new benchmark testing requirements tailored for hybrid Loads
7.2.3 new benchmark testing for routine operations and Analysis
7.3 conclusion
Chapter 1 expand sanssoucidb in cloud computing
8.1 what is cloud computing
8.2 cloud Application Types
8.3 cloud computing from the provider's perspective
8.3.1 multi-tenant
8.3.2 low-end hardware and high-end hardware
8.3.3 copy
8.3.4 improve energy efficiency with memory computing technology
8.4 conclusion
Chapter 2 The revolution in memory computing technology has kicked off
9.1 risk-free transition to memory data management
9.1.1 The memory system works side by side with the traditional system
9.1.2 system integration and scalability
9.2 customer verification points
9.2.1 Berlin Charlotte Medical University
9.2.2 Hilti
9.3 conclusion
About the author
References
Glossary
Index

Source of this book: China Interactive publishing network