Sybase to build cloud data warehouse with us

The big data age is putting pressure on data storage, and more and more organizations need to build a storage platform that can store large amounts of structured and unstructured data on the basis of cloud computing technology. Cloud storage should be "cloud"-born.

The so-called cloud storage, refers to through the cluster application, grid technology or Distributed file system functions, the network of a large number of different types of storage devices through the application software together to work together to provide data storage and business access functions of a system. The key to implementing cloud storage that adapts to user needs is to choose a strong data architecture based on application and deployment requirements.

The industry's leading enterprise-class and mobile software company, Sybase, is partnering with the country's first cloud computing infrastructure and platform services provider, to create a cloud storage platform that combines a shared architecture with a shared-nothing architecture. The Sybase IQ and the US cloud storage software have built the world's leading cloud architecture Data Warehouse solution through the rigorous functional and stability testing of the central innovation.

The program has the following advantages:

The technical advantages of shared nothing (no sharing) and shared modifiable (shared) architecture;

The advantages of both the column storage and cloud computing architectures to meet the performance requirements of the Super large data application on the infrastructure platform;

Adopt cloud computing cluster architecture, with excellent scalability and high reliability;

Eliminates the need to purchase SAN storage and replaces minicomputer with a x86 architecture, thus significantly reducing TCO costs;

The scheme provides a global leading technical framework for data-intensive and computational-intensive applications such as the Super large data analysis platform, and solves the technical challenges encountered in the "Big Data" analysis and application. The program is cost controllable, technologically advanced, supports structured and unstructured data, and can be widely used in finance, telecommunications, government, broadcasting, energy, aviation, manufacturing, tobacco, retail and other industries.

Sybase iq--supports full shared MPP architecture

The most commonly used data architecture in the industry today is shared nothing (no sharing) and shared modifiable (all shared).

Each node in the Shared nothing architecture is independent, self-contained, and there is no single point of contention across the system. Some systems need to centrally store large amounts of state information--databases, application servers, and other similar single point competitive systems. The Shared nothing system keeps the database growing steadily, and as the number of transactions increases, additional CPU and main memory can be added to ensure that each transaction time is constant.

With the development of cloud computing technology, more and more enterprises want to virtualize enterprise resources and realize information sharing, which brings the development of shared modifiable (shared) architecture. From the "client" perspective, the core of the Shared modifiable architecture is to develop computing into a public facility, like the water and electricity we use every day; from the server side, consider shared Modifiable is the virtualization of computing resources or convergence of computing resources, and the realization of information sharing, high availability and security.

Sybase has introduced a PLEXQ distributed query platform based on the "Massively Parallel Processing" (MPP) architecture in its IQ 15.3 products, which accelerates the speed of highly complex queries by spreading tasks across multiple computers in the grid configuration. The Sybase IQ 15.3 plexq distributed query platform has better concurrency, self-service ad hoc query capabilities, and independent scalability of computing and storage resources than the traditional shared MPP architecture. With this architecture, PLEXQ can transcend the requirements of service level agreements (SLAs) by merging nodes into unified modules and allocating them to different application configurations through simple, flexible resource allocations.

Unlike traditional shared-nothing-free MPP architectures, PLEXQ employs a fully shared (shared-everything) approach to dynamically manage and balance the query task load between all compute nodes in the Sybase IQ MPP configuration. PLEXQ's automated workload equalizer is active and avoids users competing for system resources, providing predictable performance for a range of parallel workloads.

Sybase IQ 15.3--Redefine cloud architecture

Based on the changing storage and analysis requirements of the enterprise, Sybase IQ uses a unique method of column storage, in which the traditional relational database needs to read every row of the entire table slowly, which often blocks the I/O channel, memory and disk, while the Sybase IQ Only need to read the columns required by the query, its vertical partitioning strategy not only can support a large number of users, large-scale data, but also can submit a high-speed access to business information, its speed can reach the traditional relational database hundred times even thousand times. At the same time, in the storage of large data, Sybase IQ can store huge amounts of complex data, compared with the traditional database, its storage capacity can be up to 10 times times.

In addition, the multiplex technology used in Sybase IQ leverages the advantages of a single server (node) that can be shared by disk storage while performing specific functions. That is, the storage is shared, compressed, and column-partitioned, interconnected by a high-speed fibre channel with all the server nodes in the multiplex architecture. These server nodes can be run either as data-read nodes or as write nodes (or both), supporting all concurrent data loads and queries connected to shared data sources. This technology enables Sybase IQ to work in a wide range of usage scenarios, such as rapid parallel analysis and reporting, while supporting high-speed real-time data loading, bulk data integration tasks, ad hoc client loading and querying, and more (see below):

The Sybase IQ 15.3 plexq distributed Query Platform solution has the following advantages:

Y Superior Performance: Users can leverage the advantages of multiple computing resources in the Sybase IQ PLEXQ environment while handling the most demanding queries;

Y Enhanced Scalability: As user data and query complexity increase, system administrators can easily and cost-effectively use any common hardware to extend Sybase IQ PLEXQ nodes;

Y architecture Flexibility: System administrators can quickly and simply merge subsets of computing resources into logical entities in response to changing user requirements.

(Sybase IQ 15.3 plexq Distributed query Platform working principle and advantages as shown below)

The Cloud storage products

US-sen technology is the industry's leading cloud storage, cloud computing system provider, with cloud storage products, cloud machine system, two products, and the industry, as well as radio and television, internet, telecommunications, education, government, IDC, and many other industries cloud storage, cloud computing solutions.

Hao Deposit is the United States and technology independent research and development of cloud storage products, built-in independent research and development of the YFS cloud file system. The YFS cloud file system enables the management of local disks on multiple common X86 schema servers to form a large storage pool for storage virtualization. In addition, Xavier provides a standard iSCSI host interface. The cloud storage products are designed for the needs of capacity expansion, high availability, focus on storage efficiency and performance applications, very suitable for virtualization, mass data storage, OLTP business, storage resource integration and other applications, can be widely used in telecommunications, government, education, Internet, finance and other industries.

Because of its "high reliability", "easy to expand", "high random Read Performance", "Easy to manage", "automatic streamlining configuration to enhance storage resource utilization" and "localization support and customization" and other characteristics and advantages, Hao existing for Hunan satellite TV Golden Eagle Network, VERYCD, Amber Education and other well-known enterprises in China to provide high-quality storage solutions and services. (Hardware and software configuration as shown below)

A fully-meaningful cloud storage architecture for Sybase and the United States

In Beijing, the creation of a letter survey is one of the well-known High-tech enterprises in the field of telecommunications monitoring, its monitoring system because of stable performance, security and reliability, full-featured, advanced technology and access to the national Science and Technology Progress Second prize, in it monitoring, especially in the field of telecommunications signaling monitoring has a higher authority.

2010, in the second phase of the construction of China Mobile No.7 signaling monitoring system, the Central Innovation Company took Sybase IQ as the core platform, the storage and management of massive speech data, which realizes the system requirement of statistics, analysis and query while large amount of data is loaded in real time, saves a lot of hardware storage equipment. The No.7 signaling monitoring system with Sybase IQ as the core storage platform has been built in more than 10 provinces such as Shanghai, Xinjiang and Henan, and its overall performance and reliability have been verified.

Focusing on Sybase's leadership in the field of database management and analysis, as well as its successful application in the field of telecommunications, the company has integrated its self-developed cloud storage product portfolio with Sybase IQ 15.3, and has passed the joint test of the Beijing Sino-Credit Measurement Technology Co., Ltd. With Sybase IQ 15.3 plexq distributed query platform MPP sharing architecture, as well as the United States and the existence of a strong cloud storage capabilities, the two sides work together to achieve a powerful combination of cloud storage.

Sybase IQ 15, as a database of signalling systems, undertakes the processing of signalling system data. Using Mutiplex architecture and Plexq's Sybase IQ, the fast query and efficient compression of large data in signaling system is realized. In the latest technical test of the Sino-Credit survey, the role of the United States as a Sybase IQ in the "array", through the MDS cloud storage technology will be dispersed on the x86 machine local hard disk virtual for a unified large storage, to provide data storage support for the upper layer of software. For Sybase IQ, multiple PC servers have been replaced by the traditional host + array pattern of Sybase IQ.

The data processing results of Sybase IQ are stored on the Low-cost PC server through the MDS Distributed File system, and a certain amount of redundancy is used to ensure the validity and reliability of the data, thus realizing the shared modifiable compute nodes and shared The perfect combination of nothing storage nodes. In addition, through the use of virtual machines, virtual fairs, distributed queries, distributed storage and other technologies to adapt to the current requirements of cloud technology, greatly enhance the flexibility of the architecture, the rationality of resource allocation, user-friendly to make choices and adjustments.

Through the complementary advantages of both products and the technical strength of the two sides, the project has achieved great success in the test of middle-record-testing. Sybase IQ and the United States and the cloud storage software cooperation, has passed the stability test, the lab environment 5 nodes can do 2 nodes down without loss of data, and can increase the physical resources in the case, to achieve the linear increase in processing performance.

After 3 months of hard work, the combination of Sybase IQ and the US cloud storage software enables a fully meaningful cloud architecture solution based on the X86 architecture, with fast data loading, massive data high-speed querying, cluster linear scaling, and support for Hadoop to build a cloud storage platform for businesses to meet the challenges of big data.