SQL Server parallel data warehouse (PDW)

Recently, the concept of big data has become very popular. Every manufacturer thinks that big data is an important direction of it in the future. Therefore, every manufacturer wants to make a difference in this field. I attended the IBM Big Data seminar a few days ago. At the Conference, IBM released their big data solutions, three integrated machines (puresystem), and IBM released DB2 V10, the purescale specifically designed for Oracle RAC was officially added to the DB2 major version ).

 

In terms of MPP architecture, Microsoft was previously criticized and lacks products to cope with the challenges of big data. I checked it from the Internet and found that Microsoft also released the MPP data warehouse architecture after 2008 R2, and will launch its own all-in-one machine this year.

 

For SQL Server PDW institutions, there is an article about very detailed, you are interested can look at :( the article comes from: http://www.jamesserra.com/archive/2011/08/microsoft-sql-server-parallel-data-warehouse-pdw-explained)

 

Microsoft SQL Server parallel data warehouse (PDW), formally called by its code name "project Madison", is an edition of Microsoft's SQL Server 2008 R2 that was released in December 2010. PDW is Microsoft's reworking of
Datallegro Inc. Massive Parallel Processing (MPP) product that Microsoft
Acquired in July 2008. It only works with certain hardware (two so far), the first of which is

HP enterprise data warehouse appliance (Dell is the other, with a couple more to come in the near future: IBM and bull). This edition of SQL Server can't
Be Bought as an independent piece of software, it has to be bought along with the hardware.

So what is MPP? Until now, SQL server has been a specified Ric multiprocessing (SMP) solution, which essential tially means it uses one server. MPP provides scalability and query performance by running independent servers in parallel. that is the quick definition.
For more details, read
What MPP means to SQL Server parallel data warehouse.

MPP is also available from other companies such
EMC greenplum, teradata,
Oracle exadata, HP vertica, and
IBM netezza, but those use proprietary systems, where PDW can be used with commodity hardware, providing a much lower cost per terabyte. but it's still not in-expensive: the hardware and installation will cost around $2
Million (not including software licenses), but gets you "200 times faster queries and 10 times the scalable than traditional Microsoft SQL Server deployments" (see

Press release). PDW also comes with its own support model.

Microsoft has had clustering capabilities in SQL Server for a while, but the scalability part was lacking. this is where PDW comes in. scalability in PDW means handling tens of terabytes of data and then moving to hundreds of terabytes worth (up to 600
TB ). at about 50 terabytes to 60 terabytes of data, clustering is needed; thereafter, clustering starts to approach its limits, and that is when you need move to PDW. clustering brings concurrency to the system and reduces load, but it can't reduce
Time that a single query wocould take without any resource latency. to break this barrier, parallelism wocould be required to execute bits of the same request simultaneously and this is what exactly this setup wocould bring to the table. PDW partitions large tables
Using SS multiple physical nodes, each having its own dedicated CPU, memory, storage, and each running its own instance of SQL Server in a parallel shared nothing design. tables can either be replicated, where a copy will be on each node (usually for Dimension
Tables), or distributed, where portions of a table are uniformly distributed partition SS all nodes (usually for fact tables ).

One drawback to PDW is that it does not use SQL Server Management studio, but uses a third-party tool called
Nexus chameleon (this third-party tool is needed because SSMs hasn't been reworked to connect directly to the control node of the parallel data warehouse ). it also uses its own query engine and not all features
Of SQL Server are supported. so, you might not be able to use all your DBA tricks. and you wouldn't want to build a solution against SQL server and then just hope to upsize it to parallel data warehouse edition.

PDW uses multiple servers within the appliance, receivalized as if they were one uniied Data Warehousing resource available. can use up to 480 cores. PDW works by controlling several different physical servers each running their own instance of SQL
Server 2008 R2. The database and it's tables are spread into SS these physical servers but appear as one database and table (s) to the end-user.
Data Warehouse appliance or brain of the PDW manages query execution and the meta data for what is stored and processed on what portion of the PDW.

Microsoft it's experience with PDW showed when they migrated information security extends lidated Event Management (ICE) to PDW, they saw query performance improve to an average of 15-20 times faster in PDW, SSIS data load throughput of up to 285 GB/hour (
Minimal query performance impact), and support for up to 12 TB/day in throughput in SSIs. See

Video.

Part of the technology inconfigurated into PDW partition des a parallel database copy that enables rapid data movement and consistency between PDW and data marts used by SSAs.

In short, PDW is ideal for large data warehouses and Bi, but not for OLTP systems. write one check, and you get a complete soup-to-nuts data warehouse storage engine that includes des everything from the servers, San, configuration, and training.

HP callpdw by a different name: Enterprise Data Warehouse (EDW). Here is the layout of the HP enterprise data warehouse appliance (full specs
Here

Review and
Architecture Overview and performance guide). The architecture is
Hub-and-spoke and supports up to 47 servers, made up of
Control Rack and
Data rack. A one rack system has 17 servers, 22 processors/132 cores, and 125 TB and can be scaled out to a four rack system with 47 servers, 82 processors/492 cores, & 500 TB:

The future road map for PDW des
Column store, petabyte scalability, real-time data warehousing, MDM, and data quality.