Microsoft's chief architect Yaniv Pessach talks about Microsoft's big data Cosmos (Chinese meaning universe)

Microsoft's chief architect Yaniv Pessach talks about Microsoft's big data Cosmos (Chinese meaning universe)

Reference: http://www.quora.com/Distributed-Systems/What-is-Microsofts-Cosmos

Universe is Microsoft's internal big data analysis platform.

Cosmos universeIt is a distributed computing component (which is somewhat equivalent to hadoop's map/reduce solution using Microsoft Dryad, which (unlike map-Reduce) allows arbitrary Dag computing.
Cosmos universeSupports SQL-like syntaxes (similar to hive/pig), including distributed storage components (and HDFS). In general, the universe provides highly scalable and reliable, fault Tolerance and automatic scaling of massive datasets.

According to the articles [1] and [3], the universe allows the use of SQL-like syntaxes, such

Source = extract col1, col2 from ""
Data = select * from source where (condition)

It is used to extract data without explicit MAP/reduce.

Scope languageIn SQL, [3] and other operators are also supported, such as joining, downgrading, and user-defined operators. These include user code [1, 3] in programmable operations such as reducers (basically, input parsing), and parallel, optimized [1] execution plan-defined queries.

Cosmos universeMicrosoft is widely used internally. It stores a large amount of data on a large number of servers [4] and processes a large amount of data every day: "Every day we process hundreds of petabytes of data from Bing, adcenter, MSN, and Windows Live." [Software Development Engineer, Principal-bing job].

I think this is what we have published so far.

Refer to public resources: page on Microsoft? Stuff Yaron finds interesting? Http://hpts.ws/papers/201 1/Sessi ...? Page on Microsoft? Cloud storage @ Microsoft (with the cosmos Team) is hiring!

 

Microsoft's chief architect Yaniv Pessach talks about Microsoft's big data Cosmos (Chinese meaning universe)