Wal-Mart Big Data Technology Truth

Wal-Mart, which created the classic business case of "beer and diapers", is one of the first traditional business giants to start investing and deploying large data applications, and it is popular to say that big data is not Wal-Mart, but Wal-Mart is a natural big data.

Before the big data concept detonated the popular industry, Wal-Mart had already started the website database Integration migration and the Hadoop cluster expansion work, acquired Kosmix, built on this foundation Walmart Labs, and in recent years has embarked on the acquisition of focus on data mining or mobile social start-up companies such as Oneops, Inkiru,tasty Labs,oneriot, enter the Internet.

Wal-Mart hopes to make consumers a bigger spender with big data applications, while catching up with the leader Amazon in the E-commerce arena.

Through its own data accumulation and mergers and acquisitions research and development, Wal-Mart already has a large database that covers consumer offline trading data, Wal-Mart's network mall electronic data and social media application data as a whole, to minimize the implementation cost and create new consumption opportunities for Wal-Mart before making decisions.

Can be a virtual scene to say Wal-Mart data operations:

Sam, a senior member of Wal-Mart, has been buying a variety of goods, quantity, model, time information, payment method information, merchandise distribution information, membership card information, address, contact details and even Sam's shopping flow monitoring video at Wal-Mart in the past five years. Access control data, such as offline consumption information has been detailed records of Wal-Mart's information system.

When Sam recently drove to the nearest Wal-Mart store, the Wal-Mart app, which has just entered the parking lot, Sam's mobile phone or mobile terminal such as the ipad, may have received information on the recommended and electronic coupons for more than 80% of the items on the shopping list. App also labels the location of each item in the store flow, model information, Wal-Mart test "Scan and Go" system in the future allows Sam to scan the product with a mobile phone, and then the cashier phone scan payment. At the instant of the move payment, Wal-Mart has started updating Sam's related consumer information on the data side, and has started to give Sam the time and quantity to buy the next commodity, such as milk and beer, and submit data to the app referral business.

In addition, Sam loved a 899-dollar Wilson tennis racket but did not join the scan list, the plastic sports model next to the racquet recorded his movements, including Sam's Stay, the number of times he took his tennis racket and his eye view of the racket, and even his left hand grip or right grip. In the second month, Sam's mobile phone message already had a Babolat light carbon racket recommended by Wal-Mart.

Behind the SAM case scene is the Wal-Mart labs and related teams that maintain their large data work. Walmart Labs embarked on the integration of Wal-Mart's Web site, will be the pilot Hadoop data node expansion of the formal cluster, decentralized storage in Oracle, Neteeza, Greenplum and other databases in a unified repository of resources in the Hadoop data Warehouse, and try to conduct internal data sharing analysis. Through the development of large data tools to load transaction data, the volume of relatively small but highly structured retail data combined with E-commerce site data for the forward shopping selection forecast provides a basic support.

The 2000 online store as an occasional product online, but ebay and Amazon let Wal-Mart recognized the terrible E-commerce, then in the next few years began to force the line. The Social genome, launched by Walmart Labs, allows Wal-Mart to combine consumer offline shopping data, Web browsing click Data, social networking data, and consumer personal data into a powerful database with extended and directed predictive capabilities, Gradually solve the online link and offline channel data matching difficulties, and with the help of consumers on the social network to show interest, predict they may be in Walmart next purchase products.

Through the deep data mining to the social network, realizes the website homepage, the product search personalization, arranges the result on the page according to the consumer's interest. Data show that Wal-Mart's global internet sales growth of 30 in the first time in ten years overtook Amazon as of January 31, 13.

And then Sam's Wal-Mart shopping story:

The end of the month Sam car was borrowed, so chose to go to Wal-Mart's online mall, the cookie automatically logged in after the last landing of the Wal-Mart shop was six months ago. Sam's home page recommendation has a member 80 percent canned Wilson tennis, as well as a new special price of all steel Siemens Juicer, Sam smile: Neighbor Peter on Twitter complaining about their own juice machine really rubbish, the day before yesterday, the praise, Wal-Mart know! Sam then discovered that his only Facebook-focused Robbie Williams new album was in sight.

In addition, a nearby gym year card is on sale, Sam touched the belly of the weight of a bite to add the year card to the shopping cart, the heart also complained about Wal-Mart you know so much.

In fact, Wal-Mart's consumer online data such as browsing the type of time, shopping cart status and payment methods, search records, etc. will be structured and then integrated into the global Hadoop data Warehouse, Six months ago Sam's search for tennis rackets through the Polaris search engine was ready for Wal-Mart's physical store to push Sam Babolat light carbon rackets through apps. Wal-Mart's big online data has been able to generate incremental sales for its offline revenues.

Incorporated into the Kosmix team, Wal-Mart values the semantic analysis of social media kosmix, using Facebook and Twitter data, engineers from the Daily hot news, the introduction of social events echoed in the product, create consumer demand, while, for social networks, the nature of fast message flow, Wal-Mart's large data labs have developed a tracking system that combines mobile Internet access, and specializes in tracking huge social dynamics, with more than 1 billion volumes of information to handle each day.

Wal-Mart Information Center deputy general Manager Turrell pointed out, "Wal-Mart itself is a massive data system, applicable to a variety of commercial analytical behavior." The media said that Wal-Mart has nearly 70% Americans of relevant consumer data, more than 80% of the domestic mailbox information. Through its powerful database algorithm recommendation and prediction program, personal consumption data can predict a series of highly sensitive personal information, Wal-Mart will clearly know who you are, what you want to buy, and even what you want to do next.

Due to the lack of personal information protection awareness, a lot of colored, low-income, marginalized communities in the community of personal data information is violated by large data business plan, and vulnerable people in Wal-Mart data tagging classification is easy to distinguish between, suffer from the weak discriminatory treatment.

Big data will be more "big" and "fast" as the data is structured and scaled, and the world's largest retailer is increasingly worried about the use of data technology to chase profits:

Technology in the development of business will not produce a similar "1984" in the Big Brother 2.0.

Note: This article starts idonews column, reprint please indicate the source and the origin.