Kafka Deployment

Kafka configures SASL authentication and permission fulfillment documentation. First, the release notes This example uses: zookeeper-3.4.10, kafka_2.11-0.11.0.0. zookeeper version no requirements, kafka must use version 0.8 or later. Second, zookeeper configuration SASLzookeeper cluster or single node configuration the same. Specific steps are as follows: 1, zoo.cfg file configuration add the following configuration: authProvider.1 = org.apa ...

"Editor's note" Shopify is a provider of online shop solutions company, the number of shops currently serving more than 100,000 (Tesla is also its users). The main frame of the website is Ruby on rails,1700 kernel and 6TB RAM, which can respond to 8,000 user requests per second. In order to expand and manage the business more easily, Shopify began to use Docker and CoreOS technology, Shopify software engineer Graeme Johnson will write a series of articles to share their experience, this article is the department ...

First, the cache or persistence RDD and similar, DStreams also allows developers to persist streaming data to memory. Use the persist () method on DStream to automatically persist RDDs in DStream into memory. This is useful if the data in DStream needs to be calculated more than once. Like reduceByWindow and reduceByKeyAndWindow this window operation, updateStateByKey this state-based operation, persistent ...

Flume-based Log collection system (i) architecture and Design Issues Guide: 1. Flume-ng and scribe contrast, flume-ng advantage in where? 2. What questions should be considered in architecture design? 3.Agent crash how to solve? Does 4.Collector crash affect? What are the 5.flume-ng reliability (reliability) measures? The log collection system in the United States is responsible for the collection of all business logs from the United States Regiment and to the Hadoop platform respectively ...

December 2014 12-14th, hosted by the China Computer Society (CCF), CCF large data Expert committee, the Chinese Academy of Sciences and CSDN co-organizer of the 2014 China Large Data Technology conference (DA data Marvell Conference 2014,BDTC 2014 will be opened at Crowne Plaza Hotel, New Yunnan, Beijing. The three-day conference aims to promote the development of large data technology in the industry, and to set up "large data Infrastructure" and "large data ..."

December 2014 12-14th, hosted by the China Computer Society (CCF), CCF large data Expert committee, the Chinese Academy of Sciences and CSDN co-organizer of the 2014 China Large Data Technology conference (DA data Marvell Conference 2014,BDTC 2014 will be opened at Crowne Plaza Hotel, New Yunnan, Beijing. The three-day conference aims to promote the development of large data technology in the industry, and to set up "large data Infrastructure" and "large data ..."

Star Ring Technology's core development team participated in the deployment of the country's earliest Hadoop cluster, team leader Sun Yuanhao in the world's leading software development field has many years of experience, during Intel's work has been promoted to the Data Center Software Division Asia Pacific CTO. In recent years, the team has studied large data and Hadoop enterprise-class products, and in telecommunications, finance, transportation, government and other areas of the landing applications have extensive experience, is China's large data core technology enterprise application pioneers and practitioners. Transwarp Data Hub (referred to as TDH) is the most cases of domestic landing ...

The drawbacks of "editor's note" Hadoop are also as stark as its virtues--large latency, slow response, and complex operation. is widely criticized, but there is demand for the creation, in Hadoop basically laid a large data hegemony, many of the open source project is to make up for the real-time nature of Hadoop as the goal is created, Storm is at this time turned out, Storm is a free open source, distributed, A highly fault-tolerant real-time computing system. The storm makes continuous flow calculation easy, making up for the real-time ...

As the big data ecosystem becomes more and more perfect, more and more users with different backgrounds will join the ecosystem. How to reduce the user's entry threshold and facilitate users to use big data resources quickly and conveniently is also a problem to be considered.

"Editor's note" in the famous tweet debate: MicroServices vs. Monolithic, we shared the debate on the microservices of Netflix, Thougtworks and Etsy engineers. After watching the whole debate, perhaps a large majority of people will agree with the service-oriented architecture. In fact, however, MicroServices's implementation is not simple. So how do you build an efficient service-oriented architecture? Here we might as well look to mixrad ...