kafka across data centers

Speaking of headings, this is only a small part of the real-time architecture. Download the latest version flume:apache-flume-1.6.0-bin.tar.gz Unzip, modify Conf/flume-conf.properties name can write casually. What I currently achieve is to read the data from the directory to write to the Kafka, the principle of the east of the Internet a lot of, only to connect the code: a1.sources = R1 a1.sinks = K1 a1.cha

A recent project uses the Kafka client to receive messages, requiring that they be written to the file (in order). There are 2 ideas: 1. Use log4j to write the file, the advantage is stable and reliable, the file according to the setting, automatically separates the size. The disadvantage is that there is no way to do when writing files to a certain number or a certain amount of time, automatically switch the function of the directory. If you are loop

", "total": {"pct": 0. 000000}}, "Memory": {"RSS": {"bytes": 7495680, "pct": 0.000400}, "Share": 0, "size": 1806336}, "name": " SearchFilterHost.exe "," Pgid ": 0," pid ": 8776," ppid ": 2524," state ":" Running "," username ":" NT Authority\\System "}," Type ":" Metricsets "} val lines = kafkautils.createdirectstream[string, String,stringdecoder, Stringdecoder] (SSC, Kafkaparams,topicset)//val message = Lines.map (_._1) map (_._1) data is empty null

Training Big Data architecture development, mining and analysis!from zero-based to advanced, one-to-one technical training! Full Technical guidance! [Technical qq:2937765541] https://item.taobao.com/item.htm?id=535950178794-------------------------------------------------------------------------------------Java Internet Architect Training!https://item.taobao.com/item.htm?id=536055176638Big Data Architectu