Using the shell to read a text file sent to Kafka

#!/bin/SH# # parameter definition dt=`Date+"%y%m%d"-D"-1 days"' Outpath=/xxxx_log_${dt}.txt brokerlist=192.168.1.:9092,192.168.1.101:9092,192.168.1.102:9092Echo$dt $outpath $brokerlist # # Query hive table, output file to the specified path hive-E"Selectname,age from Test "> ${outpath}
# # Determine the file size, if it is greater than 0, load the file, write Kafka
# # Note the end of the | > OUT.txt to add, otherwise there will be a lot of strange greater than the number fileSize=`du-B ${outpath} |awk '{print $}'`if[$fileSize-GT0] Then    Cat${outpath} |./kafka_2. One-1.0.0/bin/kafka-console-producer.SH--broker-list ${brokerlist}--Sync--topic Test1 |> out.txt fi

Using the shell to read a text file sent to Kafka