A1.sources = R1
A1.sinks = S1
A1.channels = C1
A1.sources.r1.type = Netcat
A1.sources.r1.bind = localhost
A1.sources.r1.port = 44444
A1.sinks.s1.type = Hive
a1.sinks.s1.type.hive.metastore=thrift://master:9083
A1.sinks.s1.type.hive.datebase=bd14
A1.sinks.s1.type.hive.table=flume_user
a1.sinks.s1.serializer=delimited
A1.sinks.s1.serializer.delimiter= "\ T"
A1.sinks.s1.serializer.serdeseparator= ' \ t '
A1.sinks.s1.serializer.fieldnames=user_id,user_name,age
A1.channels.c1.type = Memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
A1.sources.r1.channels = C1
A1.sink.s1.channel = C1
Save
Run Flume-ng agent-c conf-f netcat_to_hive_user.conf--name A1
Hive Directory-->hcatalogatalog/The following package is copied to flume Lib.
In Hive
Create a hive table name is Flume_user and the field is User_id,user_name,age
CREATE TABLE Flume_user (
user_id int
, user_name string
, age int
)
Clustered by (user_id) to 2 buckets
Stored as Orc
Tableproperties ("Transactional" = ' true ')
Set Hive.support.concurrency=true
Set Hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.dbtxnmanager;
SELECT * FROM Flume_user
Open a connection to the server and enter:
telnet localhost 44444
then enter the corresponding character
1 Tab key Harry Tab key
Flume sink write to hive table
