Fluentd + fluentd in RedhatLinux for bridging

The experiment environment consists of two testing machines: Server201 (192.168.10.201) and Server202 (192.168.19.202). fluentd is installed respectively. The configuration on the two machines is described in detail. The purpose is to place the logs on Server202 in a directory under Server201 through fluentd bridging. 1. fluentd. conf on Server202: & lt; source & gt;

The experiment environment consists of two testing machines: Server201 (192.168.10.201) and Server202 (192.168.19.202). fluentd is installed respectively.

The configuration on the two machines is described in detail. The purpose is to place the logs on Server202 in a directory under Server201 through fluentd bridging.

1. fluentd. conf on Server202:

Type tail

# Log 1 to be dumped
Path/home/aircom/ mongodb-linux-i686-2.0.6/bin/logs/1001.log
Format/^ *(? . *) $/
Tag mongo. apache


Type tail

# Logs to be dumped 2
Path/home/aircom/ mongodb-linux-i686-2.0.6/bin/logs/1002.log
Format/^ *(? . *) $/
Tag mongo. apache

The source log contains two log files. Path/home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001. log and path/home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

Tag mongo. apache to facilitate match.

Type forward

# Dumping Server
Host 192.168.19.201

Host 192.168.19.201

The two machines can be added as backup machines. The IP addresses of the master and slave machines are the same, and different IP addresses can be added in the production environment.

2. Configure fluent. conf on Server201

To facilitate the test, we directly modify and open the comment, which means to match logs without match and write them to the file.

Type file

# Storage path
Path/var/log/fluent/else
Compress gz

Start fluentd

201 view the configuration path

[Root @ Server201 fluentd-0.10.6] #Ll/var/log/fluent/
Total 68
-Rw-r -- 1 root 61675 Sep 28 :20 else.20120928.b4cab973e8d931ee9

Generated the file else.20120928.b4cab973e8d931ee9

3. Start fluentd: fluentd-c fluent. conf-deamon-o log/log. log under Server202.

Tail input log to get the last few lines of output

09:58:45 + 0800: adding match pattern = "mongo. **" type = "forward"
09:58:45 + 0800: 'host' option in forward output is obsoleted. use' Host xxx 'Instead.
0800 09:58:45 + 192: adding forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224 weight = 60
09:58:45 + 0800: 'host' option in forward output is obsoleted. use' Host xxx 'Instead.
0800 09:58:45 + 192: adding forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224 weight = 60
09:58:45 + 0800: listening fluent socket on 0.0.0.0: 24224
09:58:45 + 0800: following tail of/home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
09:58:45 + 0800: following tail of/home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

Two lines marked with colors indicate two machines that accept data. The port number is 24224,

When sending data, the log will also get the following output:

0800 10:02:07 + 192: detached forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224 phi = 8.280932634573869
0800 10:02:07 + 192: detached forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224 phi = 8.280785193737733
0800 10:02:42 + 192: recovered forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224
0800 10:02:42 + 192: recovered forwarding server '192. 168.19.201: 24224 'host = "192.168.19.201" port = 24224

4. view the logs after the bridge dump on Server201

Tail-f/var/log/fluent/else.20120928.b4cab973e8d931ee9

The following output is displayed. The log refresh frequency is 1 minute.

10:23:09 + 0800: plugin/in_forward.rb: 140: initialize: accepted fluent socket object_id = 242397940
10:23:09 + 0800: plugin/in_forward.rb: 181: on_close: closed fluent socket object_id = 242397940
2012-09-28T10: 23: 09 + 08: 00 fluent. trace {"message": "accepted fluent socket object_id = 242397940 "}
2012-09-28T10: 22: 23 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:23 [conn369] end connection 192.168.19.201: 51962 "}
2012-09-28T10: 22: 23 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:23 [initandlisten] connection accepted from 192.168.19.201: 51968 #372 "}
2012-09-28T10: 22: 27 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:27 [conn370] end connection 192.168.19.201: 51963 "}
2012-09-28T10: 22: 27 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:27 [initandlisten] connection accepted from 192.168.19.201: 51969 #373 "}
2012-09-28T10: 22: 33 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:33 [conn371] end connection 192.168.19.202: 39038 "}
2012-09-28T10: 22: 33 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:33 [initandlisten] connection accepted from 192.168.19.202: 39045 #374 "}
2012-09-28T10: 22: 34 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:34 [conn378] end connection 192.168.19.202: 55169 "}
2012-09-28T10: 22: 34 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:34 [initandlisten] connection accepted from 192.168.19.202: 55176 #381 "}
2012-09-28T10: 22: 35 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:35 [conn379] end connection 192.168.19.201: 41497 "}
2012-09-28T10: 22: 35 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:35 [initandlisten] connection accepted from 192.168.19.201: 41503 #382 "}
2012-09-28T10: 22: 37 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:37 [conn380] end connection 192.168.19.201: 41498 "}
2012-09-28T10: 22: 37 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:37 [initandlisten] connection accepted from 192.168.19.201: 41504 #383 "}
2012-09-28T10: 22: 53 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:53 [conn372] end connection 192.168.19.201: 51968 "}
2012-09-28T10: 22: 53 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:53 [initandlisten] connection accepted from 192.168.19.201: 51974 #375 "}
2012-09-28T10: 22: 56 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:56 [clientcursormon] mem (MB) res: 76 virt: 419 mapped: 224 "}
2012-09-28T10: 22: 57 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:57 [conn373] end connection 192.168.19.201: 51969 "}
2012-09-28T10: 22: 57 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:22:57 [initandlisten] connection accepted from 192.168.19.201: 51975 #376 "}
2012-09-28T10: 23: 01 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:01 [clientcursormon] mem (MB) res: 44 virt: 329 mapped: 160 "}
2012-09-28T10: 23: 03 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:03 [conn374] end connection 192.168.19.202: 39045 "}
2012-09-28T10: 23: 03 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:03 [initandlisten] connection accepted from 192.168.19.202: 39051 #377 "}
2012-09-28T10: 23: 04 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:04 [conn381] end connection 192.168.19.202: 55176 "}
2012-09-28T10: 23: 04 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:04 [initandlisten] connection accepted from 192.168.19.202: 55182 #384 "}
2012-09-28T10: 23: 05 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:05 [conn382] end connection 192.168.19.201: 41503 "}
2012-09-28T10: 23: 05 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:05 [initandlisten] connection accepted from 192.168.19.201: 41509 #385 "}
2012-09-28T10: 23: 07 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:07 [conn383] end connection 192.168.19.201: 41504 "}
2012-09-28T10: 23: 07 + 08: 00 mongo. apache {"message": "Fri Sep 28 10:23:07 [initandlisten] connection accepted from 192.168.19.201: 41510 #386 "}
2012-09-28T10: 23: 09 + 08: 00 fluent. trace {"message": "closed fluent socket object_id = 242397940 "}

So far, two fluentd bridges have been fully implemented on two servers.