Several problems in building a log system using Fluency + MongoDB

1. the following error occurs during running,

Error = "Unknown output plugin 'mongo '. run'gem search-rd fluent-In in' to find plugins"
Search for mongodb plug-ins
Gem search-rd fluent-plugin-mongo
Expected result
* ** Remote gems ***


Fluent-plugin-mongo (0.6.7)
Author: Masahiro Nakagawa
Homepage: https://github.com/fluent/fluent-plugin-mongo


MongoDB plugin for Fluent event collector
Install this plugin
Gem install fluent-plugin-mongo


2. The following prompt may be displayed during running:
** Notice: C extension not loaded. This is required for optimum MongoDB Ruby driver performance.
You can install the extension as follows:
Gem install bson_ext


If you continue to receive this message after installing, make sure that
Bson_ext gem is in your load path and that the bson_ext and mongo gems are of the same version.
Run the gem install bson_ext command as prompted.

3. flunt. conf

<Source>

Type tail
Path/home/aircom/ mongodb-linux-i686-2.0.6/bin/logs/1001.log
Format/^ *(? <Message>. *) $/
# Format/^ (? <Time> [^] * [^] * [^] *) (? <Host> [^] *) (? <Ident> [a-zA-Z0-9 _ \/\. \-] *) (? :\[(? <Pid> [0-9] +) \])? [^ \:] * \: * (? <Message>. *) $/
# Time_format % B % d % H: % M: % S
Tag mongo. apache
</Source>
<Match mongo. **>
Type mongo
Database apache
Collection fmongo
Host Server202
Port 10001
Flush_interval 10 s
</Match>

4. login primary node run command

PRIMARY> db. fmongo. find (). sort ({"_ id":-1 })
{"_ Id": ObjectId ("5062d3700ce0ea43fc000387"), "message": "Wed Sep 26 18:05:28 [initandlisten] connection accepted from 192.168.19.201: 34114 #3204", "time ": ISODate ("2012-09-26T10: 05: 28Z ")}
{"_ Id": ObjectId ("5062d3700ce0ea43fc000386"), "message": "Wed Sep 26 18:05:28 [conn3201] end connection 192.168.19.201: 34108", "time ": ISODate ("2012-09-26T10: 05: 28Z ")}
{"_ Id": ObjectId ("5062d3700ce0ea43fc000385"), "message": "Wed Sep 26 18:05:28 [initandlisten] connection accepted from 192.168.19.201: 34113 #3203", "time ": ISODate ("2012-09-26T10: 05: 28Z ")}
{"_ Id": ObjectId ("5062d3700ce0ea43fc000384"), "message": "Wed Sep 26 18:05:28 [conn3200] end connection 192.168.19.201: 34107 ",

5. startup logs when fluentd is configured properly

Fluentd-c fluent. conf-vv &
[1] 15847
[Root @ Server202 fluentd-0.10.6] #2012-09-26 16:28:26 + 0800: fluent/supervisor. rb: 153: supervise: starting fluentd-0.10.25
16:28:26 + 0800: fluent/supervisor. rb: 235: read_config: reading config file path = "fluent. conf"
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered buffer plugin 'file'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered buffer plugin 'memory'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'exec'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'forward'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'gc _ stat'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'http'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'object _ space'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'status'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'tcp'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'unix'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'syslog'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered input plugin 'tail'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'copy'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'exec'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'exec _ filter'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'file'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'forward'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'null'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'roundrobin'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'stdout'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'tcp'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'unix'
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'test'
16:28:26 + 0800: fluent/engine. rb: 63: block in configure: adding source type = "forward"
16:28:26 + 0800: fluent/engine. rb: 63: block in configure: adding source type = "http"
16:28:26 + 0800: fluent/engine. rb: 63: block in configure: adding source type = "tail"
16:28:26 + 0800: plugin/in_tail.rb: 49: configure: 'pos _ file path' parameter is not set to a 'tail' source.
16:28:26 + 0800: plugin/in_tail.rb: 50: configure: this parameter is highly recommended to save the position to resume tailing.
16:28:26 + 0800: fluent/engine. rb: 79: block in configure: adding match pattern = "mongo. **" type = "mongo"
16:28:26 + 0800: fluent/plugin. rb: 85: register_impl: registered output plugin 'mongo'
16:28:27 + 0800: plugin/out_0000.rb: 70: configure: Setup mongo configuration: mode = normal
16:28:27 + 0800: fluent/engine. rb: 79: block in configure: adding match pattern = "debug. **" type = "stdout"
16:28:27 + 0800: plugin/in_forward.rb: 60: listen: listening fluent socket on 0.0.0.0: 24224
16:28:27 + 0800: plugin/in_http.rb: 74: start: listening http on 0.0.0.0: 8888
16:28:27 + 0800: plugin/in_tail.rb: 262: initialize: following tail of/home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
16:28:49 + 0800: fluent/parser. rb: 38: call: pattern not match: Wed Sep 26 16:28:49 [FileAllocator] allocating new datafile/home/aircom/mongodb-linux-i686-2.0.6/bin/db/apache. ns, filling with zeroes...
16:28:49 + 0800: fluent/parser. rb: 38: call: pattern not match: Wed Sep 26 16:28:49 [FileAllocator] creating directory/home/aircom/mongodb-linux-i686-2.0.6/bin/db/_ tmp
16:28:49 + 0800: fluent/parser. rb: 38: call: pattern not match: Wed Sep 26 16:28:49 [FileAllocator] allocating new datafile/home/aircom/mongodb-linux-i686-2.0.6/bin/db/apache.0, filling with zeroes...
16:28:50 + 0800: fluent/parser. rb: 38: call: pattern not match: Wed Sep 26 16:28:50 [FileAllocator] allocating new datafile/home/aircom/mongodb-linux-i686-2.0.6/bin/db/apache.1, filling with zeroes...
16:28:50 + 0800: fluent/parser. rb: 38: call: pattern not match: Wed Sep 26 16:28:50 [conn2616] build index done 0 records 0.001 secs