Fluentd

Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data. Fluentd offers features such as a community-driven support, ruby gems installation, self-service configuration, OS default Memory allocator, C & Ruby language, 40mb memory, requires a certain number of gems and Ruby interpreter and more than 650 plugins available.

Fluentd tries to structure data as JSON as much as possible which allows Fluentd to unify all facets of processing log data such as collecting, filtering, buffering, and outputting logs across multiple sources and destinations (Unified Logging Layer).

Fluentd has a flexible plugin system that allows the community to extend its functionality where the 500+ community-contributed plugins connect dozens of data sources and data outputs and by leveraging the plugins, users can start making better use of their logs right away.

Fluentd is written in a combination of C language and Ruby, and requires very little system resource. Fluentd supports memory- and file-based buffering to prevent inter-node data loss. Fluentd also support robust failover and can be set up for high availability.

The architecture of Fluentd (Sponsored by Treasure Data) is very similar to Apache Flume or Facebook’s Scribe but Fluentd is easier to install and maintain and has better documentation and support than Flume and Scribe. Fluentd helps users collect and analyze an app log and midleware log efficiently and as a result, it improves daily operation and services. Fluentd can also be used to tally action logs and monitor action logs for puzzle games.