Open Source Linux Information Analysis Software

Elasticsearch is a distributed, RESTful search and analytics engine that lets you store, search and analyze with ease at scale. It lets you perform and combine many types of searches; it scales seamlessly, and offers answers incredibly fast with search results you can rank based on a variety of factors. Elasticsearch can be used for a wide variety of use cases, from maps and metrics to site search and workplace search, and with all data types.

An eclipse plugin providing tail in view component.

The Genalyze log analyzer, a simple yet highly extensible framework for parsing log files, was designed for parsing T-Server log files generated by Genesys contact center software platform, yet is generic and easy enough to adapt for other log formats.

This very simple perl script parses you iptables log files and produces a report in text format with a summary based on the prefix of the log ( --log-prefix option of iptables ). Prefix description is allowed.

The Open Log Engine is a powerful tool for advanced logging in games or programs. It easily allows you to make logging outputs in your program and analyse them afterwards with the Open Log Viewer, which provides you a powerful filtering system.

PerfParse. Storage and analysis of binary performance data produced by Nagios. High quality accurate graphs of live data from standard Nagios plugins. Permanent history of plugin results with advanced analysis tools.

Retrieve statistics from SNMP-aware devices, store them in a database and graphically display them for monitoring and notification purposes.

Plinko was originally an experiment with Prefix Trees and log parsing. The general concept is to have a single end point you can send any data to, in a "fire and forget" fashion. Plinko should identify and parse the data completely without the sending system caring what it sent. The latest version supports named fields in the STL files for tagging data parsed in the Prefix Tree and anonymous functions for parsing dynamic message payloads. We now output JSON objects and I'm working on HBase integration. By outputting to JSON it also leaves open the possibility for on the fly in memory correlation between events. Read the included README before starting, it has a quick start guide and info on the constructors.

A logging shell that can log commands from a shell. Designed to act as an automated lab-book for people that conduct experiments in their standard shell environments. Users can log and re-run experiments with little/no manual effort or interference.

The ProM Import Framework allows to extract process enactment event logs from a set of information systems. These can be exported in the MXML format, which is the standard event log data format for Process Mining analysis techniques.

PyLogAnalyzer is a tool that receives an input log in black and white, a configuration INI file, which contains the list of rules to process the input, and an output file where to save the results. These rules permit to detect an input line according to a regular expression (regex) or line number range, filter it, pass it, colorise in foreground and background, columnise the groups of the regex and enable or disable the rule. The final goal of this tool is to ameliorate reading long and difficult b/w logs, speed up the analysis and enhance day to day wearying activities. As common application, you can colorise adb logcat logs taking the example files and using them (and also, filter unnecessary lines). You can INSTALL it from PyPI: $ python -m pip install pyloganalyser

py-rrdtool is a Python module provides an interface to RRDTool, the well known graphing/logging tool.

Searches arbitrary text input (logs, etc) for regular expression(s) matching, and then executes arbitrary SQL statement(s) with matched substrings used as parameters. Offers daemon, incremental and transactional features.

This project is a python script to abstract S.M.A.R.T. messages from /var/log/messages (from the smartd daemon) into .csv file(s), one for each disk, suitable for graphing.

The Internet Censor is a multi-platform, Internet clustering program, for which the resulting data will be used in the creation of a non-profit content-filtering Internet Search Engine for children.