Open Source Log Analysis Software

Sarg - Squid Analysis Report Generator is a tool that allow you to view "where" your users are going to on the Internet. Sarg generate HTML reports, with informations about users, IP Addresses, bytes, sites and times.

ProM is the comprehensive, extensible framework for process mining. Process Mining deals with the a-posteriori analysis of (business) processes using enactment logs.

Fluent Bit is a super-fast, lightweight, and highly scalable logging and metrics processor and forwarder. It is the preferred choice for cloud and containerized environments. A robust, lightweight, and portable architecture for high throughput with low CPU and memory usage from any data source to any destination. Proven across distributed cloud and container environments. Highly available with I/O handlers to store data for disaster recovery. Granular management of data parsing and routing. Filtering and enrichment to optimize security and minimize cost. The lightweight, asynchronous design optimizes resource usage: CPU, memory, disk I/O, network. No more OOM errors! Integration with all your technology, cloud-native services, containers, streaming processors, and data backends. Fully event-driven design leverages the operating system API for performance and reliability. All operations to collect and deliver data are asynchronous.

Comprehensive & high performance data distribution &transformation system. Features a simple, user-friendly event driven scripting interface transparently generates & execs highly efficient Perl/C code. Uses:ETL,datawarehousing,statistics,data-cleansing.

SRG is a log file analyser and report generator for the Squid web proxy. It is fast and was created to allow easy integration with other authentication systems. SRG is flexible and can report right down the the level of individual files requested.

Use Pentaho open source business intelligence tools and MySQL to collect & distribute web analytics (clickstream) data. Extract data from logs, load database tables, & present the information in dashboards, analysis cubes, and reports for business users. This project has been moved to github - https://github.com/cjlavigne/breadboard

Data mining tool for sequences (e.g. trajectories on a map, visited web pages, etc.) that creates a succinct description of the sequences, given a taxonomy (e.g. regions and sub-regions in the map, categories and sub-categories of pages, etc.).

A tool to visualize interactively huge amounts of eventdata. Uses an innovative hierarchical zoomlens scaling from quarters down to 50 ms.

AWStats Enterprise Manager is a tool for managing awstats configuration creation and logfile processing, in a multi-server environment. This script is designed to pull all the webserver logs, for every server, and parse them with awstats.

Biik: A Zoomable, Context-Aware Visualization System for SNORT IDS

Platform based on JMS queues to centralize System Event Messages (syslog) and Application Layer Messages, persisting them in several Databases and providing several pre- and post- processes according the message's nature, like encryption, mail.

Windows monitoring agent

DAD is a Windows event log and syslog management tool that allows you to aggregate logs from hundreds to thousands of systems in real time. DAD requires no agents on the servers or workstations. Correlation and analysis is driven through a web front end.

FAST library for storing and displaying various types of statistics. Rock stable and robust and well-designed image output.

An extended version of GNU tail, with features targeted primarily (though not exclusively) at bandwidth and latency analysis of Financial Information eXchange (FIX) protocol log files.

Flamingo is a open-source Big Data Platform that combine a Ajax Rich Web Interface + Workflow Engine + Workflow Designer + MapReduce + Hive Editor + Pig Editor. 1. Easy Tool for big data 2. Use comfortable in Hadoop EcoSystem projects 3. Based GPL V3 License Supporting Pig IDE, Hive IDE, HDFS Browser, Scheduler, Hadoop Job Monitoring, Workflow Engine, Workflow Designer, MapReduce.

FwPlatinum provides real time data analysis of CheckPoint FireWall-1 connection logs. Data is stored for up to 2 years, and is accessed by CGI generated graphs which can drill down to the specific time that you are interested in.

LGX Report (freeware based) Web Application Starter Kits - Server Log Analysis, Time Tracker, Portal Base.

Concatenate successive lines within a text file, with an option to skip a number of subsequent line(s), and an option to insert a character or string between lines. Useful for turning multi-line log files into single line files (think CSV!)

Visualize Application and Firewall Logs with Google Earth. A web based PHP parser feeds Google Earth with near realtime data.

Logbus-ng consists in a set of tools to aid developers perform Log Analysis in all the stages: log generation, collection, distribution, storage and analysis. It is designed specifically for Field Failure Data Analysis in critical distributed systems

A code toolkit for collecting usability data on the web. Tools range from an entirely client side solution for measuring scrolling and render time to a hybrid solution logging user activity every 200 milliseonds. Currently IE only.

NARC Network Analysis Reporting Console takes output from security tools like Nessus, and NMap & populates a database via automated scripts for reporting purposes. Version 0.DC14 also includes rudimentary reporting capabilities. New from kaos.theory

OCD is a simple web UI written in PHP [and little CSS]. It displays dashboard of current OSSEC activity. Originally written to allow NOC to monitor OSSEC events in realtime.

The ProM Import Framework allows to extract process enactment event logs from a set of information systems. These can be exported in the MXML format, which is the standard event log data format for Process Mining analysis techniques.