Open Source Algorithms

Library and command line tools for XZ and LZMA compressed files

LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). A high compression derivative, called LZ4_HC, is available, trading customizable CPU time for compression ratio. LZ4 library is provided as open-source software using a BSD license. This benchmark simulates simple "static content transfer" scenario such as OS Kernel compression or video game's static assets (text/images/tables/scripts/etc) which loading from Flash Memory / HDD / SSD. In this case, compression time is completely ignored. Because only content developers compress the data at once and usually they don't care about its computational cost. But they always care end user's experience a.k.a. "loading time" and bandwidth. Please pay attention to "LZ4HC -9" which is quite faster than other methods.

RHash (Recursive Hasher) is a console utility for computing and verifying hash sums of files. It supports CRC32, CRC32C, MD4, MD5, SHA1, SHA256, SHA512, SHA3, AICH, ED2K, DC++ TTH, BTIH, Tiger, GOST R 34.11-2012, RIPEMD-160, HAS-160, EDON-R, and Whirlpool.

Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. Its format is stable and published as IETF RFC 8478. The negative compression levels, specified with --fast=#, offer faster compression and decompression speed in exchange for some loss in compression ratio compared to level 1, as seen in the table above. Zstd can trade compression speed for stronger compression ratios. It is configurable by small increment. Decompression speed is preserved and remain roughly the same at all settings, a property shared by most LZ compression algorithms, such as zlib or lzma.

FileVerifier++ is a Windows utility for calculating hashes using a number of algorithms including CRC32, MD5, SHA-1, SHA-256/224/384/512, WHIRLPOOL, and RIPEMD-128/160/256/320. Supported hash file formats include MD5SUM .MD5, SFV, BSD CKSUM, and others.

Fsum Frontend is a files integrity checker. It can calculate 96 hash and checksum algorithms(CRC32, MD5, SHA1, SHA2, ADLER, DHA256, FORK256, ...). You can verify your files using a .sfv/.md5/.sha1/.sha2 file or create your own checksum file.

A slightly reduced strength bulk encryption. In exchange for the reduced strength, you get the ability to rsync the encrypted files, so that local changes in the plaintext file will result in (relatively) local changes to the cyphertext file.

JDIFF is a program that outputs the differences between two binary files, either in binary format or in human readable format (detailed or summarized) and then allows to reconstruct the second file from the first one and the diff-file.

ArpON (ARP handler inspection) is a Host-based solution that make the ARP standardized protocol secure in order to avoid the Man In The Middle (MITM) attack through the ARP spoofing, ARP cache poisoning or ARP poison routing attack.

FRODO is a Java platform to solve Distributed Constraint Satisfaction Problems (DisCSPs) and Optimization Problems (DCOPs). It provides implementations for a variety of algorithms, including DPOP (and its variants), ADOPT, SynchBB, DSA...

minidjvu is a DjVu encoder for black-and-white images.

CloudI is an open-source private cloud computing framework for efficient, secure, and internal data processing. CloudI provides scaling for previously unscalable source code with efficient fault-tolerant execution of ATS, C/C++, Erlang/Elixir, Go, Haskell, Java, JavaScript/node.js, OCaml, Perl, PHP, Python, Ruby, or Rust services. The bare essentials for efficient fault-tolerant processing on a cloud!

The Simple, Scalable, Script-based Science Processor for Measurements (S4PM) is a system for highly automated processing of science data, scalable up to large processing systems and scalable down to small, special-purpose processing strings.

Genetic Algorithm File Fitter, GAFFitter for short, is a tool based on a genetic algorithm (GA) that tries to fit a collection of items, such as files/directories, into as few as possible volumes of a specific size (e.g. CDs or DVDs).

(Production / Stable) Command line tool to compute the 64 bit CRC of one or more files.

CADI is an implementation of the JPIP standard (JPEG2000 Interactive Protocol - Part 9). CADI includes a JPIP server, a JPIP proxy, and a JPIP Client. CADI is designed and programmed with the aim to provide a flexible framework to test and develop ne

OpenDark is a free simplification of 'dark'. It's the first open BWT-DC compressor, which has a simple code, high speed and compression ratio. And it's waiting for your ideas! :)

AVL Array is a sequence container (like std::vector or std::list) that allows fast insert/remove AND fast random access. Shiftable Files offers the usual file primitives plus fast insert/remove. Get the latest version via BZR in the Develop section.

This project uses massively parallel Graphics Processing Units(GPU) for neural network(Backpropagation) purposes.

TER implements the CCSDS Recommendation for Image Data Compression (CCSDS 122.0-B-1). TER also adds new features to the Recommendation. TER is designed and programmed with the aim to provide a good basis to test and develop the CCSDS Recommendation.

Random Thoughts On Distributed Algorithms: a collection of tools and ideas for the development of distributed applications. The first released part is an RxRPC security model based on OpenSSL and elliptic curve criptography. Please check docs for info...

Tpl makes it easy to serialize your C data using just a handful of API functions. The data is stored in its native binary form for maximum efficiency. C, Perl and XML supported. Data is portable across CPU types and OS's from Unix to Mac to Windows.

OpenSource C++ libraries. Full list of classes and functions listed at yf-library.sourceforge.net

This CPU Benchmark is written in .NET C# and use the Linpack and Whetstone algorithm.

a framework library with a template method interface and inheritable classes for an executable process design which produces an executable which is both a GUI and a console application. It provides multi-threading for work items of the provider process .