Open Source Big Data Tools - Page 4
Sort By:
Big Data Tools
View 281 business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
-
deskbird is the most intuitive desk booking app for your hybrid office.For companies in need of a people-centric workplace management solution so employees can see who is in the office, schedule their office and work-from-home days, and book resources for office days.
-
1
PROPER is a package for visual evaluation of ranking classifiers for biological big data mining studies in the mathematical language MATLAB. It is an efficient tool for optimization and comparison of the state-of-the-art ranking classifiers by generating over 20 different high quality two- and three-dimensional performance curves.
-
2
PROPER is a package for visual evaluation of ranking classifiers for biological big data mining studies in the mathematical language MATLAB. It is an efficient tool for optimization and comparison of the state-of-the-art ranking classifiers by generating over 20 different high quality two- and three-dimensional performance curves.
-
3
PanoramaServer
Open Source Panorama Server for free virtual tour of 360 degrees views
Ideal for creating virtual tours of panoramic views for all sorts including property exhibition for brokers at real estate agencies/property agents, tour guide for indoor/outdoor venues, information to public/private facilities for curators, travel journal for tourist as log book, backdrop setting for storytelling, treasure hunt like games, big data mining for pattern through computer vision in artificial intelligence, etc. It is like creating your own Google Map Street View. All is required by the user is to have photos of equirectangular format (panorama) taken from 3D cameras common for on-site premises. These images can be referenced by the PanoramaServer to create virtual travels with 360 degrees view where viewers can navigate to different locations, view information, etc. If made available online to general public over the internet, can even share the link of your virtual trips. PanoramaServer is free as it is open source licensed. -
4
PartialLoader
A class library for issuing big data in parts.
-
The Industry Leading Platform for eCommerce Enablement and AnalyticsMikMak’s Where to Buy Shoppable Solutions help multichannel brands drive sales, grow market share, and increase profitability while reducing costs across categories such as CPG, Grocery, Alcohol, Beauty, Personal Care, Pet Care, Home Care, Consumer Electronics, Home Appliances, Toys, and more.
-
5
Random Bits Forest
RBF: a Strong Classifier/Regressor for Big Data
We present a classification and regression algorithm called Random Bits Forest (RBF). RBF integrates neural network (for depth), boosting (for wideness) and random forest (for accuracy). It first generates and selects ~10,000 small three-layer threshold random neural networks as basis by gradient boosting scheme. These binary basis are then feed into a modified random forest algorithm to obtain predictions. In conclusion, RBF is a novel framework that performs strongly especially on data with large size. -
6
Random Bits Regression
Random Bits Regression is a strong general predictor.
We proposed an accurate, robust and fast general predictor (RBR) for regression and classification in big data era. The application of this method is very broad, from science to industry, finance and health. The accuracy and robustness improvement of our method over existing method could bring huge benefits in some critical applications. For example, natural disaster prediction, stock price prediction, personal/population disease prediction. The fast-speed nature of our method not only allows big data analysis but also enables real-time recognition and predictions. The RBR framework also hints the mechanism of brain function and leads to a "wide learning" hypothesis. We believe that this method will make a great impact and enable many downstream applications. -
7
Red-RF
Reduced Random Forest for big data
Red(uced)-RF, a new type of Random Forests that adopts dynamic data reduction and weighted upvoting techniques. Red-RF is favorably applicable to big data: it demonstrates an accurate and efficient performance while achieving a considerable data reduction w.r.t. dataset size. Manuscripts available on IEEE Xplore: H. Mohsen, H. Kurban, K. Zimmer, M. Jenne and M. Dalkilic. Red-RF: Reduced Random Forests using priority voting & dynamic data reduction. In IEEE BigData Congress'2015. H. Mohsen, H. Kurban, M. Jenne and M. Dalkilic (2014). A New Set of Random Forests with Varying Dynamic Data Reduction and Voting Techniques. In IEEE DSAA'2014. Code, README file, and a sample input file are available in Files/ directory above. For inquiries, please contact us at hmohsen@imail,iu.edu (or @indiana.edu). -
8
Redis Desktop Manager
:wrench: Cross-platform GUI management tool for Redis
Redis Desktop Manager is a fast, open source Redis database management application based on Qt 5. It's available for Windows, Linux and MacOS and offers an easy-to-use GUI to access your Redis DB. With Redis Desktop Manager you can perform some basic operations such as view keys as a tree, CRUD keys and execute commands via shell. It also supports SSL/TLS encryption, SSH tunnels and cloud Redis instances, such as: Amazon ElastiCache, Microsoft Azure Redis Cache and Redis Labs. -
9
Relation Tags
Source code for be able to use Relation Tags.
Source code for be able to use Relation Tags. It is part of project VocabularyMem but can be used separately. Relation Tags are tags which can be relationed together . For example tag "Paris" and tag "France" can be relationed with a relation "is part of". This code is created from 0 and is able to define which type of relation we use, using most elemental mathematic properties. It is strongly recommended to read "Relation Tags guide for programmers". Inside source zip, also contains dialogs for set properties of this extended tags. All this dialogs files finish either with "...dlg.cpp" or ",,,dlg.h". Please read "readme" file. It is recommended to use a binary matrix class like BinMatrix in order to have enough speed for calculations of implicit relations in a system of bogus tags with big data. Need to be compiled with C++11 and Qt libraries -
E-commerce Fulfillment For Scaling BrandsFlowspace delivers fulfillment excellence by pairing powerful software and on-the-ground logistics know-how. Our platform provides automation, real-time control, and reliability beyond traditional 3PL capabilities—so you can scale smarter, faster, and easier.
-
10
Sample Level Musical Timeline
Sample Level Modulation of Musical Timeline
Sample Level Modulation of Musical Timeline Mingfeng Zhang Dept. of Electrical and Computer Engineering, University of Rochester In this toolbox we provide signal processing tools to allocate music events (samples of musical notes) to specified time locations with sample level accuracy. In this implementation, we use computational tools to add in micro-timing variations in J.S. Bach four-part chorales as a "visualizer" for big data. By extracting data patterns from multiple time scales, we implement a tool that musicians can perform the big data at different resolutions. This toolbox will need the following supporting toolboxes: MIDI TOOLBOX https://www.jyu.fi/hum/laitokset/musiikki/en/research/coe/materials/miditoolbox MIR TOOLBOX https://www.jyu.fi/hum/laitokset/musiikki/en/research/coe/materials/mirtoolbox Please add the path in MATLAB for these two toolbox. Please also read the project document file (readme.doc/pdf) for more details -
11
SentimentAnalysis-Rick&Morty
Rick & Morty Sentiment Analysis - End-of-Degree Project - UNIR
The remarkable progress in the field of Big Data has driven the development of new technologies in natural language processing and data analysis. Text mining is a fascinating application of data analysis that extracts relevant information from related writings in different linguistic contexts. And therefore, in natural language processing, sentiment analysis and classification stands out as a key application supported by text mining. Through the extraction of information from textual data, it becomes possible to identify and comprehend the sentiments and emotions conveyed. In this end-of-degree work, we analyze and classify the dialogue of characters in an English-language television series as "Rick and Morty" using Python. The objective is to identify and categorize the feelings and emotions expressed in the text, comparing the human perception of the characters' personalities with the results obtained using natural language processing techniques. -
12
Snowplow Analytics
Enterprise-strength marketing and product analytics platform
Snowplow is ideal for data teams who want to manage the collection and warehousing of data across all their platforms and products. -
13
Streams for IBM i
Batch performance boosting and Big Data framework for IBM i
Streamd for IBM i is a suite of tools for IBM i (previously known as AS/400 and iSeries) that can significantly improve performance characteristics of batch processes. Due to extensive use of parallel programming techniques Streams for IBM i delivers significant performance improvements for single streamed batch jobs. Streams for IBM i can split an existing batch process into a number of concurrent streams, completely eliminate backup-related delays, introduce new robust recovery policies and even modify the program logic of existing applications - all without any code modifications. Streams for IBM i includes a feature allowing manipulations of batch job QTEMP libraries. -
14
TensorBase
TensorBase is a new big data warehousing with modern efforts
TensorBase hopes the open source not become a copy game. TensorBase has a clear-cut opposition to fork communities, repeat wheels, or hack traffic for so-called reputations (like Github stars). After thoughts, we decided to temporarily leave the general data warehousing field. For people who want to learn how a database system can be built up, or how to apply modern Rust to the high-performance field, or embed a lightweight data analysis system into your own big one. You can still try, ask or contribute to TensorBase. The committers are still around the community. We will help you in all kinds of interesting things pursued in the project by us and maybe you. We still maintain the project to look forward to meeting more database geniuses in this world, although no new feature will be added in the near future. -
15
The intellectual system of Eidos
http://lc.kubagro.ru/
http://lc.kubagro.ru/ http://lc.kubagro.ru/aidos/index.htm http://lc.kubagro.ru/aidos/_Aidos-X.htm On the IBM PC, the Eidos system started working in 1992. MS Windows has been running since 2012. Implemented in Alaska+Express. I want to try to translate some modes, and maybe all of them, to the Harbor. The full source text in a single file is here: http://lc.kubagro.ru/__AIDOS-X.txt Responsible Secretary Kubgau scientific journal, Professor of computer science Department Kubgau technologies and systems, doctor of Economics, candidate of technical Sciences, Professor E. V. Lutsenko http://lc.kubagro.ru/ http://ej.kubagro.ru/ https://www.researchgate.net/profile/Eugene_Lutsenko https://www.facebook.com/groups/558866657885969/ Quick free publication of articles in the RSCI with DOI: http://lc.kubagro.ru/ResearchGate.doc -
16
Universal Java Matrix Package
sparse and dense matrix, linear algebra, visualization, big data
The Universal Java Matrix Package (UJMP) is an open source Java library which provides sparse and dense matrix classes, as well as a large number of calculations for linear algebra such as matrix multiplication or matrix inverse. Operations such as mean, correlation, standard deviation, replacement of missing values or the calculation of mutual information are supported, too. The Universal Java Matrix Package provides various visualization methods, import and export filters for a large number of file formats, and even the possibility to link to JDBC databases. Multi-dimensional matrices as well as generic matrices with a specified object type are supported and very large matrices can be handled even when they do not fit into memory. -
17
Vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python
Data science solutions, insights, dashboards, machine learning, deployment. We start at 100GB. Vaex is a high-performance Python library for lazy Out-of-Core data frames (similar to Pandas), to visualize and explore big tabular datasets. It calculates statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid for more than a billion (10^9) samples/rows per second. Visualization is done using histograms, density plots and 3d volume rendering, allowing interactive exploration of big data. Vaex uses memory mapping, zero memory copy policy and lazy computations for best performance (no memory wasted). Cut development cut development time by 80%. Your prototype is your solution. Create automatic pipelines for any model. -
18
abu
Abu quantitative trading system (stocks, options, futures, bitcoin)
Abu Quantitative Integrated AI Big Data System, K-Line Pattern System, Classic Indicator System, Trend Analysis System, Time Series Dimension System, Statistical Probability System, and Traditional Moving Average System conduct in-depth quantitative analysis of investment varieties, completely crossing the user's complex code quantification stage, more suitable for ordinary people to use, towards the era of vectorization 2.0. The above system combines hundreds of seed quantitative models, such as financial time series loss model, deep pattern quality assessment model, long and short pattern combination evaluation model, long pattern stop-loss strategy model, short pattern covering strategy model, big data K-line pattern Historical portfolio fitting model, trading position mentality model, dopamine quantification model, inertial residual resistance support model, long-short swap revenge probability model, strong and weak confrontation model, trend angle change rate model, etc. -
19
ankus
Data Mining and Machine Learning Algorithms based on MapReduce
[The feature of ankus] * ankus is a 'web-based big data mining project and tool'. - MapReduce-based data mining/machine learning algorithms library - Hadoop-based distributed bigdata system - offering a web-based GUI for easy use [The ankus project & License] * The ankus project consists of three as an open source. * ankus has Dual licensed under the community and commercial licenses. * community license is following GPLv3 - Some algorithms in Core Project do not under the OSS License [Demonstration Site] http://www.openankus.org:18080 [Official website & E-mail] www.openankus.org ankus@openankus.org [ankus video list] http://bit.ly/ankus_video [community] http://www.facebook.com/groups/openankus (Korean Groups) http://www.facebook.com/openankus (English Groups) http://bit.ly/ankus_forum (Google groups user forum) -
20
apache spark data pipeline osDQ
osDQ dedicated to create apache spark based data pipeline using JSON
This is an offshoot project of open source data quality (osDQ) project https://sourceforge.net/projects/dataquality/ This sub project will create apache spark based data pipeline where JSON based metadata (file) will be used to run data processing , data pipeline , data quality and data preparation and data modeling features for big data. This uses java API of apache spark. It can run in local mode also. Get json example at https://github.com/arrahtech/osdq-spark How to run Unzip the zip file Windows : java -cp .\lib\*;osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c .\example\samplerun.json Mac UNIX java -cp ./lib/*:./osdq-spark-0.0.1.jar org.arrah.framework.spark.run.TransformRunner -c ./example/samplerun.json For those on windows, you need to have hadoop distribtion unzipped on local drive and HADOOP_HOME set. Also copy winutils.exe from here into HADOOP_HOME\bin -
21
decd
R package for complex disease analysis
This is a R packages designed for complex disease when large scale expression data is available -
22
deshang
Software to support deshang research
Deshang research project mainly focus on collecting students' behaviors and using big data technologies to analyze the factors which might make effects on behavior changing and to build strategies set of parents and teacher guiding. This SF project aims to provide interface and backend analysis functionalities for project Deshang. The softwares used are WAMP (Window Apache + MySQL + PHP) with phpMyAdmin (web base MySQL admin console) included, WordPress (3.8.1 chinese version), Sphinx as search engine and libMMSeg chinese directionary for Sphinx. -
23
giServer
giServer the easy to use and extensible batch and integration server
The giServer is an easy-to-use integration server for process automation and event-driven or scheduled execution of batch jobs. Instead of using complex XML configuration files an elaborate GUI for batch job management is included. Some possible usage scenarios are: - Automatic processing of incoming data files - Big Data applications - Process automation - Data Mining/Aggregation applications - Automatic Reporting - Processing and analysis of database records -
24
iOVFDT
iOVFDT algorithm of incremental decision tree
How to extract meaningful information from big data has been a popular open problem. Decision tree, which has a high degree of knowledge interpretation, has been favored in many real world applications. However noisy values commonly exist in high-speed data streams, e.g. real-time online data feeds that are prone to interference. When processing big data, it is hard to implement pre-processing and sampling in full batches. To solve this trade-off, we propose a new decision tree so called incrementally optimized very fast decision tree (iOVFDT). Inheriting the use of Hoeffding bound in VFDT algorithm for node-splitting check, it contains four optional strategies of functional tree leaf, which improve the classifying accuracy. In addition, a multi-objective incremental optimization mechanism investigates a balance among accuracy, mode size and learning speed... -
25
inMap
Rich layers, better user experience, big data geographic visualization
inMap is a big data visualization library based on Baidu Map. It focuses on the display of scatter, heat map, grid, and aggregation in the direction of big data. It is committed to making big data visualization easy to use.