Datapipe Files

Real-time, incremental ETL library for ML with record-level depend

Status: Alpha

Brought to you by: otatarinova

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
README.md	2021-10-13	511 Bytes	0
v0.9.8 - fix #63 source code.tar.gz	2021-10-13	731.7 kB	0
v0.9.8 - fix #63 source code.zip	2021-10-13	753.7 kB	0
Totals: 3 Items		1.5 MB	0

0.9.8

Фикс работы с SQLite для чанков больше 1000 (https://github.com/epoch8/datapipe/issues/63)

0.9.7

Новый CLI параметр --debug-sql
Стриминг реализация чтения данных из БД в ExternalTable сценарии

Minor breaking changes: code

теперь нужно создавать отдельный DBConn для данных и метаданных из-за стримингового чтения

Source: README.md, updated 2021-10-13

You Might Also Like

Gen AI apps are built with MongoDB Atlas Icon

Gen AI apps are built with MongoDB Atlas

Build gen AI apps with an all-in-one modern database: MongoDB Atlas

MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.

Start Free

Simple, Secure Domain Registration Icon

Simple, Secure Domain Registration

Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.

Sign up for free

Recommended Projects

CloverDX
Please, visit www.cloverdx.com for latest product versions. Data integration platform; can be used to transform/map/manipulate data in batch and near-realtime modes. Suppors various input/output formats (CSV,FIXLEN,Excel,XML,JSON,Parquet, Avro,EDI/X12,HL7,COBOL,LOTUS, etc.). Connects to...
Apache Kafka
Apache Kafka is a distributed streaming platform built around durable, partitioned logs called topics, enabling high-throughput, fault-tolerant event pipelines. Producers append records to partitions, brokers replicate them for durability, and consumer groups read them at their own pace while...
StarRocks
StarRocks is the next generation of real-time SQL engines for enterprise analytics. Real-time analytics is notoriously difficult. Complex data pipelines and de-normalized tables have always been a necessary evil. Processing any updates or deletes once data arrives has not been possible- until...
apache spark data pipeline osDQ
This is an offshoot project of open source data quality (osDQ) project https://sourceforge.net/projects/dataquality/ This sub project will create apache spark based data pipeline where JSON based metadata (file) will be used to run data processing , data pipeline , data quality and data...
Mage.ai
Open-source data pipeline tool for transforming and integrating data. The modern replacement for Airflow. Effortlessly integrate and synchronize data from 3rd party sources. Build real-time and batch pipelines to transform data using Python, SQL, and R. Run, monitor, and orchestrate thousands of...