Delta Lake

📣 Join us for our next 𝗢𝗽𝗲𝗻 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 + 𝗔𝗜 webinar on October 30 at 9AM PT! Most lineage graphs are overly complex and lack meaningful context, and years of chasing “total visibility” proved that more metadata doesn’t create more clarity. 🔎 Collecting metadata isn’t the problem—understanding it is. This session explains why the future of observability isn’t another graph, but a reasoning layer that connects lineage and telemetry into one continuous context—so teams can detect, diagnose, and adapt in real time. Register ➡️ https://lnkd.in/eihgBzdV 🎙️ Featuring Willy Lulciuc, Co-Founder & CEO of oleander 💬 Hosted by Lisa N. Cao, Developer Relations at Databricks #opensource #metadata #openlakehouse #ai #datalineage #oss

Come learn, connect, and celebrate the open source community in Mountain View on Nov 13 at the 𝗢𝗽𝗲𝗻 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 + 𝗔𝗜 𝗠𝗶𝗻𝗶 𝗦𝘂𝗺𝗺𝗶𝘁! 🚀 Explore AI infrastructure, interoperable data systems, and more — plus lunch, swag, and great conversations. 🔗 RSVP on Luma: https://luma.com/OLMS-1113 #opensource #oss #deltalake #unitycatalog #apachespark #apacheiceberg #ai

One of the most powerful new capabilities in Delta Lake 4.0 is support for collations, giving you much finer control over how text is compared and sorted. ✅ In this clip, Youssef Mrini explains how to enable collations: 🔹 Define a default collation at the table level 🔹 Customize collations for individual columns 🔹 Use ALTER TABLE or ALTER COLUMN to add them to existing tables Whether you’re handling multilingual datasets or need case-insensitive searches, collations make your queries more accurate and flexible — with minimal changes to your code. 🎥 Watch the full webinar to explore more: https://lnkd.in/eKgJ3tXM #opensource #oss #deltalake #linuxfoundation Scott Haines

One week to go! 🎉 Curious how data engineering design patterns translate to Delta Lake, and how Delta fits into streaming architectures? Join Scott Haines (Buf) and Bartosz Konieczny (waitingforcode.com) for a practical walkthrough of the patterns, features, and best practices powering reliable streaming with Delta Lake. 🗓️ Tuesday, Oct 14 🕝 9:00AM PT 🔗 Register: https://lnkd.in/eSRiXpDF #DeltaLake #opensource #oss #dataarchitecture #dataengineering

Paris! 🇫🇷 The Open Lakehouse + AI Meetup is coming November 24, co-located with the Forward Data Conference. Hear from Youssef Mrini (Databricks) on the next era of open table formats, Alexandre BERGERE (DataGalaxy) on scaling usage analytics through Delta Sharing, & more! Connect with open source builders, explore hands-on architectures, and shape the future of data and AI. 📅 Monday, Nov 24 🕡 6:30–10 PM 📍 Maison Internationale, Paris 🔗 RSVP: https://lu.ma/OLM-1124 #openlakehouse #opensource #lakehouse #ai #deltalake #apachespark Hymaïa Yoann Benoit

Exciting news — the Japanese edition of 𝗗𝗲𝗹𝘁𝗮 𝗟𝗮𝗸𝗲: 𝗧𝗵𝗲 𝗗𝗲𝗳𝗶𝗻𝗶𝘁𝗶𝘃𝗲 𝗚𝘂𝗶𝗱𝗲 will be released this November! 🎉 Great work by the team to expand the original guide, sharing new perspectives on why #Lakehouses are the backbone of #AI, and how community collaboration drives data innovation. 🙌 Stay tuned for more! #opensource #deltalake #lakehouse #oss

Tomorrow, Oct 7 at 9AM PT—last chance to register! ➡️ 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀 𝘁𝗼 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀: 𝗥𝗲𝗶𝗺𝗮𝗴𝗶𝗻𝗶𝗻𝗴 𝘁𝗵𝗲 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 𝗳𝗼𝗿 𝗮𝗻 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗙𝘂𝘁𝘂𝗿𝗲 🎟️ Register: https://luma.com/OLAI-107 Modern data teams still battle fragmented interfaces and brittle handoffs. What’s missing? Simple, uniform ergonomics: human‑friendly, machine‑usable. Ciro Greco and Jacopo Tagliabue from bauplan will show how function‑based execution, Git‑for‑Data, and full programmability make lakehouses ready for real AI agents—reducing complexity now and enabling safe, autonomous data ops tomorrow. ✅ Agents need more than query-writing—they must safely manage ingestion, testing, and deployment in reproducible environments. Otherwise, it’s just legacy rebranded. Join us and see the path forward! 🤝 #opensource #lakehouse #oss #agents #aiagents

Delta Connect is a plugin atop Spark Connect, introduced in Spark 3.4, and enables gRPC communication using Protocol Buffers. ✅ This allows client implementations in languages such as Rust and Go to interact with Spark outside the JVM, provided they support the protocol. 🎥 Explore a detailed walk-through and the expanded feature set of Delta 4 in this webinar: https://lnkd.in/eyyud3pP cc Scott Haines (Buf), Youssef Mrini (Databricks) #opensource #oss #deltalake #spark

Are you wondering if general concepts like data engineering design patterns can help you learn about #DeltaLake? Or, if it's possible to leverage Delta Lake within your streaming data architecture? In this webinar, Scott Haines and Bartosz Konieczny will answer these two questions. Scott, who gained streaming expertise at Yahoo, Twilio, and Nike, will share with you best practices for leveraging Delta Lake as a component of your streaming architecture. ✅ Bartosz, who recently published Data Engineering Design Patterns, will reverse-engineer a few of these design patterns to explain which Delta Lake features make everything tick. 🗓️ Tuesday, Oct 14 🕝 9AM PT Don't miss it! 🔗 Register today: https://lnkd.in/eSRiXpDF #opensource #oss #dataarchitecture #dataengineering

What happens when robust, deterministic lakehouse design meets the power of AI agents? 👉 Reduced tool fragmentation 👉 Safer, reproducible automation 👉 Data teams enabled for true innovation On Tuesday, October 7 at 9AM PT, bauplan founders Ciro Greco and Jacopo Tagliabue will break down how function-based execution and Git-for-Data semantics pave the way for autonomous agentic workflows—cutting complexity and multiplying impact. 🎤 Moderated by Lisa N. Cao 🔗 Register here to reserve your spot: https://luma.com/OLAI-107 #openlakehouse #opensource #oss #aiagents