Nile

We are hiring!

We just published a new doc on how to do schema migrations with Django and Nile's Postgres https://lnkd.in/eV5gFMPV

Things I have learnt building a database company - It does not matter how many times you have built a database in the past. They are still really hard to build - Build it right the first time. There are no shortcuts to building a reliable database - Multitenancy, sharding, storage decoupling are some of the hardest infra problems - Explaining clearly why you built it is 10x harder than building it - Building for prototypes is easy, building for production scale is the real deal - There are many database problems in B2B companies that are unsolved - OLTP is the most critical piece of infrastructure in a company and 90% adoption is based on trust - The early customers are truly angels. You have to make them successful at any cost - Number of databases is not a metric of success. Usage is king - Love for Postgres is amazing - You need to be crazy to start a database company as your first company - You owe everything to the folks who trusted you and joined in the journey - Patience and more patience

Here are my notes based on building and scaling a vector database for AI companies. - It needs to do the basics really well (store and query vectors) - Highly available and battle tested - Support millions of namespaces or tenants - Scale to 100 million vectors or more per namespace - Have the ability to vertically scale per namespace and horizontally scale to support billions of vectors - Performance isolation across namespaces. Vector queries are resource intensive and you don’t want them to affect or starve others - Fast vector index rebuilds. Maintaining and configuring indexes per namespace is better than one large monolith index - Low latency with p99 < 20 ms for fully cached dataset vs p99 < 500 ms for cache+ disk (very query dependent) - Cost effective. Storage and query cost quickly blows up. Ability to cheaply store vectors that are less frequently used and paying exactly for the resources used reduces cost - Serverless vs provisioned compute. This depends on the usecase and you want the ability to configure this per namespace - Metadata filtering. Have the ability to filter vector results by metadata. Storing the metadata along with the vectors is a prerequisite - Store vectors along with primary data to avoid complex data pipelines or the necessity to keep multiple systems in sync - Supports hybrid search. You want to combine full text and vector search for more accurate and relevant search - There is huge appreciation for managing a single database and avoiding system sprawl, if possible - Developers love Postgres and love to extend it to do vector search using the full Postgres toolchain at massive scale Nile has been designed with these learnings and design principles for vector use cases What has your experience been? Would love to hear from others

We keep shipping

🚢We shipped Nile Auth a couple of weeks ago. We have been seeing folks build actual B2B apps with it, which is kinda wild. If you're working on something multitenant, give it a spin. It handles the annoying stuff so you don’t have to. https://lnkd.in/exyFPFm2 🧙♂️Org + user flows out of the box 🔓Open source - read code, fork it, run it 💸Unlimited active users — no per-seat pricing traps 🧰Email/PWD, SSO, User mgmt — actual B2B stuff 🪄Plug-and-play UI modules — login in minutes 📦Users live in your DB — no syncing, no black boxes We built it because we needed it. Maybe you do too:

One of the goals at Nile is to make it easy to use the entire stack for local development. We now have it for Nile Auth. You can start Nile’s Postgres and Auth locally, add a few lines of code to your app and get multi-tenant auth working locally I tried this and it took me 5-10 minutes https://lnkd.in/e_gfm8GV

One of the goals at Nile is to make it easy to use the entire stack for local development. We now have it for Nile Auth. You can start Nile’s Postgres and Auth locally, add a few lines of code to your app and get multi-tenant auth working locally I tried this and it took me 5-10 minutes https://lnkd.in/e_gfm8GV