Manu Kumar

On-Prem is the New Cloud: Why Enterprises are Reversing Stance in the Age of AI Remember when cloud was the battleground for innovation, agility, and scale? Everyone was tracking the % of on-prem compute moving to the cloud and it was growing every year? Well now there’s a reversal. The reality today? On-prem is the new cloud. Here’s why. AI’s Dark Side: Legal Risk and Data Governance Headaches Enterprises are excited about AI, but also uneasy as vendors like OpenAI face intensifying legal pressure. In a high-stakes lawsuit, a judge ordered OpenAI to preserve all ChatGPT chats (even deleted ones), overriding its 30-day deletion policy. Imagine that: you delete sensitive conversations, but they’re still sucked into legal limbo. The precedent is clear: AI prompts and outputs are discoverable records, demanding integration into enterprise ESI (Electronic Stored Information) policies. For large organizations handling sensitive data, the risk is simple: you can’t afford your AI tools to turn into legal liabilities. Especially when global regulations (GDPR, etc.) and internal governance collide with unpredictably broad discovery orders. Sovereignty, Control, Trust Many enterprises still ban ChatGPT or impose strict rules on usage. Even when allowed, token limits are throttled. Enterprises are cautiously letting AI in (as they should). Enterprises need to have control and trust. SAP is leaning into this with its Sovereign Cloud On-Site, deployable on customer premises with SAP-managed infrastructure. It is built to deliver data, operational, technical, and legal sovereignty in one package. Doubt it will be the last. Scaling Pains Talking to founders, three reasons stand out why cloud isn’t scaling with AI: 1/ The rate at which AWS/GCP lets you provision compute and the compute it wants you to provision both falter as AI and AI driven developers deploy a lot more code more frequently than before. So sometimes it’s not about data governance, but actually about performance and functionality. 2/ Shadow AI is rampant—employees paste sensitive info into models despite restrictions—creating demand for better permissioning tools. 3/ Data leakage remains a live risk, like the August 2025 Grok incident where hundreds of thousands of private chats became publicly accessible and indexed. One thing is for sure - founders selling into the enterprise are scaling faster if they offer the ability to run on-prem. The Future is Hybrid Don’t get me wrong - cloud is not going anywhere. But neither is on prem. A few years agoI thought we were headed for a full cloud shift, but now I think the future is truly hybrid.

34

4 Comments