Rithwik Kollipara

There is significant unmet demand for developers who understand AI. At the same time, because most universities have not yet adapted their curricula to the new reality of programming jobs being much more productive with AI tools, there is also an uptick in unemployment of recent CS graduates. When I interview AI engineers — people skilled at building AI applications — I look for people who can: - Use AI assistance to rapidly engineer software systems - Use AI building blocks like prompting, RAG, evals, agentic workflows, and machine learning to build applications - Prototype and iterate rapidly Someone with these skills can get a massively greater amount done than someone who writes code the way we did in 2022, before the advent of Generative AI. I talk to large businesses every week that would love to hire hundreds or more people with these skills, as well as startups that have great ideas but not enough engineers to build them. As more businesses adopt AI, I expect this talent shortage only to grow! At the same time, recent CS graduates face an increased unemployment rate, though the underemployment rate — of graduates doing work that doesn’t require a degree — is still lower than for most other majors. This is why we hear simultaneously anecdotes of unemployed CS graduates and also of rising salaries for in-demand AI engineers. When programming evolved from punchcards to keyboard and terminal, employers continued to hire punchcard programmers for a while. But eventually, all developers had to switch to the new way of coding. AI engineering is similarly creating a huge wave of change. There is a stereotype of “AI Native” fresh college graduates who outperform experienced developers. There is some truth to this. Multiple times, I have hired, for full-stack software engineering, a new grad who really knows AI over an experienced developer who still works 2022-style. But the best developers I know aren’t recent graduates (no offense to the fresh grads!). They are experienced developers who have been on top of changes in AI. The most productive programmers today deeply understand computers, how to architect software, and how to make complex tradeoffs — and who additionally are familiar with cutting-edge AI tools. Sure, some skills from 2022 are becoming obsolete. For example, a lot of coding syntax that we had to memorize back then is no longer important, since we no longer need to code by hand as much. But even if, say, 30% of CS knowledge is obsolete, the remaining 70% — complemented with modern AI knowledge — is what makes really productive developers. Without understanding how computers work, you can’t just “vibe code” your way to greatness. Fundamentals are still important, and for those who additionally understand AI, job opportunities are numerous! [Original text: https://lnkd.in/gHSWcKhr ]

10,846

633 Comments

I just found the fastest serving engine for LLMs: (fully open-source & makes vLLM 7x faster) LMCache is an LLM serving engine designed to reduce time-to-first-token and increase throughput, especially under long-context scenarios. At a high level, LMCache is like a CDN of KV caches. It sits between the LLM serving engine, like vLLM, and the external data by storing the KV caches of reusable texts across various locations like GPU, CPU, and Local Disk. This way, it can boost vLLM with 7x faster access to 100x more KV caches. Find a link to the GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

1,240

37 Comments

On arXiv today, I posted a note https://lnkd.in/gxqWZB7t that describes how to build Josephson junction, inductor, and mutual inductance transformer models from basic library components in Keysight ADS. This is a follow up on our work to enable flux quantization in frequency domain circuit simulations https://lnkd.in/g5nkYNyD . Hopefully, this is detailed enough to allow users of other commercial or open-source circuit simulators to reproduce these results.

156

4 Comments

🎉 Exciting milestone: my first research paper is published! The Aryabhata 1.0 Tech Report is now live on arXiv and Hugging Face A few weeks ago, we open-sourced Aryabhata 1.0, a compact 7B math reasoning LLM built for Indian academic exams like JEE. Today, we share the full technical journey behind it. What’s inside: 🔀 Model merging ⚙️ Data curation 🎯 SFT with curriculum learning 🤖 RLVR (A2C + group relative advantage, with our novel tweaks) 📊 Evaluation pipeline 📄 Read the paper: https://lnkd.in/gSPvKk8G 🛠️ Try the model: https://lnkd.in/de3aJyFF Grateful to my amazing team Sandeep Varma Sachin Dharashivkar Tejas Chaudhari VISHAL SINGH for making this possible. 🙏 It’s been a rewarding journey, from data curation to model training to evaluations, and seeing it documented as my first paper makes it even more special. #AI #MachineLearning #LLM #Research #OpenSource #NLP #MathReasoning #JEE #PhysicsWallah #HuggingFace

428

38 Comments

It’s funny how things come full circle: When I first joined Meta and said I was working on RL for PyTorch, people got wide-eyed: "Wow, Reality Labs? That’s huge!" One teammate even thought we were building a PyTorch library just for AR/VR (I kind of wish, that'd have been pretty cool). I also recall that at the very beginning, when the AR/VR org was being named RL, a manager suggested thinking carefully about that acronym not to confuse it with Reinforcement Learning but it didn't really click with anyone... Back then, reinforcement learning was niche, Reality Labs was hype. Fast forward 4 years: by the time I left, there was zero confusion. Today, if you say you work on RL, people assume DeepSeek R1-style intelligence. I wonder how many folks inside Reality Labs now face the opposite situation 🤔.

227

10 Comments

Speculative Decoding before: limited choices, the draft model must have the same tokenizer 😬 Speculative Decoding now: unlimited choices, ANY draft model can be used and better speedup opportunities 😎 The folks at Intel have been cooking, and Speculative Decoding (with speculative sampling!) now works with ANY draft model through "Universal Assisted Generation". In a nutshell, a vocabulary mapping is built, which enables us to map the probability distributions from one model to the other 🧠 If the large model you're using already has a good draft sidekick, you can use the smallest models on the Hub with Universal Assisted Generation to probably get even better throughput. If the model you're using doesn't have a draft model, then the benefits are obvious 😉 Available in 🤗 transformers v4.50. In the comments, I'm leaving a link to a blog post that explains and benchmarks Universal Assisted Generation. Kudos to the UAG team: Jonathan Mamou ( Jonathan Mamou ) Oren Pereg ( Oren Pereg ) Nadav Timor ( Nadav Timor ) Daniel Korat ( Daniel Korat ) Moshe Berchansky ( Moshe Berchansky ) Moshe Wasserblat ( Moshe Wasserblat ) Roy Schwartz ( Roy Schwartz )

275

14 Comments

Sharing my thoughts on function inlining in C++ in this short article. I discuss how the compiler handles inlining, the impact of the One Definition Rule (ODR), and the effect on cache performance. A very commonly asked question in Quant Developer interviews. Big thanks to Hari Hara Naveen S for reviewing the post. Read here: https://lnkd.in/gKK6Xksm

137

6 Comments

Recent research by UIUC and Intel Labs has introduced a new jailbreak technique for Large Language Models (LLMs) known as InfoFlood. This method takes advantage of a vulnerability termed "Information Overload," where excessive linguistic complexity can circumvent safety mechanisms without the need for traditional adversarial prefixes or suffixes.   InfoFlood operates through a three-stage process: Linguistic Saturation, Rejection Analysis, and Saturation Refinement. Initially, it reformulates potentially harmful queries into more complex structures. If the first attempt does not succeed, the system analyzes the response to iteratively refine the query until a successful jailbreak is achieved. Empirical validation across four notable LLMs—GPT-4o, GPT-3.5-turbo, Gemini 2.0, and LLaMA 3.1—indicates that InfoFlood significantly surpasses existing methods, achieving success rates up to three times higher on various benchmarks.   The study underscores significant vulnerabilities in current AI safety measures, as widely used defenses, such as OpenAI’s Moderation API, proved ineffective against InfoFlood attacks. This situation raises important concerns regarding the robustness of AI alignment systems and highlights the necessity for more resilient safety interventions. As LLMs become increasingly integrated into diverse applications, addressing these vulnerabilities is crucial for ensuring the responsible deployment of AI technologies and enhancing their safety against emerging adversarial techniques.  Arxiv: https://lnkd.in/eBty6G7z

53

4 Comments