Megan O'Keefe

🚀 We are excited to announce the release of MobileLLM-R1 — a series of sub-billion parameter reasoning model. Despite the common belief that reasoning only emerges after training on massive amounts of data, we prove otherwise. With just 4.2T pre-training tokens and a small amount of post-training, MobileLLM-R1 demonstrates state-of-the-art reasoning ability. Here is some highlights: 🎉 100% Open Source: model, training recipe, and data sources. https://lnkd.in/gG-WmYgz 🌟 2x-5x performance boost over other fully open-source models: MobileLLM-R1-950M model scores ~5× higher on MATH vs. Olmo 1.24B and ~2× higher vs. SmolLM2 1.7B—despite being smaller. ⚡ 1/10 of the pre-training token count compared to Qwen: Trained only on 4.2T Tokens total (11.7% compared to Qwen’s 36T tokens), MobileLLM-R1-950M matches or surpasses Qwen3-0.6B on multiple reasoning benchmarks. Collaborating with Ernie, Changsheng, Chia-Jung, Wei, Yuandong, et al. And thanks for the support from Raghuraman, Yangyang, Vikas

539

6 Comments

Happy Friday! This week in #learnwithmz, I’m building on my recent post about running LLMs/SLMs locally: https://lnkd.in/gpz3kXhD Since sharing that, the landscape has rapidly evolved, local LLM tooling is more capable and deployment-ready than ever. In fact, at a conference last week, I was asked twice about private model hosting. Clearly, the demand is real. So let's dive deeper into the frameworks making local inference faster, easier, and more scalable. Ollama (Most User-Friendly) Run models like llama3, phi-3, and deepseek with one command. https://ollama.com/ llama.cpp (Lightweight & C++-based) Fast inference engine for quantized models. https://lnkd.in/ghxrSnY3 MLC LLM (Cross-Platform Compiler Stack) Runs LLMs on iOS, Android, and Web via TVM. https://mlc.ai/mlc-llm/ ONNX Runtime (Enterprise-Ready) Cross-platform, hardware-accelerated inference from Microsoft. https://onnxruntime.ai/ LocalAI (OpenAI API-Compatible Local Inference) Self-hosted server with model conversion, whisper integration, and multi-backend support. https://lnkd.in/gi4N8v5H LM Studio (Best UI for Desktop) A polished desktop interface to chat with local models. https://lmstudio.ai/ Qualcomm AI Hub (For Snapdragon-powered Devices) Deploy LLMs optimized for mobile and edge hardware. https://lnkd.in/geDVwRb7 LiteRT (short for Lite Runtime), formerly known as TensorFlow Lite Still solid for embedded and mobile deployments. https://lnkd.in/g2QGSt9H CoreML (Apple) Optimized for deploying LLMs on Apple devices using Apple Silicon + Neural Engine. https://lnkd.in/gBvkj_CP MediaPipe (Google) Optimized for LLM inference on Android devices. https://lnkd.in/gZJzTcrq Nexa AI SDK (Nexa AI) Cross-platform SDK for integrating LLMs directly into mobile apps. https://lnkd.in/gaVwv7-5 Why Local LLMs Matter? - Edge AI and privacy-first features are rising - Cost, latency, and sovereignty concerns are real - Mobile + Desktop + Web apps need on-device capabilities - Developers + PMs: This is your edge. Building products with LLMs doesn't always need the cloud. Start testing local-first workflows. What stack are you using or exploring? #AI #LLMs #EdgeAI #OnDeviceAI #AIInfra #ProductManagement #Privacy #AItools #learnwithmz

66

2 Comments

For many of us working with LLMs, especially on proprietary models, the current reality of domain adaptation feels like an uphill battle. I have seen many customers invest months of effort into preparing datasets and meticulously validating fine-tuned model. Yet, a new, better base #LLM ships every 2-3 months, often making previous investments obsolete. Memory Decoder (has nothing to do with the "memory" ^_^) is a new approach that seems promising: 1. Plug-and-Play, no parameter modification 2. One-time domain knowledge compression: It learns to imitate the behavior of external non-parametric retrievers, effectively compressing domain #knowledge into a compact decoder model.  3. Works across models, as long as they share the same #tokenizer, and regardless of its size. 4. Preserves general reasoning From the paper, I can see that Memory Decoder makes continuous adaptation far more manageable, especially for enterprise solutions relying on rapidly evolving base models. Read more about this paper: https://lnkd.in/gkepvn8y #domainadaptation #finetuning #sft #lora

52

2 Comments

AWS has made real progress on its mission to make the cloud the cleanest and most-energy efficient, while supporting customers on their path to digital transformation. And with our Compute for Climate Change Fellowship, we want to take an additional step by supporting some of the world’s most exciting startups on their journey to develop new climate technologies. This programme will invest up to $4 million in AWS credits to help climate tech startups build solutions that use advanced cloud computing and AI to develop solutions for the climate crisis that is gripping our planet. I’m personally proud that among the 20 global cohort members is AURA-AERO from France, an electric aircraft manufacturer that could reimagine air travel. Through this programme, they will receive guided mentorship from AI and sustainability experts to help them bring their ideas to life and make a real change to the climate. Wishing them all the success on their journey to build a sustainable future with AWS! https://lnkd.in/dp4kPTuM

101

1 Comment