Jiaqi Zhang

Google Research Director Denny Zhou, who founded the LLM Reasoning Team at DeepMind, recently gave a great talk at Stanford’s CS25 class: Large Language Model Reasoning. This was one of the most intuitive talks about reasoning model training and application. He outlined new directions for training models to handle questions where answers aren’t easily verifiable. My team is currently working on an enterprise LLM project focused on using reasoning to extract relevant context and generate responses for unverifiable cases. It’s a tough but promising area, and it’s an interesting time to be building. [Link to Denny Zhou's talk: https://lnkd.in/gdJ6Mzi8]

1,023

8 Comments

For over 60 years, Dijkstra's algorithm has been the undisputed champion for finding the shortest path in graphs. A new paper from researchers at Tsinghua University, Stanford University, and the Max Planck Institute for Multidisciplinary Sciences proves that even champions can be beaten. They've achieved the first major breakthrough for the directed Single-Source Shortest Path (SSSP) problem on real-weighted graphs, breaking the long-standing "sorting barrier" that made Dijkstra's algorithm seem optimal. Their method ingeniously combines the logic of Dijkstra's and the Bellman-Ford algorithms. Through a clever recursive technique, it avoids the need to fully sort vertices by distance, which was the core bottleneck. The result is a faster, deterministic algorithm that runs in O(mlog2/3n) time. This is a huge deal for theoretical computer science and has practical implications for speeding up route calculations in GPS, optimizing data flow in computer networks, and improving efficiency in logistics. A fundamental problem just got a new, faster solution. #Algorithms #ComputerScience #GraphTheory #Dijkstra #Research #Innovation

7,064

293 Comments

Algorithmic History Made — After 41 Years! Researchers from Tsinghua University have done the unthinkable — breaking Dijkstra’s dominance in shortest path algorithms. 📌 What changed? Dijkstra (1984): O(m + n log n) New algorithm (2025): O(m log^(2/3) n) 💡 Why it matters: log^(2/3) n grows slower than log n, meaning on massive graphs, the new approach crushes Dijkstra in speed. Even after decades, core CS algorithms are still being reinvented. This is why computer science never stops blowing our minds 🤯

522

16 Comments

🚨Throwback #2: Semantic Search for RAG Workshop Back in late 2023, I ran a technical workshop series on the fundamentals of semantic search and the role it plays in modern LLM applications. The session unpacks why semantic search in RAG matters, then shows how to do it—from smart chunking and vector indexing to fast retrieval and LLM reasoning. Along the way it benchmarks classic BM25 against bi-encoders, cross-encoder re-rankers, hybrid search, SPLADE, HyDE, and domain-adapted models, spelling out the real-world trade-offs behind each approach. While newer methods have emerged since then, the fundamentals still hold - so I'm sharing the slides + recording in case its helpful to anyone tackling RAG use cases today. 🔗 Project page: https://lnkd.in/eMM2X2TN 🔗 Slides: https://lnkd.in/e_BP2PA9 🔗 Recording: https://lnkd.in/eQMzKPzk

83

1 Comment

🔎 Recent Observations in Embedding Modeling (Gemini & Qwen3) The frontier in embedding modeling is increasingly shaped by large LLMs like Google Gemini (https://lnkd.in/dp8ygAuy) and Alibaba Qwen3 (https://lnkd.in/dREPHW8m). Both teams are converging on a similar recipe for building state-of-the-art embeddings for retrieval, ranking, and search. 🎯 Common Patterns Emerging: • Leverage LLM Backbones: Both approaches start from large, pretrained LLMs (decoder-only, causal architectures) rather than building encoder models from scratch. • Synthetic Data Generation: Synthetic positive and hard negative pairs are created in bulk—using the LLM itself—to augment or even replace scarce labeled data. • Contrastive Fine-Tuning: Embedding quality is improved by fine-tuning with contrastive objectives, with in-batch / hard negatives and multi-stage training. • Task Conditioning: Simple instruction prompts (like "query:" or "passage:") help the model generalize across tasks without re-architecting. • Pooling over Causal Outputs: Instead of modifying the attention mask to be bidirectional, these models pool the outputs of a decoder-only, left-to-right model for sequence-level representations. 💡 As these practices become standard, a tough question remains for downstream adoption: how can we fine-tune LLM-based embedding models for a specific task without catastrophic forgetting, if we lack the diverse multi-task or general-domain data that major labs have access to? • Mixing in original language modeling or general tasks is a proven way to retain model generality, but it requires substantial extra data and resources. • In practice, many organizations have only narrow downstream data. This creates a real risk of overfitting or catastrophic forgetting when fine-tuning for new embedding tasks. Would love to hear thoughts, practical experiences, or solutions from those tackling this problem!

55

5 Comments

There’s a new RL algorithm in town… and it’s already shaking up the LLM research community. 🤯 Group Sequence Policy Optimization (GSPO) was created by the research team at Qwen and is purpose built for operating at the sequence level rather than the token level. This means it evaluates, clips, and optimizes entire response sequences by their overall likelihood. The important upshot is that it directly aligns the optimization process with how rewards are actually assigned in real world scenarios! Some key reasons to be excited! 1. Better stability. GSPO improves training stability and helps avoids model collapse that can happen with algorithms like GRPO. 2. Scalable RL for mixture of experts (MoE) models. GSPO inherently solves stability challenges teams face when scaling RL for MoE architectures. Large scale RL becomes more practically feasible! 3. Simplifies Infra. Sequence-level optimization is inherently less sensitive to small numerical differences between training and inference systems compared to token-level methods, which should lead to more robust RL pipelines! We can already see signs that this has paid off in big ways for the open-source community. Qwen3 models are doing really well on benchmarks, boasting competitive performance with top models like Gemini-2.5 Pro on GPQA, AIME25, and LiveCodeBench! https://lnkd.in/gCZMQvUj P.S. 👀 If you want to run RL on your own Qwen3 model with GRPO/GSPO, keep an eye on our open-source post-building repo 🧡 : https://lnkd.in/gXZ32qkm #AI #LLM #ReinforcementLearning

81

10 Comments