Smitha Venkat

𝗛𝗼𝘄 𝗠𝗖𝗣 𝗦𝗲𝗿𝘃𝗲𝗿𝘀 𝗠𝗮𝗻𝗮𝗴𝗲 𝗦𝗲𝘀𝘀𝗶𝗼𝗻𝘀 𝘄𝗶𝘁𝗵 𝗟𝗟𝗠𝘀 𝗮𝗻𝗱 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗪𝗶𝗻𝗱𝗼𝘄𝘀 Managing sessions in LLMs isn’t just about keeping track of prompts. It is about orchestrating memory, context, and tools seamlessly across multiple agents. That’s where MCP servers come in. Here’s the flow I captured visually → Context Window as Working Memory: Holds system prompt, user inputs, assistant responses, tool calls, and rolling context → MCP Servers: Each server maintains its own session state, ensuring consistency across interactions → System Prompt + Tools + Code Loops: Work together to power agentic behavior while grounding outputs in real tasks → LLM Integration: The context is constantly summarized and updated before being sent to the model This structure ensures scalable, multi-session orchestration, which is critical for enterprise AI where multiple tasks and tools interact simultaneously.

149

14 Comments

As we move past the initial excitement of how to build and experiment with agents, we are now facing the classic day-two operations problems at enterprise scale: how do we meaningfully observe systems whose failure modes are more semantic than syntactic?     Traditional metrics like latency, error rates, and token counts feel like trying to debug a complex distributed system with only “ping”. They tell you if a service is alive, but not if it’s lucid. Businesses need to answer higher-order questions about factual accuracy and coherence.     Today at .conf, Splunk and AGNTCY | A Linux Foundation Project announced their collaboration to:   — Drive agentic semantic conventions directly into the OpenTelemetry (OTel) schema to establish a common language  —Integrate the AGNTCY Metrics Compute Engine (MCE) that provides the tooling, using an LLM-as-a-judge approach to evaluate qualitative signals like coherence alongside the operational metrics we already know. Consider a banking AI agent handling loan applications. Your operational dashboards might be all green, but the MCE could flag a persistent dip in "factual accuracy" for queries about variable APRs, revealing a silent, compliance-critical failure mode. Now you are no longer just monitoring the health of the container; you're monitoring the integrity of its reasoning.  This foundation becomes even more critical as we move from single agents to multi-agent systems, where specialized agents collaborate to execute complex jobs. Get your hands on the code. Join the workgroup meetings. Provide feedback.   Start with the open-source AGNTCY Observe toolkit on GitHub and instrument an agent here https://lnkd.in/gCUqVsff Photo from .conf today on the main stage highlighting where AGNTCY is delivering with agent observability

109

5 Comments

Efficient and effective 4-bit compute across the pipeline is a critical frontier in AI workload optimization 🚀 Not long ago, the idea of lower precision pretraining created real anxiety for us engineers. It felt like the math should not work. Just a few years back, we were still struggling to get models to converge with mixed precision FP32 and BF16. Then DeepSeek v3 demonstrated that native FP8 pretraining was possible. Stable gradient computations at such low precision seemed out of reach, yet that breakthrough changed everything. Now 4-bit training is starting to look very similar. With careful scaling of weights and activations, stochastic rounding, and new ways to stabilize gradients, we can achieve convergence that resembles higher precision while gaining the efficiency benefits of NVFP4. Learn more in our NVFP4 Training Blog: https://lnkd.in/e6CwaNkb #ai #nvidia #gpu #llm #modeltraining #nvfp4 #optimization

36

For many of us working with LLMs, especially on proprietary models, the current reality of domain adaptation feels like an uphill battle. I have seen many customers invest months of effort into preparing datasets and meticulously validating fine-tuned model. Yet, a new, better base #LLM ships every 2-3 months, often making previous investments obsolete. Memory Decoder (has nothing to do with the "memory" ^_^) is a new approach that seems promising: 1. Plug-and-Play, no parameter modification 2. One-time domain knowledge compression: It learns to imitate the behavior of external non-parametric retrievers, effectively compressing domain #knowledge into a compact decoder model.  3. Works across models, as long as they share the same #tokenizer, and regardless of its size. 4. Preserves general reasoning From the paper, I can see that Memory Decoder makes continuous adaptation far more manageable, especially for enterprise solutions relying on rapidly evolving base models. Read more about this paper: https://lnkd.in/gkepvn8y #domainadaptation #finetuning #sft #lora

52

2 Comments