Anna Berenberg

High availability architectures need to be tested. If your plan for a bad day doesn't get exercised often, it's much more likely to let you down when you really need it. Aurora DSQL now works with AWS Fault Injection Service (FIS) to make it much easier to test your multi-region applications: https://lnkd.in/gPjy8ZSj

261

2 Comments

My run-mcp-servers-with-aws-lambda library now supports Bedrock AgentCore Gateway! Run your existing stdio-based MCP server in a Lambda function, and let the Gateway take care of implementing the MCP HTTP transport protocol https://lnkd.in/g-Q67G3N

375

8 Comments

Engagement in IXPs: beyond #technology, an ecosystem of trust and collaboration. Internet Exchange Points are not just pieces of #infrastructure: they are living communities where trust, #neutrality, and human relationships matter as much as technical performance. In my latest article on RIPE NCC blog, I explore: - How neutrality enables network operators of all sizes to join and grow - Why strategic alignment and local collaboration are key to ecosystem growth - The role of engagement, active listening, and #networking in strengthening IXP communities Read the full article here: https://lnkd.in/dCvdfY5s #IXP #peering #networking #community

64

4 Comments

Spent the afternoon at the Amazon Web Services (AWS) Builder Loft in #SanFrancisco getting a deep dive into #AWSTrainium and the #NeuronSDK — not just how to run models on them, but how the stack works end to end. Trainium is AWS’s custom chip for training and inference, built with multiple NeuronCores, each with its own high-bandwidth memory (HBM). With tensor parallelism, you can split a model across cores to get more compute and memory bandwidth working in parallel — though at some point, communication overhead can offset the gains. The Neuron SDK is the software layer that makes this possible: - Neuron Compiler turns a PyTorch/TensorFlow model into optimized, hardware-specific kernels. - Neuron Runtime loads those kernels onto NeuronCores for execution. - The compiled model is stored as a NEFF file — the binary the hardware actually runs, which can be reused on the same hardware type without recompiling. - Profiling tools let you see latency, utilization, and memory use in real time. We covered how serving choices affect performance just as much as hardware: - Bucketing requests by sequence length so cores aren’t wasting cycles on padding. - Too few buckets = wasted compute, too many = possible latency hit if buckets don’t fill fast enough. - Continuous batching to keep hardware busy between requests — especially effective with steady traffic, but under bursty load it works best when tuned alongside bucket sizes and max sequence length. - Disaggregated prefill to separate prompt processing from decoding, allowing work to be overlapped across cores on long prompts. - PJRT integration so Trainium works with PyTorch XLA, JAX, and vLLM without heavy code changes. - Serving framework integrates cleanly with vLLM, Hugging Face TGI, and Ray Serve, so you can pick the serving layer that matches your latency and concurrency needs. In the workshop, we deployed a Qwen model with vLLM + NeuronX Distributed Inference on an Inf2 instance, tuned tensor parallelism, adjusted bucket configurations, experimented with continuous batching, and watched the changes show up in TTFT, tokens/sec, and utilization. If you’ve worked with Trainium or Neuron SDK, I’d love to hear — what applications are you considering them for, and how are you tuning for your workloads? Thanks to Annapurna Labs for organizing, Ruchi Bhatia for hosting, Kamran Khan and Sindhura Palakodety for helping me get things running smoothly, and Jim Burtoft, Pinak Panigrahi, and Mrinal Shukla for the technical deep dives that connected the hardware and software pieces. #AWS #AWSCloud #AWSTrainium #NeuronSDK #AnnapurnaLabs #vLLM #AI #AIAgents #Amazon

49

2 Comments

Amazon Web Services (AWS) just launched general availability of new R8gn instances - though the way that I found out about them wasn't from the official Amazon blog post, it was from the new automated newsletter for EC2instances.info. EC2instances.info is regularly rebuilding itself every few hours and now also checks for two things: 1) new instance types and 2) any pricing changes with existing instance types. You can sign up to receive email alerts about these two changes as often as the site finds them. If you're curious about getting the notifications sent your way to see what Amazon is launching, especially leading up to AWS re:Invent, I included the link for how to set this up in the comments below.

33

3 Comments

This is a great example of the real-world impact of AI capabilities within AlloyDB for PostgreSQL ! Target set out to modernize its digital search experience to better match guest expectations and support more intuitive discovery across millions of products. To meet that challenge, they rebuilt their platform with hybrid search powered by filtered vector queries and AlloyDB AI. The result: a faster, smarter, more resilient search experience that’s already improved product discovery relevance by 20% and delivered measurable gains in performance and guest satisfaction. #alloydb #postgresql #gcp

49

AWS transform was featured in the Amazon Q2 quarterly earnings release as "General availability of AWS Transform, an AWS agent that cuts mainframe modernization timelines from years to months, completes VMware network conversions up to 80x faster, and reduces operating costs for .NET applications by up to 40%." See https://lnkd.in/gSrsSC72.

85