Joshua Bell

Bringing 3D Gen AI to the Metaverse I've posted about our work in 3D Gen AI here before, but this week was especially exciting since our work on 3D Gen AI tools was featured in Mark's Keynote at Meta Connect. Our teams have built and deployed models for generating 3D assets, textures, audio, and avatars that are now powering hundreds of worlds in the metaverse. You can read more about all the exciting AI tools in Horizon Worlds Studio here: https://lnkd.in/g6p4wR3v

199

1 Comment

We teamed up with Stanford Institute for Human-Centered Artificial Intelligence (HAI) in their effort to train a fully open source Foundation Model. This means that not only the weights, but also the code, data, and processes are fully open sourced. To optimize efficiency and performance, the team combined the power of JAX/XLA and TPUs. Read all the gory details of the journey and the different choices made in our new blog post: https://lnkd.in/gTGdURZ5

185

9 Comments

There's a new Medium post from Valeriy Manokhin, PhD, MBA, CQF, who you should all be following here for his snarky but deep wisdom on the details of ML, especially but not exclusively on time series. The post is about why GANs have pretty much disappeared from the landscape. tl;dr:, if you have a model learning learning from its own output, eventually Ouroboros dies of malnutrition. Jump over and read it: https://lnkd.in/gBPNrwaY My POV: Yes, in a GAN you have two models (one predicting and one judging), but if they are learning from each other then they are essentially acting as a single ensemble with two sets of outputs: one being the prediction and one being the judgement. Why is this relevant to Gen AI? For two reasons. First is something that's been catching on in the discussion space recently but which I wrote about here ~2 years ago: if LLMs are trained on LLM-generated text, then the process will accumulate errors / misconceptions / biases. Just like in standard ML, where we have to be careful about overfitting, we need to hold data out of the process. Training an LLM on LLM-generated text is similar in a way as generating synthetic data to a tabular dataset using a technique like SMOTE: you're not really generating any new information, since you're extrapolating from the existing information. Second, "LLMs-as-judges" have been a hot topic for the last year or so, as of course is the closely adjacent topic of using RL to continue model training after initial corpus training and RLHF. See for example the STAR paper. LLMs as judges can have the same problem as GANs, IF the feedback is used to continue to train the original model. This is why the teams building your favorite LLMs write in their papers that they are very careful not to go too far with this training step, lest results begin to get worse. Yes, "distilling" a large model into a small model is adjacent to this, but at least in that case we are very intentionally trying to copy the behavior of the larger model, warts and all.

54

8 Comments

At PolyAI we’ve believed from day one that dialogue design (or conversation design) is one of the most critical ingredients for successful AI Agents—especially voice. Why? Because the perceived quality of an AI Agent and the business outcomes it delivers are only as strong as the craft behind its conversations. Designing for voice AI is often harder than people think. Great dialogue designers must balance several challenges: 1️⃣ Consumers first – Every caller is reaching out because something went wrong. They may be impatient, frustrated, or even in trouble. A good design makes the agent empathetic, human-like, and supportive—because conversations aren’t just about solving problems, they’re about making people feel heard. 2️⃣ Business goals – No two enterprises are alike. Some want to prevent churn, others focus on upsell, premium experiences, or sheer efficiency. Dialogue design ensures the AI delivers value that’s aligned with the brand’s objectives. 3️⃣ Technology limitations – LLMs are powerful, but not perfect. Designers need to build conversations that feel natural and flexible while introducing smart guardrails to prevent hallucinations and unwanted behaviors. 4️⃣ Legacy impressions – Let’s face it, IVRs have trained us all to hate “talking to machines.” Overcoming this bias takes intentional design, anticipating resistance, and proving through experience that AI conversations can actually be pleasant—and effective. This is why conversation design has been a core practice at PolyAI from the very beginning. It’s the reason our AI voice agents have been successfully deployed across thousands of real-world use cases. 👉 If you’re considering implementing voice AI, invest in dialogue design. It’s not optional—it’s the difference between frustration and delight, between wasted investment and measurable ROI. Check out our latest discussion here by Jenn Cunningham Arkadiusz Kwapiszewski Harry Swanson Eshan Singhal https://lnkd.in/eyNQTt9C

41

1 Comment

Most AI tools today are thin LLM wrappers. Here’s how we built Vectice differently. When people ask, “Why not just use GPT to write my model documentation directly?” it sounds reasonable. Until you look closer at what’s really required. If we take the example of a model validation documentation, the document must show: - Which dataset version was used - The exact code and parameters that produced the results - Metrics and thresholds agreed with governance - Plots that match the evidence - Section content aligned with internal policies and regulators’ templates An LLM on its own can’t guarantee any of that. At best, it produces well-worded but disconnected content. At worst, it hallucinates numbers, forgets context across sections, or drifts in tone. That’s why Vectice looks more like a multi-agent orchestration platform than a LLM wrapper: - Autolog listeners capture raw evidence across notebooks, CI/CD, and registries ( code, data, iterations, metrics... ) without manual copy/paste. - An autonomous evidence graph normalizes artifacts, assigns stable IDs, and enables point-in-time reconstruction. - Agents review content, decide which sections can be generated safely, and insert plots and tables directly from evidence, not memory. - A knowledge base aligns prompts with regulatory standards like SR 11-7 or the EU AI Act. - The orchestrator ensures determinism with retries, timeouts, parallelization, and smallest-capable LLM routing for cost control. The result isn’t “a single shot LLM wrote this.” It’s a multi-agent choreography where every sentence, figure, and table is anchored to evidence: traceable, citable, and reproducible. That’s where the new generation of vertical startups are building their defensive moat. They aren’t threatened by LLMs, or vibe coding, they’re anchored in defensive IP, industry-specific workflows, and months of prompt tuning and orchestration. The next generation of LLMs may be smarter tomorrow. But will they know how to generate a compliant SR 11-7 document you can trust on their own? We’re confident they won’t.

51

3 Comments

AI for code editing is not perfect and often times frustrating. But as an OSS maintainer, there are things that are starting to come into view. What tools like Cursor can help you do is shift your role - rather than the code producer, you become the compiler. The AI takes the seat of engineer and QA. Below is an example of cursor background agents and bugbot being used in conjunction. Without these two tools, this would have been extremely frustrating. But with the two working together, I am able to get the PR in a ready state without ever creating my own git branch. Yes, this PR could have been done by myself pretty easily. But you can see the potential. In some ways it's starting to obscure away things like branches. You start thinking about agents working on tasks rather than feature branches. I was a huge proponent of making sure ideas "compile" before working on things. I never knew I might quite literally become an idea "compiler"

36

2 Comments