LLM Financial Applications

I frequently see conversations where terms like LLMs, RAG, AI Agents, and Agentic AI are used interchangeably, even though they represent fundamentally different layers of capability. This visual guides explain how these four layers relate—not as competing technologies, but as an evolving intelligence architecture. Here’s a deeper look: 1. 𝗟𝗟𝗠 (𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹) This is the foundation. Models like GPT, Claude, and Gemini are trained on vast corpora of text to perform a wide array of tasks: – Text generation – Instruction following – Chain-of-thought reasoning – Few-shot/zero-shot learning – Embedding and token generation However, LLMs are inherently limited to the knowledge encoded during training and struggle with grounding, real-time updates, or long-term memory. 2. 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻) RAG bridges the gap between static model knowledge and dynamic external information. By integrating techniques such as: – Vector search – Embedding-based similarity scoring – Document chunking – Hybrid retrieval (dense + sparse) – Source attribution – Context injection …RAG enhances the quality and factuality of responses. It enables models to “recall” information they were never trained on, and grounds answers in external sources—critical for enterprise-grade applications. 3. 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 RAG is still a passive architecture—it retrieves and generates. AI Agents go a step further: they act. Agents perform tasks, execute code, call APIs, manage state, and iterate via feedback loops. They introduce key capabilities such as: – Planning and task decomposition – Execution pipelines – Long- and short-term memory integration – File access and API interaction – Use of frameworks like ReAct, LangChain Agents, AutoGen, and CrewAI This is where LLMs become active participants in workflows rather than just passive responders. 4. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 This is the most advanced layer—where we go beyond a single autonomous agent to multi-agent systems with role-specific behavior, memory sharing, and inter-agent communication. Core concepts include: – Multi-agent collaboration and task delegation – Modular role assignment and hierarchy – Goal-directed planning and lifecycle management – Protocols like MCP (Anthropic’s Model Context Protocol) and A2A (Google’s Agent-to-Agent) – Long-term memory synchronization and feedback-based evolution Agentic AI is what enables truly autonomous, adaptive, and collaborative intelligence across distributed systems. Whether you’re building enterprise copilots, AI-powered ETL systems, or autonomous task orchestration tools, knowing what each layer offers—and where it falls short—will determine whether your AI system scales or breaks. If you found this helpful, share it with your team or network. If there’s something important you think I missed, feel free to comment or message me—I’d be happy to include it in the next iteration.

If you’re building anything with LLMs, your system architecture matters more than your prompts. Most people stop at “call the model, get the output.” But LLM-native systems need workflows, blueprints that define how multiple LLM calls interact, how routing, evaluation, memory, tools, or chaining come into play. Here’s a breakdown of 6 core LLM workflows I see in production: 🧠 LLM Augmentation Classic RAG + tools setup. The model augments its own capabilities using: → Retrieval (e.g., from vector DBs) → Tool use (e.g., calculators, APIs) → Memory (short-term or long-term context) 🔗 Prompt Chaining Workflow Sequential reasoning across steps. Each output is validated (pass/fail) → passed to the next model. Great for multi-stage tasks like reasoning, summarizing, translating, and evaluating. 🛣 LLM Routing Workflow Input routed to different models (or prompts) based on the type of task. Example: classification → Q&A → summarization all handled by different call paths. 📊 LLM Parallelization Workflow (Aggregator) Run multiple models/tasks in parallel → aggregate the outputs. Useful for ensembling or sourcing multiple perspectives. 🎼 LLM Parallelization Workflow (Synthesizer) A more orchestrated version with a control layer. Think: multi-agent systems with a conductor + synthesizer to harmonize responses. 🧪 Evaluator–Optimizer Workflow The most underrated architecture. One LLM generates. Another evaluates (pass/fail + feedback). This loop continues until quality thresholds are met. If you’re an AI engineer, don’t just build for single-shot inference. Design workflows that scale, self-correct, and adapt. 📌 Save this visual for your next project architecture review. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

Agentic AI: The Iceberg of Core Components Agentic AI is more than just powerful models, it’s a layered ecosystem of interconnected components that work together to create intelligent, autonomous systems. Think of it like an iceberg: the visible part (applications we interact with) is only the surface, while the real power lies beneath in the hidden infrastructure and models that make everything possible. To truly understand Agentic AI, we need to look at both the Application Layer (above the surface) and the Model Layer (below the surface). 🔹 Application Layer (Above the Surface) This is where users and businesses experience Agentic AI directly. It’s the layer that adds intelligence, usability, and trust. •Communication Protocols – Enable smooth interaction and task handoff between multiple agents. • Memory – Tools like Memo, Cognne, Letta allow agents to retain knowledge, context, and long-term reasoning. • LLM Security – Platforms like Lakera, WhyLabs, and NVIDIA ensure safe, reliable, and compliant AI operations. • Model Routing – Directs tasks to the most suitable models, improving efficiency and accuracy. • Orchestration Frameworks – LangChain, Haystack, and LlamaIndex connect agents, tools, and workflows into a seamless system. • LLM Evaluation – Tools such as Arize, Langfuse, and Galileo test accuracy, performance, and robustness of AI agents. • LLM Observability – Braintrust, Traceloop, and similar tools track metrics and provide visibility into AI decision-making. • Data Storage – Vector databases like Chroma and Pinecone enable retrieval, grounding, and context storage for agents. 🔹 Model Layer (Below the Surface) This is the hidden foundation, the computational and model infrastructure that powers everything above. • Foundation Models – Core LLMs from OpenAI, Anthropic, Cohere, DeepSeek, Mistral, and Gemini serve as the intelligence engine. • Base Infrastructure – Kubernetes, Docker, Slurm, and vLLM provide orchestration, scaling, and deployment environments. • GPU/CPU Compute – Heavy lifting is done here with compute from Azure, Google Cloud, Groq, and NVIDIA to support training, inference, and scaling. Together, these two layers create the backbone of Agentic AI. The Model Layer provides raw intelligence and compute power, while the Application Layer adds orchestration, security, memory, and usability. When combined, they transform isolated AI models into autonomous, reliable, and scalable Agentic systems. #AgenticAI

If you're building AI agents to in Finance, this hands-on notebook tutorial by Hanane D. is an excellent place to start. It takes you through these two systems -- System 1: Multi-Agent Financial Analysis (LlamaIndex AgentWorkflow) This system breaks down fundamental analysis into specialised agents: Fundamentals, Profitability, Liquidity, and Supervisor. Each one calculates and interprets financial ratios, then passes state and results downstream in a structured workflow. It uses: - Tool calling (e.g. FinanceToolkit) - Threshold-based checks - State updates and agent coordination System 2: Agentic RAG with ReAct This one combines retrieval + reasoning to answer questions over 10-Ks like: - "What was Apple’s revenue growth from 2023 to 2024?" - "Who had higher net income — Apple or Nvidia?" How it works: - Two separate QueryEngines (Apple and Nvidia) - ReAct agent chooses tools, retrieves data, reasons through the answer What’s covered in the notebook: - How to structure multi-agent systems (very important to understand) - ReAct-based agentic RAG - How agents overcome LLM limitations - Key design patterns for automating finance workflows Link to the notebook: https://lnkd.in/gafd5gMd #AI #LLMs #GenAI #Finance

How to get the most out of LLMs for FP&A use cases? One way is using Copilot tuning! Most AI tools rely on generic large language models (LLMs) that pull from public datasets. While powerful, they often fall short when it comes to your business’s unique terminology, financial workflows, and reporting style. Instead of relying on generic AI responses, Copilot can be tuned to your specific finance tasks — from budget analysis to forecasting to explaining variances — with minimal effort from IT. With Copilot Tuning, you can: ✅ Train Copilot on your internal policies, templates, and financial logic ✅Auto-draft budget narratives, variance reports, or planning decks ✅Create Q&A agents that speak your org’s language (cost centers, planning rules, etc.) ✅Empower analysts to move faster — without waiting on IT Here is my guide to get started with it: 1) Data Ingestion Enterprise Data is gathered from your internal systems — things like reports, documents, templates, and business logic. This data goes through Data Preparation, which formats and cleans it for model training. 2) Training A Training Set is automatically generated from the prepared data. A Fine-tuning Recipe is applied — this is a guided, low-code configuration that tells Copilot how to learn from your data. The system then performs Fine-tuning, creating a custom model that understands your business context, terminology, and tasks. 3) Inference When a User Request is made (e.g., “Summarize this variance report”), Copilot uses a Task Inference Recipe — essentially, a playbook for how to handle the request using your fine-tuned model. The model delivers a tailored response, driving Task Completion with speed and accuracy.

Focusing on AI’s hype might cost your company millions… (Here’s what you’re overlooking) Every week, new AI tools grab attention—whether it’s copilot assistants or image generators. While helpful, these often overshadow the true economic driver for most companies: AI automation. AI automation uses LLM-powered solutions to handle tedious, knowledge-rich back-office tasks that drain resources. It may not be as eye-catching as image or video generation, but it’s where real enterprise value will be created in the near term. Consider ChatGPT: at its core, there is a large language model (LLM) like GPT-3 or GPT-4, designed to be a helpful assistant. However, these same models can be fine-tuned to perform a variety of tasks, from translating text to routing emails, extracting data, and more. The key is their versatility. By leveraging custom LLMs for complex automations, you unlock possibilities that weren’t possible before. Tasks like looking up information, routing data, extracting insights, and answering basic questions can all be automated using LLMs, freeing up employees and generating ROI on your GenAI investment. Starting with internal process automation is a smart way to build AI capabilities, resolve issues, and track ROI before external deployment. As infrastructure becomes easier to manage and costs decrease, the potential for AI automation continues to grow. For business leaders, identifying bottlenecks that are tedious for employees and prone to errors is the first step. Then, apply LLMs and AI solutions to streamline these operations. Remember, LLMs go beyond text—they can be used in voice, image recognition, and more. For example, Ushur is using LLMs to extract information from medical documents and feed it into backend systems efficiently—a task that was historically difficult for traditional AI systems. (Link in comments) In closing, while flashy AI demos capture attention, real productivity gains come from automating tedious tasks. This is a straightforward way to see returns on your GenAI investment and justify it to your executive team.

Generative AI in Accounting: Jobs To Be Done (JTBD) 🔸 Data collection and ingestion Data collection and ingestion is a fundamental aspect of accounting, requiring professionals to gather data from various sources such as bank statements, general ledgers, and commerce platforms to consolidate performance metrics and resolve discrepancies, a process known as reconciliation. Traditionally, this is a time-consuming task involving manual comparisons and inputs from multiple teams. However, advancements in Open Banking and universal APIs have already simplified data importation. The introduction of large language models (LLMs) further streamlines this process by enabling the extraction of data from unstructured formats like contracts and invoices. This allows for more centralized data management in enterprises and offers SMBs the ability to resolve discrepancies quickly through AI copilots that can generate audit trails, enhancing overall efficiency. 🔸 Research Research is another area where LLMs offer significant benefits. CPAs often need to classify, report, and determine the tax implications of various revenue and expense line items, a task that involves navigating complex tax codes, accounting standards, SEC filings, and extensive guidance documents. Previously, this required manual searches and consultations with colleagues. LLMs, trained on these comprehensive data sets, can now provide precise answers and adapt to the judgment calls specific to a firm or professional, vastly improving the speed and accuracy of research tasks. 🔸 Report generation and filing Report generation and filing involve analyzing categorized data to produce various internal and external reports, such as journal entries, audit checklists, and technical memos. While some tasks, like populating tax forms, may not need genAI, LLMs excel at drafting summaries and ensuring documents adhere to a firm's distinctive style. This capability allows for the automated generation of audit-ready reports and checklists. 🔸 Client service and advice Client service and advice stands to gain the most from generative AI. Accountants aim to convert annual, transactional client interactions into ongoing engagements by providing regular financial analysis and optimization advice. GenAI can help achieve this by producing high-quality, scalable insights monthly, thereby offering value that clients may be willing to pay for. This is especially beneficial for accounting firms serving startups and SMBs lacking sophisticated in-house finance functions. The challenge lies in developing fine-tuned models that integrate industry-specific details with quantitative analysis capabilities. Although general-purpose LLMs perform well in many tasks, the future lies in specialized models trained on relevant data sets. 👉 Subscribe for more insights https://lnkd.in/d94JgWBU Source a16z #fintech #ai #accounting Leda Florian Alex Ali

I'm excited to share my first open-source project, the fmp-data package, a Python client for Financial Modeling Prep! To make it even easier to use, I've created a new tutorial that demonstrates a novel approach to financial data analysis using natural language processing. This simple Colab notebook shows how to combine fmp-data with LangChain to build an AI-powered financial assistant. This agent, powered by Financial Modeling Prep's extensive data and LangChain's easy to use orchestration layer, can process complex queries, provide instant answers, and engage in multi-turn conversations. The tutorial covers: - Managing 100+ financial endpoints efficiently with LLMs - Setting up natural language processing for market data queries - Building multi-turn financial conversations - Making the system production-ready The code is open source and fully documented. I encourage you to try it out and share your feedback and comments on all aspects of the project, from the fmp-data package to the tutorial itself. I love to hear your thoughts! https://lnkd.in/eMUXms86 #Python #FinancialData #API #LangChain #MachineLearning #DataScience #FinTech #FinancialModelingPrep #OpenSource

What are the building blocks behind autonomous AI agents with #𝗔𝗜𝗔𝗴𝗲𝗻𝘁𝘀𝗟𝗮𝘆𝗲𝗿𝗲𝗱𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 and 𝗧𝗼𝗼𝗹𝘀 driving them? Understanding the building blocks behind #autonomousAIagents is essential for any professional working at the intersection of AI agents, and product development. This layered architecture provides a structured roadmap, from foundational models to governance — helping us build safer, more powerful, and context-aware #AIagents. Here’s a quick breakdown of each layer and the tools driving them. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟭: 𝗟𝗟𝗠 (𝗙𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗟𝗮𝘆𝗲𝗿) This is the reasoning and language core. Large Language Models like GPT-4, Claude, Mistral, and LLaMA form the foundation for text generation and understanding. 𝗧𝗼𝗼𝗹𝘀: OpenAI GPT-4, Claude, Cohere, Gemini, LLaMA, Mistral. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟮: 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗕𝗮𝘀𝗲 (𝗞𝗕) Provides external context (structured/unstructured) for better decisions. 𝗧𝗼𝗼𝗹𝘀: Chroma, Pinecone, Redis, PostgreSQL, Weaviate. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟯: 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) Retrieves relevant data before generation to improve factual accuracy. 𝗧𝗼𝗼𝗹𝘀: LangChain RAG, LlamaIndex, Haystack, Unstructured .io. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟰: 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝗜𝗻𝘁𝗲𝗿𝗳𝗮𝗰𝗲 Where users and agents meet —via text, voice, or tools. 𝗧𝗼𝗼𝗹𝘀: OpenAI Assistant API, Streamlit, Gradio, LangChain Tools, Function Calling. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟱: 𝗘𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻𝘀 Agents connect with CRMs, APIs, browsers, and other services to take action. 𝗧𝗼𝗼𝗹𝘀: Zapier, Make .com, Serper API, Browserless, LangChain Agents, n8n. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟲: 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗟𝗼𝗴𝗶𝗰 & 𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆 The brain of autonomous agents — task planning, decision-making, execution. 𝗧𝗼𝗼𝗹𝘀: AutoGen, CrewAI, MetaGPT, LangGraph, Autogen Studio. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟳: 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 & 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 Ensures traceability, ethical alignment, and debugging. 𝗧𝗼𝗼𝗹𝘀: Helicone, LangSmith, PromptLayer, WandB, Trulens. 🔹 𝗟𝗮𝘆𝗲𝗿 𝟴: 𝗦𝗮𝗳𝗲𝘁𝘆 & 𝗘𝘁𝗵𝗶𝗰𝘀 Builds trust by preventing toxic, biased, or unsafe behavior. 𝗧𝗼𝗼𝗹𝘀: Azure Content Filter, OpenAI Moderation API, GuardrailsAI, Rebuff. This architecture is more than just a stack — it’s a blueprint for responsible AI innovation. Whether you're building internal copilots, autonomous agents, or customer-facing assistants, understanding these layers ensures reliability, compliance, and contextual intelligence.

"This report covers findings from 19 semi-structured interviews with self-identified LLM power users, conducted between April and July of 2024. Power users are distinct from frontier AI developers: they are sophisticated or enthusiastic early adopters of LLM technology in their lines of work, but do not necessarily represent the pinnacle of what is possible with a dedicated focus on LLM development. Nevertheless, their embedding across a range of roles and industries makes them excellently placed to appreciate where deployment of LLMs create value, and what the strengths and limitations of them are for their various use cases.  ... Use cases We identified eight broad categories of use case, namely: - Information gathering and advanced search - Summarizing information - Explaining information and concepts - Writing - Chatbots and customer service agents - Coding - code generation, debugging/troubleshooting, cleaning and documentation - Idea generation - Categorization, sentiment analysis, and other analytics ... In terms of how interviewees now approached their work (vs. before the advent of LLMs), common themes were: - For coders, less reliance upon forums, searching, and asking questions of others when dealing with bugs - A shift from more traditional search processes to one that uses an LLM as a first port of call - Using an LLM to brainstorm ideas and consider different solutions to problems as a first step - Some workflows are affected by virtue of using proprietary tools within a company that reportedly involve LLMs (e.g., to aid customer service assistants, deal with customer queries) ... Most respondents had not developed or did not use fully automated LLM-based pipelines, with humans still ‘in the loop’. The greatest indications of automation were in customer service oriented roles, and interviewees in this sector expected large changes and possible job loss as a result of LLMs. Several interviewees felt that junior, gig, and freelance roles were most at risk from LLMs ... These interviews reveal that LLM power users primarily employed the technology for core tasks such as information gathering, writing, and coding assistance, with the most advanced applications coming from those with coding backgrounds. Although users reported significant productivity gains, they usually maintained human oversight due to concerns about accuracy and hallucinations. The findings suggest LLMs were primarily being used as sophisticated assistants rather than autonomous replacements, but many interviewees remained concerned that their jobs might be at risk or dramatically changed with improvements to or wider adoption of LLMs. By Jamie Elsey Willem Sleegers David Moss Rethink Priorities