Documentation↗
Book a Demo

LLM Loan Processing LLM Loan Processing Pipeline

About

Learn how to build a production-ready ML pipeline that combines large language models (LLMs) with traditional ML using Seldon Core 2. In this demo-led session, the Seldon team shows how to seamlessly integrate an LLM from Hugging Face with a classic scikit-learn model inside a modular, observable, and fully traceable pipeline all running on Kubernetes.

Whether you’re aiming to improve unstructured data handling or simplify complex workflows, this demo offers a real-world example of LLM orchestration that’s scalable, explainable, and production-ready.

Learnings

LLMs Meet Traditional ML: Use LLMs like Mistral to extract structured data from text, then pass to high-performance models like Random Forest for final decision-making.

Modular Pipeline Deployment: Build a multi-step pipeline (LLM → Preprocessor → ML model) using manifests and Seldon’s custom resource definitions on Kubernetes.

Fully Observed and Auditable: Integrates with tools like K9s, Grafana, and model performance tracking for real-time observability and compliance.

GPU-Aware and Persistent: Deploy models using private registries, control GPU allocation, and cache models via persistent volumes for fast startup.

End-to-End Use Case Demo: See a live walkthrough of a loan application use case—structured entity extraction, automated decisioning, and real-time inference.

Supports Local and Third-Party LLMs: Deploy from Hugging Face, or bring your own LLM, with support for OpenAI keys and multi-GPU workloads.

Built on Seldon Core v2: Leverage new features like enhanced pipeline control, runtime modularity, and minimal manual setup.

Picture of Alex Housley

Alex Housley

Alex has been founding and building technology companies since 2003, specializing in mobile, data, and machine learning (ML). He experienced the challenges of scaling model deployment infrastructure whilst building a startup that served billions of personalized news article recommendations. In 2014, Alex founded Seldon with the aim of democratizing ML operations to solve the world's most pressing issues. He served as CEO until 2023, shaping the MLOps industry and delivering measurable, meaningful results for clients worldwide. As Founder and Seldon Board Member, Alex is responsible for the product and technology strategy and execution, focusing on the needs of customers today and in the future.

Complex, real-time use cases is what we do best

Talk with a expert to explore how Seldon can support more streamlined deployments for real-time, complex projects like fraud detection, personalization, and so much more.

Book a Demo

MORE RECENT WEBINARS & EVENTS

Building Trust in Production ML: A Complete Guide to Model Observability
Production ML Serving & Monitoring with Seldon & Kubernetes
Europe’s Landmark Regulation: What You Should Know
The Foundation of AI-Enabled Apps
An Essential Guide to ML Model Serving Strategies (Including LLMs)
DeepSeek Reasoning, the Cake Mystery
Stay Ahead in MLOps with our
Monthly Newsletter!

Join over 25,000 MLOps professionals with Seldon’s MLOps Monthly Newsletter. Your source for industry insights, practical tips, and cutting-edge innovations to keep you informed and inspired. You can opt out anytime with just one click.

Email Signup Form