Software Engineer, LLM Pipeline & AI Agents

Overview

TSOLife's production LLM pipeline processes audio interviews between senior living communities and their residents — transcribing and diarizing the audio, indexing it for retrieval, and using RAG to answer 85+ structured questions about each resident's history, preferences, and well-being. This data builds a rich, personalized profile for every resident, helping communities provide better, more tailored care and programming.

We're looking for a backend engineer who can operate and scale this production RAG system, while also designing and building stateful AI agents using LangGraph for a related project. You will own the full lifecycle of these AI features: from prompt design and model routing to operating the FastAPI and Celery backend services on Amazon EKS.

What You'll Do Day-to-Day

• Operate & Scale Services: Ship, maintain, and monitor a production FastAPI + Celery backend service running on Amazon EKS.
• Optimize the RAG Pipeline: Continuously improve dialogue chunking, vector retrieval (Aurora pgvector), context assembly, and prompt design to ensure strict JSON adherence and high faithfulness.
• Build AI Agents: Design and maintain stateful AI agents using LangGraph for a sibling product initiative.
• Evaluate & Benchmark: Compare prompts, retrieval strategies, and models. You'll run consensus-style accuracy benchmarks and safely roll out per-question routing changes.
• Manage Audio Processing: Maintain background jobs (Celery + Redis) for transcription submission and handle webhook post-processing from our speech-to-text/diarization provider (Pyannote AI).
• Work with Cloud & Data: Query MariaDB for reviewer feedback and training data, and manage pipeline document storage using Amazon Aurora PostgreSQL.

Required Experience

Core Backend & ML Stack

• Python Ecosystem: 3+ years of production experience (Python 3.12, FastAPI, Uvicorn, Pydantic).
• LLM & RAG Tooling: Hand