https://liyuan.org/tags/rag/Recent content in RAG on Yuan's BlogHugoen-usSun, 10 May 2026 00:00:00 +0000https://liyuan.org/posts/ai/rag-improvement-via-18-failures/Sun, 10 May 2026 00:00:00 +0000https://liyuan.org/posts/ai/rag-improvement-via-18-failures/This project details the refinement of an agentic RAG system for financial Q&amp;A, boosting test accuracy from <strong>0.871 to ~0.919</strong> by systematically diagnosing 18 failure cases. Rather than blind model tuning, the author prioritized a &quot;diagnose-first&quot; approach: resolving &quot;judge-side&quot; discrepancies with deterministic numeric prefiltering, then implementing structural improvements like query translation, anti-refusal checks, and a five-layer fix for superlative ambiguities. The results highlight that while prompt-based reflection is helpful, structural, schema-enforced changes offer superior reliability. Ultimately, the author demonstrates engineering pragmatism by consciously leaving eight failures unfixed—due to dataset noise or unfavorable ROI—distinguishing between &quot;fixing everything&quot; and strategic, production-oriented optimization.https://liyuan.org/posts/ai/rag-eval-metrics-selection/Tue, 05 May 2026 00:00:00 +0000https://liyuan.org/posts/ai/rag-eval-metrics-selection/This article outlines a pragmatic, tiered approach to evaluating Retrieval-Augmented Generation (RAG) agents, specifically within the context of complex financial document analysis (FinanceBench). The author argues that effective evaluation is not about maximizing the number of metrics, but about selecting signals that provide clear, actionable insights at different stages of the development lifecycle.