Cisco Aims to Reduce Manual Debugging in Multi-Step AI Pipelines With Open-Source FAPO
Cisco launches open-source FAPO to simplify AI pipeline debugging, improving efficiency and workflow performance.
San Jose, California based Cisco has introduced FAPO, an open-source framework designed to automate prompt optimization for complex, multi-step large language model (LLM) pipelines. The company said the system aims to reduce manual prompt engineering by identifying failures, refining prompts, and restructuring AI workflows when necessary.
FAPO, short for Fully Automated Prompt Optimization, is orchestrated through Claude Code agents and supports Codex as an alternative optimization engine. The framework is released under the Apache 2.0 license, an open-source software license that allows developers and enterprises to deploy and modify it for production AI applications.
The framework addresses one of the biggest challenges in enterprise generative AI deployments. Multi-step AI pipelines often involve retrieval, reasoning, formatting, and validation stages. When outputs fail, developers typically inspect every intermediate step manually to identify the root cause. Cisco said, “FAPO automates that debugging process through step-level failure attribution.”
According to Cisco, developers provide an initial prompt and a labeled dataset. FAPO then evaluates model performance, identifies the stage responsible for errors, proposes prompt variations, validates improvements, and repeats the cycle until a target accuracy threshold is achieved. If prompt modifications alone cannot improve performance, the framework escalates to changing model parameters or restructuring the AI pipeline itself.
The system organizes optimization projects into isolated “tenants,” each containing prompts, datasets, scoring functions, and workflow definitions. Its execution engine, named Hephaestus, supports multiple deployment environments, including OpenAI models, Baseten, and Amazon SageMaker.
Cisco reported that FAPO outperformed the existing prompt optimization framework GEPA (Generalized Evolutionary Prompt Architecture) in 15 of 18 model-benchmark comparisons, delivering an average improvement of 14.1 percentage points. On the HoVer and IFBench benchmarks, where structural workflow changes were introduced, FAPO achieved an average gain of 33.8 percentage points over GEPA.
Cisco also tested FAPO on CTIBench-RCM, a benchmark for evaluating AI performance on cybersecurity tasks. Rather than changing the AI models themselves, FAPO improved the prompts they received, helping them generate more accurate responses with no additional model training.
- GPT-5 → 4 percentage points improvement
- Foundation-Sec-8B-Instruct → 7.1 percentage points improvement
- Foundation-Sec-8B-Reasoning → 2 percentage points improvement
To reduce overfitting during optimization, FAPO separates training and testing datasets, stores prompt variants as immutable files, and includes an independent review stage before accepting optimization proposals. Cisco said these safeguards help ensure improvements generalize beyond the evaluation dataset.
The framework reflects a broader shift toward agentic AI development, where software agents increasingly automate engineering tasks beyond code generation. Rather than relying on manual prompt engineering, systems like FAPO seek to continuously evaluate and improve AI workflows through iterative testing and validation.
Cisco has made the FAPO framework available as an open-source project, enabling developers to integrate automated prompt optimization into enterprise AI applications and research workflows.
