Proposal Generation Engine
A production LLM reasoning system that reads raw government solicitations and returns scored, schema-validated output through an API.
The problem
Government solicitation documents are long, inconsistently formatted PDFs. Responding to one means reading it end to end, pulling out every requirement, and producing a structured response that maps back to those requirements. Done by hand it is slow; done naively with a single LLM call it is unreliable — the model skips requirements, invents structure, and returns output no downstream system can safely parse.
Approach
Multi-step reasoning instead of one big prompt
The system is an autonomous agent that works in stages: ingest the raw PDF or JSON, structure the unstructured content, reason over the structured form in multiple steps, then score and format the result. Splitting the work this way means each stage has a narrow, checkable job rather than one prompt carrying the entire task.
A JSON schema at every boundary
Every input and output stage has a defined JSON schema and a validation step. Anything that does not conform is caught at the boundary it failed at rather than surfacing as a confusing error three stages later. This is the single change that did the most for reliability.
Retrieval over noisy real data
Ingestion, preprocessing, embedding and semantic-retrieval pipelines run over real-world documents, with explicit handling for the edge cases and failure modes that only appear once you are processing genuine files rather than clean samples.
Evaluation
Quality was not assumed — it was measured. I established repeatable evaluation benchmarks that every release runs against, so a prompt change or model swap produces a number rather than an argument. Iterating against those benchmarks, with prompt optimisation and structured output validation, is what moved the system from acceptable to dependable.
Result: output quality moved from roughly 30% to 90%+ against the evaluation criteria defined for the system.
Serving it
The system is exposed as a FastAPI service deployed on AWS with Docker. It was engineered for about 100 concurrent users: Redis caching on repeated work, horizontal scaling, and latency monitoring to keep responses inside their target window under load. I also authored the system-architecture specs and methodology documentation that other engineering teams build against.
Stack
Note on detail
This is client work, so the repository and document corpus are not public. The architecture, evaluation methodology and results above are described at the level I can share.