HARPO: an LLM agent that repairs and optimizes HLS code under a tool budget

July 2026 · code on GitHub · preprint: doi:10.5281/zenodo.21366536

I spent the last month building HARPO — an autonomous agent that takes broken or unoptimized HLS C/C++, runs the actual toolchain (csim, then real Vitis HLS synthesis), reads the reports, and iterates: fix correctness first, then improve performance and area, all under a hard budget on how many tool invocations it's allowed to spend. It started as a competition entry; I ended up caring more about what the benchmarks revealed than the competition. Everything below runs on a local open-weights model (qwen3.6-35B on my own GPU box) — total LLM cost for every result in the paper: $0.

What it does

Two loops sharing one budget. The repair loop takes code that fails simulation, diagnoses the failure from parsed tool output, asks the model for a patch, applies it in an isolated candidate copy, and re-verifies. On a planted arithmetic bug, the local 35B model diagnosed and fixed it in one LLM call, about 18 seconds end to end. The optimize loop takes already-correct code and proposes pragmas (pipelining, array partitioning, unrolling), re-synthesizes, and keeps a change only if simulation still passes and the score strictly improves. Non-improving and correctness-breaking patches are discarded — the baseline is never lost.

The result I didn't expect: deterministic recipes win where the LLM over-parallelizes

I gave the optimize loop two patch providers and benchmarked them head-to-head on real Vitis synthesis. One is the LLM. The other is a small deterministic recipe library — precise, parameterized pragma transformations (e.g. ARRAY_PARTITION cyclic factor=8 on the right array), tried best-first. Throughput is scored on the whole design's initiation interval (interval_max), and a candidate is kept only if it meets the throughput target and shrinks area.

On this kernel the recipe applies the one pragma that unblocks the pipeline — cutting the design interval from 1024 to 256 while also dropping area (369 → 315 LUTs), for zero LLM tokens. The raw LLM's only move is to fully unroll the loop; that inflates area for no honest throughput gain, so the scorer rejects it and the baseline stands. The same pattern held on a 2-D matmul kernel. The lesson generalizes past HLS: when a domain has a small set of known-good transformations, a deterministic library with the LLM as selector beats the LLM as generator. Put the intelligence at design time, not runtime.

Historical note — the 42× result that exposed a flaw in my original scoring. An earlier version of the scorer ranked throughput on each loop's per-loop initiation interval instead of the whole design's interval. Under that flawed metric the LLM's fully-unrolled design counted as a win, and the same head-to-head came out at 315 LUTs (recipe) versus 13,194 LUTs (LLM) for comparable throughput — a 42× area gap. That number is real, but it measures the scoring bug as much as the model: fixing the metric is exactly what the next section is about, and under the corrected scoring the LLM produces no accepted improvement on this kernel.

The bug that taught me the most

Midway through, the agent started accepting designs that were obviously worse. Root cause: my score ranked throughput by per-loop initiation interval — and a fully unrolled loop has no II. It reports as none, my comparison treated none as best-possible, so the score literally rewarded the area blow-up. The agent accepted a design whose real interval was 3× worse than baseline at 100× the LUTs.

The fix was two separate things, and keeping them separate mattered: (1) score throughput on the design-level interval, which always exists; (2) change the objective from "fastest at any cost" to satisfice-then-minimize-area — meet a throughput target, then shrink. On a matrix-multiply kernel, speed-first chased interval 19 at 5,689 LUTs; satisfice stopped at the target with interval 44 at 3,121 LUTs — 45% less area for throughput that was already good enough. The target itself is derived automatically by a capped, zero-token probe of safe single-pragma variants.

Honest scope

This is a small, honest setup: eight kernels (including three PolyBench ports), one planted-bug repair family, csim-first verification, one local model. It is workshop-tier evidence, not a thesis — but every number is from real Vitis HLS runs, the full evidence tables are in the repo, and the head-to-head recipe-vs-LLM area comparison is one I haven't seen quantified elsewhere in the LLM-for-HLS literature.

Paper (9 pages, with per-phase token accounting and all ablations): DOI link (coming via Zenodo). Code, tasks, and evidence logs: github.com/phucducnguyen/harpo (MIT). If you work on HLS, FPGA toolchains, or LLM agents for EDA, I'd genuinely like to hear where this breaks — open an issue or reach me via GitHub.