Developer Tools & APIs

Tools that expose their own state or evidence were easier to trust in these tests. Hindsight’s visible memory usage, Denser AI’s consistently visible source citations, and Anomaly AI’s visible SQL all helped make their outputs auditable. The tests rewarded tools that showed what they were doing, not just tools that produced a final answer.

Multi-turn consistency mattered as much as single-turn correctness. The memory benchmark focused on selective retrieval, scope isolation, and delete-or-forget behavior across sessions, while the chatbot and text-to-SQL tests both checked follow-up handling. In practice, staying stable after the first response was a deciding factor in several categories.

Preserving document structure was more important than chasing perfect surface fidelity. The PDF benchmarks favored tools that kept reading order, tables, and headings usable for downstream work, even when other elements were imperfect. Tensorlake and docling won because the extracted output stayed practical for RAG, search, or reuse.

Every winner still had a sharp edge-case failure that mattered for real deployment. Skyvern was slower and had recording-sync fragility, Tensorlake failed on scanned multilevel tables, docling dropped charts and standalone images, LlamaParse had inconsistent keys and some value drift, and Anomaly AI could shift context between turns. The tests show that picking the winner still requires matching it to your hardest inputs and workflow constraints.