Reading / 2026-06/2026-06-10t221112-estimating-no-cot-task-completion-time-horizons-of-frontier
Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models
Measures how well frontier LLMs complete tasks without chain-of-thought reasoning, finding GPT-5.5 handles ~3-minute human tasks at 50% reliability — a capability doubling roughly every year since 2019, with safety implications for CoT-based monitoring.
Jun 10, 2026 · tech · Anders Cairns Woodruff et al., LessWrong
Topics
- ai-safety
- benchmarks
- llm-inference
- ai-agents
- llm-engineering
Cited by
- AI agents
Autonomous systems that plan, act, and verify across tool calls and multi-step workflows, with active debate over architecture choices, coordination costs, memory design, state management, and the governance infrastructure needed to make them reliable.
- AI safety
AI safety spans containment of agentic systems, epistemic harms from sycophancy, skill atrophy from unreviewed code generation, and macro-level risks from rapid capability growth — each requiring different mitigations.
- Benchmarks
Benchmarks measure model or system capability, but their results are only as meaningful as their design — a recurring problem across LLM, multi-agent, and vision tasks, where tests built for one context are routinely applied to contexts they cannot capture.
- LLM engineering
LLM engineering spans the full stack of building with large language models: training, inference optimization, agent architecture, harness design, and the operational tradeoffs that determine whether model capability translates into reliable software.
- LLM inference
LLM inference covers how language models generate tokens from a prompt — spanning hardware constraints, serving architecture, caching strategies, quantization, routing, and cost — and has become its own engineering discipline as scale and cost pressures intensify.
Related
- Unslothtopic
- The Orchestrator Isn't Your Moattopic
- Agentic Coding is a Traptopic
- Vibe Training: Auto Train a Small Language Model for Your Use Casetopic
- Vision Language Models (Better, Faster, Stronger)topic
- How to Build Scalable Web Apps with OpenAI's Privacy Filtertopic
- CanItRun — Can my GPU run this LLM?topic
- What CI Actually Looks Like at a 100-Person Teamtopic