DHH horizontal testing programming model: DeepSeek ran out of $23, Fable about $550

source··08:27 编辑

According to monitoring, Ruby on Rails author DHH used the same Python to Rust task to test multiple models. The DeepSeek Pro V4 Max was completed in about 2.5 hours, and the total token cost was $23. Grok 4.6 cost $55, about 1.5 hours, GPT-5.6 Sol cost $43; DeepSeek V4 Flash and GPT-5.6 Luna failed to complete. The first round was dominated by Fable, and DHH statistics cost around $550 and took about 45 minutes. It rewrote TerminalTextEffects from Python into Rust, and shared approximately 11 million tokens. Startup time was reduced from 87ms to 2ms, and rendering was 9.6x faster. However, there is a premise for comparison. Fable first wrote a 483-line implementation plan, and after Codex xhigh's review, all other models directly followed this plan. So it's more like comparing the cost and efficiency of different models implementing the same solution. The project is open source. The repository contains 354 conformance tests that compare the Rust version and the original Python version frame by frame, byte by byte, and the final product can be compiled into a single static file of about 3.3 MB.

Original Link
说明: All Bitpush articles reflect the author's views only and do not constitute investment advice.

Related

Loading...