Skip to content

[Performance]: reduce token usage & improve performance, such as with rust token killer rtk #2916

@ahundt

Description

@ahundt

Performance Issue Description

Right now cli commands use more tokens than necessary, increasing costs and runtimes.

https://github.com/rtk-ai/rtk aka "rust token killer" is a rust cli command wrapper that recoverably filters cli commands to the necessary outputs, saving tokens, and in many cases improving model performance due to decreased context filler.

Token Savings (30-min Claude Code Session)

Operation Frequency Standard rtk Savings
ls / tree 10x 2,000 400 -80%
cat / read 20x 40,000 12,000 -70%
grep / rg 8x 16,000 3,200 -80%
git status 10x 3,000 600 -80%
git diff 5x 10,000 2,500 -75%
git log 5x 2,500 500 -80%
git add/commit/push 8x 1,600 120 -92%
cargo test / npm test 5x 25,000 2,500 -90%
ruff check 3x 3,000 600 -80%
pytest 4x 8,000 800 -90%
go test 3x 6,000 600 -90%
docker ps 3x 900 180 -80%
Total ~118,000 ~23,900 -80%

Of course, I saw you take care to optimize for model idiosyncrasies per your blog and I expect rtk integration would need some optimization to work properly.

Issue Type

Other

Steps to Reproduce

see tokens savings table, cli commands like ls / tree produce lots of extraneous output.

Expected Performance

see tokens savings table

Actual Performance

see token savings table

Performance Measurements

Project Size

No response

Environment Details

Macos M3 Max, but applies across all platforms.

Configuration

Profiling Data

Frequency

Always

Workarounds

No response

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions