Article
A retry can work and still be unjustified: introducing RetryBench
uuhugg
• • 3You Liu PRO
uuhugg
AI & ML interests
None yet
Recent Activity
repliedto their post 12 days ago
A retry can succeed and still be unjustified.
RetryBench separates whether an agent’s retry is supported by the visible contract from what happens afterward. After a timeout, success alone does not tell us whether retrying was justified.
Try the interactive demo (English / 中文):
https://huggingface.co/spaces/uuhugg/retrybench-demo
Read the findings and reproduction guide:
https://huggingface.co/blog/uuhugg/retrybench-retry-permission
How do you evaluate retries in your agents? authored a paper 14 days ago
RetryBench: Separating Retry Permission from Recovery Outcomes in GPT Tool Use reacted to theirpost with 🔥 15 days ago
A retry can succeed and still be unjustified.
RetryBench separates whether an agent’s retry is supported by the visible contract from what happens afterward. After a timeout, success alone does not tell us whether retrying was justified.
Try the interactive demo (English / 中文):
https://huggingface.co/spaces/uuhugg/retrybench-demo
Read the findings and reproduction guide:
https://huggingface.co/blog/uuhugg/retrybench-retry-permission
How do you evaluate retries in your agents?Organizations
published an article 15 days ago