Orca-Bench: How Ready Are Language Model Agents for Oncall?
15 points - today at 6:32 PM
SourceComments
4di today at 7:25 PM
looks like the public bench link in the paper was taken down. https://hub.harborframework.com/datasets/orca-bench/ORCA-ben...
This doesn't work anymore. Is there a newer link?
dash2 today at 7:01 PM
Seems like there's a big attack-defence asymmetry at present: models are great at exploiting systems and poor at fixing them.