Runtime security enforcement and capability scoping for agents

Hi everyone. We're AI researchers at Harvard and Carnegie Mellon working on a project to advance the state of agent security. Currently, many systems rely on static sandboxing, which in long-running sessions enables agents to understand the safeguards holding them in place and break out of them. We've found vulnerabilities across over a dozen agent providers and frameworks (practically every one we tested) displaying this behavior (eg. a model fraudulently splitting payments to avoid a company-set payment limit, multi-model agent teams infecting each with injections, MCP rug pulls, etc).

We've developed an architecture that does two things: 1) instead of setting a sandbox for a session and leaving it in place, dynamically scoping the sandbox to cover the minimum subset of capabilities and file accesses that are needed for solving a particular problem set by the user, and continuously moving that sandbox to be in line with what the user wants. Think of this as, instead of a large stationary box, being a smaller, faster, moving container around the agent; 2) monitoring strictly speaking benign behavior (accepted tool calls, accepted file access) for suspicious behavior, borrowing techniques my partner and I developed in AML research. Together, those components have been able to mitigate almost every common attack class against models that we've evaluated so far.

Our system has performed very well on open benchmarks and data we've been able to evaluate it on, but our goal is to evaluate it on production data. We hope to release a paper/open-source project as an output of this, but really need production data to verify that our method works as well on real production data as it does on open benchmarks.

If you're interested in testing it, we'd love it if you signed up for our waitlist.

Thank you, and hope to hear from you!