The Short Answer
Put agents behind a consistent-hash router keyed by conversation id, scale stateless workers horizontally, and pin only stateful sessions. Add a health-check aware queue to drain overloaded instances gracefully.
Stateless workers handle the bulk of requests and scale linearly; only sessions carrying in-memory state need pinning.
Pro Tip: Drain, Don't Kill
Implement a drain window so in-flight work finishes before an instance is retired. Sudden kills cause retry storms.