@Damian, my suggestion would be to add a whitelist. We have a handful of clients that have multiple VPSs with us and run scripts like MTPuTTY so the script will see them with XX connections while they really only have 2 or 3 connections per VPS.
Also, I don't know if SFTP is handled the same as...