When you make a lot of SSL/TLS (don't know the right terminology for it) connections, it still takes longer time to connect than normally. I think this can't be solved easily, because it needs to transmit certificate/key during to handshake.
But what's really wrong is that CPU usage raises up to 100% during this operation. And this should be fixed.
i thought it is caused by fast looping in ConnectionManager::run() when processing too many accept()'s, but inserting Sleep(1000) doesn't change anything, so it must be anywhere else.
It can be easily reproduced when you make secure connections to all users in DCDev public (for example by "Browse file list"). While all filelists display immediately with very low CPU usage when TLS is disabled, it will take longer time with 100% CPU usage when TLS is enabled.
you haven't run this through a profiler, have you? it could be that creating up ephemeral keys for dhe takes some time...