Convergence: Add waiting time throttle for sync point

Current retry method on get sync with sync point and update database table by a while loop.
When a convergence stack with huge number of resources doing any action, all resource try to grep the sync point and cause contentions. This reduce the performance of total sync action.

We should add some method to reduce pointless contentions caused by unlimited retry.

We can add a time throttle and limit the unlimited retry. For example if we give a limit to 5 times and with time throttle set to random number between 0.5 -1 sec.

After five times of retry, resource should consider under a contention, therefore start add a waiting time throttle before next retry.

Tested by a devstack vcpu2 memory 4G
config:(without improvement)
max_resources_per_stack = -1
rpc_response_timeout = 3600 (to avoid hitting [2])

 -------------------------------------------------------------------
| Size | Time taken (secs) | max_db_cons |
 -------------------------------------------------------------------
| 100 | 70 | 90 |
 -------------------------------------------------------------------
| 200 | 180 | 123 |
 -------------------------------------------------------------------
| 400 | 665 | 125 |
 ------------------------------------------------------------------
| 600 | 950 | 199 |
 ------------------------------------------------------------------

config:(with improvement)
free_sync_retry_limit = 5
time_throttle = 0.5
max_resources_per_stack = -1
rpc_response_timeout = 3600 (to avoid hitting [2])

 -------------------------------------------------------------------
| Size | Time taken (secs) | max_db_cons |
 -------------------------------------------------------------------
| 100 | 44 | 100 |
 -------------------------------------------------------------------
| 200 | 88 | 172 |
 -------------------------------------------------------------------
| 400 | 190 | 142 |
 ------------------------------------------------------------------
| 600 | 330 | 240 |
 ------------------------------------------------------------------

Same size are tested together in different time, so we won't compare the increase ratio between size.

We can tell that with proper waiting time the overall cost of time get a pretty good improvement.

The db connection look also grep a little advance on this, but only make sure through more detail test.

[1] https://github.com/openstack/heat/blob/master/heat/engine/sync_point.py#L121

[2] https://bugs.launchpad.net/heat/+bug/1491185