50 nodes bootstrap slow (tftp server problems)

My suggestion is to remove 5-10 nodes in cycle without sleeps.
Also need to make this parameter configurable as it was done for nodes deployment
https://github.com/stackforge/fuel-astute/blob/master/lib/astute/config.rb#L74

If we try to remove nodes without sleep, we will get a same problem, because remove operation take very little time and Cobbler is going to be loaded as well as before.

I suggest also add second parameter: remove_interval in seconds between remove operations and set it to 10 second as default. For the fifth dozen it will get 40 second difference which can solve this problem.

https://review.openstack.org/#/c/103518/

Timur Nurlygayanov, can you help with a test?

we will test it when we will use Fuel 5.0 with MOX project.

Reviewed: https://review.openstack.org/103518
Committed: https://git.openstack.org/cgit/stackforge/fuel-astute/commit/?id=2b3a8592cc71c6883d82f5bc4820641fba9292a2
Submitter: Jenkins
Branch: master

commit 2b3a8592cc71c6883d82f5bc4820641fba9292a2
Author: Vladimir Sharshov <email address hidden>
Date: Mon Jun 30 13:56:42 2014 +0400

    Avoid high load for Cobbler TFTP when delete many nodes.

    Delete operation does not have limit for nodes, but it
    erase disks and reboot nodes after which they try to boot
    using network. Cobbler installed at master node provide
    ability to boot using network and have limited resources.

    To prevent high load for Cobbler this changes add two things:
    * split all nodes to groups (default - 10) which process
    in series;
    * wait some time (default - 10 sec) between such groups.

    Both of this parameters can be changed in config.

    Closes-Bug: #1330938

    Change-Id: I9c3af6e8ab3c7c610e31baa6e58ec86aae20708d

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/105459

Reviewed: https://review.openstack.org/105459
Committed: https://git.openstack.org/cgit/stackforge/fuel-astute/commit/?id=741b4fa9b964d73d9ab8f2fbcf5bb02836c98da1
Submitter: Jenkins
Branch: stable/5.0

commit 741b4fa9b964d73d9ab8f2fbcf5bb02836c98da1
Author: Vladimir Sharshov <email address hidden>
Date: Mon Jun 30 13:56:42 2014 +0400

    Avoid high load for Cobbler TFTP when delete many nodes.

    Delete operation does not have limit for nodes, but it
    erase disks and reboot nodes after which they try to boot
    using network. Cobbler installed at master node provide
    ability to boot using network and have limited resources.

    To prevent high load for Cobbler this changes add two things:
    * split all nodes to groups (default - 10) which process
    in series;
    * wait some time (default - 10 sec) between such groups.

    Both of this parameters can be changed in config.

    Closes-Bug: #1330938
    Closes-Bug: #1339024

    Backport from 5.1

    Change-Id: I9c3af6e8ab3c7c610e31baa6e58ec86aae20708d

Fix proposed to branch: master
Review: https://review.openstack.org/178119

Change abandoned by Igor Shishkin (<email address hidden>) on branch: master
Review: https://review.openstack.org/178119
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

Change abandoned by Fuel DevOps Robot (<email address hidden>) on branch: master
Review: https://review.openstack.org/178119
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.