Further logrotate fixes

Fix proposed to branch: master
Review: https://review.openstack.org/187910

Folks, what is the impact of this bug? It looks not high, in fact. I suggest to lower this to medium and move to the 7.0

Bogdan, unless you get written confirmation here from QA & support folks (Andrey Epifanov was the engaged person), let's NOT silently deprioritize and move to 7.0

Sorry, I missed the comment sent to me from Vladimir Khlyunev:
"As I investigated, the changes are close related to some changes in
https://bugs.launchpad.net/fuel/+bug/1461400. Current version of fuel-
logrotate works and there is some risks to change and debug it while HCF
in progress. But current version is too complicated (run twice an hour,
check if logrotate is running via pgrep) and contains pitfalls. I
suggest to move changes listed here to 7.0."

Here it is https://bugs.launchpad.net/fuel/+bug/1462050/comments/5
These two bugs looks related. And there is a common thing: absolutely no impact specified.

So, I'm marking this as a duplicate of https://bugs.launchpad.net/fuel/+bug/1462050 and suggest to continue the development process in the scope of the latter one.

The "delaycompress" parameter is the main issue that should be fixed. /etc/logrotate.d/fuel.nodaily does not contain this parameter, but /etc/logrotate.conf contains, so there is a possibiliny to lock fuel-logrotate by daily logrotate. This bug should be fixed, not moved.

Bogdan, please ask Andrew Maksimov if you want to change the status or the milestone of this bug.

Please review, https://review.openstack.org/#/c/187910/ for this bug.

Related fix proposed to branch: master
Review: https://review.openstack.org/191582

Reviewed: https://review.openstack.org/187910
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=dd0eacc4606308c6ef54f86d2d6bf2c224a8c713
Submitter: Jenkins
Branch: master

commit dd0eacc4606308c6ef54f86d2d6bf2c224a8c713
Author: Bartłomiej Piotrowski <email address hidden>
Date: Wed Jun 3 09:17:19 2015 +0200

    Make log rotation more bulletproof

    - use flock to ensure that only one instance of fuel-logrotate is running
    - combine centos and ubuntu versions of fuel-logrotate to a single file
    - remove unused versions of fuel-logrotate file
    - enable delayed compression in system-wide logrotate.conf to avoid race
      condition between fuel-logrotate and regular logrotate
    - enable delaycompress also on master node
    - default minsize (30M) and maxsize (100M) in system-wide logrotate.conf
    - use file_line for logrotate.conf updates
    - use bash to execute fuel-logrotate
    - fix path to cron's logrotate script in logrotate-debug

    This commit changes default behavior of logrotate by enabling compress
    and delaycompress. The result is that compression is enabled for all log
    files managed by logrotate and rotated logs are compressed only on the next
    rotation. It allows us to resolve rotation cycle that might happen when
    both regular logrotate and fuel-logrotate were executed at the same
    time, causing both to rotate the same file.

    Closes-Bug: 1461400
    Doc-Impact: ops guide
    Change-Id: I94c497a8916fa6fac87ea2ddc51f6de56ab73f98
    Co-Authored-By: Alex Schultz <email address hidden>

Fix proposed to branch: stable/6.1
Review: https://review.openstack.org/191690

Reviewed: https://review.openstack.org/191690
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=ce272320f45c88a5e66f4762a15fcf02ce7c217b
Submitter: Jenkins
Branch: stable/6.1

commit ce272320f45c88a5e66f4762a15fcf02ce7c217b
Author: Bartłomiej Piotrowski <email address hidden>
Date: Wed Jun 3 09:17:19 2015 +0200

    Make log rotation more bulletproof

    - use flock to ensure that only one instance of fuel-logrotate is running
    - combine centos and ubuntu versions of fuel-logrotate to a single file
    - remove unused versions of fuel-logrotate file
    - enable delayed compression in system-wide logrotate.conf to avoid race
      condition between fuel-logrotate and regular logrotate
    - enable delaycompress also on master node
    - default minsize (30M) and maxsize (100M) in system-wide logrotate.conf
    - use file_line for logrotate.conf updates
    - use bash to execute fuel-logrotate
    - fix path to cron's logrotate script in logrotate-debug

    This commit changes default behavior of logrotate by enabling compress
    and delaycompress. The result is that compression is enabled for all log
    files managed by logrotate and rotated logs are compressed only on the next
    rotation. It allows us to resolve rotation cycle that might happen when
    both regular logrotate and fuel-logrotate were executed at the same
    time, causing both to rotate the same file.

    Closes-Bug: 1461400
    Doc-Impact: ops guide
    Change-Id: I94c497a8916fa6fac87ea2ddc51f6de56ab73f98
    Co-Authored-By: Alex Schultz <email address hidden>
    (cherry picked from commit dd0eacc4606308c6ef54f86d2d6bf2c224a8c713)

Reviewed: https://review.openstack.org/191582
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=e642a55e1485c478f1701a74b9dde4983bdda604
Submitter: Jenkins
Branch: master

commit e642a55e1485c478f1701a74b9dde4983bdda604
Author: Alex Schultz <email address hidden>
Date: Sun Jun 14 13:07:25 2015 -0500

    Add copytruncate to conntrackd logrotate config

    Currently the service restart of conntrackd does not always result
    in the log files properly being rotated. We have seen instances where
    the file will continue to be written to even after rotation. In order
    to address this, this change adds a copytruncate to the logrotation
    configuration.

    Change-Id: I9832021c88d8dde87e9b391b50b479e81f630402
    Closes-Bug: 1464932
    Related-Bug: 1461400

Related fix proposed to branch: stable/6.1
Review: https://review.openstack.org/191760

Reviewed: https://review.openstack.org/191760
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=483b543f8b81840be3808395edf7359d80923921
Submitter: Jenkins
Branch: stable/6.1

commit 483b543f8b81840be3808395edf7359d80923921
Author: Alex Schultz <email address hidden>
Date: Sun Jun 14 13:07:25 2015 -0500

    Add copytruncate to conntrackd logrotate config

    Currently the service restart of conntrackd does not always result
    in the log files properly being rotated. We have seen instances where
    the file will continue to be written to even after rotation. In order
    to address this, this change adds a copytruncate to the logrotation
    configuration.

    Change-Id: I9832021c88d8dde87e9b391b50b479e81f630402
    Closes-Bug: 1464932
    Related-Bug: 1461400
    (cherry picked from commit e642a55e1485c478f1701a74b9dde4983bdda604)

Related fix proposed to branch: stable/6.0
Review: https://review.openstack.org/199026

Reviewed: https://review.openstack.org/199026
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=cf13078c4fe492ea13a6698b8ed765bc1a2cdb82
Submitter: Jenkins
Branch: stable/6.0

commit cf13078c4fe492ea13a6698b8ed765bc1a2cdb82
Author: Bartłomiej Piotrowski <email address hidden>
Date: Tue Jul 7 11:12:53 2015 +0200

    Backport logrotate changes from 6.1

    * use tabooext to ensure no overlap between Fuel's cron job and the
      system one
    * run logrotate twice an hour
    * merge weekly and daily configuration files
    * rotate logs basing on two criteria: age and size
    * use dateformat to make rotated logs' filename more readable
    * enable delayed compression to avoid race condition between
      fuel-logrotate and regular logrotate
    * combine CentOS and Ubuntu logrotate wrappers to a single file

    Related-Bug: 1382515
    Related-Bug: 1450039
    Related-Bug: 1461400
    Closes-Bug: 1472179
    Change-Id: I58caf3c93943af818a03e07d534ff8c42a63d2cb

Customer found in 6.0 and 5.1 Please backport

Verified on ISO#292

Verified for 6.1

Won't Fix for 5.1.1-updates and 6.0-updates as there is no delivery channel for fuel-library fixes

Related fix proposed to branch: stable/5.1
Review: https://review.openstack.org/263639

Change abandoned by Tony Breeds (<email address hidden>) on branch: stable/5.1
Review: https://review.openstack.org/263639
Reason: This branch (stable/5.1) is at End Of Life