database-nodemgr doesn't get number correctly

Bug #1552477 reported by Tony Liu
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
High
Nikhil Bansal
Trunk
Fix Committed
High
Nikhil Bansal

Bug Description

database_nodemgr/database_event_manager.py

========
    popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{s+=$4}END{print s}'` && echo $1"
========

# df -Pk /var/lib/cassandra/data/ContrailAnalyticsCql
Filesystem 1024-blocks Used Available Capacity Mounted on
/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /

# df -Pk /var/lib/cassandra/data/ContrailAnalyticsCql | grep % | awk '{s+=$4}END{print s}'
6.74076e+09

I fixed it with this patch.
========
- popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{s+=$4}END{print s}'` && echo $1"
+ popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{print $4}'` && echo $1"
========

The same issue may happens to other fields of df output.

Tags: analytics
Changed in juniperopenstack:
assignee: nobody → Raj Reddy (rajreddy)
tags: added: analytics
Revision history for this message
Raj Reddy (rajreddy) wrote :

Tony, which OS is this, I don't see this on ubuntu 14 or cento 7.1

In any case can you see if change
{print s}
to
{printf "%i", s}
works.

Revision history for this message
Tony Liu (taoliu-7) wrote :

The server where we see this issue is 5b4s8 10.87.64.132.
You can login to see why this happens there.

root@5b4s8:~# uname -a
Linux 5b4s8 3.13.0-40-generic #69-Ubuntu SMP Thu Nov 13 17:53:56 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@5b4s8:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
root@5b4s8:~# echo "/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /" | awk '{s+=$4}END{print s}'
6.74076e+09
root@5b4s8:~# echo "/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /" | awk '{s+=$4}END{printf "%i", s}'
2147483647

Your suggestion works.

Revision history for this message
Raj Reddy (rajreddy) wrote :

where the problem is reported, there's mawk is installed and it's getting called for awk..

root@5b4s8:~# ls -l /usr/bin/awk
lrwxrwxrwx 1 root root 21 Mar 1 17:35 /usr/bin/awk -> /etc/alternatives/awk
root@5b4s8:~# ls -l /etc/alternatives/awk
lrwxrwxrwx 1 root root 13 Mar 1 17:35 /etc/alternatives/awk -> /usr/bin/mawk
root@5b4s8:~# dpkg -S /usr/bin/mawk
mawk: /usr/bin/mawk

Where as in our regular machines it's gawk -- hence the difference..

Regardless,
. we can use popen call as in

import subprocess
df = subprocess.Popen(["df", "filename"], stdout=subprocess.PIPE)
output = df.communicate()[0]
device, size, used, available, percent, mountpoint = \
    output.split("\n")[1].split()

OR

use os.statvfs()..

import os
statvfs = os.statvfs('/home/foo/bar/baz')

statvfs.f_frsize * statvfs.f_blocks # Size of filesystem in bytes
statvfs.f_frsize * statvfs.f_bfree # Actual number of free bytes

------
There's additional issue that data_file_directories can be multiple and we should be summ'ing up the space on all of them..

Changed in juniperopenstack:
assignee: Raj Reddy (rajreddy) → Nikhil Bansal (nikhilb-u)
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/18203
Submitter: Nikhil Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18203
Committed: http://github.org/Juniper/contrail-controller/commit/e68c499aaf89a2eb404abcf700828a8bb4274685
Submitter: Zuul
Branch: master

commit e68c499aaf89a2eb404abcf700828a8bb4274685
Author: Nikhil B <email address hidden>
Date: Mon Mar 7 10:04:38 2016 +0530

Fixing the parse issue due to mawk

In some setups, if mawk is installed, awk does not output numbers correctly.
With this patch, awk is being avoided altogether to alleviate such issues

Change-Id: I14befc9a72202bfe7c602b004f7ac6972a8efc60
Closes-Bug: 1552477

Changed in juniperopenstack:
milestone: none → r3.1.0.0-fcs
Raj Reddy (rajreddy)
information type: Proprietary → Public
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/18421
Submitter: Nikhil Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18421
Committed: http://github.org/Juniper/contrail-controller/commit/7b6ce73ff5a7e6b287b73fe904ecdb95e3f2191c
Submitter: Zuul
Branch: R3.0

commit 7b6ce73ff5a7e6b287b73fe904ecdb95e3f2191c
Author: Nikhil B <email address hidden>
Date: Mon Mar 7 10:04:38 2016 +0530

Fixing the parse issue due to mawk

In some setups, if mawk is installed, awk does not output numbers correctly.
With this patch, awk is being avoided altogether to alleviate such issues

Change-Id: I14befc9a72202bfe7c602b004f7ac6972a8efc60
Closes-Bug: 1552477
(cherry picked from commit e68c499aaf89a2eb404abcf700828a8bb4274685)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.