Lshw incorrectly detects amount of RAM in system

Bug #1752440 reported by Douglas
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Invalid
Medium
Unassigned
lshw (Ubuntu)
New
Undecided
Unassigned

Bug Description

When commissioning our new servers, we are only seeing 50% of the memory show up in the MaasUI. Looking at lshw, it looks like only 12 of the 24 DIMMs are listed (we see 384GB but we have 768GB). Everything else seems to be fine, number of cores, disks, etc... When running free on the machine, we get the expected amount of RAM (768GB)

Current:
384GB

Expecting
768GB

root@maas:~# dpkg -l '*maas*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=======================-================-================-===================================================
ii maas 2.3.0-6434-gd354 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.3.0-6434-gd354 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.3.0-6434-gd354 all MAAS server common files
ii maas-dhcp 2.3.0-6434-gd354 all MAAS DHCP server
ii maas-dns 2.3.0-6434-gd354 all MAAS DNS server
ii maas-proxy 2.3.0-6434-gd354 all MAAS Caching Proxy
ii maas-rack-controller 2.3.0-6434-gd354 all Rack Controller for MAAS
ii maas-region-api 2.3.0-6434-gd354 all Region controller API service for MAAS
ii maas-region-controller 2.3.0-6434-gd354 all Region Controller for MAAS
un maas-region-controller- <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisionin <none> <none> (no description available)
ii python3-django-maas 2.3.0-6434-gd354 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.3.0-6434-gd354 all MAAS python API client (Python 3)
ii python3-maas-provisioni 2.3.0-6434-gd354 all MAAS server provisioning libraries (Python 3)

Revision history for this message
Douglas (dgrosvenor) wrote :
Revision history for this message
Andres Rodriguez (andreserl) wrote :

So this is definitely and issue in LSHW. MAAS directly grabs the information from there.

That said, I think it would be nice for MAAS to have an alternate method of gathering the memory in case lshw is completely wrong as is this case.

Changed in maas:
importance: Undecided → Medium
status: New → Triaged
milestone: none → 2.4.x
summary: - MaaS/lshw incorrectly detects amount of RAM in system
+ Lshw incorrectly detects amount of RAM in system
Changed in lshw (Ubuntu):
importance: Undecided → High
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Actually, I think I may have spoken too soon. Lshw shows two banks of memory:

 lshw:node:
            id: bank:1
            claimed: true
            class: memory
            handle: DMI:001E
            - lshw:description:
              DIMM Synchronous 2400 MHz (0.4 ns)
            - lshw:product:
              72ASS8G72LZ-2G3B2
            - lshw:vendor:
              Micron
            - lshw:physid:
              1
            - lshw:serial:
              191285AE
            - lshw:slot:
              P1-DIMME1
            - lshw:size:
              units: bytes
              34358689792
            - lshw:width:
              units: bits
              64
            - lshw:clock:
              units: Hz
              2400000000
          - lshw:node:
            id: bank:2
            claimed: true
            class: memory
            handle: DMI:0020
            - lshw:description:
              DIMM Synchronous 2400 MHz (0.4 ns)
            - lshw:product:
              72ASS8G72LZ-2G3B2
            - lshw:vendor:
              Micron
            - lshw:physid:
              2
            - lshw:serial:
              191282FF
            - lshw:slot:
              P1-DIMMF1
            - lshw:size:
              units: bytes
              34358689792

Changed in lshw (Ubuntu):
importance: High → Undecided
status: New → Incomplete
Revision history for this message
Lee Trager (ltrager) wrote :

Could you please post the lshw output in XML form to allow us to create a unit test?

You can download this from the commissioning tab or using the CLI with
maas <profile> node-script-result download <system_id> current-commissioning filters=00-maas-01-lshw > lshw.log

Revision history for this message
Douglas (dgrosvenor) wrote :
Revision history for this message
Lee Trager (ltrager) wrote :

lshw shows two banks of memory but MAAS is still correctly calculating the output from lshw. There are twelve occurrences of the line <size units="bytes">34358689792</size> which is 384GB. I looked up the part number reported by lshw, 72ASS8G72LZ-2G3B2. According to [1] each DIM should be 64G, not 32G. There is a commit upstream [2] which adds support for DDR4 RAM and another [3] which adds support for larger amounts of RAM found in DDR4 RAM. As [1] says this module is DDR4 I suspect the version of lshw in Ubuntu needs both these patches.

@dgrosvenor could you try building lshw in rescue mode to confirm?

sudo apt install git gcc make
git clone https://github.com/lyonel/lshw.git
make
sudo ./lshw -xml > lshw.log

[1] https://www.micron.com/parts/modules/ddr4-sdram/mta72ass8g72lz-2g3
[2] https://github.com/lyonel/lshw/commit/0ea2e95b3ccdeb3834bec6fe22d0794a65e8c37a
[3] https://github.com/lyonel/lshw/commit/ab43c697ebaff3e5138c43a39a17f69859cde87d

Revision history for this message
Lee Trager (ltrager) wrote :

It look like these patches have already been accepted into the Bionic version of lshw.

@dgrosvenor you don't have to compile lshw. You could also deploy Bionic, install lshw, and run sudo lshw -xml > lshw.log to comfirm.

Revision history for this message
Douglas (dgrosvenor) wrote :

I tried deploying Bionic out through MaaS but it doesn't seem to be working. Looking at the console of the system, I'm getting a "failed posting event: start: init-local/check-cache: attempting to read from cache [trust]". Screenshot is attached as well (let me know if you want me to open up a separate ticket for this). Since I can't deploy out Bionic, I'll try compiling lshw from scratch.

Revision history for this message
Andres Rodriguez (andreserl) wrote : Re: [Bug 1752440] Re: Lshw incorrectly detects amount of RAM in system

Are you using a mirror ? That could be due to a mirror not up to date with
the Ubuntu images but I wouldn’t be sure without a full log.

On Fri, Mar 2, 2018 at 4:01 PM Douglas <email address hidden> wrote:

> I tried deploying Bionic out through MaaS but it doesn't seem to be
> working. Looking at the console of the system, I'm getting a "failed
> posting event: start: init-local/check-cache: attempting to read from
> cache [trust]". Screenshot is attached as well (let me know if you want
> me to open up a separate ticket for this). Since I can't deploy out
> Bionic, I'll try compiling lshw from scratch.
>
> ** Attachment added: "maas18.04deploy.png"
>
> https://bugs.launchpad.net/maas/+bug/1752440/+attachment/5067453/+files/maas18.04deploy.png
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1752440
>
> Title:
> Lshw incorrectly detects amount of RAM in system
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1752440/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=maas; milestone=2.4.x; status=Triaged;
> importance=Medium; assignee=None;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=lshw; component=main;
> status=Incomplete; importance=Undecided; assignee=None;
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: andreserl dgrosvenor ltrager
> Launchpad-Bug-Reporter: Douglas (dgrosvenor)
> Launchpad-Bug-Modifier: Douglas (dgrosvenor)
> Launchpad-Message-Rationale: Subscriber (MAAS)
> Launchpad-Message-For: andreserl
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

Lee Trager (ltrager)
Changed in lshw (Ubuntu):
status: Incomplete → New
Changed in maas:
status: Triaged → Invalid
Revision history for this message
Douglas (dgrosvenor) wrote :

@andreserl I'm not sure if I'm using a mirror or not, I'm just using the built in maas.io images. Where would I see that/what log are you looking for? Once I get Bionic installed or the latest lshw compiled, I'll get you all the logs. Thanks!

Revision history for this message
Douglas (dgrosvenor) wrote :

Attached is the latest lshw.log from lshw version B.02.18.76. Looks like its getting the right DIMM size (64GB each).

Changed in maas:
milestone: 2.4.x → none
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.