MAAS 3.3 RC shows incorrect storage amount

Hi Kevin!

I'm currently failing to reproduce the issue, but maybe you can help to narrow the scope.
Can you please examine websocket messages that MAAS UI makes towards MAAS API and the data that is being returned?

JSON data returned for `machine.list` and `machine.get` methods might be helpful.

E.g.
Thats the UI requesting summary for the machine with id 4g3pr6
{"method":"machine.get","type":0,"params":{"system_id":"4g3pr6"},"request_id":7}

Then there will be an response message starting with:
{"type": 1, "request_id": 7, "rtype": 0, "result": {...}}

Can you please check if properties `physical_disk_count` and `storage` are also different for different method calls?

Hi Anton!

I'm unable to find where I can get that data for you. Is there any way I can find it in the Chrome Developer Tools window?

Otherwise, I can get you temporary credentials to this MaaS server so that you can interrogate it yourself?

In the Chrome Developer Tools it is available under Network -> WS

If there is no `ws` socket available on the left, just refresh the page with the Developer Tools being opened and it should appear.

Ahh ok, here you go. Here's the entire object for bare metal server bmhv7.maas.
I'll include 2 screenshots that also show the output on the overview screen and the output for machine.get on this server.

output for machine.get:
{
    "type": 1,
    "request_id": 14,
    "rtype": 0,
    "result": {
        "id": 15,
        "created": "Tue, 13 Dec. 2022 17:17:57",
        "updated": "Thu, 12 Jan. 2023 05:21:25",
        "system_id": "4erkye",
        "hardware_uuid": "00000000-0000-0000-0000-ac1f6bad3dfe",
        "hostname": "bmhv7",
        "description": "",
        "pool": {
            "id": 0,
            "name": "default"
        },
        "domain": {
            "id": 0,
            "name": "maas"
        },
        "status": "Deployed",
        "owner": "maasadmin",
        "bios_boot_method": "uefi",
        "osystem": "custom",
        "distro_series": "u-2004-0-k-12212-0",
        "architecture": "amd64/generic",
        "min_hwe_kernel": "",
        "hwe_kernel": "focal (ga-20.04)",
        "node_type": 0,
        "parent": null,
        "error_description": "",
        "zone": {
            "id": 2,
            "name": "atl2"
        },
        "cpu_count": 32,
        "cpu_speed": 2400,
        "memory": 64,
        "swap_size": 0,
        "bmc": 13,
        "power_state": "on",
        "error": "All scripts successfully ran",
        "license_key": "",
        "boot_disk": null,
        "default_user": "",
        "current_commissioning_script_set": 68,
        "current_installation_script_set": 297,
        "current_testing_script_set": 69,
        "locked": false,
        "last_applied_storage_layout": "lvm",
        "enable_hw_sync": false,
        "permissions": [
            "edit",
            "delete"
        ],
        "fqdn": "bmhv7.maas",
        "actions": [
            "on",
            "off",
            "release",
            "test",
            "rescue-mode",
            "mark-broken",
            "lock",
            "tag",
            "untag",
            "clone",
            "set-zone",
            "set-pool",
            "delete"
        ],
        "node_type_display": "Machine",
        "link_type": "machine",
        "tags": [],
        "physical_disk_count": 2,
        "storage": 2560.5,
        "storage_tags": [
            "ssd"
        ],
        "commissioning_status": {
            "status": 2,
            "pending": 0,
            "running": 0,
            "passed": 12,
            "failed": 0
        },
        "commissioning_start_time": "Tue, 13 Dec. 2022 20:59:06",
        "testing_status": {
            "status": 2,
            "pending": 0,
            "running": 0,
            "passed": 2,
            "failed": 0
        },
        "testing_start_time": "Tue, 13 Dec. 2022 21:00:10",
        "installation_start_time": "Thu, 12 Jan. 2023 05:11:38",
        "has_logs": true,
        "status_code": 6,
        "simple_status": "Deployed",
        "subnets": [
            "10.10.0.0/23",
            "74.50.109.192/28"
        ],
        "fabrics": [
            "fabric-0",
            "fabric-2"
        ],
        "spaces": [
            "BGP",
            "Default"
        ],
        "extra_macs": ...

Thanks Kevin!

I confirm that `machine.list` and `machine.get` return different values for `physical_disk_count` and `storage`.

You mentioned:
> bmhv9.maas 10.24TB 2560.5GB over 2 disks (1x 512GB + 1x 2048GB)

And how many VMs do you have on `bmhv9.maas`? It seems that all the values are simply multiplied by 4 (do you have 4 VMs there?)

For the reference:
In both cases `dehydrate` method of NodeHandler is called (websockets/handlers/node.py)

But for `list` we take values from `physical_disk_count` and `total_storage` returned by Meta.list_queryset (websockets/handlers/machine.py)

physical_disk_count=Count(
  "current_config__blockdevice__physicalblockdevice"
),
total_storage=Sum(
  "current_config__blockdevice__physicalblockdevice__size"
)

And for `get` we do calculation in a bit difference way (which seems to be a correct one).

There are no VMs running on them, all the "bmhv..." nodes are bare metal servers running Kubernetes.

That said, it seems that the storage calculation is happening correctly for nodes that are actually virtual machines, while not happening correctly for nodes that are actually bare metal servers.

Kevin, can you please check if the same behaviour happens if you fetch machines list and machine summary data using CLI?

Hi Anton, I checked using "maas admin nodes read" and the storage information returned by the CLI is correct.

When I saw this happening in 3.3.2 yesterday, I could see the values flapping between correct ones and "double" when refreshing the page.

Specially if I entered the storage config, changed something (like a mount point) when going back I would see different numbers.