Murano OSTF check fail

Bug #1264292 reported by Bogdan Dobrelya
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Timur Nurlygayanov
4.1.x
Invalid
High
Timur Nurlygayanov
5.0.x
Fix Released
High
Timur Nurlygayanov

Bug Description

Murano OSTF fails with Fuel 4.0.
env: iso #202 {"build_id": "2013-12-25_01-01-41", "ostf_sha": "a27665008e961b773861b95969acf61d25f0f977", "build_number": "202", "nailgun_sha": "d4aecc4f1b60d66021bac195800952fb94cbc627", "fuelmain_sha": "15f575db2d35e3374a217e29a38091152801cfe5", "astute_sha": "9000b12aa0f3639bd4a18efeb82ef554fdbde9c0", "release": "4.0", "fuellib_sha": "cc0301c5f0399e7567230d5cf261f0796005d91c"}

Testing was prepared according to http://docs.mirantis.com/fuel/fuel-4.0/user-guide.html#platform-tests-description

(see ostf* and murano* logs for errors)

Additional info:
Looks like the issue is related to flavor settings OSTF use for launching testing instances.
I've managed to spawn IIS service in Win2012std Murano environment manually, only with flavor of 1/1/50/50 (cpu/RAM/root/ephemeral), and failed with 1/1/40/40 and 1/1/30/30 as well (nova-compute error:Instance type's disk is too small for requested image).
Note, free disk space consumed at /var/lib/nova (42G total) max was not more than 60% all the time.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
description: updated
Changed in fuel:
assignee: nobody → Timur Nurlygayanov (tnurlygayanov)
importance: Undecided → High
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Bogdan, what the 'correct' flavor we can use for Windows instances? Can we use m1.large for it?

tags: added: ostf
removed: ost
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

I've managed to spawn IIS service in Win2012std Murano environment manually only with flavor of 1/1/50/50 (cpu/RAM/root/ephemeral), and failed with 1/1/40/40 and 1/1/30/30 as well (nova-compute error:Instance type's disk is too small for requested image)

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

I can not reproduce this issue with the standard 'm1.medium' flavor with 2 vCPU, 40 GB root disk and 4 GB of RAM and OSTF tests use this flavor for all Windows-based services. The one equirement for Windows VM is minimum 40GB of hard disk and 2 GB of RAM.

It will be very helpfull if we can collect logs from Murano/Heat and Nova to find the root of this problem. The additional fix to Murano OSTF tests to collect more information about FAILED tests: https://review.openstack.org/74322

Also, Bogdan, can you please recheck this issue for the latest Fuel 4.1 with Murano 0.4.1, using the updated user manual: https://github.com/stackforge/fuel-docs/blob/master/pages/user-guide/post-install-healthchecks.rst#preparing-murano-for-testing

I can help you with Windows Server 2012 image and with logs analysis.

Changed in fuel:
assignee: Timur Nurlygayanov (tnurlygayanov) → Bogdan Dobrelya (bogdando)
status: In Progress → Incomplete
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Here is a logs snapshot from #211 ISO. Murano OSTF tests failed, there was also the errors mentioned in https://bugs.launchpad.net/fuel/+bug/1272217 found (Could not decode response body as JSON).

This time I cannot manually spawn IIS service in Win2012std neither with m1.medium, nor 1/1/50/50 (test_murano custom flavor)

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Changed in fuel:
status: Incomplete → Confirmed
Changed in fuel:
assignee: Bogdan Dobrelya (bogdando) → Tatyana (tatyana-leontovich)
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

After several retries, I've managed to manually spawn IIS service again, see details below (still no luck with OSTF tho)

{"build_id": "2014-02-26_13-39-45", "mirantis": "yes", "build_number": "211", "nailgun_sha": "ea08cef3e06a72f47cfaa8cd8fe6d034e2cf722e", "ostf_sha": "8e6681b6d06c7cb20a84c1cc740d5f2492fb9d85", "fuelmain_sha": "baa8bb07393698f1186cb67bb65f1b93907c59bd", "astute_sha": "10cccc87f2ee35510e43c8fa19d2bf916ca1fced", "release": "4.1", "fuellib_sha": "0a2e5bdc01c1e3bb285acb7b39125101e950ac72"}

Reproduce:
* Deploy Centos simple, Neutron with GRE, Murano and Ceilometer, Cinder LVM for volumes, Ceph RBD for images;
nodes: 1 controller, 1 compute, 1 cinder, 2 ceph
* Prepare Murano OSTF tests https://github.com/stackforge/fuel-docs/blob/master/pages/user-guide/post-install-healthchecks.rst#preparing-murano-for-testing
* Manually create Murano environment and deploy Win2012std IIS service using 1 vcpu/1G ram/50G disk/50G ephemeral/10000MB swap (murano_test) flavor.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Is it means that we should just change m1.medium flavor to m1.large ?

Changed in fuel:
status: Confirmed → In Progress
assignee: Tatyana (tatyana-leontovich) → Timur Nurlygayanov (tnurlygayanov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ostf (master)

Fix proposed to branch: master
Review: https://review.openstack.org/77113

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Probably we should:
1) For Murano OSTF based on KVM CI tests, use custom flavor 2/2/50/50/10000 and increase compute node powers approriately in devops templates.
2) For Murano OSTF based on BM labs, no comments because I've not tested Murano OSTF for BM case.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ostf (stable/4.1)

Fix proposed to branch: stable/4.1
Review: https://review.openstack.org/77116

Changed in fuel:
milestone: 4.1 → 4.1.1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-ostf (master)

Fix proposed to branch: master
Review: https://review.openstack.org/81746

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-ostf (master)

Reviewed: https://review.openstack.org/81746
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=f6f25d85c2f8505bb4b234bc0b06a0b47eb0141f
Submitter: Jenkins
Branch: master

commit f6f25d85c2f8505bb4b234bc0b06a0b47eb0141f
Author: TimurNurlygayanov <email address hidden>
Date: Thu Mar 20 13:39:02 2014 +0400

    Fixed issue with failed OSTF tests

    Change-Id: Ife17c274fed3be2fbfdb6f83a39fef2b311ca280
    Related-Bug: #1264292

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

backport to 4.1.1, please

Changed in fuel:
milestone: 4.1.1 → 5.0
tags: added: backports-4.1.1
Changed in fuel:
milestone: 5.0 → 4.1.1
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

backport to stable/4.1 please

Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Vladimir, how we can backport this fix to 4.1.1?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-ostf (stable/4.1)

Related fix proposed to branch: stable/4.1
Review: https://review.openstack.org/86247

Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Related fix merged to fuel-ostf (stable/4.1)

Reviewed: https://review.openstack.org/86247
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=3690709d95c1f42506b6d955b44221910cec8527
Submitter: Jenkins
Branch: stable/4.1

commit 3690709d95c1f42506b6d955b44221910cec8527
Author: Sergey Murashov <email address hidden>
Date: Wed Apr 9 11:27:27 2014 +0400

    Fixed issue with failed OSTF tests

    Change-Id: I428ece5d7e0cda6c7c8eb91623018d95b9e4998d
    Related-Bug: #1264292

Changed in fuel:
milestone: 4.1.1 → 5.0
Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

can not verify on 127 iso - murano service do not start - [root@node-1 ~]# /etc/init.d/openstack-murano-api status
murano-api dead but subsys locked
[root@node-1 ~]#

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

should be re-verified after https://bugs.launchpad.net/fuel/+bug/1311590 will be fixed

Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Need to retest.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.