Provision failed with 404 via empty local mos and auxilary repos

Bug #1572684 reported by Artem Hrechanychenko
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Confirmed
High
Fuel Documentation Team
Mitaka
Won't Fix
High
Evgeny Konstantinov

Bug Description

Detailed bug description:
Provision has failed. Failed to execute hook 'shell' Failed to run command cd / && fa_build_image --image_build_dir /var/lib/fuel/ibp --log-file /var/log/fuel-agent-env-1.log --data_driver nailgun_build_image --input_data '{"image_data": {"/boot": {"container": "gzip", "uri": "http://10.20.0.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz", "format": "ext2"}, "/": {"container": "gzip", "uri": "http://10.20.0.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz", "format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos": [{"name": "ubuntu", "section": "main universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty", "type": "deb"}, {"name": "ubuntu-updates", "section": "main universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty-updates", "type": "deb"}, {"name": "ubuntu-security", "section": "main universe multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty-security", "type": "deb"}, {"name": "mos", "section": "main restricted", "uri": "http://10.20.0.2:8080/mitaka-9.0/ubuntu/x86_64", "priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name": "mos-updates", "section": "main restricted", "uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security", "section": "main restricted", "uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security", "type": "deb"}, {"name": "mos-holdback", "section": "main restricted", "uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1100, "suite": "mos9.0-holdback", "type": "deb"}, {"name": "Auxiliary", "section": "main restricted", "uri": "http://10.20.0.2:8080/mitaka-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary", "type": "deb"}], "packages": ["acl", "anacron", "bash-completion", "bridge-utils", "bsdmainutils", "build-essential", "cloud-init", "curl", "daemonize", "debconf-utils", "gdisk", "grub-pc", "hpsa-dkms", "hwloc", "i40e-dkms", "linux-firmware", "linux-firmware-nonfree", "linux-headers-generic-lts-trusty", "linux-image-generic-lts-trusty", "lvm2", "mcollective", "mdadm", "multipath-tools", "multipath-tools-boot", "nailgun-agent", "nailgun-mcagents", "network-checker", "ntp", "openssh-client", "openssh-server", "puppet", "python-amqp", "ruby-augeas", "ruby-ipaddress", "ruby-json", "ruby-netaddr", "ruby-openstack", "ruby-shadow", "ruby-stomp", "telnet", "ubuntu-minimal", "ubuntu-standard", "uuid-runtime", "vim", "virt-what", "vlan"], "codename": "trusty"}'

from Astute logs:
[18638] Error running provisioning: Image build task failed. Please check build log here for details: /var/log/fuel-agent-env-1.log. Hint: restart deployment can help if no error in build log was found, trace:

From fuel-agent-logs:
2016-04-20 13:45:48.686 6096 ERROR fuel_agent.cmd.agent [-] 404 Client Error: Not Found for url: http://10.20.0.2:8080/mitaka-9.0/ubuntu/x86_64/dists/mos9.0/Release
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent Traceback (most recent call last):
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/cmd/agent.py", line 121, in main
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent getattr(mgr, action)()
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 1046, in do_build_image
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent direct_repo_addrs=driver_os.proxies.direct_repo_addr_list)
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 491, in _set_apt_repos
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent proxies=proxies, direct_repo_addrs=direct_repo_addrs)
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 469, in add_apt_preference
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent direct_repo_addrs=direct_repo_addrs))
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 394, in get_release_file
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent noproxy_addrs=direct_repo_addrs).text
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/utils/utils.py", line 265, in init_http_request
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent response_obj.raise_for_status()
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/requests/models.py", line 837, in raise_for_status
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent raise HTTPError(http_error_msg, response=self)
2016-04-20 13:45:48.686 6096 TRACE fuel_agent.cmd.agent HTTPError: 404 Client Error: Not Found for url: http://10.20.0.2:8080/mitaka-9.0/ubuntu/x86_64/dists/mos9.0/Release

ls /var/www/nailgun/mitaka-9.0/ubuntu/x86_64/
images

Steps to reproduce:
1) Create VM - 2VCPU, 4096 RAM 50HDD, Centos Minimal Install, 3 network interfaces
2) download http://mirror.fuel-infra.org/mos-repos/centos/mos9.0-centos7/os/x86_64/Packages/fuel-release-9.0.0-1.mos6341.noarch.rpm
3) yum install fuel-release
4) yum install fuel-setup
5) bootstrap_admin_node.sh
6) check fuel-services
7) bootstrap 2 slaves
8) create env with controller and compute
9) Deploy changes
10 Verify networks

Expected results:
Deployment successfully finished

Actual result:
 Failed to provision with 404 error

Reproducibility:
100% on KVM, VBOX

Workaround:
Change mos and Auxiliary repos from local to online

Impact:
Separate master node deployment feature

Description of the environment:
     cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-misc-9.0.0-1.mos8309.noarch
 fuel-bootstrap-cli-9.0.0-1.mos273.noarch
 python-fuelclient-9.0.0-1.mos312.noarch
 fuel-9.0.0-1.mos6341.noarch
 fuel-ui-9.0.0-1.mos2668.noarch
 fuel-notify-9.0.0-1.mos8309.noarch
 fuel-release-9.0.0-1.mos6341.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8653.noarch
 python-packetary-9.0.0-1.mos128.noarch
 fuelmenu-9.0.0-1.mos269.noarch
 fuel-utils-9.0.0-1.mos8309.noarch
 fuel-migrate-9.0.0-1.mos8309.noarch
 nailgun-mcagents-9.0.0-1.mos724.noarch
 shotgun-9.0.0-1.mos88.noarch
 network-checker-9.0.0-1.mos72.x86_64
 fuel-openstack-metadata-9.0.0-1.mos8653.noarch
 fuel-setup-9.0.0-1.mos6341.noarch
 fuel-library9.0-9.0.0-1.mos8309.noarch
 fuel-mirror-9.0.0-1.mos128.noarch
 fuel-agent-9.0.0-1.mos273.noarch
 fuel-nailgun-9.0.0-1.mos8653.noarch
 fuel-ostf-9.0.0-1.mos918.noarch
 rubygem-astute-9.0.0-1.mos724.noarch

Tags: area-docs
Revision history for this message
Artem Hrechanychenko (agrechanichenko) wrote :
Changed in fuel:
milestone: none → 9.0
assignee: nobody → Vitaly Parakhin (vparakhin)
summary: - Provision failed with 404
+ Provision failed with 404 via empty local mos and auxilary repos
description: updated
Revision history for this message
Vitaly Parakhin (vparakhin) wrote :

I don't think it should be considered as bug. In case of deploying master on a pre-provisioned server, one should use fuel-mirror tool to create the local MOS Ubuntu packages mirror, and deploy environments from it. This should be reflected in the documentation.

tags: added: area-docs
removed: area-build
Changed in fuel:
assignee: Vitaly Parakhin (vparakhin) → Fuel Documentation Team (fuel-docs)
Changed in fuel:
milestone: 9.0 → 10.0
Revision history for this message
Dmitry Meytin (meytin) wrote :

This bug affects any environments which has not direct access to the internet (e.g. located behind corporate proxy). For this use-case the bug is critical.

Revision history for this message
Dmitry Meytin (meytin) wrote :
Changed in fuel:
status: New → Confirmed
Revision history for this message
Evgeny Konstantinov (evkonstantinov) wrote :

if it's already in the docs, marking this as wontfix.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.