| 2017-07-13 15:04:11 |
Zoltan Langi |
bug |
|
|
added bug |
| 2017-07-13 15:05:15 |
Zoltan Langi |
description |
Detailed bug description:
Installed openstack Ocata with fuel 11, installation finished. After a few hours of finishing the install, I've added more compute nodes, that succeeded.
After a few days I wanted to add even more compute nodes, but the instalation fails with the following message:
Error
All nodes are finished. Failed tasks: Task[hosts/1], Task[hosts/3], Task[hosts/2] Stopping the deployment process!
In the puppet.log on the nodes, found this:
2017-07-13 14:49:30 +0000 Puppet (err): Duplicate declaration: Host[node-13.xx.com] is already declared; cannot redeclare at /etc/puppet/modules/osnailyfacter/manifests/hosts/hosts.pp:25 on node node-1.xx.com
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:500:in `fail_on_duplicate_type_and_title'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:86:in `add_one_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:72:in `block in add_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `each'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:88:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:59:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions/create_resources.rb:73:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:43:in `block (2 levels) in <top (required)>'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `each'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/function.rb:42:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/ifstatement.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `evaluate_classes'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:60:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:12:in `block in each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:426:in `evaluate_main'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:121:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:34:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:95:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:93:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:92:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:52:in `find'
/usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in `find'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:222:in `block in main'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:159:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:507:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:496:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:146:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:in `execute'
/usr/bin/puppet:8:in `<main>'
Checking the source code, found this file:
root@node-1:~# cat /etc/hiera/deleted_nodes.yaml
deleted_nodes:
- node-13.xx.com
- node-31.xx.com
- node-30.xx.com
- node-11.xx.com
- node-35.xx.com
- node-17.xx.com
- node-16.xx.com
- node-10.xx.com
According to this file, these nodes are failed but that's not true because they're online and running right now, fuel shows them as online.
Impact:
Currently I can't add or remove any nodes from my environment because it fails on this error.
Version: fuel 11 ocata
Thank you for looking into it in advance,
Zoltan |
Detailed bug description:
Installed openstack Ocata with fuel 11, installation finished. After a few hours of finishing the install, I've added more compute nodes, that succeeded.
After a few days I wanted to add even more compute nodes, but the instalation fails with the following message:
Error
All nodes are finished. Failed tasks: Task[hosts/1], Task[hosts/3], Task[hosts/2] Stopping the deployment process!
In the puppet.log on the nodes, found this:
2017-07-13 14:49:30 +0000 Puppet (err): Duplicate declaration: Host[node-13.xx.com] is already declared; cannot redeclare at /etc/puppet/modules/osnailyfacter/manifests/hosts/hosts.pp:25 on node node-1.xx.com
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:500:in `fail_on_duplicate_type_and_title'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:86:in `add_one_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:72:in `block in add_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `each'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:88:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:59:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions/create_resources.rb:73:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:43:in `block (2 levels) in <top (required)>'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `each'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/function.rb:42:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/ifstatement.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `evaluate_classes'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:60:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:12:in `block in each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:426:in `evaluate_main'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:121:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:34:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:95:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:93:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:92:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:52:in `find'
/usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in `find'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:222:in `block in main'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:159:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:507:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:496:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:146:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:in `execute'
/usr/bin/puppet:8:in `<main>'
Checking the source code, found this file:
root@node-1:~# cat /etc/hiera/deleted_nodes.yaml
deleted_nodes:
- node-13.xx.com
- node-31.xx.com
- node-30.xx.com
- node-11.xx.com
- node-35.xx.com
- node-17.xx.com
- node-16.xx.com
- node-10.xx.com
According to this file, these nodes are deleted but that's not true because they're online and running, serving right now, fuel node list shows them as online.
Impact:
Currently I can't add or remove any nodes from my environment because it fails on this error.
Version: fuel 11 ocata
Thank you for looking into it in advance,
Zoltan |
|
| 2017-07-14 09:59:03 |
Zoltan Langi |
summary |
adding compute nodes after deplpoyment finished failes because some hosts are already declared |
adding nodes after deplpoyment finished fails because some hosts are already declared (or appears to be deleted) |
|
| 2017-07-14 09:59:31 |
Zoltan Langi |
summary |
adding nodes after deplpoyment finished fails because some hosts are already declared (or appears to be deleted) |
adding nodes after the deplpoyment finished fails, because some active hosts are already declared (or appears to be deleted) |
|
| 2017-07-14 10:00:28 |
Zoltan Langi |
description |
Detailed bug description:
Installed openstack Ocata with fuel 11, installation finished. After a few hours of finishing the install, I've added more compute nodes, that succeeded.
After a few days I wanted to add even more compute nodes, but the instalation fails with the following message:
Error
All nodes are finished. Failed tasks: Task[hosts/1], Task[hosts/3], Task[hosts/2] Stopping the deployment process!
In the puppet.log on the nodes, found this:
2017-07-13 14:49:30 +0000 Puppet (err): Duplicate declaration: Host[node-13.xx.com] is already declared; cannot redeclare at /etc/puppet/modules/osnailyfacter/manifests/hosts/hosts.pp:25 on node node-1.xx.com
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:500:in `fail_on_duplicate_type_and_title'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:86:in `add_one_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:72:in `block in add_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `each'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:88:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:59:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions/create_resources.rb:73:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:43:in `block (2 levels) in <top (required)>'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `each'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/function.rb:42:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/ifstatement.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `evaluate_classes'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:60:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:12:in `block in each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:426:in `evaluate_main'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:121:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:34:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:95:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:93:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:92:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:52:in `find'
/usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in `find'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:222:in `block in main'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:159:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:507:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:496:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:146:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:in `execute'
/usr/bin/puppet:8:in `<main>'
Checking the source code, found this file:
root@node-1:~# cat /etc/hiera/deleted_nodes.yaml
deleted_nodes:
- node-13.xx.com
- node-31.xx.com
- node-30.xx.com
- node-11.xx.com
- node-35.xx.com
- node-17.xx.com
- node-16.xx.com
- node-10.xx.com
According to this file, these nodes are deleted but that's not true because they're online and running, serving right now, fuel node list shows them as online.
Impact:
Currently I can't add or remove any nodes from my environment because it fails on this error.
Version: fuel 11 ocata
Thank you for looking into it in advance,
Zoltan |
Detailed bug description:
Installed openstack Ocata with fuel 11, installation finished. After a few hours of finishing the install, I've added more compute nodes, that succeeded.
After a few days I wanted to add even more compute nodes, but the instalation fails with the following message:
Error
All nodes are finished. Failed tasks: Task[hosts/1], Task[hosts/3], Task[hosts/2] Stopping the deployment process!
In the puppet.log on the nodes, found this:
2017-07-13 14:49:30 +0000 Puppet (err): Duplicate declaration: Host[node-13.xx.com] is already declared; cannot redeclare at /etc/puppet/modules/osnailyfacter/manifests/hosts/hosts.pp:25 on node node-1.xx.com
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:500:in `fail_on_duplicate_type_and_title'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:86:in `add_one_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:72:in `block in add_resource'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `each'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:71:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:88:in `add_resource'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:59:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions/create_resources.rb:73:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:43:in `block (2 levels) in <top (required)>'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `each'
/etc/puppet/modules/stdlib/lib/puppet/parser/functions/ensure_resource.rb:36:in `block in <top (required)>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:164:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:157:in `block in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/function.rb:42:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/ifstatement.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:236:in `evaluate_classes'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:60:in `block (3 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:64:in `exceptwrap'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:43:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:42:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:12:in `block in each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/branch.rb:11:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `collect'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/resource.rb:25:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:11:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:61:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:129:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:77:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:426:in `evaluate_main'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:135:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:121:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:34:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:95:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:93:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:92:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:52:in `find'
/usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in `find'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:222:in `block in main'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:159:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:507:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:496:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:146:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:in `execute'
/usr/bin/puppet:8:in `<main>'
Checking the source code, found this file:
root@node-1:~# cat /etc/hiera/deleted_nodes.yaml
deleted_nodes:
- node-13.xx.com
- node-31.xx.com
- node-30.xx.com
- node-11.xx.com
- node-35.xx.com
- node-17.xx.com
- node-16.xx.com
- node-10.xx.com
According to this file, these nodes are deleted but that's not true because they're online and running, serving right now, fuel node list shows them as online.
Impact:
Currently I can't add or remove any nodes from my environment or do any changed like password, certificate change because it fails on this error.
Version: fuel 11 ocata
Thank you for looking into it in advance,
Zoltan |
|
| 2017-11-20 18:12:45 |
Oleksiy Molchanov |
fuel: status |
New |
Incomplete |
|
| 2017-11-20 18:12:50 |
Oleksiy Molchanov |
fuel: milestone |
|
11.x-updates |
|
| 2017-11-20 18:12:59 |
Oleksiy Molchanov |
fuel: assignee |
|
MOS Maintenance (mos-maintenance) |
|
