Juniper Openstack

[2.20-Build 64] Contrail API sometimes return 'NoIdError: Unknown id: routing_instance'

Series r3.0
Bug #1476493

Bug #1476493 reported by Daisuke Nakajima on 2015-07-21

This bug affects 2 people

	Status	Importance	Assigned to
Juniper Openstack	Status tracked in Trunk
R2.20	Incomplete	Medium	Prakash Bailkeri
R3.0	New	Medium	Prakash Bailkeri
R3.1	New	Medium	Prakash Bailkeri
Trunk	Incomplete	Medium	Prakash Bailkeri

Bug Description

While creating Virtual-network and Virtual-machine-interface without delay, Contrail API sometimes return "NoIdError: Unknown id: routing_instance".

Sent JSON messages;
{"virtual-network": {"virtual_network_properties": {"vxlan_network_identifier": 30001,"forwarding_mode": "l2_l3","allow_transit": false},"display_name": "testvn01","is_shared": false,"router_external": false,"parent_type": "project","uuid": "00000000-9999-0000-0001-000000000001","route_target_list": {"route_target": ["target:64512:30001"]},"flood_unknown_unicast": true,"fq_name": ["default-domain","demo","testvn01"],"routing_instances": [{"to": ["default-domain","demo","testvn01","testvn01"]}],"network_ipam_refs": [{"to": ["default-domain","default-project","default-network-ipam"],"attr": {"ipam_subnets": [{"subnet": {"ip_prefix": "30.1.1.0","ip_prefix_len": 24},"enable_dhcp": true,"default_gateway": "30.1.1.253","allocation_pools": [],"dhcp_option_list": {"dhcp_option": []},"host_routes": {"route": []},"addr_from_start": true}]}}],"name": "testvn01"}}

'{"virtual-machine-interface": {"display_name": "testvni01","name": "testvni01","virtual_machine_interface_device_owner": "","security_group_refs": [{"to": ["default-domain","demo","default"]}],"fq_name": ["default-domain","demo","testvni01"],"routing_instance_refs": [{"to": ["default-domain","demo","testvn01","testvn01"],"attr": {"direction": "both","protocol": null,"dst_mac": null,"mpls_label": null,"vlan_tag": null,"src_mac": null,"service_chain_address": null}}],"virtual_network_refs": [{"to": ["default-domain","demo","testvn01"]}], "uuid": "00000001-9999-0000-0001-000000000001","parent_type": "project","virtual_machine_interface_mac_addresses": {"mac_address": ["30:00:01:01:00:01"]}}}

###API log
07/21/2015 02:55:35 PM [contrail-03:contrail-api:Config:0]: __default__ [SYS_ERR]: VncApiError: <class 'cfgm_common.exceptions.NoIdError'>
Python 2.7.6: /usr/bin/python
Tue Jul 21 14:55:35 2015

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py in handler_trap_exception(*args=(), **kwargs={})
  534
  535 trace = self._generate_rest_api_request_trace()
  536 response = handler(*args, **kwargs)
  537 self._generate_rest_api_response_trace(trace, response)
  538
response undefined
handler = <bound method VncApiServer.virtual_machine_inter...i_server.vnc_cfg_api_server.VncApiServer object>>
args = ()
kwargs = {}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/gen/vnc_api_server_gen.py in virtual_machine_interfaces_http_post(self=<vnc_cfg_api_server.vnc_cfg_api_server.VncApiServer object>)
20762 if refs:
20763 for ref in refs:
20764 ref_uuid = db_conn.fq_name_to_uuid(obj_type, ref['to'])
20765 (ok, status) = self._permissions.check_perms_link(request, ref_uuid)
20766 if not ok:
ref_uuid = '00000000-9999-0000-0001-000000000001'
db_conn = <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>
db_conn.fq_name_to_uuid = <bound method VncDbClient.fq_name_to_uuid of <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>>
obj_type = 'routing_instance'
ref = {'attr': {'direction': 'both', 'dst_mac': None, 'mpls_label': None, 'protocol': None, 'service_chain_address': None, 'src_mac': None, 'vlan_tag': None}, 'to': ['default-domain', 'demo', 'testvn01', 'testvn01']}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in fq_name_to_uuid(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>, obj_type='routing_instance', fq_name=['default-domain', 'demo', 'testvn01', 'testvn01'])
1493
1494 def fq_name_to_uuid(self, obj_type, fq_name):
1495 obj_uuid = self._cassandra_db.fq_name_to_uuid(obj_type, fq_name)
1496 return obj_uuid
1497 # end fq_name_to_uuid
obj_uuid undefined
self = <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>
self._cassandra_db = <vnc_cfg_api_server.vnc_cfg_ifmap.VncServerCassandraClient object>
self._cassandra_db.fq_name_to_uuid = <bound method VncServerCassandraClient.fq_name_t...r.vnc_cfg_ifmap.VncServerCassandraClient object>>
obj_type = 'routing_instance'
fq_name = ['default-domain', 'demo', 'testvn01', 'testvn01']

/usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py in fq_name_to_uuid(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerCassandraClient object>, obj_type='routing_instance', fq_name=['default-domain', 'demo', 'testvn01', 'testvn01'])
  248
  249 if len(col_infos) == 0:
  250 raise NoIdError('%s %s' % (obj_type, fq_name))
  251
  252 for (col_name, col_val) in col_infos:
global NoIdError = <class 'cfgm_common.exceptions.NoIdError'>
obj_type = 'routing_instance'
fq_name = ['default-domain', 'demo', 'testvn01', 'testvn01']
<class 'cfgm_common.exceptions.NoIdError'>: Unknown id: routing_instance ['default-domain', 'demo', 'testvn01', 'testvn01']
    __class__ = <class 'cfgm_common.exceptions.NoIdError'>
    __delattr__ = <method-wrapper '__delattr__' of NoIdError object>
    __dict__ = {'_unknown_id': "routing_instance ['default-domain', 'demo', 'testvn01', 'testvn01']"}
    __doc__ = None
    __format__ = <built-in method __format__ of NoIdError object>
    __getattribute__ = <method-wrapper '__getattribute__' of NoIdError object>
    __getitem__ = <method-wrapper '__getitem__' of NoIdError object>
    __getslice__ = <method-wrapper '__getslice__' of NoIdError object>
    __hash__ = <method-wrapper '__hash__' of NoIdError object>
    __init__ = <bound method NoIdError.__init__ of NoIdError()>
    __module__ = 'cfgm_common.exceptions'
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of NoIdError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of NoIdError object>
    __repr__ = <method-wrapper '__repr__' of NoIdError object>
    __setattr__ = <method-wrapper '__setattr__' of NoIdError object>
    __setstate__ = <built-in method __setstate__ of NoIdError object>
    __sizeof__ = <built-in method __sizeof__ of NoIdError object>
    __str__ = <bound method NoIdError.__str__ of NoIdError()>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of NoIdError object>
    __weakref__ = None
    _unknown_id = "routing_instance ['default-domain', 'demo', 'testvn01', 'testvn01']"
    args = ()
    message = ''

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 536, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/gen/vnc_api_server_gen.py", line 20764, in virtual_machine_interfaces_http_post
    ref_uuid = db_conn.fq_name_to_uuid(obj_type, ref['to'])
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1495, in fq_name_to_uuid
    obj_uuid = self._cassandra_db.fq_name_to_uuid(obj_type, fq_name)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/vnc_cassandra.py", line 250, in fq_name_to_uuid
    raise NoIdError('%s %s' % (obj_type, fq_name))
NoIdError: Unknown id: routing_instance ['default-domain', 'demo', 'testvn01', 'testvn01']

Tags:

Daisuke Nakajima (dnakajima) on 2015-07-21

information type:	Proprietary → Public
tags:	added: customer
tags:	added: bms

Nagabhushana R (bhushana) on 2015-07-21

tags:

added: config

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

Is this HA setup?
>> Yes HA seup

How many configuration nodes?
>> Three

To which node did you send 1000 requests?
>> To Virtual IP address, so three API servers were used.

Did you repeat sending the same request 1000 times or send 1000 different requests?
>> same request 1000 times

Did you send requests back-to-back or wail for response before sending the next request?
>> No delay. back-to-back.

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

config_1547.log Edit (1.4 MiB, text/plain)

contrail-logs --object-type config --last 10m

Revision history for this message

Prakash Bailkeri (prakashmb) wrote on 2015-07-21:

REST data for VMI create is referring to Routing instance. It is possible that this is not yet created when the POST for VMI is received.

You should not specify routing instance ref in VMI POST data.

Please note: On VN create contrail-schema creates the Routing instance for the VN. And linking between VMI to routing instance is also done by contrail-schema. Since the VMI POST json data has ref to routing instance(which is not yet created), the error is thrown while handling VMI post request.

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

Download full text (11.9 KiB)

I modiried JSON messages as above solution, however, other error messages were seen.

07/21/2015 05:53:47 PM [contrail-03:contrail-api:Config:0]: __default__ [SYS_ERR]: VncApiError: Failed to invoke type specific dbe_update_notification
07/21/2015 05:53:47 PM [contrail-03:contrail-api:Config:0]: __default__ [SYS_ERR]: VncApiError: <type 'exceptions.AttributeError'>
Python 2.7.6: /usr/bin/python
Tue Jul 21 17:53:47 2015

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in _dbe_subscribe_callback(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>, oper_info={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  814 self._dbe_create_notification(oper_info)
  815 if oper_info['oper'] == 'UPDATE':
  816 self._dbe_update_notification(oper_info)
  817 elif oper_info['oper'] == 'DELETE':
  818 self._dbe_delete_notification(oper_info)
self = <vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>
self._dbe_update_notification = <bound method VncServerKombuClient._dbe_update_n...erver.vnc_cfg_ifmap.VncServerKombuClient object>>
oper_info = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in _dbe_update_notification(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>, obj_info={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  876 r_class = self._db_client_mgr.get_resource_class(obj_info['type'])
  877 if r_class:
  878 r_class.dbe_update_notification(obj_info)
  879 except:
  880 msg = "Failed to invoke type specific dbe_update_notification"
r_class = <class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>
r_class.dbe_update_notification = <bound method type.dbe_update_notification of <c..._api_server.vnc_cfg_types.VirtualNetworkServer'>>
obj_info = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py in dbe_update_notification(cls=<class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>, obj_ids={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  599 @classmethod
  600 def dbe_update_notification(cls, obj_ids):
  601 cls.addr_mgmt.net_update_notify(obj_ids)
  602 # end dbe_update_notification
  603
cls = <class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>
cls.addr_mgmt = <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>
cls.addr_mgmt.net_update_notify = <bound method AddrMgmt.net_update_notify of <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>>
obj_ids = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_...

I modiried JSON messages as above solution, however, other error messages were seen.

07/21/2015 05:53:47 PM [contrail-03:contrail-api:Config:0]: __default__ [SYS_ERR]: VncApiError: Failed to invoke type specific dbe_update_notification  
07/21/2015 05:53:47 PM [contrail-03:contrail-api:Config:0]: __default__ [SYS_ERR]: VncApiError: <type 'exceptions.AttributeError'>
Python 2.7.6: /usr/bin/python
Tue Jul 21 17:53:47 2015

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in _dbe_subscribe_callback(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>, oper_info={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  814                 self._dbe_create_notification(oper_info)
  815             if oper_info['oper'] == 'UPDATE':
  816                 self._dbe_update_notification(oper_info)
  817             elif oper_info['oper'] == 'DELETE':
  818                 self._dbe_delete_notification(oper_info)
self = <vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>
self._dbe_update_notification = <bound method VncServerKombuClient._dbe_update_n...erver.vnc_cfg_ifmap.VncServerKombuClient object>>
oper_info = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in _dbe_update_notification(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncServerKombuClient object>, obj_info={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  876             r_class = self._db_client_mgr.get_resource_class(obj_info['type'])
  877             if r_class:
  878                 r_class.dbe_update_notification(obj_info)
  879         except:
  880             msg = "Failed to invoke type specific dbe_update_notification"
r_class = <class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>
r_class.dbe_update_notification = <bound method type.dbe_update_notification of <c..._api_server.vnc_cfg_types.VirtualNetworkServer'>>
obj_info = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py in dbe_update_notification(cls=<class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>, obj_ids={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  599     @classmethod
  600     def dbe_update_notification(cls, obj_ids):
  601         cls.addr_mgmt.net_update_notify(obj_ids)
  602     # end dbe_update_notification
  603 
cls = <class 'vnc_cfg_api_server.vnc_cfg_types.VirtualNetworkServer'>
cls.addr_mgmt = <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>
cls.addr_mgmt.net_update_notify = <bound method AddrMgmt.net_update_notify of <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>>
obj_ids = {u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py in net_update_notify(self=<vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>, obj_ids={u'oper': u'UPDATE', u'type': u'virtual-network', u'uuid': u'00000000-9999-0000-0001-000000000001'})
  512         vn_dict = result
  513         vn_fq_name_str = ':'.join(vn_dict['fq_name'])
  514         self._create_subnet_objs(vn_fq_name_str, vn_dict)
  515     # end net_update_notify
  516 
self = <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>
self._create_subnet_objs = <bound method AddrMgmt._create_subnet_objs of <vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>>
vn_fq_name_str = u'default-domain:demo:testvn01'
vn_dict = {'display_name': u'testvn01', 'flood_unknown_unicast': True, 'fq_name': [u'default-domain', u'demo', u'testvn01'], 'id_perms': {u'created': u'2015-07-21T08:53:44.864508', u'creator': None, u'description': None, u'enable': True, u'last_modified': u'2015-07-21T08:53:46.405641', u'permissions': {u'group': u'admin', u'group_access': 7, u'other_access': 7, u'owner': u'admin', u'owner_access': 7}, u'user_visible': True, u'uuid': {u'uuid_lslong': 281474976710657, u'uuid_mslong': 2576941056}}, 'is_shared': False, 'network_ipam_refs': [{'attr': {u'ipam_subnets': [{u'addr_from_start': True, u'allocation_pools': [], u'default_gateway': u'30.1.1.253', u'dhcp_option_list': {...}, u'dns_server_address': u'30.1.1.2', u'enable_dhcp': True, u'host_routes': {...}, u'subnet': {...}, u'subnet_uuid': u'989f76ae-c86a-4c84-8d2f-84532fe9ad2d'}]}, 'href': 'http://0.0.0.0:9100/network-ipam/644962c8-b54f-497b-b3b1-7cb901fe59b6', 'to': [u'default-domain', u'default-project', u'default-network-ipam'], 'uuid': '644962c8-b54f-497b-b3b1-7cb901fe59b6'}], 'parent_href': u'http://0.0.0.0:9100/project/c05418b9-2f32-47f3-a085-a004dae65e49', 'parent_type': u'project', 'parent_uuid': 'c05418b9-2f32-47f3-a085-a004dae65e49', 'route_target_list': {u'route_target': [u'target:64512:30001']}, ...}

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py in _create_subnet_objs(self=<vnc_cfg_api_server.vnc_addr_mgmt.AddrMgmt object>, vn_fq_name_str=u'default-domain:demo:testvn01', vn_dict={'display_name': u'testvn01', 'flood_unknown_unicast': True, 'fq_name': [u'default-domain', u'demo', u'testvn01'], 'id_perms': {u'created': u'2015-07-21T08:53:44.864508', u'creator': None, u'description': None, u'enable': True, u'last_modified': u'2015-07-21T08:53:46.405641', u'permissions': {u'group': u'admin', u'group_access': 7, u'other_access': 7, u'owner': u'admin', u'owner_access': 7}, u'user_visible': True, u'uuid': {u'uuid_lslong': 281474976710657, u'uuid_mslong': 2576941056}}, 'is_shared': False, 'network_ipam_refs': [{'attr': {u'ipam_subnets': [{u'addr_from_start': True, u'allocation_pools': [], u'default_gateway': u'30.1.1.253', u'dhcp_option_list': {...}, u'dns_server_address': u'30.1.1.2', u'enable_dhcp': True, u'host_routes': {...}, u'subnet': {...}, u'subnet_uuid': u'989f76ae-c86a-4c84-8d2f-84532fe9ad2d'}]}, 'href': 'http://0.0.0.0:9100/network-ipam/644962c8-b54f-497b-b3b1-7cb901fe59b6', 'to': [u'default-domain', u'default-project', u'default-network-ipam'], 'uuid': '644962c8-b54f-497b-b3b1-7cb901fe59b6'}], 'parent_href': u'http://0.0.0.0:9100/project/c05418b9-2f32-47f3-a085-a004dae65e49', 'parent_type': u'project', 'parent_uuid': 'c05418b9-2f32-47f3-a085-a004dae65e49', 'route_target_list': {u'route_target': [u'target:64512:30001']}, ...})
  416                         dns_nameservers=nameservers,
  417                         alloc_pool_list=allocation_pools,
  418                         addr_from_start=addr_start)
  419                     self._subnet_objs[vn_fq_name_str][subnet_name] = \
  420                          subnet_obj
addr_from_start undefined
addr_start = True

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py in __init__(self=<vnc_cfg_api_server.vnc_addr_mgmt.Subnet object>, name=u'default-domain:demo:testvn01:30.1.1.0/24', prefix=u'30.1.1.0', prefix_len='24', gw=u'30.1.1.253', service_address=u'30.1.1.2', enable_dhcp=True, dns_nameservers=None, alloc_pool_list=[{'end': '30.1.1.254', 'start': '30.1.1.0'}], addr_from_start=True)
  248         # reserve excluded addresses
  249         for addr in exclude:
  250             self._db_conn.subnet_alloc_req(name, int(addr))
  251 
  252         self._name = name
self = <vnc_cfg_api_server.vnc_addr_mgmt.Subnet object>
self._db_conn = <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>
self._db_conn.subnet_alloc_req = <bound method VncDbClient.subnet_alloc_req of <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>>
name = u'default-domain:demo:testvn01:30.1.1.0/24'
builtinint = <type 'int'>
addr = IPAddress('30.1.1.253')

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in subnet_alloc_req(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>, subnet=u'default-domain:demo:testvn01:30.1.1.0/24', addr=503382525)
 1466 
 1467     def subnet_alloc_req(self, subnet, addr=None):
 1468         return self._zk_db.subnet_alloc_req(subnet, addr)
 1469     # end subnet_alloc_req
 1470 
self = <vnc_cfg_api_server.vnc_cfg_ifmap.VncDbClient object>
self._zk_db = <vnc_cfg_api_server.vnc_cfg_ifmap.VncZkClient object>
self._zk_db.subnet_alloc_req = <bound method VncZkClient.subnet_alloc_req of <vnc_cfg_api_server.vnc_cfg_ifmap.VncZkClient object>>
subnet = u'default-domain:demo:testvn01:30.1.1.0/24'
addr = 503382525

/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py in subnet_alloc_req(self=<vnc_cfg_api_server.vnc_cfg_ifmap.VncZkClient object>, subnet=u'default-domain:demo:testvn01:30.1.1.0/24', addr=503382525)
  991         try:
  992             if addr is not None:
  993                 if allocator.read(addr) is not None:
  994                     return addr
  995                 else:
allocator = None
allocator.read undefined
addr = 503382525
builtinNone = None
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'read'
    __class__ = <type 'exceptions.AttributeError'>
    __delattr__ = <method-wrapper '__delattr__' of exceptions.AttributeError object>
    __dict__ = {}
    __doc__ = 'Attribute not found.'
    __format__ = <built-in method __format__ of exceptions.AttributeError object>
    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.AttributeError object>
    __getitem__ = <method-wrapper '__getitem__' of exceptions.AttributeError object>
    __getslice__ = <method-wrapper '__getslice__' of exceptions.AttributeError object>
    __hash__ = <method-wrapper '__hash__' of exceptions.AttributeError object>
    __init__ = <method-wrapper '__init__' of exceptions.AttributeError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of exceptions.AttributeError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.AttributeError object>
    __repr__ = <method-wrapper '__repr__' of exceptions.AttributeError object>
    __setattr__ = <method-wrapper '__setattr__' of exceptions.AttributeError object>
    __setstate__ = <built-in method __setstate__ of exceptions.AttributeError object>
    __sizeof__ = <built-in method __sizeof__ of exceptions.AttributeError object>
    __str__ = <method-wrapper '__str__' of exceptions.AttributeError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __unicode__ = <built-in method __unicode__ of exceptions.AttributeError object>
    args = ("'NoneType' object has no attribute 'read'",)
    message = "'NoneType' object has no attribute 'read'"

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 816, in _dbe_subscribe_callback
    self._dbe_update_notification(oper_info)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 878, in _dbe_update_notification
    r_class.dbe_update_notification(obj_info)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py", line 601, in dbe_update_notification
    cls.addr_mgmt.net_update_notify(obj_ids)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 514, in net_update_notify
    self._create_subnet_objs(vn_fq_name_str, vn_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 418, in _create_subnet_objs
    addr_from_start=addr_start)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 250, in __init__
    self._db_conn.subnet_alloc_req(name, int(addr))
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1468, in subnet_alloc_req
    return self._zk_db.subnet_alloc_req(subnet, addr)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 993, in subnet_alloc_req
    if allocator.read(addr) is not None:
AttributeError: 'NoneType' object has no attribute 'read'

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

config_1801.log Edit (2.6 MiB, text/plain)

Revision history for this message

Tony Liu (taoliu-7) wrote on 2015-07-21:

Did you repeat sending the same request 1000 times or send 1000 different requests?
>> same request 1000 times

The same VN can not be created 1000 times.

If there is any scripts to reproduce this issue, please share it.

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

>>The same VN can not be created 1000 times.
Before sending same VN/VNI, I deleted them.

I sent following command repeatedly.

Revision history for this message

Daisuke Nakajima (dnakajima) wrote on 2015-07-21:

j=1
h=1
while [ $h -ne 0 ]
do
echo "cycle $j"
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-networks -b '{"virtual-network": {"virtual_network_properties": {"vxlan_network_identifier": 30001,"forwarding_mode": "l2_l3","allow_transit": false},"display_name": "testvn01","is_shared": false,"router_external": false,"parent_type": "project","uuid": "00000000-9999-0000-0001-000000000001","route_target_list": {"route_target": ["target:64512:30001"]},"flood_unknown_unicast": true,"fq_name": ["default-domain","demo","testvn01"],"routing_instances": [{"to": ["default-domain","demo","testvn01","testvn01"]}],"network_ipam_refs": [{"to": ["default-domain","default-project","default-network-ipam"],"attr": {"ipam_subnets": [{"subnet": {"ip_prefix": "30.1.1.0","ip_prefix_len": 24},"enable_dhcp": true,"default_gateway": "30.1.1.253","allocation_pools": [],"dhcp_option_list": {"dhcp_option": []},"host_routes": {"route": []},"addr_from_start": true}]}}],"name": "testvn01"}}' -o post
$i=`./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-machine-interfaces -b '{"virtual-machine-interface": {"display_name": "testvni01","name": "testvni01","virtual_machine_interface_device_owner": "","security_group_refs": [{"to": ["default-domain","demo","default"]}],"fq_name": ["default-domain","demo","testvni01"],"virtual_network_refs": [{"to": ["default-domain","demo","testvn01"]}], "uuid": "00000001-9999-0000-0001-000000000001","parent_type": "project","virtual_machine_interface_mac_addresses": {"mac_address": ["30:00:01:01:00:01"]}}}' -o post` > /dev/null 2>&1
$h=`echo $i | grep Internal | wc -l` > /dev/null 2>&1
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-machine-interface/00000001-9999-0000-0001-000000000001 -o delete
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-network/00000000-9999-0000-0001-000000000001 -o delete
j=`expr $j + 1`
done

Contrail_json.py is contrail REST/API helper.
https://github.com/nakadaisuke/Contrail_json/blob/master/Contrail_json.py

j=1
h=1
while [ $h -ne 0 ]
do
echo "cycle $j" 
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-networks -b '{"virtual-network": {"virtual_network_properties": {"vxlan_network_identifier": 30001,"forwarding_mode": "l2_l3","allow_transit": false},"display_name": "testvn01","is_shared": false,"router_external": false,"parent_type": "project","uuid": "00000000-9999-0000-0001-000000000001","route_target_list": {"route_target": ["target:64512:30001"]},"flood_unknown_unicast": true,"fq_name": ["default-domain","demo","testvn01"],"routing_instances": [{"to": ["default-domain","demo","testvn01","testvn01"]}],"network_ipam_refs": [{"to": ["default-domain","default-project","default-network-ipam"],"attr": {"ipam_subnets": [{"subnet": {"ip_prefix": "30.1.1.0","ip_prefix_len": 24},"enable_dhcp": true,"default_gateway": "30.1.1.253","allocation_pools": [],"dhcp_option_list": {"dhcp_option": []},"host_routes": {"route": []},"addr_from_start": true}]}}],"name": "testvn01"}}' -o post
$i=`./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-machine-interfaces -b '{"virtual-machine-interface": {"display_name": "testvni01","name": "testvni01","virtual_machine_interface_device_owner": "","security_group_refs": [{"to": ["default-domain","demo","default"]}],"fq_name": ["default-domain","demo","testvni01"],"virtual_network_refs": [{"to": ["default-domain","demo","testvn01"]}], "uuid": "00000001-9999-0000-0001-000000000001","parent_type": "project","virtual_machine_interface_mac_addresses": {"mac_address": ["30:00:01:01:00:01"]}}}' -o post` > /dev/null 2>&1
$h=`echo $i | grep Internal | wc -l` > /dev/null 2>&1
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-machine-interface/00000001-9999-0000-0001-000000000001 -o delete 
./Contrail_json.py -k 172.27.113.201 -c 172.27.113.209 -d virtual-network/00000000-9999-0000-0001-000000000001 -o delete
j=`expr $j + 1`
done

Contrail_json.py is contrail REST/API helper.
https://github.com/nakadaisuke/Contrail_json/blob/master/Contrail_json.py

Revision history for this message

Prakash Bailkeri (prakashmb) wrote on 2015-07-23:

Tried to recreate the error without success.
Can you please recreate the issue and provide the log files from all 3 cfgm and contrail-log output?

We trying to find the root cause for following excetion. Original reported issue (NoIdError) is resolved by fixing the json data sent to VMI create.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 816, in _dbe_subscribe_callback
    self._dbe_update_notification(oper_info)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 878, in _dbe_update_notification
    r_class.dbe_update_notification(obj_info)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_types.py", line 601, in dbe_update_notification
    cls.addr_mgmt.net_update_notify(obj_ids)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 514, in net_update_notify
    self._create_subnet_objs(vn_fq_name_str, vn_dict)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 418, in _create_subnet_objs
    addr_from_start=addr_start)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_addr_mgmt.py", line 250, in __init__
    self._db_conn.subnet_alloc_req(name, int(addr))
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 1468, in subnet_alloc_req
    return self._zk_db.subnet_alloc_req(subnet, addr)
  File "/usr/lib/python2.7/dist-packages/vnc_cfg_api_server/vnc_cfg_ifmap.py", line 993, in subnet_alloc_req
    if allocator.read(addr) is not None:
AttributeError: 'NoneType' object has no attribute 'read'

Revision history for this message

OpenContrail Admin (ci-admin-f) wrote on 2016-04-19: [Bug update]

#10

bug update...

Revision history for this message

Satish Venkatasubramanian (satish-venkatasubramanian-7) wrote on 2016-08-09:

#11

We too have encountered this bug in version 2.2. Can you let me know the work around to fix this issue.

Can we identify the related neutron port and delete it.

Please advice

Revision history for this message

Prakash Bailkeri (prakashmb) wrote on 2016-08-09:

#12

Can you specify what error you are hitting? The NoIdError in "bug description" was due to port create json data. It was specifying the routing instance ref wrongly.