07:07:18 <joehuang> #startmeeting multisite 07:07:18 <collabot`> Meeting started Thu Jun 16 07:07:18 2016 UTC. The chair is joehuang. Information about MeetBot at http://wiki.debian.org/MeetBot. 07:07:18 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic. 07:07:18 <collabot`> The meeting name has been set to 'multisite' 07:07:26 <SAshish> yeah. Its better 07:07:39 <sorantis> good 07:07:42 <SAshish> I had tennis matches continuous three days 07:07:47 <SAshish> so its effect 07:07:57 <joehuang> Good 07:08:07 <sorantis> i see 07:08:43 <sorantis> I’ve finished the Kingbird installation script that can be used in OPNFV CI 07:09:00 <sorantis> Having one issue though 07:09:00 <joehuang> That's cool! 07:09:10 <SAshish> okay. what is that? 07:09:12 <sorantis> not sure I’m populating the right conf parameters 07:09:29 <joehuang> what's the issue 07:09:35 <sorantis> the issue is related to starting kingbird-engine 07:09:50 <sorantis> can i paste here? 07:10:09 <sorantis> root@node-9:~# kingbird-engine --config-file /etc/kingbird/kingbird.conf 07:10:09 <sorantis> 2016-06-15 15:23:02.307 12577 INFO kingbird.engine.engine_config [-] Logging enabled! 07:10:11 <sorantis> 2016-06-15 15:23:02.307 12577 INFO kingbird.engine.engine_config [-] /usr/local/bin/kingbird-engine version 0.1.1.dev36 07:10:12 <sorantis> 2016-06-15 15:23:02.308 12577 INFO kingbird.engine [-] Server on http://kingbird.host with 2 07:10:14 <sorantis> 2016-06-15 15:23:02.309 12577 CRITICAL kingbird.engine [req-3fbcef4a-3dac-455a-9fdb-8e9f4ad05baf - - - - -] AttributeError: 'NoneType' object has no attribute 'rstrip' 07:10:15 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine Traceback (most recent call last): 07:10:17 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/bin/kingbird-engine", line 10, in <module> 07:10:18 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine sys.exit(main()) 07:10:20 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/cmd/engine.py", line 55, in main 07:10:21 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine engine_service = service.create_service() 07:10:23 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/service.py", line 60, in create_service 07:10:24 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine engine_manager = EngineManager() 07:10:26 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/listener.py", line 54, in __init__ 07:10:27 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine self.qm = QuotaManager() 07:10:29 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 64, in __init__ 07:10:30 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine self.endpoints = endpoint_cache.EndpointCache() 07:10:32 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 30, in __init__ 07:10:33 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine self._update_endpoints() 07:10:35 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 83, in _update_endpoints 07:10:36 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine endpoint_map = EndpointCache._get_endpoint_from_keystone(self) 07:10:38 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 48, in _get_endpoint_from_keystone 07:10:39 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine EndpointCache._get_admin_token(self)) 07:10:41 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 43, in _get_admin_token 07:10:42 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return sess.get_token() 07:10:44 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/debtcollector/removals.py", line 242, in wrapper 07:10:45 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return f(*args, **kwargs) 07:10:47 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 643, in get_token 07:10:48 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return (self.get_auth_headers(auth) or {}).get('X-Auth-Token') 07:10:50 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 617, in get_auth_headers 07:10:51 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return auth.get_headers(self, **kwargs) 07:10:53 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/base.py", line 142, in get_headers 07:10:54 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine token = self.get_token(session) 07:10:56 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 200, in get_token 07:10:57 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return self.get_access(session).auth_token 07:10:59 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 240, in get_access 07:11:00 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine self.auth_ref = self.get_auth_ref(session) 07:11:02 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 184, in get_auth_ref 07:11:03 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine token_url = self.token_url 07:11:05 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 84, in token_url 07:11:06 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine return '%s/auth/tokens' % self.auth_url.rstrip('/') 07:11:08 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine AttributeError: 'NoneType' object has no attribute 'rstrip' 07:11:09 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine 07:11:11 <sorantis> ok, apparently yes 07:11:12 <sorantis> so, here 07:11:14 <SAshish> have you polulated cache and keystone_authtoken sections properly 07:11:21 <sorantis> and? 07:11:22 <SAshish> in conf file 07:11:29 <sorantis> I thought only cache is sufficient 07:11:33 <SAshish> no 07:11:36 <joehuang> both 07:11:43 <SAshish> it uses keystone middleware 07:11:47 <sorantis> speaking of which, why do we need both? 07:11:47 <SAshish> so both 07:11:50 <joehuang> correct 07:12:12 <sorantis> why do we need cache if this information is already present in keystone_authtoken 07:12:38 <sorantis> I’ve raised a bug https://bugs.launchpad.net/kingbird/+bug/1592762 07:13:02 <SAshish> sorry, the engine does not need keystone_authtoken section 07:13:07 <SAshish> its for APIs 07:13:26 <SAshish> kb-api needs keystone_authtoken 07:13:30 <joehuang> only one kingbird.conf 07:13:37 <SAshish> engine only needs cache 07:13:41 <SAshish> section 07:13:48 <sorantis> I’m confused now 07:14:02 <sorantis> can’t it reuse the keystone_authtoken section? 07:14:14 <sorantis> is the information written is both different? 07:16:33 <SAshish> endpoint cache module uses cache section, Now I am also getting doubt If separate cache section is actually required 07:16:41 <sorantis> here’s what I have 07:16:44 <sorantis> [cache] 07:16:45 <sorantis> auth_url = http://192.168.0.8:5000/v3 07:16:46 <sorantis> admin_password = keystone_kb_pass 07:16:47 <sorantis> admin_user = kingbird 07:16:48 <sorantis> admin_tenant_name = services 07:17:14 <joehuang> it's different, keystone_authtoken is for token validation, if you want to access other region's endpoint information 07:17:17 <SAshish> I am quite sure, they error is with few configurations 07:17:28 <joehuang> need account to access info 07:17:34 <SAshish> [cache] # # From kingbird.common.config # # Keystone authorization url (string value) auth_url = http://172.16.73.113:35357/v3 # Keystone service url (string value) identity_url = http://172.16.73.113:35357/v3 # Username of admin account, needed when auto_refresh_endpoint set to True # (string value) admin_username = admin # Password of admin account, needed when auto_refresh_endpoint set to True # (string value) admin_pas 07:17:42 <SAshish> auth_url = http://172.16.73.113:35357/v3 07:17:44 <joehuang> just look at nova.conf 07:17:46 <SAshish> identity_url = http://172.16.73.113:35357/v3 07:17:49 <SAshish> admin_username = admin 07:17:51 <SAshish> admin_password = openstack 07:17:54 <SAshish> admin_tenant = admin 07:17:57 <SAshish> admin_user_domain_name = Default 07:17:59 <SAshish> admin_tenant_domain_name = Default 07:18:27 <SAshish> you have domain information in cache section? 07:18:36 <SAshish> @Dimitri 07:18:36 <collabot`> SAshish: Error: "Dimitri" is not a valid command. 07:19:08 <sorantis> no, why, the default value is ‘Default’ 07:19:15 <sorantis> check config.py 07:20:00 <joehuang> keystone_authtoken is for token validation 07:20:18 <SAshish> identity _url ? 07:20:20 <joehuang> cache is the user to access other regions 07:21:27 <SAshish> identity_url?? 07:21:34 <sorantis> but isn’t it the same user? 07:21:44 <sorantis> I mean we have one authentication endpoint 07:21:52 <sorantis> ah 07:21:53 <sorantis> ok 07:22:10 <sorantis> but still doesn’t make sense 07:22:31 <sorantis> we can always get info about other regions (endpoint) 07:22:48 <sorantis> ok, let me try with updated config 07:23:53 <sorantis> hot a different error 07:24:05 <joehuang> this is different purpose, and may be configured with different user and role, I don't think it's good idea to merge the config 07:24:28 <joehuang> althoug in test, often use same user/role 07:24:44 <SAshish> what is the error now? 07:24:48 <sorantis> root@node-9:~# kingbird-engine --config-file /etc/kingbird/kingbird.conf 07:24:49 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine.engine_config [-] Logging enabled! 07:24:51 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine.engine_config [-] /usr/local/bin/kingbird-engine version 0.1.1.dev37 07:24:52 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine [-] Server on http://kingbird.host with 2 07:24:54 <sorantis> 2016-06-16 07:23:30.147 16649 CRITICAL kingbird.engine [req-9677b61c-7062-429f-ae6d-00a40c97b1b9 - - - - -] BadRequest: Expecting to find id or name in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-3cd7c7d6-d38e-4750-8987-f11306303a58) 07:24:55 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine Traceback (most recent call last): 07:24:57 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/bin/kingbird-engine", line 10, in <module> 07:24:58 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine sys.exit(main()) 07:25:00 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/cmd/engine.py", line 55, in main 07:25:01 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine engine_service = service.create_service() 07:25:03 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/service.py", line 60, in create_service 07:25:04 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine engine_manager = EngineManager() 07:25:06 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/listener.py", line 54, in __init__ 07:25:07 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine self.qm = QuotaManager() 07:25:09 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 64, in __init__ 07:25:10 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine self.endpoints = endpoint_cache.EndpointCache() 07:25:12 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 30, in __init__ 07:25:13 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine self._update_endpoints() 07:25:15 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 83, in _update_endpoints 07:25:16 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine endpoint_map = EndpointCache._get_endpoint_from_keystone(self) 07:25:18 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 48, in _get_endpoint_from_keystone 07:25:19 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine EndpointCache._get_admin_token(self)) 07:25:21 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 43, in _get_admin_token 07:25:22 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return sess.get_token() 07:25:24 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/local/lib/python2.7/dist-packages/debtcollector/removals.py", line 242, in wrapper 07:25:25 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return f(*args, **kwargs) 07:25:27 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 643, in get_token 07:25:28 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return (self.get_auth_headers(auth) or {}).get('X-Auth-Token') 07:25:30 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 617, in get_auth_headers 07:25:31 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return auth.get_headers(self, **kwargs) 07:25:33 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/base.py", line 142, in get_headers 07:25:34 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine token = self.get_token(session) 07:25:36 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 200, in get_token 07:25:37 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return self.get_access(session).auth_token 07:25:39 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 240, in get_access 07:25:40 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine self.auth_ref = self.get_auth_ref(session) 07:25:42 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 190, in get_auth_ref 07:25:43 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine authenticated=False, log=False, **rkwargs) 07:25:45 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 501, in post 07:25:46 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return self.request(url, 'POST', **kwargs) 07:25:48 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner 07:25:49 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine return func(*args, **kwargs) 07:25:51 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 401, in request 07:25:52 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine raise exceptions.from_response(resp, method, url) 07:25:54 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine BadRequest: Expecting to find id or name in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-3cd7c7d6-d38e-4750-8987-f11306303a58) 07:25:55 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine 07:25:58 <sorantis> the kingbird-api started wthout a problem 07:26:56 <sorantis> however.. 07:27:08 <sorantis> let me check one thing 07:27:54 <sorantis> none of the other users are associated with a project 07:28:01 <sorantis> root@node-9:~# openstack user list --long 07:28:02 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+ 07:28:03 <sorantis> | ID | Name | Project | Email | Enabled | 07:28:05 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+ 07:28:06 <sorantis> | 0fcd67f7c6bb4e2a8bcf9b2dd0fe65a0 | heat-cfn | | heat-cfn@localhost | True | 07:28:08 <sorantis> | 20ec521e945347398566b63c75f958a5 | swift | | swift@localhost | True | 07:28:10 <sorantis> | 227663fd419342f99a8419d91460b5b8 | admin | | admin@localhost | True | 07:28:11 <sorantis> | 29896d5a8ac840599a884f5679b25e76 | neutron | | neutron@localhost | True | 07:28:12 <sorantis> | 70324433be434edba5043c1e0e7de97e | heat | | heat@localhost | True | 07:28:14 <sorantis> | 7263d5aec4ed463f94a483c0d4ca1e71 | glance | | glance@localhost | True | 07:28:15 <sorantis> | 90a58db6bbbb40f5944cce6297dbb890 | cinder | | cinder@localhost | True | 07:28:17 <sorantis> | c9a6400e89854e93ac38e059583bd6f0 | kingbird | services | None | True | 07:28:18 <sorantis> | d9528f33771c4720aa161ca4f8fe69f8 | nova | | nova@localhost | True | 07:28:19 <sorantis> | e3cd9e251aac4239a6fcca51169b47cd | fuel_stats_user | | | True | 07:28:20 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+ 07:28:23 <sorantis> which is strange, because there is a project named services 07:29:53 <sorantis> [cache] 07:29:53 <sorantis> identity_url = http://192.168.0.8:5000/v3 07:29:55 <sorantis> auth_url = http://192.168.0.8:5000/v3 07:29:56 <sorantis> admin_password = keystone_kb_pass 07:29:57 <sorantis> admin_user = kingbird 07:29:58 <sorantis> admin_tenant_name = services 07:31:17 <joehuang> cache is for access other regions's info, so use user 'admin', tenant_name = 'admin' 07:31:53 <sorantis> i have created a user with admin role 07:32:17 <joehuang> could you run "openstack project list" 07:32:33 <sorantis> root@node-9:~# openstack project list 07:32:33 <sorantis> +----------------------------------+----------+ 07:32:35 <sorantis> | ID | Name | 07:32:36 <sorantis> +----------------------------------+----------+ 07:32:38 <sorantis> | 1af277a011034cca8ae0eda41ae8ccfc | admin | 07:32:39 <sorantis> | 875f8a213e24427c8af2d3ab5199001e | services | 07:32:40 <sorantis> +----------------------------------+----------+ 07:33:55 <SAshish> cfg.StrOpt('admin_tenant', help='Tenant name of admin account, needed when' ' auto_refresh_endpoint set to True'), 07:34:01 <SAshish> it is admin_tenant 07:34:12 <SAshish> you have admin_tenant_name = services 07:34:29 <SAshish> can you please check that once 07:34:39 <SAshish> admin_tenant_name = services => admin_tenant = services 07:34:41 <sorantis> crap, you’re right 07:36:40 <sorantis> nope 07:36:44 <sorantis> didn’t help 07:38:01 <sorantis> found it 07:38:05 <sorantis> another bug 07:40:04 <sorantis> ok 07:40:05 <sorantis> it’s up 07:40:11 <joehuang> KB-api/engine run in Shanghai lab correctly, will check and paste the conf 07:40:14 <joehuang> good 07:40:26 <sorantis> I misconfigured rabbitmq though 07:40:54 <SAshish> Joe, is rebalancing happening there? 07:40:59 <joehuang> db/mq are common configure items 07:41:03 <SAshish> or did you try running any API there? 07:41:14 <joehuang> not yet, some test cases failed 07:41:20 <SAshish> not testcase 07:41:28 <SAshish> just a manual run of any API 07:41:41 <joehuang> only two test cases run ok 07:41:59 <SAshish> please, try with latest code once 07:42:00 <joehuang> could you share the scripts to run manually 07:42:10 <SAshish> yeah, will mail 07:42:27 <SAshish> it just the curl commands 07:42:33 <sorantis> so i made it quite simple 07:42:34 <sorantis> [oslo_messaging_rabbit] 07:42:35 <sorantis> rabbit_password = guest 07:42:36 <sorantis> rabbit_userid = guest 07:42:43 <sorantis> of course it didn’t work 07:42:48 <SAshish> but, Joe one more question 07:43:08 <SAshish> you said engine started, so periodic sync happening? 07:45:20 <joehuang> will check, but the vpn to lab is not working today 07:45:38 <SAshish> okay. you can see that on console itself. 07:45:47 <SAshish> when u run engine 07:45:51 <joehuang> I remember saw the print info in the console 07:46:50 <SAshish> okay, If you get access, please mail me the logs on console after starting engine 07:47:33 <SAshish> Dimitri, Engine up?? 07:48:25 <joehuang> test I pasted last time : http://paste.openstack.org/show/508827/ 07:48:56 <SAshish> yeah. this is tempest logs 07:49:14 <SAshish> just engine logs 07:50:23 <joehuang> will paste the engine log 07:51:00 <sorantis> engine up 07:51:23 <SAshish> okay. can you see logs on console 07:51:50 <sorantis> yes 07:51:53 <sorantis> some errors 07:51:59 <SAshish> okay 07:52:37 <sorantis> 2016-06-16 07:50:36.732 8579 INFO kingbird.engine.quota_manager [-] Quota sync Called for Project: 1af277a011034cca8ae0eda41ae8ccfc 07:52:38 <sorantis> 2016-06-16 07:50:36.732 8579 INFO kingbird.drivers.openstack.sdk [-] Using cached OS client objects 07:52:39 <sorantis> 2016-06-16 07:50:36.911 8579 INFO kingbird.drivers.openstack.sdk [-] Using cached OS client objects 07:52:41 <sorantis> 2016-06-16 07:50:37.036 8579 INFO kingbird.engine.quota_manager [-] Reading quota usage for 1af277a011034cca8ae0eda41ae8ccfc in RegionOne 07:52:41 <joehuang> have you configured the db part? 07:52:42 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.engine.quota_manager [-] Reading quota usage for 1af277a011034cca8ae0eda41ae8ccfc in RegionTwo 07:52:44 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.drivers.openstack.sdk [-] Creating fresh OS Clients objects 07:52:45 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.drivers.openstack.sdk [-] Creating fresh OS Clients objects 07:52:47 <sorantis> Exception in thread Thread-4: 07:52:48 <sorantis> Traceback (most recent call last): 07:52:50 <sorantis> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner 07:52:51 <sorantis> self.run() 07:52:53 <sorantis> File "/usr/lib/python2.7/threading.py", line 763, in run 07:52:54 <sorantis> self.__target(*self.__args, **self.__kwargs) 07:52:56 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 109, in read_quota_usage 07:52:57 <sorantis> os_client = sdk.OpenStackDriver(region) 07:52:59 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/drivers/openstack/sdk.py", line 64, in __init__ 07:53:00 <sorantis> self.keystone_client.session) 07:53:02 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/drivers/openstack/neutron_v2.py", line 33, in __init__ 07:53:03 <sorantis> self.extension_list = self.neutron.list_extensions() 07:53:05 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 608, in list_extensions 07:53:06 <sorantis> return self.get(self.extensions_path, params=_params) 07:53:08 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 342, in get 07:53:09 <sorantis> headers=headers, params=params) 07:53:11 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 319, in retry_request 07:53:12 <sorantis> headers=headers, params=params) 07:53:14 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 282, in do_request 07:53:15 <sorantis> self._handle_fault_response(status_code, replybody, resp) 07:53:17 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 257, in _handle_fault_response 07:53:19 <sorantis> exception_handler_v20(status_code, error_body) 07:53:21 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 84, in exception_handler_v20 07:53:22 <sorantis> request_ids=request_ids) 07:53:23 <sorantis> Unauthorized: Authentication required 07:53:55 <sorantis> but that’s probably because of another thing 07:53:57 <sorantis> ok 07:54:04 <sorantis> so far so good 07:54:13 <sorantis> at least it’s up and failing ) 07:54:20 <joehuang> :) 07:54:21 <sorantis> I’ll reconfigure regions 07:54:29 <SAshish> it reuses endpoint cache module 07:54:46 <sorantis> yesterday I messed up one of the controllers, so had to reinstall whole setup 07:55:04 <sorantis> I didn’t configure services in region 2 to use keystone from region 1 07:55:10 <sorantis> this is most probably the issue 07:55:18 <sorantis> I’ll do it today 07:55:20 <joehuang> should use 07:55:32 <sorantis> yes, I know 07:55:40 <sorantis> I just didn’t have the time to do it 07:55:52 <joehuang> and especially check that in keystone configuration the url return should not be 127.0.0.1 07:56:01 <joehuang> otherwise will fail 07:56:08 <sorantis> what do you mean? 07:56:22 <sorantis> I don’t even use keystone in region 2 07:56:36 <joehuang> In region one Keystone 07:56:53 <joehuang> you need to check the configuration in keystone.confg 07:57:38 <joehuang> there is one default url returned, it should not be 127.0.0.1, otherwise the service in region two will access 127.0.0.1, it's local host will fail 07:57:53 <sorantis> it isn't 07:58:15 <joehuang> good if it isn't 07:58:57 <joehuang> be careful not to use 127.0.0.1 like address, use real IP instead 07:59:23 <sorantis> all services use real IPs 07:59:38 <joehuang> that's great helpful 07:59:53 <joehuang> any other topic 08:00:16 <SAshish> I have to work a little on neutron and cinder tempests 08:00:30 <SAshish> will push it soon 08:00:42 <joehuang> good 08:00:57 <SAshish> then will verify tempests on LAB setups 08:01:12 <sorantis> ok 08:01:17 <sorantis> thanks for help guys 08:01:20 <joehuang> so see you next week 08:01:20 <sorantis> talk soon 08:01:23 <sorantis> bye 08:01:26 <joehuang> bye 08:01:27 <SAshish> you are welcome :) 08:01:30 <SAshish> bye 08:01:34 <joehuang> #endmeeting