07:07:18 <joehuang> #startmeeting multisite
07:07:18 <collabot`> Meeting started Thu Jun 16 07:07:18 2016 UTC.  The chair is joehuang. Information about MeetBot at http://wiki.debian.org/MeetBot.
07:07:18 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
07:07:18 <collabot`> The meeting name has been set to 'multisite'
07:07:26 <SAshish> yeah. Its better
07:07:39 <sorantis> good
07:07:42 <SAshish> I had tennis matches continuous three days
07:07:47 <SAshish> so its effect
07:07:57 <joehuang> Good
07:08:07 <sorantis> i see
07:08:43 <sorantis> I’ve finished the Kingbird installation script that can be used in OPNFV CI
07:09:00 <sorantis> Having one issue though
07:09:00 <joehuang> That's cool!
07:09:10 <SAshish> okay. what is that?
07:09:12 <sorantis> not sure I’m populating the right conf parameters
07:09:29 <joehuang> what's the issue
07:09:35 <sorantis> the issue is related to starting kingbird-engine
07:09:50 <sorantis> can i paste here?
07:10:09 <sorantis> root@node-9:~# kingbird-engine --config-file /etc/kingbird/kingbird.conf
07:10:09 <sorantis> 2016-06-15 15:23:02.307 12577 INFO kingbird.engine.engine_config [-] Logging enabled!
07:10:11 <sorantis> 2016-06-15 15:23:02.307 12577 INFO kingbird.engine.engine_config [-] /usr/local/bin/kingbird-engine version 0.1.1.dev36
07:10:12 <sorantis> 2016-06-15 15:23:02.308 12577 INFO kingbird.engine [-] Server on http://kingbird.host with 2
07:10:14 <sorantis> 2016-06-15 15:23:02.309 12577 CRITICAL kingbird.engine [req-3fbcef4a-3dac-455a-9fdb-8e9f4ad05baf - - - - -] AttributeError: 'NoneType' object has no attribute 'rstrip'
07:10:15 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine Traceback (most recent call last):
07:10:17 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/bin/kingbird-engine", line 10, in <module>
07:10:18 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     sys.exit(main())
07:10:20 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/cmd/engine.py", line 55, in main
07:10:21 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     engine_service = service.create_service()
07:10:23 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/service.py", line 60, in create_service
07:10:24 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     engine_manager = EngineManager()
07:10:26 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/listener.py", line 54, in __init__
07:10:27 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     self.qm = QuotaManager()
07:10:29 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 64, in __init__
07:10:30 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     self.endpoints = endpoint_cache.EndpointCache()
07:10:32 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 30, in __init__
07:10:33 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     self._update_endpoints()
07:10:35 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 83, in _update_endpoints
07:10:36 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     endpoint_map = EndpointCache._get_endpoint_from_keystone(self)
07:10:38 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 48, in _get_endpoint_from_keystone
07:10:39 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     EndpointCache._get_admin_token(self))
07:10:41 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 43, in _get_admin_token
07:10:42 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return sess.get_token()
07:10:44 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/debtcollector/removals.py", line 242, in wrapper
07:10:45 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return f(*args, **kwargs)
07:10:47 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 643, in get_token
07:10:48 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return (self.get_auth_headers(auth) or {}).get('X-Auth-Token')
07:10:50 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 617, in get_auth_headers
07:10:51 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return auth.get_headers(self, **kwargs)
07:10:53 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/base.py", line 142, in get_headers
07:10:54 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     token = self.get_token(session)
07:10:56 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 200, in get_token
07:10:57 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return self.get_access(session).auth_token
07:10:59 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 240, in get_access
07:11:00 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     self.auth_ref = self.get_auth_ref(session)
07:11:02 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 184, in get_auth_ref
07:11:03 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     token_url = self.token_url
07:11:05 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 84, in token_url
07:11:06 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine     return '%s/auth/tokens' % self.auth_url.rstrip('/')
07:11:08 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine AttributeError: 'NoneType' object has no attribute 'rstrip'
07:11:09 <sorantis> 2016-06-15 15:23:02.309 12577 ERROR kingbird.engine
07:11:11 <sorantis> ok, apparently yes
07:11:12 <sorantis> so, here
07:11:14 <SAshish> have you polulated cache and keystone_authtoken sections properly
07:11:21 <sorantis> and?
07:11:22 <SAshish> in conf file
07:11:29 <sorantis> I thought only cache is sufficient
07:11:33 <SAshish> no
07:11:36 <joehuang> both
07:11:43 <SAshish> it uses keystone middleware
07:11:47 <sorantis> speaking of which, why do we need both?
07:11:47 <SAshish> so both
07:11:50 <joehuang> correct
07:12:12 <sorantis> why do we need cache if this information is already present in keystone_authtoken
07:12:38 <sorantis> I’ve raised a bug https://bugs.launchpad.net/kingbird/+bug/1592762
07:13:02 <SAshish> sorry, the engine does not need keystone_authtoken  section
07:13:07 <SAshish> its for APIs
07:13:26 <SAshish> kb-api needs keystone_authtoken
07:13:30 <joehuang> only one kingbird.conf
07:13:37 <SAshish> engine only needs cache
07:13:41 <SAshish> section
07:13:48 <sorantis> I’m confused now
07:14:02 <sorantis> can’t it reuse the keystone_authtoken section?
07:14:14 <sorantis> is the information written is both different?
07:16:33 <SAshish> endpoint cache module uses cache section, Now I am also getting doubt If separate cache section is actually required
07:16:41 <sorantis> here’s what I have
07:16:44 <sorantis> [cache]
07:16:45 <sorantis> auth_url = http://192.168.0.8:5000/v3
07:16:46 <sorantis> admin_password = keystone_kb_pass
07:16:47 <sorantis> admin_user = kingbird
07:16:48 <sorantis> admin_tenant_name = services
07:17:14 <joehuang> it's different, keystone_authtoken is for token validation, if you want to access other region's endpoint information
07:17:17 <SAshish> I am quite sure, they error is with few configurations
07:17:28 <joehuang> need account to access info
07:17:34 <SAshish> [cache]  # # From kingbird.common.config #  # Keystone authorization url (string value) auth_url = http://172.16.73.113:35357/v3  # Keystone service url (string value) identity_url = http://172.16.73.113:35357/v3  # Username of admin account, needed when auto_refresh_endpoint set to True # (string value) admin_username = admin  # Password of admin account, needed when auto_refresh_endpoint set to True # (string value) admin_pas
07:17:42 <SAshish> auth_url = http://172.16.73.113:35357/v3
07:17:44 <joehuang> just look at nova.conf
07:17:46 <SAshish> identity_url = http://172.16.73.113:35357/v3
07:17:49 <SAshish> admin_username = admin
07:17:51 <SAshish> admin_password = openstack
07:17:54 <SAshish> admin_tenant = admin
07:17:57 <SAshish> admin_user_domain_name = Default
07:17:59 <SAshish> admin_tenant_domain_name = Default
07:18:27 <SAshish> you have domain information in cache section?
07:18:36 <SAshish> @Dimitri
07:18:36 <collabot`> SAshish: Error: "Dimitri" is not a valid command.
07:19:08 <sorantis> no, why, the default value is ‘Default’
07:19:15 <sorantis> check config.py
07:20:00 <joehuang> keystone_authtoken is for token validation
07:20:18 <SAshish> identity _url ?
07:20:20 <joehuang> cache is the user to access other regions
07:21:27 <SAshish> identity_url??
07:21:34 <sorantis> but isn’t it the same user?
07:21:44 <sorantis> I mean we have one authentication endpoint
07:21:52 <sorantis> ah
07:21:53 <sorantis> ok
07:22:10 <sorantis> but still doesn’t make sense
07:22:31 <sorantis> we can always get info about other regions (endpoint)
07:22:48 <sorantis> ok, let me try with updated config
07:23:53 <sorantis> hot a different error
07:24:05 <joehuang> this is different purpose, and may be configured with different user and role, I don't think it's good idea to merge the config
07:24:28 <joehuang> althoug in test, often use same user/role
07:24:44 <SAshish> what is the error now?
07:24:48 <sorantis> root@node-9:~# kingbird-engine --config-file /etc/kingbird/kingbird.conf
07:24:49 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine.engine_config [-] Logging enabled!
07:24:51 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine.engine_config [-] /usr/local/bin/kingbird-engine version 0.1.1.dev37
07:24:52 <sorantis> 2016-06-16 07:23:30.131 16649 INFO kingbird.engine [-] Server on http://kingbird.host with 2
07:24:54 <sorantis> 2016-06-16 07:23:30.147 16649 CRITICAL kingbird.engine [req-9677b61c-7062-429f-ae6d-00a40c97b1b9 - - - - -] BadRequest: Expecting to find id or name in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-3cd7c7d6-d38e-4750-8987-f11306303a58)
07:24:55 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine Traceback (most recent call last):
07:24:57 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/bin/kingbird-engine", line 10, in <module>
07:24:58 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     sys.exit(main())
07:25:00 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/cmd/engine.py", line 55, in main
07:25:01 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     engine_service = service.create_service()
07:25:03 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/service.py", line 60, in create_service
07:25:04 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     engine_manager = EngineManager()
07:25:06 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/listener.py", line 54, in __init__
07:25:07 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     self.qm = QuotaManager()
07:25:09 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 64, in __init__
07:25:10 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     self.endpoints = endpoint_cache.EndpointCache()
07:25:12 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 30, in __init__
07:25:13 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     self._update_endpoints()
07:25:15 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 83, in _update_endpoints
07:25:16 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     endpoint_map = EndpointCache._get_endpoint_from_keystone(self)
07:25:18 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 48, in _get_endpoint_from_keystone
07:25:19 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     EndpointCache._get_admin_token(self))
07:25:21 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/kingbird/common/endpoint_cache.py", line 43, in _get_admin_token
07:25:22 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return sess.get_token()
07:25:24 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/local/lib/python2.7/dist-packages/debtcollector/removals.py", line 242, in wrapper
07:25:25 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return f(*args, **kwargs)
07:25:27 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 643, in get_token
07:25:28 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return (self.get_auth_headers(auth) or {}).get('X-Auth-Token')
07:25:30 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 617, in get_auth_headers
07:25:31 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return auth.get_headers(self, **kwargs)
07:25:33 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/base.py", line 142, in get_headers
07:25:34 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     token = self.get_token(session)
07:25:36 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 200, in get_token
07:25:37 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return self.get_access(session).auth_token
07:25:39 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 240, in get_access
07:25:40 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     self.auth_ref = self.get_auth_ref(session)
07:25:42 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 190, in get_auth_ref
07:25:43 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     authenticated=False, log=False, **rkwargs)
07:25:45 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 501, in post
07:25:46 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return self.request(url, 'POST', **kwargs)
07:25:48 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
07:25:49 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     return func(*args, **kwargs)
07:25:51 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine   File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 401, in request
07:25:52 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine     raise exceptions.from_response(resp, method, url)
07:25:54 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine BadRequest: Expecting to find id or name in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-3cd7c7d6-d38e-4750-8987-f11306303a58)
07:25:55 <sorantis> 2016-06-16 07:23:30.147 16649 ERROR kingbird.engine
07:25:58 <sorantis> the kingbird-api started wthout a problem
07:26:56 <sorantis> however..
07:27:08 <sorantis> let me check one thing
07:27:54 <sorantis> none of the other users are associated with a project
07:28:01 <sorantis> root@node-9:~# openstack user list --long
07:28:02 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+
07:28:03 <sorantis> | ID                               | Name            | Project  | Email              | Enabled |
07:28:05 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+
07:28:06 <sorantis> | 0fcd67f7c6bb4e2a8bcf9b2dd0fe65a0 | heat-cfn        |          | heat-cfn@localhost | True    |
07:28:08 <sorantis> | 20ec521e945347398566b63c75f958a5 | swift           |          | swift@localhost    | True    |
07:28:10 <sorantis> | 227663fd419342f99a8419d91460b5b8 | admin           |          | admin@localhost    | True    |
07:28:11 <sorantis> | 29896d5a8ac840599a884f5679b25e76 | neutron         |          | neutron@localhost  | True    |
07:28:12 <sorantis> | 70324433be434edba5043c1e0e7de97e | heat            |          | heat@localhost     | True    |
07:28:14 <sorantis> | 7263d5aec4ed463f94a483c0d4ca1e71 | glance          |          | glance@localhost   | True    |
07:28:15 <sorantis> | 90a58db6bbbb40f5944cce6297dbb890 | cinder          |          | cinder@localhost   | True    |
07:28:17 <sorantis> | c9a6400e89854e93ac38e059583bd6f0 | kingbird        | services | None               | True    |
07:28:18 <sorantis> | d9528f33771c4720aa161ca4f8fe69f8 | nova            |          | nova@localhost     | True    |
07:28:19 <sorantis> | e3cd9e251aac4239a6fcca51169b47cd | fuel_stats_user |          |                    | True    |
07:28:20 <sorantis> +----------------------------------+-----------------+----------+--------------------+---------+
07:28:23 <sorantis> which is strange, because there is a project named services
07:29:53 <sorantis> [cache]
07:29:53 <sorantis> identity_url = http://192.168.0.8:5000/v3
07:29:55 <sorantis> auth_url = http://192.168.0.8:5000/v3
07:29:56 <sorantis> admin_password = keystone_kb_pass
07:29:57 <sorantis> admin_user = kingbird
07:29:58 <sorantis> admin_tenant_name = services
07:31:17 <joehuang> cache is for access other regions's info, so use user 'admin', tenant_name = 'admin'
07:31:53 <sorantis> i have created a user with admin role
07:32:17 <joehuang> could you run "openstack project list"
07:32:33 <sorantis> root@node-9:~# openstack project list
07:32:33 <sorantis> +----------------------------------+----------+
07:32:35 <sorantis> | ID                               | Name     |
07:32:36 <sorantis> +----------------------------------+----------+
07:32:38 <sorantis> | 1af277a011034cca8ae0eda41ae8ccfc | admin    |
07:32:39 <sorantis> | 875f8a213e24427c8af2d3ab5199001e | services |
07:32:40 <sorantis> +----------------------------------+----------+
07:33:55 <SAshish> cfg.StrOpt('admin_tenant',                help='Tenant name of admin account, needed when'                     ' auto_refresh_endpoint set to True'),
07:34:01 <SAshish> it is admin_tenant
07:34:12 <SAshish> you have admin_tenant_name = services
07:34:29 <SAshish> can you please check that once
07:34:39 <SAshish> admin_tenant_name = services => admin_tenant = services
07:34:41 <sorantis> crap, you’re right
07:36:40 <sorantis> nope
07:36:44 <sorantis> didn’t help
07:38:01 <sorantis> found it
07:38:05 <sorantis> another bug
07:40:04 <sorantis> ok
07:40:05 <sorantis> it’s up
07:40:11 <joehuang> KB-api/engine run in Shanghai lab correctly, will check and paste the conf
07:40:14 <joehuang> good
07:40:26 <sorantis> I misconfigured rabbitmq though
07:40:54 <SAshish> Joe, is rebalancing happening there?
07:40:59 <joehuang> db/mq are common configure items
07:41:03 <SAshish> or did you try running any API there?
07:41:14 <joehuang> not yet, some test cases failed
07:41:20 <SAshish> not testcase
07:41:28 <SAshish> just a manual run of any API
07:41:41 <joehuang> only two test cases run ok
07:41:59 <SAshish> please, try with latest code once
07:42:00 <joehuang> could you share the scripts to run manually
07:42:10 <SAshish> yeah, will mail
07:42:27 <SAshish> it just the curl commands
07:42:33 <sorantis> so i made it quite simple
07:42:34 <sorantis> [oslo_messaging_rabbit]
07:42:35 <sorantis> rabbit_password = guest
07:42:36 <sorantis> rabbit_userid = guest
07:42:43 <sorantis> of course it didn’t work
07:42:48 <SAshish> but, Joe one more question
07:43:08 <SAshish> you said engine started, so periodic sync happening?
07:45:20 <joehuang> will check, but the vpn to lab is not working today
07:45:38 <SAshish> okay. you can see that on console itself.
07:45:47 <SAshish> when u run engine
07:45:51 <joehuang> I remember saw the print info in the console
07:46:50 <SAshish> okay, If you get access, please mail me the logs on console after starting engine
07:47:33 <SAshish> Dimitri, Engine up??
07:48:25 <joehuang> test I pasted last time : http://paste.openstack.org/show/508827/
07:48:56 <SAshish> yeah. this is tempest logs
07:49:14 <SAshish> just engine logs
07:50:23 <joehuang> will paste the engine log
07:51:00 <sorantis> engine up
07:51:23 <SAshish> okay. can you see logs on console
07:51:50 <sorantis> yes
07:51:53 <sorantis> some errors
07:51:59 <SAshish> okay
07:52:37 <sorantis> 2016-06-16 07:50:36.732 8579 INFO kingbird.engine.quota_manager [-] Quota sync Called for Project: 1af277a011034cca8ae0eda41ae8ccfc
07:52:38 <sorantis> 2016-06-16 07:50:36.732 8579 INFO kingbird.drivers.openstack.sdk [-] Using cached OS client objects
07:52:39 <sorantis> 2016-06-16 07:50:36.911 8579 INFO kingbird.drivers.openstack.sdk [-] Using cached OS client objects
07:52:41 <sorantis> 2016-06-16 07:50:37.036 8579 INFO kingbird.engine.quota_manager [-] Reading quota usage for 1af277a011034cca8ae0eda41ae8ccfc in RegionOne
07:52:41 <joehuang> have you configured the db part?
07:52:42 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.engine.quota_manager [-] Reading quota usage for 1af277a011034cca8ae0eda41ae8ccfc in RegionTwo
07:52:44 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.drivers.openstack.sdk [-] Creating fresh OS Clients objects
07:52:45 <sorantis> 2016-06-16 07:50:37.037 8579 INFO kingbird.drivers.openstack.sdk [-] Creating fresh OS Clients objects
07:52:47 <sorantis> Exception in thread Thread-4:
07:52:48 <sorantis> Traceback (most recent call last):
07:52:50 <sorantis> File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
07:52:51 <sorantis> self.run()
07:52:53 <sorantis> File "/usr/lib/python2.7/threading.py", line 763, in run
07:52:54 <sorantis> self.__target(*self.__args, **self.__kwargs)
07:52:56 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/engine/quota_manager.py", line 109, in read_quota_usage
07:52:57 <sorantis> os_client = sdk.OpenStackDriver(region)
07:52:59 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/drivers/openstack/sdk.py", line 64, in __init__
07:53:00 <sorantis> self.keystone_client.session)
07:53:02 <sorantis> File "/usr/local/lib/python2.7/dist-packages/kingbird/drivers/openstack/neutron_v2.py", line 33, in __init__
07:53:03 <sorantis> self.extension_list = self.neutron.list_extensions()
07:53:05 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 608, in list_extensions
07:53:06 <sorantis> return self.get(self.extensions_path, params=_params)
07:53:08 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 342, in get
07:53:09 <sorantis> headers=headers, params=params)
07:53:11 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 319, in retry_request
07:53:12 <sorantis> headers=headers, params=params)
07:53:14 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 282, in do_request
07:53:15 <sorantis> self._handle_fault_response(status_code, replybody, resp)
07:53:17 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 257, in _handle_fault_response
07:53:19 <sorantis> exception_handler_v20(status_code, error_body)
07:53:21 <sorantis> File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 84, in exception_handler_v20
07:53:22 <sorantis> request_ids=request_ids)
07:53:23 <sorantis> Unauthorized: Authentication required
07:53:55 <sorantis> but that’s probably because of another thing
07:53:57 <sorantis> ok
07:54:04 <sorantis> so far so good
07:54:13 <sorantis> at least it’s up and failing )
07:54:20 <joehuang> :)
07:54:21 <sorantis> I’ll reconfigure regions
07:54:29 <SAshish> it reuses endpoint cache module
07:54:46 <sorantis> yesterday I messed up one of the controllers, so had to reinstall whole setup
07:55:04 <sorantis> I didn’t configure services in region 2 to use keystone from region 1
07:55:10 <sorantis> this is most probably the issue
07:55:18 <sorantis> I’ll do it today
07:55:20 <joehuang> should use
07:55:32 <sorantis> yes, I know
07:55:40 <sorantis> I just didn’t have the time to do it
07:55:52 <joehuang> and especially check that in keystone configuration the url return should not be 127.0.0.1
07:56:01 <joehuang> otherwise will fail
07:56:08 <sorantis> what do you mean?
07:56:22 <sorantis> I don’t even use keystone in region 2
07:56:36 <joehuang> In region one Keystone
07:56:53 <joehuang> you need to check the configuration in keystone.confg
07:57:38 <joehuang> there is one default url returned, it should not be 127.0.0.1, otherwise the service in region two will access 127.0.0.1, it's local host will fail
07:57:53 <sorantis> it isn't
07:58:15 <joehuang> good if it isn't
07:58:57 <joehuang> be careful not to use 127.0.0.1 like address, use real IP instead
07:59:23 <sorantis> all services use real IPs
07:59:38 <joehuang> that's great helpful
07:59:53 <joehuang> any other topic
08:00:16 <SAshish> I have to work a little on neutron and cinder tempests
08:00:30 <SAshish> will push it soon
08:00:42 <joehuang> good
08:00:57 <SAshish> then will verify tempests on LAB setups
08:01:12 <sorantis> ok
08:01:17 <sorantis> thanks for help guys
08:01:20 <joehuang> so see you next week
08:01:20 <sorantis> talk soon
08:01:23 <sorantis> bye
08:01:26 <joehuang> bye
08:01:27 <SAshish> you are welcome :)
08:01:30 <SAshish> bye
08:01:34 <joehuang> #endmeeting