07:10:33 <joehuang> #startmeeting multisite 07:10:33 <collabot> Meeting started Thu Aug 4 07:10:33 2016 UTC. The chair is joehuang. Information about MeetBot at http://wiki.debian.org/MeetBot. 07:10:33 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 07:10:33 <collabot> The meeting name has been set to 'multisite' 07:10:44 <sorantis> #info dimitri 07:10:51 <joehuang> #info joehuang 07:10:56 <May-meimei> #info meimei 07:11:11 <May-meimei> I talked to Ashish while ago 07:11:44 <joehuang> how about the functest in CI 07:11:57 <joehuang> Hello, Ashish 07:12:03 <May-meimei> he is trouble-shooting the access problem 07:12:14 <SAshish> Hi Joe, By mistake I connected to some other group 07:12:23 <SAshish> yeah, Hi All 07:12:38 <joehuang> no problem, welcome back :) 07:12:51 <SAshish> we need to resolve one the access problem, then tempest can work 07:13:08 <SAshish> other things are happening with tempest, where we create user/project for kb 07:13:13 <joehuang> do you mean the 8118 port can't be accessed in the container 07:13:35 <SAshish> I think, the mapping from public ip to internal ip is not proper 07:13:46 <SAshish> 192. internal Ip 07:13:52 <SAshish> 172. is public ip 07:14:00 <joehuang> is this the mapping by Fuel? 07:14:09 <SAshish> HA in fuel 07:14:35 <SAshish> I have seen configuration for the mapping only in HA configuration files 07:14:39 <sorantis> try grepping on the ip and port number of nova 07:14:39 <joehuang> Is kingbird running with Nova in same node? 07:14:43 <SAshish> yes 07:14:51 <sorantis> see where it’s set and make sure that kb follows the same 07:15:03 <SAshish> it is same 07:15:11 <sorantis> start with the /etc 07:15:16 <SAshish> iptables/haconfigurations/kb.conf 07:15:23 <SAshish> all are same 07:15:26 <sorantis> is it only this? 07:15:38 <SAshish> I think so 07:15:50 <joehuang> I have an idea 07:15:55 <SAshish> okay 07:16:18 <joehuang> change the port of kingbird to one of other service port which works in functest 07:16:55 <SAshish> 1 sec 07:16:56 <SAshish> okay 07:17:21 <SAshish> http://192.168.0.2:9292 07:17:25 <SAshish> this url is registered in keystone for glance 07:18:09 <SAshish> but the bind host is 192.168.0.5 07:18:15 <SAshish> so ours is also same 07:18:26 <SAshish> bind host in kb.conf is 192.168.0.5 07:18:46 <SAshish> url in keystone is http://192.168.0.2:8118/v1.0 07:18:49 <SAshish> now 07:19:21 <joehuang> really strange, I have no idea about Fuel's setting why. 07:19:22 <SAshish> from the controller glance is accessible with both the addresses 07:19:35 <SAshish> i mean, telnet 192.168.0.5 8774 07:19:36 <SAshish> works 07:19:41 <SAshish> telnet 192.168.0.2 8774 07:19:42 <SAshish> works 07:19:57 <SAshish> but for kingbird it doesn not, it works only with the bind host registered in kb.conf 07:20:12 <SAshish> not the url mentioned in keystone 07:20:20 <joehuang> you can use 0.0.0.0 for bind host 07:20:30 <joehuang> other wise I guess you can't access it 07:20:49 <joehuang> for 0.0.0.0 means all IP address for this host works for the service 07:21:13 <SAshish> in all other configurations it is the ip 07:21:36 <joehuang> how many ip address the host have? 07:21:40 <SAshish> so kept the same for kb as well 07:22:39 <joehuang> can Haproxy forward(load balancing) request to kingbird? 07:22:51 <SAshish> that is the problem I suppose 07:22:56 <joehuang> or haproxy drops the packet? 07:23:19 <SAshish> If it forwards and it is discarded at some point I am not sure 07:24:46 <joehuang> for kb, can you try the kb functest bypass the haproxy 07:25:11 <joehuang> from curl http://192.168.0.5:8118/v1.0 first 07:25:26 <SAshish> no 192 series is not accessible 07:25:33 <sorantis> Have you seen this? /etc/apache2/sites-available/25-apache_api_proxy.conf:29: AllowCONNECT 443 563 5000 6385 8000 8003 8004 8042 8080 8082 8386 8773 8774 8776 8777 9292 9494 9696 07:25:53 <joehuang> no 8118 port allowed! 07:26:19 <joehuang> to Dimitri, this may be the issue 07:27:14 <sorantis> Ashish, try adding 8118 to the list and restart apache. see if it fixes this 07:27:33 <SAshish> it does not 07:27:49 <joehuang> grep 8774 and add 8118 accordding 07:28:07 <SAshish> I feel mapping for 192.168.0.5 => 192.168.0.2 is not happening for 8118 07:29:37 <SAshish> as we are not able to access kb from 192.168.0.2 07:29:42 <sorantis> Iptable /etc/iptables/rules.v4:115:-A INPUT -p tcp -m multiport --ports 9292,9494,9191,8773 -m comment --comment "104 glance" -j ACCEPT 07:29:53 <sorantis> I couldn’t fined an entry for kb 07:30:08 <SAshish> ACCEPT tcp -- anywhere anywhere multiport ports 8118 /* 106 kingbird */ 07:30:19 <SAshish> root@node-4:~# iptables -L | grep 8118 07:30:41 <sorantis> ok, shouldn’t it be in the file? 07:31:35 <sorantis> also this 07:31:41 <sorantis> /etc/haproxy/conf.d/080-glance-api.cfg:3: bind 172.16.0.3:9292 07:31:49 <sorantis> /etc/haproxy/conf.d/080-glance-api.cfg:4: bind 192.168.0.2:9292 07:31:50 <sorantis> /etc/haproxy/conf.d/080-glance-api.cfg:10: server node-4 192.168.0.5:9292 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 07:32:24 <sorantis> i guess this is the mapping 07:32:42 <SAshish> yes 07:32:50 <SAshish> we have file simillary 07:33:34 <sorantis> yes, looking at it 07:34:21 <joehuang> have you restared haproxy after adding the kb configuration? 07:35:04 <SAshish> yes 07:36:42 <sorantis> cat /etc/keystone/default_catalog.templates 07:36:49 <sorantis> weird file 07:38:46 <SAshish> may be it is used during deployment 07:39:02 <SAshish> thats it 07:43:49 <joehuang> ok 07:44:26 <joehuang> this file is used for batch endpont create 07:45:08 <joehuang> can the kb service pingable now from functest container? 07:45:55 <SAshish> no 07:46:44 <SAshish> it is not pingable from controller also on the other internal ip 07:46:46 <joehuang> BTW, the installation and configuration guide has been submitted for review, please have a look, thanks 07:46:57 <SAshish> this is the problem 07:47:49 <SAshish> yeah, will do it. Have you updated comments for the commit done is kb repo as well 07:47:52 <joehuang> we need to have 0.2.0 tag in kingbird after the functest works, and test cases pass 07:48:22 <SAshish> few of my comments are valid on https://review.openstack.org/#/c/349762/ as well 07:48:24 <joehuang> will do that after one is merged, so update once in kb reposity 07:48:39 <SAshish> okay 07:48:44 <joehuang> understand, so review the multisite first 07:48:53 <SAshish> fine 07:49:22 <joehuang> could you add the user guide, you have writen lots of API example, it will be easy for you 07:50:04 <SAshish> yeah, tell me the procedure I will do it 07:50:11 <joehuang> hi dimitri and meiemi, how do you think about it? 07:51:34 <sorantis> about what 07:52:09 <joehuang> tagging and user guide 07:52:47 <joehuang> the colorado branch is planned on Aug 15 07:53:59 <sorantis> I’ve prepared the milestone on the launchpad already. Including the tag that we will use in this release 07:54:00 <joehuang> o, one more, I mentioned in the mail, that some times the on demand sync doesn't sync immediatly 07:54:04 <sorantis> it will 1.0 07:54:19 <sorantis> trying to align with newton release cycle 07:54:32 <joehuang> seems a little early for 1.0 version number 07:54:41 <sorantis> you might have hit the time when on-demand collided with periodic 07:54:45 <sorantis> I don’t think so 07:54:51 <sorantis> we have quota management in place 07:55:03 <SAshish> one usecase we solve then it can be 1.0 07:55:09 <SAshish> the a management 07:55:14 <SAshish> quota management 07:55:22 <sorantis> this could be the first proper release 07:55:32 <joehuang> not reach stable yet 07:55:49 <joehuang> for example, 0.9 also acceptable 07:56:05 <joehuang> beta release 07:56:34 <joehuang> not been tested by user yet 07:57:43 <joehuang> a little worry on the lock 07:58:47 <sorantis> have you checked if you might have hit the time when on-demand collided with the periodic run? 07:58:58 <SAshish> that should not happen 07:59:03 <SAshish> it should not collide 07:59:20 <SAshish> as only in periodic sync I am acquring lock and release it 07:59:23 <joehuang> when kb-engine restart, often can't get the lock 07:59:30 <SAshish> that is because 07:59:34 <joehuang> the message printed 07:59:55 <SAshish> the earlier run has acquired and due to some error it halted 08:00:00 <SAshish> now the lock is there in db 08:00:08 <SAshish> then subsequent runs will fail 08:00:29 <SAshish> before releasing lock, engine is stopped 08:01:07 <SAshish> so next time when we start engine, db has lock acquired earlier, which will not allow to acquire and sync 08:01:09 <joehuang> ok, Ashish, is there any risk for you to submit the userguide and get merged before Aug.15 08:02:28 <joehuang> currently there are quite few projects( in one cycle), so the lock should be released soon 08:02:32 <sorantis> the risk is the immediate problems with testing 08:02:40 <joehuang> but seems it doesn't 08:02:40 <sorantis> you can see that there are issues 08:02:54 <joehuang> test is the first priority 08:03:48 <joehuang> I want to help, but I am not able to access the lab in intel pod 08:04:33 <joehuang> so Ashish and Dimitri fix the test environment issue first 08:04:47 <sorantis> you can help by managing the docs. If the issue persist, Ashish may not have the time to write it 08:05:11 <joehuang> I'll try 08:05:17 <SAshish> I can help with you the API structures and all 08:05:31 <joehuang> ok 08:05:38 <SAshish> thanks 08:05:40 <sorantis> great. I think we can close 08:05:44 <joehuang> #info fix the test issue first 08:05:57 <joehuang> #info joe help to add the userguide 08:06:21 <joehuang> ok, let's close, and hope the test can pass ASAP 08:06:31 <joehuang> #endmeeting