17:07:21 <narinder`> #startmeeting joid 17:07:21 <collabot`> Meeting started Wed Nov 16 17:07:21 2016 UTC. The chair is narinder`. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:07:21 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic. 17:07:21 <collabot`> The meeting name has been set to 'joid' 17:07:46 <narinder`> bryan_att: no i am in India and working from India for 4 weeks 17:08:04 <bryan_att> narinder`: ok, thanks 17:08:28 <bryan_att> #info Bryan Sullivan 17:08:29 <narinder`> bryan_att: what is username and password for organisors fot gotomeeting 17:08:38 <narinder`> #info Narinder Gupta 17:08:46 <bryan_att> I'll send it direct 17:08:55 <narinder`> i think i got it 17:13:05 <rprakash> #info rprakash 17:14:05 <narinder`> https://etherpad.opnfv.org/p/joid 17:49:42 <narinder`> sorry got disconnected\ 17:50:00 <narinder`> can not hear anyone now 18:30:09 <bryan_att> narindergupta: any idea why MAAS can't power-on the bootstrap node? https://www.irccloud.com/pastebin/y3fUM5kh/ 18:30:38 <narindergupta> bryan_att, currently in meeting as soon as done i will look into it 18:30:48 <bryan_att> ok, thanks 21:07:44 <narinder> bryan_att, i was looking into the labconfig.yaml 21:08:13 <narinder> bryan_att, i think you should have bridge: <bridge name> for public network so that maas can do dhcp over there 21:08:27 <narinder> rest looks of to me 21:10:52 <bryan_att> I tried that but it complained there was no brPublic bridge. So I guess I need to define one but I'm not sure how with only one nic on the jumphost. 21:12:12 <bryan_att> It would need to be defined in /etc but I don't know how to add one nic to two bridges or if that would work. 21:15:15 <narinder> bryan_att, in /etc/network/interfaces file you have to make changes 21:15:59 <narinder> also send me deployconfig.yaml and deployment.yaml form joid/ci when u add the second bridge in labconfig.yaml 21:18:09 <bryan_att> Ok will do. How do I set the interfaces file? 22:10:27 <narinder> bryan_att, https://help.ubuntu.com/community/NetworkConfigurationCommandLine/Automatic 20:18:15 <narinder> jmorgan1, hi 20:18:35 <narinder> jmorgan1, it seems intel pod6 5 out of 6 servers are not responding to ipmi 20:19:00 <narinder> jmorgan1, can you power reset all servers in pod? 00:06:29 <jmorgan1> narinder`: looks like node2,3,4,5,6 are not responding. I'll take a look 00:15:34 <DanSmithEricsson> hello..anyone who's installed JOID online / around? i have a couple silly questions 02:39:53 <DanSmithEricsson> If anyone is / does listen in.. i wanted to know if the labconfig.yaml for JOID only support IMPI - i have done up a virtual install, but its a hack of JOID cause the installer created stops if there is no IMPI enpoint found.. I hacked a way around, but if there is a better method please let me know. 16:53:26 <bryan_att> narinder`: hi quick question. Is there a MAAS command to release old DHCP addresses? I have a situation in JOID where the JuJu bootstrap node is trying to get 192.168.10.100 and the MAAS DHCP server is NAKing that as "not available" but it is (the arp entry for that address is "incomplete") 16:59:10 <bryan_att> narinder`: never mind, stupid error I had turned on my router DHCP server and forgot to turn it off... 17:40:15 <narindergupta> bryan_att, ok 19:10:38 <bryan_att> narindergupta: you said that newton was ready to test right? what are the install instructions for JOID? I'm running into issues with the 03* installer. 19:11:47 <narindergupta> ./deploy.sh -o newton -s nosdn -t nonha -l attpod1 -d xenial 19:12:12 <bryan_att> I mean for MAAS deploy 19:12:21 <bryan_att> virtual 19:12:27 <bryan_att> to start with 19:12:32 <narindergupta> bryan_att, for maas_deploy i am still working on 19:12:44 <narindergupta> and we are expecting 16.04 with newton 19:13:02 <narindergupta> then run ./03-maasdeploy.sh default 19:13:12 <narindergupta> let me know what issue you are facing? 19:16:09 <bryan_att> ok hang on 19:16:37 <bryan_att> https://www.irccloud.com/pastebin/qEG6UWm5/ 19:17:33 <bryan_att> narindergupta: can I deploy without MAAS? for virtual? 19:19:59 <bryan_att> also, when you get some time I need your help to review my 2-NIC labconfig.yaml. Aimee and I have been trying to get 3-node non-HA installs working now with the same servers as before (NUCs in my case, desktops in hers), but with 2 NICs on all nodes instead of just on the controller. 19:21:37 <narindergupta> bryan_att, looks like you are having old version of master code. will you please update? 19:22:20 <bryan_att> I just cloned 19:22:27 <bryan_att> about 30 mins ago 19:23:12 <bryan_att> this is a new 16.04 install, fresh original clone of joid 19:25:32 <narindergupta> r u trying 00-maasinstall or 03-maasinstall 19:25:44 <bryan_att> 03 19:26:06 <bryan_att> I tried 00 and it's clearly for Colorado. Figured that one out. 19:27:41 <narindergupta> bryan_att, somehow master was not cloned as i can see i am not deploying maas-deployer anyymore in script and your error is maas-deployer for xenial 19:28:10 <bryan_att> ok, let me clone again and try 19:29:14 <narindergupta> bryan_att, i uses this command to clone git clone http://gerrit.opnfv.org/gerrit/joid.git joidnew.git 19:30:03 <bryan_att> not "git clone https://gerrit.opnfv.org/gerrit/joid" ? That's what gerrit says to use 19:30:25 <bryan_att> what's "joidnew"? 19:34:50 <bryan_att> narindergupta: I tried it with that clone command - same result 19:35:04 <narindergupta> just a new target 19:35:13 <narindergupta> can u pastebin 03-maasdeploy.sh 19:35:15 <narindergupta> ? 19:35:36 <narindergupta> reason i am saying is we do not deploy maas-deployer at all now 19:35:56 <narindergupta> so error related to maas-deployer is really confusing here 19:36:30 <bryan_att> https://usercontent.irccloud-cdn.com/file/UHsJKWmC/03-maasdeploy.sh 19:39:27 <narindergupta> bryan_att, code is good but somehow you got maas-deploy repo listed. Did you tried anything else on this system after deploying xenial? 19:42:01 <bryan_att> narindergupta: no, just installed filezilla. nothing else 19:42:30 <narindergupta> i am sure something else. can u list files in /etc/apt/sources.list.d/ 19:42:56 <narindergupta> but i can not see any code in my 03-maasdeploy.sh 19:43:04 <narindergupta> which deploy maas-deployer 19:43:16 <bryan_att> https://www.irccloud.com/pastebin/ZRgQHLIg/ 19:45:26 <bryan_att> if this view helps too... https://www.irccloud.com/pastebin/5PaBTvqQ/ 19:46:34 <narindergupta> bryan_att, i am sure looks like 00-maasdeploy.sh ran here as i can see all ppa related to maas-deployer 19:46:37 <narindergupta> maas-deployers-ubuntu-stable-xenial.list 19:46:45 <narindergupta> sud o let delete all file from there 19:46:50 <narindergupta> and run sudo apt-get update 19:46:59 <narindergupta> then run 03-maasdeploy.sh default 19:47:08 <bryan_att> yes, I did try that once because I did not know there was an 03 installer... 19:47:31 <narindergupta> bryan_att, those files were side effect of that 19:47:36 <bryan_att> I didn't see any instructions yet on the wiki so I just did what we did before. 19:48:03 <narindergupta> bryan_att, yeah wiki is not updated for 16.04 MAAS yet 19:48:19 <narindergupta> as i am still working on real labconfig.yaml deployment 19:49:33 <bryan_att> ok, thanks I am trying to get a head start on the virtual deploy at least with newton Congress 19:50:36 <bryan_att> if you have a minute, can you look at my 2-NIC labconfig.yaml file and tell me if it looks OK? 19:51:19 <bryan_att> narindergupta: 2-NIC config https://usercontent.irccloud-cdn.com/file/bBe9kAMJ/labconfig.yaml 19:52:27 <bryan_att> narindergupta: updated interfaces file for 2nd NIC (single-NIC on jumphost did not work with 2 NICs defined in the labconfig.yaml file) https://usercontent.irccloud-cdn.com/file/gRJlAdPp/interfaces 19:53:03 <narindergupta> iface eth2 inet manual 19:53:11 <narindergupta> is eth2 is real interface 19:53:18 <narindergupta> labconfig.yaml looks good 19:53:30 <narindergupta> ifconfig -a will give you more info 19:54:05 <bryan_att> eth2 is the interface that showed up when I plugged in the usb-ethernet adapter. I was expecting eth1 but it was not defined. eth2 was and is working 19:54:40 <bryan_att> ok maasdeploy seems to be getting farther now 19:54:42 <narindergupta> ok 19:54:47 <narindergupta> cool 20:22:32 <bryan_att> narindergupta: I'm having issues with the 2-NIC deployment, neutron-gateway is failing with "config-changed". This is the second time I have tried to deploy. What do you need to help me debug? 20:23:05 <narindergupta> i need to know the ifconfig -a on the neutron-gateway machine? 20:23:23 <narindergupta> to see whether eth1 matches or not? 20:23:39 <narindergupta> if not then it has to changed in labconfig.yaml as well 20:25:12 <bryan_att> hmm eth1 seems not to be defined... some issue with setting up the usb-ethernet adapter I guess. Let me re-plug it and try again. 20:25:44 <bryan_att> narindergupta: but on the virtual deploy I had another error as below 20:26:08 <bryan_att> https://www.irccloud.com/pastebin/tPusHLsA/ 20:29:40 <narindergupta> bryan_att, for now create directory manually sudo mkdir ~maas/.ssh/ 20:30:05 <narindergupta> and retry i will fix this soon 20:31:11 <bryan_att> ok, trying now 20:31:20 <narindergupta> thanks 21:11:17 <bryan_att> narinder`: seems we need another file 21:11:33 <bryan_att> https://www.irccloud.com/pastebin/ZjkCaYPQ/ 00:50:21 <bryan_att> narindergupta: found the right workaround - see next 00:50:24 <bryan_att> https://www.irccloud.com/pastebin/C7H9a9fy/ 00:50:38 <bryan_att> the -y was making the command fail 01:12:22 <bryan_att> narinder`: still having a number of other issues though - we can talk when you are back at work next week. if you happen to connect over the weekend I'll keep an eye out. 04:25:20 <narindergupta> bryan_att, thanks i will propose a change and change it accordingly 05:00:20 <JingLu> Hi, my name is Jing Lu, I am from the Yardstick team. As you may notice, Yardstick daily CI job is keep failing in os-nosdn-lxd-ha and os-nosdn-lxd-noha scenarios. CI console log: https://build.opnfv.org/ci/view/yardstick/job/yardstick-joid-baremetal-daily-master/lastFailedBuild/console 05:01:01 <JingLu> As you can see from the log, the Yardstick job is failing on TC005 when yardstick container try to log onto the VM via ssh tunnel. I am working on fixing it, but we have an obstacle that we don’t have a local lxd environment to verify the work. Can your guys provide a test environment to us? Any advice will be appreciated. 06:09:30 <OPNFV-Gerrit-Bot> Narinder Gupta proposed joid: modified to accomodate the MAAS 2.0 https://gerrit.opnfv.org/gerrit/24649 06:10:33 <OPNFV-Gerrit-Bot> Merged joid: modified to accomodate the MAAS 2.0 https://gerrit.opnfv.org/gerrit/24649 08:04:05 <bryan_att> narinder`: testing latest master - two bugs: 1) syntax error "enablesubnetand dhcp(){" in 03-maasdeploy.sh line 216 08:05:16 <bryan_att> narinder`: also I can't run the 03-maasdeploy.sh multiple times as it tries to create the ubuntu user again and fails. I have to manually edit the user to change the name and email. 08:05:50 <bryan_att> narinder`: also got the error "{"__all__": ["Boot source selection with this Boot source, Os and Release already exists."]}" 08:05:59 <bryan_att> narinder`: so i 08:06:47 <bryan_att> narinder`: its difficult to make much progress if I cant run maas multiple times. when it ran in a VM at least we could destroy it and start over 16:05:20 <bryan_att> narindergupta: ping 16:05:34 <narindergupta> bryan_att, pong 16:05:54 <bryan_att> narindergupta: I sent some issues last night on the virtual install - did you see them? 16:06:17 <bryan_att> https://www.irccloud.com/pastebin/Odmkh6jv/ 16:06:39 <bryan_att> narindergupta: those are FYI - what I need help with right now is the 2-NIC bare metal deploy 16:06:57 <bryan_att> narindergupta: JuJu is not setting up the 2nd NIC as I expected. 16:07:40 <bryan_att> narindergupta: it's only setting up juju-br0 16:07:43 <bryan_att> https://www.irccloud.com/pastebin/ugjgW3q0/ 16:07:56 <bryan_att> https://www.irccloud.com/pastebin/IKXE5SxM/ 16:08:43 <bryan_att> narindergupta: and on node 2... 16:08:45 <bryan_att> https://www.irccloud.com/pastebin/Pimmaiyg/ 16:09:15 <bryan_att> https://www.irccloud.com/pastebin/3IjmAxD1/ 16:10:24 <bryan_att> narindergupta: the labconfig is the same as I sent you the other day, which you said looked fine. How can I get both NICs to be setup for the intended purpose (eth0=admin, eth1=public) ? 16:11:18 <narindergupta> bryan_att, for non virtual i am still working on 16:11:25 <narindergupta> MAAS 2.0 16:11:36 <bryan_att> narindergupta: this is stable/colorado for bare metal 16:11:53 <narindergupta> stable / colorado should use vm 16:12:02 <narindergupta> not on jumphost 16:12:31 <bryan_att> narindergupta: not sur e what you mean - use VM for what? I'm talking about the NIC use here 16:13:13 <narindergupta> bryan_att, i thought you are talking about <bryan_att> narindergupta: I sent some issues last night on the virtual install - did you see them? 16:13:49 <narindergupta> bryan_att, looks like in your MAAS install dhcp was not setup in your public network 16:13:55 <bryan_att> narindergupta: yes, I pasted them but also said - "hose are FYI - what I need help with right now is the 2-NIC bare metal deploy" - sorry for the confusion 16:14:32 <bryan_att> narindergupta: do I need a MAAS DHCP change somewhere - what file? 16:15:39 <narindergupta> bryan_att, maas is already install so go into maas gui 16:15:52 <narindergupta> click on clusters 16:16:27 <narindergupta> clieck on modify on right hand after mouse browse 16:16:41 <narindergupta> there look for interfaces section 16:17:11 <narindergupta> eth1 should be modified to dhcp only rather than unmanaged 16:17:29 <bryan_att> narindergupta: ok, I can do that but what config file sets up DHCP for the other networks? 16:17:31 <bryan_att> https://www.irccloud.com/pastebin/FSmGmJpw/ 16:17:42 <bryan_att> narindergupta: it's already set to DHCP 16:18:09 <narindergupta> actually you already have setup dhcp 16:18:31 <narindergupta> just wondering why your server is not responding and configuring dhcp 16:18:44 <narindergupta> can u check connection please? 16:19:59 <bryan_att> https://www.irccloud.com/pastebin/JcFA9W96/ 16:20:21 <bryan_att> https://www.irccloud.com/pastebin/P2QM5amd/ 16:20:30 <narindergupta> bryan_att, can u log into maas and check 16:20:53 <narindergupta> if that works then lets check the m1 eth1 16:21:02 <narindergupta> cat /etc/nbetwork/interfaces 16:21:36 <bryan_att> narindergupta: what do you want me to check in MAAS? 16:22:15 <narindergupta> ping both the gateway 16:22:35 <bryan_att> ok, from the MAAS VM you mean... 16:23:51 <bryan_att> https://www.irccloud.com/pastebin/pSqM4fQZ/ 16:24:01 <narindergupta> correct 16:24:11 <bryan_att> narindergupta: working OK 16:24:16 <narindergupta> great 16:24:23 <narindergupta> now on deployed nodes 16:24:44 <narindergupta> send me cat /etc/network/interfaces 16:25:20 <bryan_att> https://www.irccloud.com/pastebin/uoO0FwGZ/ 16:26:16 <bryan_att> https://www.irccloud.com/pastebin/2Frvpc4L/ 16:26:23 <narindergupta> ok looks like in maas eth1 is not enabled 16:26:55 <narindergupta> can u check in node details 16:27:03 <narindergupta> in maas 16:27:13 <bryan_att> yes, it says unconfigured 16:27:30 <narindergupta> can u change it to dhcp 16:27:40 <narindergupta> thats the issue 16:28:03 <bryan_att> OK, I can do that, but is there some change I need to do to make this work in MAAS? 16:28:09 <bryan_att> (in a config file) 16:28:22 <narindergupta> i need deployconfig.yaml to verify 16:29:02 <bryan_att> https://usercontent.irccloud-cdn.com/file/3Y8AjO8L/deployconfig.yaml 16:30:24 <narindergupta> this looks ok interface-enable: eth1 16:31:07 <narindergupta> it should be enabled by default. can i have maas deployment logs to see what happened wrong 16:31:10 <narindergupta> ? 16:32:09 <bryan_att> https://www.irccloud.com/pastebin/10r5CdNW/ 16:40:48 <bryan_att> narindergupta: OK, in the meantime I will update it in MAAS UI and then redeploy to see if it works. Otherwise if you want me to wait for more debugging let me know. 16:42:25 <narindergupta> bryan_att, no just update the nodes and we can re deploy 16:42:43 <narindergupta> do not redeploy MAAS 16:42:56 <bryan_att> narindergupta: ok doing that now. 16:42:58 <narindergupta> bryan_att, can u enable dhcp on eth1 for the node 16:44:02 <bryan_att> narindergupta: will do. in order to do that I need to recommission the nodes, right (to get them to "ready" state again) 16:45:00 <bryan_att> narindergupta: oh, nevermind for some reason they returned to "ready" (I guess when I ran clean.sh) 16:45:53 <bryan_att> narindergupta: should both NICs be "DHCP" (eth0 is "Auto Assign") 16:47:56 <narindergupta> bryan_att, no need to recommision. Just run clean.sh 16:48:16 <narindergupta> you can move both to auto assign 16:48:25 <narindergupta> which will use static ip should be ok 16:48:38 <narindergupta> just make sure subnet selected is good 16:48:57 <bryan_att> narindergupta: also it's strange to me that for node-2 the MAAS UI shows eth1 as the PXE-boot NIC. Does the ordering of the NICs in the MAAS UI matter? 16:49:39 <narindergupta> bryan_att, no thats fine only issue is if that node gets selected as neutron-gateway then its an issue 16:49:53 <bryan_att> narindergupta: in both nodes its eth0 that is the PXE NIC (it's a physical NIC - the other NICs are USB-ethernet adapters) 16:50:04 <narindergupta> is then its fine 16:50:29 <bryan_att> narindergupta: yes, I have been having issues with neutron-gateway, so if that happens is there a way I can force neutron-gateway onto a particular node? 16:50:51 <bryan_att> narindergupta: or is it randomly assigned (hope not) 16:53:55 <narindergupta> bryan_att, use the feature 16:54:04 <narindergupta> for not using hyperconverged 16:54:35 <bryan_att> narindergupta: OK, how do I do that 16:56:55 <narindergupta> -f dishypcon 16:57:02 <narindergupta> in ./deploy.sh 16:57:49 <bryan_att> narindergupta: OK, that will disable the variable assignment of the neutron-gateway ? how do I ensure that it gets on a specific node (or will that command fix it to a specific node?) 16:59:07 <narindergupta> bryan_att, node labelled as neutron will be used 16:59:19 <narindergupta> but you need to adjust the ext-net then 16:59:55 <bryan_att> narindergupta: ok, in what config file is the node labelled as neutron, and what is the update to ext-net needed? 17:00:19 <narindergupta> labconfig.yaml 17:00:25 <narindergupta> we have it called role 17:00:42 <narindergupta> it may be neutron or network i have to check that 17:00:45 <narindergupta> but one of it 17:01:02 <bryan_att> it's network 17:01:13 <narindergupta> then its network 17:01:19 <bryan_att> thanks that's clear - what is the update to ext-net needed? 17:04:48 <narindergupta> if it is eth0 then use eth0 otherwise eth1 is good 17:05:57 <bryan_att> OK, that's clear - you mean the " ext-port: "eth1" " setting. OK, I'll try with this guidance and see if I can get it working. Thanks for all your help! 17:13:01 <narindergupta> bryan_att, no problem Bryan 22:06:16 <OPNFV-Gerrit-Bot> Stuart Mackie proposed joid: Changed to use a local (working) cassandra charm from ovno repo instead of from Launchpad https://gerrit.opnfv.org/gerrit/24699 22:12:09 <OPNFV-Gerrit-Bot> Merged joid: Changed to use a local (working) cassandra charm from ovno repo instead of from Launchpad https://gerrit.opnfv.org/gerrit/24699 16:35:47 <bryan_att> narindergupta: I was having issues trying to get the MAAS deploy working "ssl.SSLError: [Errno 1] _ssl.c:510: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version" but that went away and now I have ... 16:36:12 <bryan_att> narindergupta: "2016-11-23 16:26:17 ERROR juju.cmd supercommand.go:429 cannot determine if environment is already bootstrapped.: could not access file 'baa89ca6-d470-4ab5-85fa-6351fc73a74a-provider-state': gomaasapi: got error back from server: 401 OK (Authorization Error: Invalid API key.)" 16:36:21 <narindergupta> bryan_att, i am also seeing same error 16:36:51 <narindergupta> i will find out the solution soon 16:36:57 <bryan_att> OK, not just me. 17:01:22 <StuartMackie> #info Stuart Mackie 17:04:34 <bryan_att> #info Bryan Sullivan 17:05:00 <Anand> #info Anand Gorti 17:05:12 <collabot`> arturt: Error: Can't start another meeting, one is in progress. Use #endmeeting first. 17:05:20 <arturt> #endmeeting