14:01:02 <fdegir> #startmeeting OpenStack 3rd Party CI 14:01:02 <collabot> Meeting started Wed Aug 31 14:01:02 2016 UTC. The chair is fdegir. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:02 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 14:01:02 <collabot> The meeting name has been set to 'openstack_3rd_party_ci' 14:01:18 <fdegir> #chair jmorgan1 Julien-zte hwoarang 14:01:18 <collabot> Current chairs: Julien-zte fdegir hwoarang jmorgan1 14:01:27 <fdegir> #topic Roll Call 14:01:32 <Julien-zte> #info Julien 14:01:36 <ChrisPriceAB> #info Chris Price 14:01:42 <fdegir> #chair ChrisPriceAB 14:01:42 <collabot> Current chairs: ChrisPriceAB Julien-zte fdegir hwoarang jmorgan1 14:01:47 <ChrisPriceAB> noooooo 14:01:49 <ChrisPriceAB> :) 14:01:50 <hwoarang> #info Markos 14:01:50 <fdegir> :) 14:01:56 <jmorgan1> #info Jack Morgan 14:01:56 <fdegir> I'm new in this so... 14:02:13 <fdegir> Yolanda is off this week so we can start 14:02:18 <fdegir> here is the agenda 14:02:19 <fdegir> #topic Agenda 14:02:19 <Julien-zte> ok 14:02:24 <fdegir> #info Welcome 14:02:28 <fdegir> #info Background Info 14:02:32 <fdegir> #info Progress: bifrost 14:02:38 <fdegir> #info Progress: puppet-infracloud 14:02:41 <fdegir> #info Infra/HW Needs 14:02:45 <fdegir> #info AOB 14:02:50 <Julien-zte> #link http://docs.openstack.org/developer/bifrost/readme.html 14:03:03 <fdegir> if anyone wants to add anything to the agenda, please feel free to do so 14:03:05 <Julien-zte> #link https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI 14:03:12 <fdegir> Julien-zte: thx 14:03:15 <fdegir> #topic Welcome 14:03:29 <fdegir> #info I want to welcome everyone and thanks for joining this effort! 14:03:46 <fdegir> and I'm really happy to see we have made a direct impact 14:03:49 <fdegir> :) 14:04:04 <fdegir> there have been some contributions to upstream already 14:04:13 <Julien-zte> thanks fgedir, great job has been done 14:04:14 <fdegir> which I'll paste the link when the topic comes there 14:04:29 <fdegir> this is really encouriging 14:04:42 <fdegir> so, thx again and ChrisPriceAB expects more! 14:04:50 <fdegir> #topic Background Info 14:05:04 <fdegir> #info There might be questions regarding why we are working on this 14:05:25 <fdegir> #info We tried to give a summary regarding this activity on Wiki so please feel free to read, comment, update 14:05:35 <fdegir> #link https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI 14:05:56 <fdegir> #info The primary tools we are looking into are bifrost and puppet-infracloud 14:06:00 <fdegir> #link https://github.com/openstack/bifrost 14:06:04 <fdegir> #link http://git.openstack.org/cgit/openstack-infra/puppet-infracloud 14:06:31 <fdegir> and we can move to real stuff now... 14:06:41 <fdegir> #topic Progress: bifrost 14:07:01 <fdegir> #info We have been evaluating/trying out bifrost and fixing what we think could be fixed/improved 14:07:25 <fdegir> #info Yolanda and Markos have been pretty active with locating and fixing nasty bugs 14:07:34 <fdegir> the rabbitmq stuff for example 14:07:54 <ChrisPriceAB> #info the objective of this activity is to make sure we can use bifrost to deploy an OPNFV scenario on a "pharos" lab 14:08:22 <ChrisPriceAB> #info using openstack native CI tools 14:08:34 <fdegir> and from master! 14:08:46 <ChrisPriceAB> ack 14:08:51 <fdegir> who likes months old sw 14:09:17 <Julien-zte> yes, from the master branch 14:09:32 <Julien-zte> good point 14:09:43 <fdegir> #info We can now provision 3 VMs using bifrost 14:09:57 <fdegir> #info Ubuntu as OS and the 3 VMs serve as jumphost, controller, and compute 14:10:06 <Julien-zte> I have not finished this yet :( 14:10:08 <fdegir> #info Work to introduce Centos and SuSe support is ongoing 14:10:22 <fdegir> hwoarang: would you like to add anything regarding SuSe support 14:10:26 <jmorgan1> which SuSE versions? 14:10:31 <fdegir> hwoarang: or any other item? 14:10:57 <hwoarang> fdegir: it's still work in progress. we are missing the ipxe package which i need to work on. but it's pretty close after that 14:11:05 <Julien-zte> many a newest LTS version 14:11:13 <hwoarang> jmorgan1: openSUSE Tumbleweed at first. then Leap. then SLES 14:11:23 <Julien-zte> many: maybe 14:11:26 <fdegir> #info Introducing SuSe support is work in progress. we are missing the ipxe package which i need to work on. but it's pretty close after that 14:11:38 <fdegir> #info openSUSE Tumbleweed at first. then Leap. then SLES 14:11:39 <jmorgan1> hwoarang: ok 14:12:23 <fdegir> #info The work we've done with bifrost is put on OPNFV Jenkins and we are provisioning VMs over and over automatically 14:12:29 <fdegir> #link https://build.opnfv.org/ci/job/infra-provision-virtual-daily-master/lastBuild/consoleFull 14:12:39 <Julien-zte> my question about bifrost is time cost to run once. 14:12:47 <fdegir> if anyone wants to try stuff, you can take a look at the script from releng repo 14:12:53 <Julien-zte> do we have some time estimation? 14:13:21 <fdegir> Julien-zte: you mean how much time it takes to provision? 14:13:25 <Julien-zte> yes 14:13:38 <fdegir> Julien-zte: https://build.opnfv.org/ci/job/infra-provision-virtual-daily-master/buildTimeTrend 14:13:47 <fdegir> but the about 45 minutes 14:13:51 <fdegir> but please note that 14:13:59 <fdegir> the time you see is everything from scratch 14:14:09 <fdegir> installation of complete toolchain (ansible and whatnot) 14:14:13 <fdegir> building diskimages 14:14:17 <fdegir> and so on 14:14:36 <fdegir> and when we do this for real, we will probably skip these and cut the time 14:14:37 <Julien-zte> good for in a hour 14:14:58 <fdegir> I expect 30 minutes or less for provisioning VMs 14:15:08 <fdegir> baremetal will probably differ 14:15:50 <fdegir> #info This is the script that is run by Jenkins 14:15:52 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/infra/infra-provision.sh 14:16:29 <Julien-zte> thanks fdegir, will check for this 14:16:31 <fdegir> anyone has any questions or comments regarding bifrost? 14:16:38 <fdegir> or wants to add anything? 14:17:06 <fdegir> I take this as no 14:17:17 <Julien-zte> for now, just some time issues 14:17:19 <Julien-zte> some tiny issues 14:17:32 <fdegir> Julien-zte: you now where we live :) 14:17:36 <fdegir> know^ 14:17:53 <fdegir> the other thing I want to highlight is our contributions to OpenStack 14:18:02 <fdegir> people generally talk about features to openstack 14:18:12 <fdegir> but I believe contributions to infra is important 14:18:31 <fdegir> so I will monitor and collect the contributions and link them here 14:18:46 <fdegir> #info bifrost contributions since 2016-06-01 14:18:47 <Julien-zte> agree, openstack infra has been evoluting, no stop 14:18:47 <fdegir> #link http://bit.ly/2c4tgC7 14:19:17 <fdegir> moving on to puppet-infracloud 14:19:31 <fdegir> #topic Progress: puppet-infracloud 14:19:40 <fdegir> the story is similar for puppet-infracloud as well 14:19:44 <fdegir> #info We have been evaluating/trying out bifrost and fixing what we think could be fixed 14:19:48 <fdegir> #undo 14:19:48 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x2c6d490> 14:19:56 <fdegir> #info We have been evaluating/trying out puppet-infracloud and fixing what we think could be fixed 14:20:23 <fdegir> #info The main pain with the puppet-infracloud is that the scripts they have for trying stuff have many hardcoded values in them 14:20:44 <fdegir> #info Yolanda spent quite some time to get rid of them to make it work for OPNFV 14:21:20 <fdegir> #info Based on her work, we can now install OpenStack on 2 node setup from stable/mitaka on Ubuntu 14:21:43 <jmorgan1> this is all manual (via script) right? 14:21:50 <Julien-zte> it's great 14:22:07 <fdegir> jmorgan1: the provisining using bifrost part is via script/totally automated 14:22:16 <fdegir> jmorgan1: but OpenStack installation lacks the full automation 14:22:45 <fdegir> jmorgan1: I can perhaps info this in 14:22:54 <jmorgan1> ok, thanks 14:23:09 <fdegir> #info Putting bifrost and puppet-infracloud together: How to provision VMs and install openstack 14:23:37 <fdegir> #info Provision VMs using bifrost (via script: https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/infra/infra-provision.sh) 14:23:54 <fdegir> #info Once the 3 VMs are provisioned, login to controller and do puppet apply 14:24:02 <fdegir> #info And then to compute and do puppet apply 14:24:05 <hwoarang> is this what openstack upstream does? 14:24:17 <fdegir> hwoarang: yes 14:24:20 <hwoarang> ok 14:24:27 <fdegir> hwoarang: or at least how they explain it in their examples 14:24:40 <fdegir> hwoarang: they probably have more automation but we haven't come that far yet 14:25:03 <hwoarang> right 14:25:11 <fdegir> they call this like "ansible driving puppet driving ansible" 14:25:14 <fdegir> or vice versa 14:25:25 <hwoarang> it's certainly not pretty :) 14:25:42 <fdegir> I see you make things pretty pretty 14:25:45 <fdegir> so :) 14:26:00 <fdegir> #info Once puppet apply is done on controller and compute 14:26:10 <fdegir> #info The installation can be verified from jumphost 14:26:30 <fdegir> #info The puppet-infracloud process for OPNFV has been documented on Gerrit Releng 14:27:08 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=prototypes/puppet-infracloud/README.md 14:27:22 <fdegir> there is a problem there which I intend to send a patch for 14:27:29 <fdegir> but if anyone does not want to wait 14:28:07 <fdegir> #info Fix puppet apply --manifests/site.pp --modulepath=/etc/puppet/modules:/opt/releng/prototypes/puppet-infracloud/modules 14:28:14 <fdegir> #info puppet apply manifests/site.pp --modulepath=/etc/puppet/modules:/opt/releng/prototypes/puppet-infracloud/modules 14:28:39 <fdegir> so we have this part also almost working but automation is missing 14:28:55 <fdegir> need to do some ansible stuff to automate puppet apply and then smoke test 14:29:12 <fdegir> anyone has anything to add? 14:29:19 <fdegir> or have you reached to this point? 14:29:26 <hwoarang> i haven't 14:29:41 <Julien-zte> just something strange for ansible + puppet :) 14:30:03 <fdegir> :) 14:30:30 <Julien-zte> shall we considering container openstack deployment? 14:30:38 <fdegir> one thing to add, if you create servers on this deployment using cirros, you might experience issues 14:30:47 <fdegir> Julien-zte: maybe in 2018 :) 14:31:06 <hwoarang> fdegir: what kind of issues? 14:31:11 <fdegir> Julien-zte: jokes aside, if we stay aligned with the upstream 14:31:21 <fdegir> Julien-zte: and if they do stuff with kolla or whatever 14:31:27 <fdegir> Julien-zte: we should be able to consume that as well 14:31:32 <fdegir> (they=openstack infra) 14:31:45 <fdegir> hwoarang: you can't ping or ssh to those cirros vms 14:31:59 <fdegir> hwoarang: according to Yolanda, xenial works fine 14:32:00 <Julien-zte> if Kolla team can contribute it will be better 14:32:10 <fdegir> after adjusting security groups 14:32:19 <hwoarang> i see 14:32:32 <fdegir> I tried but failed, probably wrong settings with security groups 14:32:42 <fdegir> Julien-zte: +1 14:33:09 <Julien-zte> in the list you provided, some of issues we found have been resolved! good work! 14:33:22 <fdegir> #info I did the same thing with puppet-infracloud and here is the link to contributions since 2016-06-01 14:33:30 <fdegir> #link http://bit.ly/2bRJWQ5 14:33:51 <fdegir> #info Some might not be for OPNFV so it is up to you to read them 14:34:04 <fdegir> we perhaps need to agree on how to track our contributions to OpenStack Infra as well 14:34:19 <fdegir> ChrisPriceAB: any hints? 14:34:58 <ChrisPriceAB> one thing to do ouwld be to gat the commit with "opnfv" or something 14:35:11 <jmorgan1> fdegir: should we get invloved with OpenStack Infra? 14:35:24 <fdegir> jmorgan1: we're already involved in 14:35:33 <ChrisPriceAB> maybe "releng" or something you specifically want to use. we can then track directly back to this initiative. 14:35:36 <Julien-zte> too much project in openstack infra 14:35:50 <fdegir> jmorgan1: Yolanda is core in puppet-infracloud I think 14:35:57 <fdegir> and some more perhaps 14:36:29 <fdegir> ChrisPriceAB: we can check with Yolanda to see how would they react if we put opnfv into our commit messages 14:36:35 <fdegir> or how else we can track them 14:36:59 <fdegir> so, this is all good and we have some progress definitely 14:37:07 <fdegir> but we want to go to baremetal obviously 14:37:07 <ChrisPriceAB> I suggest something for releng specifically. then you can map it back directly. But yeah ask Yolanda how best to do it 14:37:11 <Julien-zte> fdegir, can the search string support in" commit message" 14:37:26 <fdegir> ChrisPriceAB: will do, thx 14:37:49 <fdegir> #info We started with VMs since it is easier to work with them - at least for me personally 14:37:53 <Julien-zte> in the cli, I can not implement this requirement. if supported, we can add opnfv community in the commit message 14:37:59 <fdegir> #info But we will soon go to baremetal 14:38:14 <fdegir> #info As noted above, we currently deploy stable/mitaka on Ubuntu 14:38:36 <fdegir> #info Once the Centos support is introduced and we can deploy from latest, work to deploy on baremetal will start 14:38:46 <fdegir> in a week or two 14:39:11 <fdegir> and the aim is to have this up and running on OPNFV Jenkins/Pharos by OpenStack Summit 14:39:53 <fdegir> afterwards, the tricky parts will come in; HA support and OPNFV Networking 14:40:22 <ChrisPriceAB> But once you get that far you can enable the community to do that part 14:40:39 <fdegir> ChrisPriceAB: yep 14:40:48 <fdegir> I think OpenStack Infra is following what we are doing 14:40:57 <fdegir> they mentioned this during their Infra Meeting last week 14:41:06 <fdegir> and they were happy to have customers :) 14:41:14 <jmorgan1> how can we help out? 14:41:34 <jmorgan1> test the code in releng.git? 14:41:43 <jmorgan1> other? 14:41:48 <fdegir> jmorgan1: https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI#OpenStack3rdPartyCI-HighLevelPlan 14:41:56 <fdegir> that's a very high level plan 14:42:10 <fdegir> and we are on step 1.2 14:42:22 <fdegir> and had like 10+ fixes in upstream just for these small steps 14:42:34 <fdegir> my suggestion would be to start playing with bifrost and puppet-infracloud 14:42:48 <Julien-zte> maybe more in the later 14:42:49 <fdegir> and get things work - either yourself or using the scripts we have in releng repo 14:42:59 <jmorgan1> i have more hardware resources, i'll start taking a look 14:43:06 <fdegir> once everyone is comfortable with these tools 14:43:15 <fdegir> we can have better planning and work distribution 14:43:33 <fdegir> but until that happens, it is kind of get something work some way 14:43:41 <Julien-zte> +1 14:43:47 <jmorgan1> who will be at openstack summit? 14:43:58 <Julien-zte> I plan to go 14:44:02 <fdegir> me as well 14:44:03 <hwoarang> i may be 14:44:07 <ChrisPriceAB> me 14:44:11 <fdegir> Yolanda will be there too 14:44:13 <fdegir> and Aric 14:44:21 <jmorgan1> i'm tbd at the moment 14:44:30 <fdegir> and we want to sit down with the OpenStack Infra guys 14:44:38 <fdegir> and it is important to have real stuff to talk about 14:44:42 <fdegir> rather than wiki pages or slides 14:44:43 <jmorgan1> right 14:44:48 <ChrisPriceAB> I'll buy you a cervesa if you show up jack 14:45:04 <fdegir> I'll forget about the etherpad if you show up jack 14:45:10 <ChrisPriceAB> lol 14:45:28 <fdegir> last topic of the day 14:45:34 <fdegir> #topic Infra/HW Needs for OpenStack 3rd Party CI 14:46:00 <fdegir> #info Our aim is not to fork anything but certain things can't be upstreamed either 14:46:10 <fdegir> #info So we decided to keep them in OPNFV releng repo 14:46:33 <fdegir> #info And have a basic flow regarding how to combine upstream with releng: cp -R 14:46:42 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=prototypes 14:46:56 <fdegir> #info As you see, we have 2 folders there; bifrost and puppet-infracloud 14:47:17 <Julien-zte> hope remaining in releng is the use case, and most of them will be upstreamed 14:47:19 <fdegir> #info Each of these directories follow same directory structure as upstream so we can add new files or overwrite existing files as needed 14:47:36 <ChrisPriceAB> #info does that process allow OPNFV projects to add things easily? Can SDNVPN for instance add ODL with a specific config? 14:47:54 <ChrisPriceAB> #info is that a TBD item, or do they create a structue in their repo? 14:47:55 <fdegir> ChrisPriceAB: this is pretty basic at the moment 14:48:10 <fdegir> #info TBD item 14:48:15 <ChrisPriceAB> ok cool 14:48:32 <fdegir> #info The current structure just allows us to move on with not much hassles 14:48:43 <ChrisPriceAB> :D sounds wise 14:48:46 <fdegir> #info But needs to be improved definitely when things become serious 14:48:54 <fdegir> with some help from git gurus in community 14:49:13 <fdegir> #info We also have some hardware to use for development work 14:49:34 <fdegir> #info We currently have 4 machines in Intel POD4 to use for development purposes using VMs 14:49:55 <fdegir> #info hwoarang submitted a jira ticket to Pharos/Jack to get access to Intel POD4 and he will get pod4-node2 14:50:07 <Julien-zte> I have blades in ZTE for development 14:50:08 <fdegir> #info Yolanda uses pod4-jumphost and me pod4-node1 14:50:17 <fdegir> Julien-zte: ping us when you need machine 14:50:28 <Julien-zte> OK 14:50:29 <jmorgan1> well, i'd like to join releng project offically then 14:50:43 <fdegir> #info We also will hopefully get a full POD for development on baremetal 14:51:03 <fdegir> #info LF POD4 or POD5 - to be discussed during Infra WG meeting after this one 14:51:06 <jmorgan1> i'll have more hardware resources in Intel lab after lab move (post colorado 1.0 release) 14:51:12 <fdegir> jmorgan1: you are already part of releng 14:51:40 <fdegir> jmorgan1: if you contribute, you can perhaps become a committer one day 14:51:48 <fdegir> starting with etherpad perhaps :P 14:51:48 <jmorgan1> fdegir: ok, got it 14:51:49 <Julien-zte> :) 14:52:05 <fdegir> #topic AoB 14:52:14 <fdegir> anyone wants to say anything? 14:52:16 <fdegir> ask anything? 14:52:20 <ChrisPriceAB> plans for etherpad training? 14:52:21 <fdegir> complain about stuff? 14:52:35 <fdegir> I think we have some material on wiki 14:52:39 <Julien-zte> I will ask for help when some issue can not be resolved 14:52:40 <fdegir> for etherpad usage 14:52:59 <jmorgan1> what training do you need for etherpad? 14:53:00 <ChrisPriceAB> Can I complain about the infra WG meeting here? 14:53:04 <jmorgan1> create page, edit ;) 14:53:06 <Julien-zte> I'm working on ipxe 14:53:16 <jmorgan1> ChrisPriceAB: off topic ;) 14:53:17 <fdegir> ChrisPriceAB: nope 14:53:36 <fdegir> ChrisPriceAB: I'm the complainer 14:53:53 <jmorgan1> anyway, this looks like a lot of fun and i'm excited to work on it 14:54:03 <jmorgan1> thanks for driving it fdegir 14:54:06 <fdegir> #info Finally, I want to thank everyone again to see this happening! 14:54:07 <ChrisPriceAB> I need training for everything. 14:54:32 <hwoarang> thank you fdegir 14:54:34 <ChrisPriceAB> thanks fdegir 14:54:37 <fdegir> thanks for the day and talk to you all in 5 minutes 14:54:39 <Julien-zte> ChrisPriceAB, agree 14:54:41 <fdegir> #endmeeting