14:01:02 <fdegir> #startmeeting OpenStack 3rd Party CI
14:01:02 <collabot> Meeting started Wed Aug 31 14:01:02 2016 UTC.  The chair is fdegir. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:02 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:01:02 <collabot> The meeting name has been set to 'openstack_3rd_party_ci'
14:01:18 <fdegir> #chair jmorgan1 Julien-zte hwoarang
14:01:18 <collabot> Current chairs: Julien-zte fdegir hwoarang jmorgan1
14:01:27 <fdegir> #topic Roll Call
14:01:32 <Julien-zte> #info Julien
14:01:36 <ChrisPriceAB> #info Chris Price
14:01:42 <fdegir> #chair ChrisPriceAB
14:01:42 <collabot> Current chairs: ChrisPriceAB Julien-zte fdegir hwoarang jmorgan1
14:01:47 <ChrisPriceAB> noooooo
14:01:49 <ChrisPriceAB> :)
14:01:50 <hwoarang> #info Markos
14:01:50 <fdegir> :)
14:01:56 <jmorgan1> #info Jack Morgan
14:01:56 <fdegir> I'm new in this so...
14:02:13 <fdegir> Yolanda is off this week so we can start
14:02:18 <fdegir> here is the agenda
14:02:19 <fdegir> #topic Agenda
14:02:19 <Julien-zte> ok
14:02:24 <fdegir> #info Welcome
14:02:28 <fdegir> #info Background Info
14:02:32 <fdegir> #info Progress: bifrost
14:02:38 <fdegir> #info Progress: puppet-infracloud
14:02:41 <fdegir> #info Infra/HW Needs
14:02:45 <fdegir> #info AOB
14:02:50 <Julien-zte> #link http://docs.openstack.org/developer/bifrost/readme.html
14:03:03 <fdegir> if anyone wants to add anything to the agenda, please feel free to do so
14:03:05 <Julien-zte> #link https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI
14:03:12 <fdegir> Julien-zte: thx
14:03:15 <fdegir> #topic Welcome
14:03:29 <fdegir> #info I want to welcome everyone and thanks for joining this effort!
14:03:46 <fdegir> and I'm really happy to see we have made a direct impact
14:03:49 <fdegir> :)
14:04:04 <fdegir> there have been some contributions to upstream already
14:04:13 <Julien-zte> thanks fgedir, great job has been done
14:04:14 <fdegir> which I'll paste the link when the topic comes there
14:04:29 <fdegir> this is really encouriging
14:04:42 <fdegir> so, thx again and ChrisPriceAB expects more!
14:04:50 <fdegir> #topic Background Info
14:05:04 <fdegir> #info There might be questions regarding why we are working on this
14:05:25 <fdegir> #info We tried to give a summary regarding this activity on Wiki so please feel free to read, comment, update
14:05:35 <fdegir> #link https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI
14:05:56 <fdegir> #info The primary tools we are looking into are bifrost and puppet-infracloud
14:06:00 <fdegir> #link https://github.com/openstack/bifrost
14:06:04 <fdegir> #link http://git.openstack.org/cgit/openstack-infra/puppet-infracloud
14:06:31 <fdegir> and we can move to real stuff now...
14:06:41 <fdegir> #topic Progress: bifrost
14:07:01 <fdegir> #info We have been evaluating/trying out bifrost and fixing what we think could be fixed/improved
14:07:25 <fdegir> #info Yolanda and Markos have been pretty active with locating and fixing nasty bugs
14:07:34 <fdegir> the rabbitmq stuff for example
14:07:54 <ChrisPriceAB> #info the objective of this activity is to make sure we can use bifrost to deploy an OPNFV scenario on a "pharos" lab
14:08:22 <ChrisPriceAB> #info using openstack native CI tools
14:08:34 <fdegir> and from master!
14:08:46 <ChrisPriceAB> ack
14:08:51 <fdegir> who likes months old sw
14:09:17 <Julien-zte> yes, from the master branch
14:09:32 <Julien-zte> good point
14:09:43 <fdegir> #info We can now provision 3 VMs using bifrost
14:09:57 <fdegir> #info Ubuntu as OS and the 3 VMs serve as jumphost, controller, and compute
14:10:06 <Julien-zte> I have not finished this yet :(
14:10:08 <fdegir> #info Work to introduce Centos and SuSe support is ongoing
14:10:22 <fdegir> hwoarang: would you like to add anything regarding SuSe support
14:10:26 <jmorgan1> which SuSE versions?
14:10:31 <fdegir> hwoarang: or any other item?
14:10:57 <hwoarang> fdegir: it's still work in progress. we are missing the ipxe package which i need to work on. but it's pretty close after that
14:11:05 <Julien-zte> many a newest LTS version
14:11:13 <hwoarang> jmorgan1: openSUSE Tumbleweed at first. then Leap. then SLES
14:11:23 <Julien-zte> many: maybe
14:11:26 <fdegir> #info Introducing SuSe support is work in progress. we are missing the ipxe package which i need to work on. but it's pretty close after that
14:11:38 <fdegir> #info openSUSE Tumbleweed at first. then Leap. then SLES
14:11:39 <jmorgan1> hwoarang: ok
14:12:23 <fdegir> #info The work we've done with bifrost is put on OPNFV Jenkins and we are provisioning VMs over and over automatically
14:12:29 <fdegir> #link https://build.opnfv.org/ci/job/infra-provision-virtual-daily-master/lastBuild/consoleFull
14:12:39 <Julien-zte> my question about bifrost is time cost to run once.
14:12:47 <fdegir> if anyone wants to try stuff, you can take a look at the script from releng repo
14:12:53 <Julien-zte> do we have some time estimation?
14:13:21 <fdegir> Julien-zte: you mean how much time it takes to provision?
14:13:25 <Julien-zte> yes
14:13:38 <fdegir> Julien-zte: https://build.opnfv.org/ci/job/infra-provision-virtual-daily-master/buildTimeTrend
14:13:47 <fdegir> but the about 45 minutes
14:13:51 <fdegir> but please note that
14:13:59 <fdegir> the time you see is everything from scratch
14:14:09 <fdegir> installation of complete toolchain (ansible and whatnot)
14:14:13 <fdegir> building diskimages
14:14:17 <fdegir> and so on
14:14:36 <fdegir> and when we do this for real, we will probably skip these and cut the time
14:14:37 <Julien-zte> good for in a hour
14:14:58 <fdegir> I expect 30 minutes or less for provisioning VMs
14:15:08 <fdegir> baremetal will probably differ
14:15:50 <fdegir> #info This is the script that is run by Jenkins
14:15:52 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/infra/infra-provision.sh
14:16:29 <Julien-zte> thanks fdegir, will check for this
14:16:31 <fdegir> anyone has any questions or comments regarding bifrost?
14:16:38 <fdegir> or wants to add anything?
14:17:06 <fdegir> I take this as no
14:17:17 <Julien-zte> for now, just some time issues
14:17:19 <Julien-zte> some tiny issues
14:17:32 <fdegir> Julien-zte: you now where we live :)
14:17:36 <fdegir> know^
14:17:53 <fdegir> the other thing I want to highlight is our contributions to OpenStack
14:18:02 <fdegir> people generally talk about features to openstack
14:18:12 <fdegir> but I believe contributions to infra is important
14:18:31 <fdegir> so I will monitor and collect the contributions and link them here
14:18:46 <fdegir> #info bifrost contributions since 2016-06-01
14:18:47 <Julien-zte> agree, openstack infra has been evoluting, no stop
14:18:47 <fdegir> #link http://bit.ly/2c4tgC7
14:19:17 <fdegir> moving on to puppet-infracloud
14:19:31 <fdegir> #topic Progress: puppet-infracloud
14:19:40 <fdegir> the story is similar for puppet-infracloud as well
14:19:44 <fdegir> #info We have been evaluating/trying out bifrost and fixing what we think could be fixed
14:19:48 <fdegir> #undo
14:19:48 <collabot> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x2c6d490>
14:19:56 <fdegir> #info We have been evaluating/trying out puppet-infracloud and fixing what we think could be fixed
14:20:23 <fdegir> #info The main pain with the puppet-infracloud is that the scripts they have for trying stuff have many hardcoded values in them
14:20:44 <fdegir> #info Yolanda spent quite some time to get rid of them to make it work for OPNFV
14:21:20 <fdegir> #info Based on her work, we can now install OpenStack on 2 node setup from stable/mitaka on Ubuntu
14:21:43 <jmorgan1> this is all manual (via script) right?
14:21:50 <Julien-zte> it's great
14:22:07 <fdegir> jmorgan1: the provisining using bifrost part is via script/totally automated
14:22:16 <fdegir> jmorgan1: but OpenStack installation lacks the full automation
14:22:45 <fdegir> jmorgan1: I can perhaps info this in
14:22:54 <jmorgan1> ok, thanks
14:23:09 <fdegir> #info Putting bifrost and puppet-infracloud together: How to provision VMs and install openstack
14:23:37 <fdegir> #info Provision VMs using bifrost (via script: https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/infra/infra-provision.sh)
14:23:54 <fdegir> #info Once the 3 VMs are provisioned, login to controller and do puppet apply
14:24:02 <fdegir> #info And then to compute and do puppet apply
14:24:05 <hwoarang> is this what openstack upstream does?
14:24:17 <fdegir> hwoarang: yes
14:24:20 <hwoarang> ok
14:24:27 <fdegir> hwoarang: or at least how they explain it in their examples
14:24:40 <fdegir> hwoarang: they probably have more automation but we haven't come that far yet
14:25:03 <hwoarang> right
14:25:11 <fdegir> they call this like "ansible driving puppet driving ansible"
14:25:14 <fdegir> or vice versa
14:25:25 <hwoarang> it's certainly not pretty :)
14:25:42 <fdegir> I see you make things pretty pretty
14:25:45 <fdegir> so :)
14:26:00 <fdegir> #info Once puppet apply is done on controller and compute
14:26:10 <fdegir> #info The installation can be verified from jumphost
14:26:30 <fdegir> #info The puppet-infracloud process for OPNFV has been documented on Gerrit Releng
14:27:08 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=prototypes/puppet-infracloud/README.md
14:27:22 <fdegir> there is a problem there which I intend to send a patch for
14:27:29 <fdegir> but if anyone does not want to wait
14:28:07 <fdegir> #info Fix puppet apply --manifests/site.pp --modulepath=/etc/puppet/modules:/opt/releng/prototypes/puppet-infracloud/modules
14:28:14 <fdegir> #info puppet apply manifests/site.pp --modulepath=/etc/puppet/modules:/opt/releng/prototypes/puppet-infracloud/modules
14:28:39 <fdegir> so we have this part also almost working but automation is missing
14:28:55 <fdegir> need to do some ansible stuff to automate puppet apply and then smoke test
14:29:12 <fdegir> anyone has anything to add?
14:29:19 <fdegir> or have you reached to this point?
14:29:26 <hwoarang> i haven't
14:29:41 <Julien-zte> just something strange for ansible + puppet :)
14:30:03 <fdegir> :)
14:30:30 <Julien-zte> shall we considering container openstack deployment?
14:30:38 <fdegir> one thing to add, if you create servers on this deployment using cirros, you might experience issues
14:30:47 <fdegir> Julien-zte: maybe in 2018 :)
14:31:06 <hwoarang> fdegir: what kind of issues?
14:31:11 <fdegir> Julien-zte: jokes aside, if we stay aligned with the upstream
14:31:21 <fdegir> Julien-zte: and if they do stuff with kolla or whatever
14:31:27 <fdegir> Julien-zte: we should be able to consume that as well
14:31:32 <fdegir> (they=openstack infra)
14:31:45 <fdegir> hwoarang: you can't ping or ssh to those cirros vms
14:31:59 <fdegir> hwoarang: according to Yolanda, xenial works fine
14:32:00 <Julien-zte> if Kolla team can contribute it will be better
14:32:10 <fdegir> after adjusting security groups
14:32:19 <hwoarang> i see
14:32:32 <fdegir> I tried but failed, probably wrong settings with security groups
14:32:42 <fdegir> Julien-zte: +1
14:33:09 <Julien-zte> in the list you provided, some of issues we found have been resolved! good work!
14:33:22 <fdegir> #info I did the same thing with puppet-infracloud and here is the link to contributions since 2016-06-01
14:33:30 <fdegir> #link http://bit.ly/2bRJWQ5
14:33:51 <fdegir> #info Some might not be for OPNFV so it is up to you to read them
14:34:04 <fdegir> we perhaps need to agree on how to track our contributions to OpenStack Infra as well
14:34:19 <fdegir> ChrisPriceAB: any hints?
14:34:58 <ChrisPriceAB> one thing to do ouwld be to gat the commit with "opnfv" or something
14:35:11 <jmorgan1> fdegir: should we get invloved with OpenStack Infra?
14:35:24 <fdegir> jmorgan1: we're already involved in
14:35:33 <ChrisPriceAB> maybe "releng" or something you specifically want to use.  we can then track directly back to this initiative.
14:35:36 <Julien-zte> too much project in openstack infra
14:35:50 <fdegir> jmorgan1: Yolanda is core in puppet-infracloud I think
14:35:57 <fdegir> and some more perhaps
14:36:29 <fdegir> ChrisPriceAB: we can check with Yolanda to see how would they react if we put opnfv into our commit messages
14:36:35 <fdegir> or how else we can track them
14:36:59 <fdegir> so, this is all good and we have some progress definitely
14:37:07 <fdegir> but we want to go to baremetal obviously
14:37:07 <ChrisPriceAB> I suggest something for releng specifically.  then you can map it back directly.  But yeah ask Yolanda how best to do it
14:37:11 <Julien-zte> fdegir, can the search string support in" commit message"
14:37:26 <fdegir> ChrisPriceAB: will do, thx
14:37:49 <fdegir> #info We started with VMs since it is easier to work with them - at least for me personally
14:37:53 <Julien-zte> in the cli, I can not implement this requirement. if supported, we can add opnfv community in the commit message
14:37:59 <fdegir> #info But we will soon go to baremetal
14:38:14 <fdegir> #info As noted above, we currently deploy stable/mitaka on Ubuntu
14:38:36 <fdegir> #info Once the Centos support is introduced and we can deploy from latest, work to deploy on baremetal will start
14:38:46 <fdegir> in a week or two
14:39:11 <fdegir> and the aim is to have this up and running on OPNFV Jenkins/Pharos by OpenStack Summit
14:39:53 <fdegir> afterwards, the tricky parts will come in; HA support and OPNFV Networking
14:40:22 <ChrisPriceAB> But once you get that far you can enable the community to do that part
14:40:39 <fdegir> ChrisPriceAB: yep
14:40:48 <fdegir> I think OpenStack Infra is following what we are doing
14:40:57 <fdegir> they mentioned this during their Infra Meeting last week
14:41:06 <fdegir> and they were happy to have customers :)
14:41:14 <jmorgan1> how can we help out?
14:41:34 <jmorgan1> test the code in releng.git?
14:41:43 <jmorgan1> other?
14:41:48 <fdegir> jmorgan1: https://wiki.opnfv.org/display/INF/OpenStack+3rd+Party+CI#OpenStack3rdPartyCI-HighLevelPlan
14:41:56 <fdegir> that's a very high level plan
14:42:10 <fdegir> and we are on step 1.2
14:42:22 <fdegir> and had like 10+ fixes in upstream just for these small steps
14:42:34 <fdegir> my suggestion would be to start playing with bifrost and puppet-infracloud
14:42:48 <Julien-zte> maybe more in the later
14:42:49 <fdegir> and get things work - either yourself or using the scripts we have in releng repo
14:42:59 <jmorgan1> i have more hardware resources, i'll start taking a look
14:43:06 <fdegir> once everyone is comfortable with these tools
14:43:15 <fdegir> we can have better planning and work distribution
14:43:33 <fdegir> but until that happens, it is kind of get something work some way
14:43:41 <Julien-zte> +1
14:43:47 <jmorgan1> who will be at openstack summit?
14:43:58 <Julien-zte> I plan to go
14:44:02 <fdegir> me as well
14:44:03 <hwoarang> i may be
14:44:07 <ChrisPriceAB> me
14:44:11 <fdegir> Yolanda will be there too
14:44:13 <fdegir> and Aric
14:44:21 <jmorgan1> i'm tbd at the moment
14:44:30 <fdegir> and we want to sit down with the OpenStack Infra guys
14:44:38 <fdegir> and it is important to have real stuff to talk about
14:44:42 <fdegir> rather than wiki pages or slides
14:44:43 <jmorgan1> right
14:44:48 <ChrisPriceAB> I'll buy you a cervesa if you show up jack
14:45:04 <fdegir> I'll forget about the etherpad if you show up jack
14:45:10 <ChrisPriceAB> lol
14:45:28 <fdegir> last topic of the day
14:45:34 <fdegir> #topic Infra/HW Needs for OpenStack 3rd Party CI
14:46:00 <fdegir> #info Our aim is not to fork anything but certain things can't be upstreamed either
14:46:10 <fdegir> #info So we decided to keep them in OPNFV releng repo
14:46:33 <fdegir> #info And have a basic flow regarding how to combine upstream with releng: cp -R
14:46:42 <fdegir> #link https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=prototypes
14:46:56 <fdegir> #info As you see, we have 2 folders there; bifrost and puppet-infracloud
14:47:17 <Julien-zte> hope remaining in releng is the use case, and most of them will be upstreamed
14:47:19 <fdegir> #info Each of these directories follow same directory structure as upstream so we can add new files or overwrite existing files as needed
14:47:36 <ChrisPriceAB> #info does that process allow OPNFV projects to add things easily?  Can SDNVPN for instance add ODL with a specific config?
14:47:54 <ChrisPriceAB> #info is that a TBD item, or do they create a structue in their repo?
14:47:55 <fdegir> ChrisPriceAB: this is pretty basic at the moment
14:48:10 <fdegir> #info TBD item
14:48:15 <ChrisPriceAB> ok cool
14:48:32 <fdegir> #info The current structure just allows us to move on with not much hassles
14:48:43 <ChrisPriceAB> :D  sounds wise
14:48:46 <fdegir> #info But needs to be improved definitely when things become serious
14:48:54 <fdegir> with some help from git gurus in community
14:49:13 <fdegir> #info We also have some hardware to use for development work
14:49:34 <fdegir> #info We currently have 4 machines in Intel POD4 to use for development purposes using VMs
14:49:55 <fdegir> #info hwoarang submitted a jira ticket to Pharos/Jack to get access to Intel POD4 and he will get pod4-node2
14:50:07 <Julien-zte> I have blades in ZTE for development
14:50:08 <fdegir> #info Yolanda uses pod4-jumphost and me pod4-node1
14:50:17 <fdegir> Julien-zte: ping us when you need machine
14:50:28 <Julien-zte> OK
14:50:29 <jmorgan1> well, i'd like to join releng project offically then
14:50:43 <fdegir> #info We also will hopefully get a full POD for development on baremetal
14:51:03 <fdegir> #info LF POD4 or POD5 - to be discussed during Infra WG meeting after this one
14:51:06 <jmorgan1> i'll have more hardware resources in Intel lab after lab move (post colorado 1.0 release)
14:51:12 <fdegir> jmorgan1: you are already part of releng
14:51:40 <fdegir> jmorgan1: if you contribute, you can perhaps become a committer one day
14:51:48 <fdegir> starting with etherpad perhaps :P
14:51:48 <jmorgan1> fdegir: ok, got it
14:51:49 <Julien-zte> :)
14:52:05 <fdegir> #topic AoB
14:52:14 <fdegir> anyone wants to say anything?
14:52:16 <fdegir> ask anything?
14:52:20 <ChrisPriceAB> plans for etherpad training?
14:52:21 <fdegir> complain about stuff?
14:52:35 <fdegir> I think we have some material on wiki
14:52:39 <Julien-zte> I will ask for help when some issue can not be resolved
14:52:40 <fdegir> for etherpad usage
14:52:59 <jmorgan1> what training do you need for etherpad?
14:53:00 <ChrisPriceAB> Can I complain about the infra WG meeting here?
14:53:04 <jmorgan1> create page, edit ;)
14:53:06 <Julien-zte> I'm working on ipxe
14:53:16 <jmorgan1> ChrisPriceAB: off topic ;)
14:53:17 <fdegir> ChrisPriceAB: nope
14:53:36 <fdegir> ChrisPriceAB: I'm the complainer
14:53:53 <jmorgan1> anyway, this looks like a lot of fun and i'm excited to work on it
14:54:03 <jmorgan1> thanks for driving it fdegir
14:54:06 <fdegir> #info Finally, I want to thank everyone again to see this happening!
14:54:07 <ChrisPriceAB> I need training for everything.
14:54:32 <hwoarang> thank you fdegir
14:54:34 <ChrisPriceAB> thanks fdegir
14:54:37 <fdegir> thanks for the day and talk to you all in 5 minutes
14:54:39 <Julien-zte> ChrisPriceAB, agree
14:54:41 <fdegir> #endmeeting