08:01:16 <ollivier> #startmeeting Functest weekly meeting 29 Jan. 2019
08:01:16 <collabot`> Meeting started Tue Jan 29 08:01:16 2019 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:01:16 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:01:16 <collabot`> The meeting name has been set to 'functest_weekly_meeting_29_jan__2019'
08:01:25 <ollivier> #info Cédric Ollivier
08:01:35 <juhak> #info Juha Kosonen
08:01:39 <viktor_t> #info Viktor Tikkanen
08:02:48 <ollivier> #topic Action point follow-up
08:03:11 <ollivier> #info Gnocchi is now optional in Rally
08:03:22 <ollivier> #link https://gerrit.opnfv.org/gerrit/66653
08:03:57 <ollivier> juhak: thank you. It was right on time as Gnocchi is not so well integrated by devstack.
08:04:35 <ollivier> #info lots of Snaps subtests have been removed but it could now verify latest OpenStack
08:04:53 <ollivier> #info it's mostly unmaintained and it can't be updated in our containers (git commit id)
08:05:04 <ollivier> Steve refused to update requirements due to magnum support (Newton)
08:05:13 <ollivier> Snaps package has been renamed which may increase the work
08:06:11 <ollivier> few ressources may be uncleant after Snaps testcases (https://gerrit.opnfv.org/gerrit/#/c/66751/)
08:07:08 <ollivier> Do we enable snaps testcases or do we remove it? We can only remove the test...
08:07:31 <ollivier> We can only remov the test if an error is detected.
08:09:00 <juhak> In that case removing is the only viable option, I suppose?
08:11:01 <ollivier> yes or we do maintain by ourselves and update the magnum requirements. Hopefully upper-constraints force our versions. Normally we would have taken that decision for an external project.
08:11:46 <juhak> ok, I see
08:12:16 <ollivier> If we remove api_check, it would be put on the agenda of the TSC even if it's clear from a technical point of view
08:12:28 <ollivier> but I can deal with that :)
08:12:41 <CristinaPauna> #info Cristina Pauna
08:13:01 <ollivier> Hello Cristina.
08:13:08 <CristinaPauna> Hi guys
08:13:31 <CristinaPauna> I missed the first part of the discussion
08:14:00 <CristinaPauna> The conclusion with snaps is that it has to be removed?
08:14:14 <ollivier> I will send an overall email but yes I consider the decision is up to us.
08:15:00 <ollivier> We remove 1/3 of the test and to allow running it vs latest OpenStack. Snaps can't be updated (see 2 * -1 when updating magnum from Newton)
08:15:10 <CristinaPauna> I know
08:15:36 <CristinaPauna> do we have the option to patch snaps on the flight before using it?
08:15:47 <CristinaPauna> or you think it's not worth the bother?
08:16:13 <ollivier> Why? In such a case, we can directly maintain snaps.
08:17:05 <ollivier> I will broadcast an email. As soon as we detect an issue, we have to remove the related Class.
08:17:25 <CristinaPauna> I was under the impression that Steven doesn't want us to maintain snaps... that's why he didn't merge your patch, right?
08:18:20 <ollivier> because snaps is leveraging on magnum Newton. My patch would have broken magnum testing. As it was unmaintained, I thought he would accept.
08:18:37 <ollivier> Else our patches are always quickly merged.
08:18:49 <CristinaPauna> ok, but he said that we can drop magnum altoghether...
08:19:05 <ollivier> IT's much more complex.
08:19:31 <ollivier> magnum is part of snaps code and then it's defined in requirements whatever we call the classes or not
08:20:06 <ollivier> He asked us to maintain the magnum part.
08:20:18 <CristinaPauna> I see. So what do you suggest to do next?
08:21:45 <ollivier> It's not a big deal regarding the whole Functest scope. I don't like removing test (so bad) without being able to update. I will broadcast the issues.
08:21:54 <ollivier> Snaps will be removed. The question is when.
08:22:01 <CristinaPauna> ok
08:22:31 <CristinaPauna> we're early in the process for Hunter so I think it's not a problem to officially remove it now
08:22:38 <CristinaPauna> It's not ran anyway
08:23:15 <ollivier> The issue will be more or less political. As soon as healthcheck is changed, we do go to the TSC. Hopefully as scope is reduced, it shouldn't be a big deal.
08:23:26 <ollivier> for installers
08:23:44 <ollivier> #topic OpenStack Stein (functest-virtual is hosting an OS all-in-one master)
08:24:05 <ollivier> #info juju_epc and tempest_full pass vs OpenStack master now
08:24:21 <ollivier> #link https://gerrit.opnfv.org/gerrit/66705
08:24:35 <ollivier> #link https://gerrit.opnfv.org/gerrit/66687
08:24:48 <ollivier> #link https://gerrit.opnfv.org/gerrit/66663
08:25:12 <ollivier> #info rally_full was failing due to the default volume sizes
08:26:23 <ollivier> I think we could add rally_full in all Functest gates. The jjb ends in 4 hours in functest-virtual. No real reason to skip such a relevant test, isn't it?
08:26:48 <ollivier> #link https://build.opnfv.org/ci/view/functest/job/functest-latest-gate/29/
08:27:30 <CristinaPauna> this is the gate for verifying functest patches?
08:27:41 <ollivier> #info only SIP testing is incomplete but heat_ims is mostly considered as successful
08:27:59 <ollivier> yes. It simply tests rally patches before it were merged
08:28:27 <ollivier> That's CI/CD
08:28:37 <CristinaPauna> if you are ok with the extra time added, I don't see a reason not to add it
08:29:13 <ollivier> It will increase the duration for all installers. It's already in our gates
08:29:54 <CristinaPauna> ok, I'm not sure I follow, you say it tests rally patches
08:30:02 <CristinaPauna> installers don't have rally patches
08:30:47 <ollivier> Functest has its own independent gates which allows us verifying our patches before merge (gates). In our gates, rally_full is already included.
08:30:58 <CristinaPauna> ok
08:31:12 <ollivier> We are already verifying OS master.
08:31:34 <ollivier> Fuel and Apex don't run rally full from the time being
08:31:59 <ollivier> I think it's the right time to add that key test in installer gates.
08:32:27 <CristinaPauna> that's when you merge a patch in installers
08:32:46 <ollivier> Daily jobs. Fuel may pass it https://build.opnfv.org/ci/view/functest%20(Installer%20gates)/job/functest-fuel-baremetal-daily-master/659/console
08:33:04 <CristinaPauna> how long does it takes to run it?
08:33:31 <ollivier> 1h30 here.
08:33:52 <ollivier> less installers, less scenarios.. We could add it.
08:34:14 <CristinaPauna> well, I personally think the daily run is already too long, but that's a broader discussion
08:34:32 <CristinaPauna> you can add it for now
08:34:44 <CristinaPauna> until the broader discussion is sorted out
08:34:49 <ollivier> Our full gates run in 4 hours (rally_full included).
08:35:14 <CristinaPauna> by "our" you mean Orange open lab?
08:35:14 <ollivier> The issue is on Releng which doesn't allow testing in // as Functest supports.
08:35:20 <ollivier> Functest
08:35:37 <CristinaPauna> you run it in parrallel jobs, right?
08:35:47 <ollivier> yes. Functest supports it
08:35:51 <CristinaPauna> I know
08:36:05 <ollivier> We are fully redesigning Releng.
08:36:07 <CristinaPauna> that is cool, and I know jenkins is stupid at that
08:36:34 <CristinaPauna> who's we?
08:36:38 <ollivier> Normally Installer should run our new daily job instead of the former bash scripts.
08:37:22 <ollivier> juhak: viktor_t: are you ok to add it in installer gates?
08:37:41 <CristinaPauna> do you also use jenkins in the internal lab, or some other tool?
08:38:20 <juhak> ollivier: I'm ok with that
08:38:50 <viktor_t> ok
08:39:49 <ollivier> Great
08:40:14 <ollivier> Yes I run Functest in internal as well. I will reconnect my bot in few days :)
08:40:51 <CristinaPauna> I was wandering if you use Jenkins for it
08:41:02 <CristinaPauna> as in, can Jenkins be made to run parrallel testing
08:41:07 <ollivier> I made a demo during Hackfest but lots of people from OPNFV were missing. I'm documenting the basics on the wiki
08:41:23 <CristinaPauna> or we need zuul or some other tool
08:41:26 <ollivier> No issues with Jenkins. The issues are in releng
08:41:35 <CristinaPauna> ok, that's good news
08:41:53 <CristinaPauna> you're working with Trevor and Aric to overcome those?
08:42:41 <ollivier> I have explained my points for 1 year. As we have resources, we can easily improve the CI/CD model.
08:43:26 <ollivier> Releng is limiting Functest for a while.
08:43:41 <CristinaPauna> sorry for making you repeating, I am doing my best to catch up
08:44:02 <ollivier> no pb. I may explain deeply out of that meeting. I will send you few links
08:44:10 <CristinaPauna> awsome, thanks a lot
08:44:38 <ollivier> No Xtesting allows building CI/CD toolchains on demand. We reuse that to rewrite the Releng jjbs.
08:44:50 <ollivier> (Functest, Functest K8s, Xtesting...)
08:45:06 <ollivier> http://testresults.opnfv.org/functest/functest2019/
08:45:33 <ollivier> We are defining early gates in new OPNFV model but in fact we are runing everything even if gate job.
08:45:48 <ollivier> #topic OpenStack Stein
08:45:58 <ollivier> #undo
08:45:58 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Topic object at 0x22ae210>
08:46:04 <ollivier> #topic OpenStack Rocky
08:46:24 <ollivier> #info tempest hunter and older are buggy when multiple shared networks (Functest in parallel)
08:47:00 <ollivier> #link https://review.openstack.org/#/c/615425/3 (interfaces_by_fixed_ip)
08:47:29 <ollivier> I have to double check if dhcp testing is safe (one run failed)
08:48:12 <ollivier> I'm not clear yet if we do backport it or update tempest?
08:48:50 <ollivier> It seems Fuel faced with it too https://artifacts.opnfv.org/logs/functest/ericsson-pod1/2019-01-28_05-28-07/tempest_full/tempest-report.html
08:49:42 <ollivier> That's the only issue known in Functest (latest, hunter, gambia). +we may harden heat_ims
08:50:32 <ollivier> I would propose to split our POD to multiple all-in-one to cover all our versions (6). Are you ok?
08:50:47 <juhak> ok
08:50:54 <ollivier> juhak: as far as I understand we are still blocked in the first track.
08:51:25 <juhak> yes, nothing new related to pod5
08:51:40 <ollivier> Ok. thank you for testing the first idea.
08:52:45 <ollivier> It's quite funny because we can't really leverage in backward compatibilities to check all versions. For instance, bad filters are now forbidden by latest OpenStack.
08:53:15 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/66625/
08:54:06 <ollivier> And K8s does too fast to allow verifying the 3 Functest versions in one deployment.
08:54:42 <ollivier> #topic AoB
08:54:47 <ollivier> Anything else?
08:55:14 <ollivier> I would be afk till towmorrow evening. I will fix tempest hunter right after.
08:55:14 <juhak> not from my side
08:55:22 <CristinaPauna> I'm all set
08:55:59 <ollivier> Thank you. Functest is on steroids. Great Job!
08:56:06 <CristinaPauna> :)
08:56:11 <ollivier> #endmeeting