08:00:29 <ollivier> #startmeeting Functest weekly meeting 9 Jan. 2018
08:00:29 <collabot> Meeting started Tue Jan  9 08:00:29 2018 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:29 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:29 <collabot> The meeting name has been set to 'functest_weekly_meeting_9_jan__2018'
08:01:01 <ollivier> #info Cédric
08:01:14 <ollivier> My best wishes for the year!
08:01:22 <juhak> #info Juha Kosonen
08:01:38 <jose_lausuch> #info Jose Lausuch
08:01:39 <LindaWang> #info Linda Wang
08:01:56 <CristinaPauna> #info Cristina Pauna
08:01:56 <LindaWang> ollivier: Happy New Year!
08:02:10 <ollivier> LindaWang: thank you
08:02:11 <jose_lausuch> happy new year to all
08:02:57 <ollivier> lots of actions done during the last 3 weeks. I will try to list them quickly. Please let me know if we must detail some of them.
08:03:03 <ollivier> #info  Action point follow-up
08:03:23 <ollivier> #topic Action point follow-up
08:03:34 <ollivier> #info the 3 new repositories were created (functest-requirements, functest-xtesting and functest-kubernetes)
08:03:43 <ollivier> #info the docker container garbage collector was published  (21/11 AP1) in both master and stable/euphrates
08:03:55 <ollivier> #info release notes were updated to simply list manifests (19/12 AP1) then no need to publish additional tags (19/12 AP2)
08:04:07 <ollivier> #info getting auth token is tested when checking deployment (19/12 AP6)
08:04:19 <ollivier> #info ceilometer tests failing in rally sanity were removed (19/12 AP7)
08:04:27 <ollivier> thank you all for that!
08:05:23 <ollivier> Regarding getting auth token, it may be incomplete to only check OS_AUTH_URL. Fuel is buggy (wrong public identity endpoint) and we couldn't detect it.
08:06:08 <ollivier> #topic Euphrates
08:06:29 <ollivier> #info orchestra_* testcases were now enabled for Daisy only
08:06:40 <ollivier> #info "inject all envs in openrc into CONST" was backported
08:06:48 <ollivier> #info minor errors were fixed in documentation
08:06:49 <depo> #info Delia Popescu
08:07:33 <ollivier> I think Euphrates is done. We will only fix new bugs.
08:07:39 <ollivier> Agree?
08:07:52 <LindaWang> agree
08:08:23 <jose_lausuch> yes
08:08:45 <LindaWang> ollivier:  about getting auth token issue, we could talk about it later.
08:09:23 <LindaWang> i did not notice the bug from fuel
08:09:33 <ollivier> yes. From the time being, it's fine. Fuel bug could have been detected before running vnf testcases.
08:10:10 <ollivier> A wrong v2 url is registered in catalog. Only vnf gets the public endpoint before creating resources.
08:11:17 <ollivier> #topic functest
08:11:40 <ollivier> I think we don't list the next topic last meeting.
08:12:21 <ollivier> We need to check testcases if our testcases are able to manage unversioned endpoints (key change in Pike).
08:13:06 <ollivier> now v2 or v3 should not be listed in url. Then it could break several middleware (see juju)
08:13:32 <ollivier> #action all to check vnfs if they manage unversioned endpoints
08:14:34 <ollivier> maybe it only concerns juju but I want to share this possible side effects.
08:15:12 <ollivier> #info juju_epc has been successful vs Daisy but they still remain errors for the other installers
08:15:35 <LindaWang> ok,  i have not seen other tests with the same issue
08:17:09 <ollivier> Yes. I think *ims* testcases are fine.
08:18:10 <ollivier> #info all Functest testcases have been successful vs Daisy
08:18:48 <ollivier> Great even if it's the simplest case (http endpoint, management network reachability)
08:19:27 <ollivier> It seems that we are fine for the next release :)
08:20:30 <LindaWang> MS4 you mean?
08:20:37 <LindaWang> Jan. 15th?
08:21:01 <ollivier> My feeling is MS4 is ensured by unit tests. Then we must be successful
08:21:46 <ollivier> The issue is jsut about xtesting as we will change some python namespaces. Normally we shouldn't change the framework.
08:22:17 <jose_lausuch> the milestone is about framework ready
08:22:20 <ollivier> They are trivial modification in 2 or 3 projects. That's not a big deal
08:22:23 <jose_lausuch> minor changes can be done, specially bug fixes
08:22:30 <jose_lausuch> exactly
08:22:38 <ollivier> I reported to David. That's MS4 is ok.
08:22:43 <jose_lausuch> I saw that
08:23:24 <ollivier> Our rules are better simply because we check MS4 automatically (and before merging)
08:23:46 <LindaWang> David also mentioned the feature patch gating job in the last release meeting, need to check that with Tim from Apex,
08:24:18 <LindaWang> Tim would provide a snapshot to enable deployment openstack in a few minutes
08:24:40 <LindaWang> I am not sure about this progress
08:25:24 <ollivier> Yes we discussed in case of Functest as well. In our case, it's more about checking MS3 automatically (running healthcheck) by functional tests
08:25:34 <ollivier> That's the same topic as XCI
08:26:04 <ollivier> (during release meeting right after the last Functest meeting)
08:27:11 <ollivier> My point is we require Functional gating and Tim said it's could be possible by snapshoting APEX and by hacking the current jenkins jobs.
08:28:04 <LindaWang> which tests could be used for functional gating?
08:28:13 <LindaWang> only healthcheck is enough?
08:28:29 <ollivier> Regarding Feature, I think the issue is more about gating several features every 3 days (SDNVPN) when APEX allows that.
08:29:03 <ollivier> I would say as much as possible. I would have added tempest as well.
08:29:06 <LindaWang> the functional gating you mean is for functest patch gating?
08:29:37 <jose_lausuch> the gating tests should be simple and quick
08:29:45 <ollivier> yes. To deploy a referent platform and to check healthcheck before merging the patch
08:29:53 <jose_lausuch> healthcheck
08:29:57 <jose_lausuch> I wouldn't add tempest there
08:30:41 <LindaWang> yes, tempest tests would take a longer time
08:30:59 <ollivier> If the platform is considered as stable... Tempest could be executed.
08:32:14 <ollivier> OpenStack runs tempest before merging.
08:32:33 <LindaWang> only on nosd-nofeature-ha scenario?
08:33:27 <ollivier> We must be confident about SUT. Then I would say os-nosdn-nofeature-ha
08:33:47 <jose_lausuch> yes, me too
08:34:07 <ollivier> If SUT is buggy, we can't run implement functional gating simply because we will lose time to debug and to override jenkins vote.
08:34:08 <jose_lausuch> we can always extend it later, but let's start with the base scenario
08:34:27 <LindaWang> ok
08:34:57 <ollivier> But I fully agree. At least MS3 must be covered. Let's see for tempest later (for the time being only Functest is ready for thsi gating)
08:37:26 <ollivier> Regarding the global planning, Fraser status is more that fine as os-nosdn-nofeature is already working in the simplest scenario. We must check the https certificates and the identify endpoint management.
08:38:06 <jose_lausuch> that's very positive :)
08:38:28 <ollivier> Yes, great work!
08:38:38 <ollivier> byw, the issues during basic heat operations were fixed in XCI
08:39:05 <LindaWang> this issue is only related to vnf tests?
08:39:51 <ollivier> the policy.json generated by XCI was wrong and Functest detected that.
08:40:56 <juhak> can you share the link for the fix?
08:41:12 <LindaWang> you mean when running tempest and rally/
08:41:13 <LindaWang> ?
08:41:48 <ollivier> before. Any basic heat operation fails (connection_check)
08:42:08 <ollivier> juhak: yes. I am searching the OpenSTack review.
08:42:41 <LindaWang> i think policy.json defines the role for the user  to take some actions
08:43:36 <ollivier> #link https://review.openstack.org/#/c/529147/
08:44:20 <juhak> ollivier: thanks!
08:44:36 <ollivier> I think everyone is added as CC to Morgan's email. If not I will complete the list.
08:44:38 <LindaWang> but this patch has been abandoned?
08:44:46 <ollivier> Yes.
08:45:21 <ollivier> It's the first proposal submitted by Fatih. They simply switch the proposal but I think the topic is closed.
08:45:43 <ollivier> If I remember well it also remains one issue regarding the external network.
08:45:58 <LindaWang> It has been fixed in XCI?
08:46:31 <ollivier> Yes. From the time being Functest is used to qualify XCI. But then if the platform is stable we could apply the opposite :)
08:46:49 <ollivier> That's the purpose of snapshots.
08:47:27 <ollivier> #topic functest-kubernetes
08:47:35 <LindaWang> ollivier: could you forward Morgan's email to me?
08:47:45 <LindaWang> ollivier: thanks
08:48:14 <ollivier> Sure. I think Morgan added everyone. But I will double check and complete it if required.
08:48:34 <ollivier> #info Functest Kubernetes Dockerfile was merged
08:49:24 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50063/
08:50:20 <LindaWang> i have tested  it locally, it works well
08:50:24 <ollivier> Thank you, Konrad.
08:50:51 <ollivier> The test or the container builds ?
08:51:07 <ollivier> #info ollivier/functest-kubernetes is built automatically per change
08:51:20 <LindaWang> i have pulled ollivier/functest-kubetetes
08:52:00 <LindaWang> and  i have created a testcases.yaml to run conformance tests inheriting from feature
08:52:06 <LindaWang> it works
08:52:11 <ollivier> Great!
08:52:28 <jose_lausuch> thats good news
08:52:40 <LindaWang> But i would prefer to make it inherit from testcase.TestCase,
08:52:59 <ollivier> Thank you. It confirms the go version and we will have a coverage for K8s as well.
08:53:00 <LindaWang> an email has been sent to you
08:53:26 <ollivier> ok. I will check it. Then we will have to implement the releng part
08:53:39 <jose_lausuch> are we inheriting from bashfeature?
08:54:01 <LindaWang> the testcases.yaml has not be published
08:54:19 <ollivier> first to build the containers (I think we can simply add functest-kubernetes in the classical releng build)
08:54:28 <ollivier> second to run it.
08:54:42 <LindaWang> ollivier: ok
08:54:48 <jose_lausuch> I can take care of adding that to the classical build
08:54:55 <ollivier> thank you
08:55:12 <LindaWang> even though the testcases.yaml has not be published, i thought you wanted to make it inherit form feature?
08:55:17 <ollivier> #action Jose to take care of adding functest-kubernetes to the classical build
08:55:19 <LindaWang> am i wrong?
08:55:32 <ollivier> Now this proposal could be fine as well.
08:55:36 <LindaWang> jose_lausuch: thanks
08:55:42 <jose_lausuch> feature or bashfeature?
08:55:50 <LindaWang> bashFeature
08:55:53 <ollivier> It allows to manage deeper data sucvh as detail when publishing in DB.
08:56:38 <jose_lausuch> ok
08:57:07 <ollivier> Jose could we first finish xtesting (next topic) before releng. There are many releng tasks but I think we should first focus on framework (MS4 related) Agree?
08:57:17 <ollivier> #topic xtesting
08:57:23 <jose_lausuch> ok
08:57:30 <ollivier> #info lots of changes have been published to break dependencies to OpenStack or to OPNFV utils
08:57:32 <jose_lausuch> addding the classical build is only 5 min I think
08:57:39 <jose_lausuch> I need to finish my patch about push to db
08:57:44 <jose_lausuch> sorry for being late with it
08:57:48 <ollivier> no pb.
08:58:36 <ollivier> It's the last one. I can help you this afternoon if needed.
08:58:51 <jose_lausuch> ok, thank you
08:59:00 <jose_lausuch> it could take until tomorrow as I have a workshop this afternoon
08:59:03 <jose_lausuch> but will do my best
09:00:36 <ollivier> I think your proposal in comment is fine and you have done the biggest part.
09:00:50 <jose_lausuch> I will merge all in 1 function
09:00:54 <ollivier> ok.
09:01:01 <jose_lausuch> instead of 1 static and 1 class method
09:01:43 <ollivier> Yes. Your proposal is fine. I'm fine to wait tomorrow. Then I will publish my local changes in functest-xtesting (19/12 AP5)
09:01:46 <jose_lausuch> let's discuss it later if needed, thanks :)
09:02:02 <ollivier> #topic AoB
09:03:03 <ollivier> I also noted actions on releng that I will share later by email.
09:03:46 <ollivier> We are still working on fixing the issues when compiling ruby modules on ARM (mismatch in dependency versions).
09:04:04 <ollivier> (vnf + testapi)
09:04:49 <ollivier> I will be partially off the next 3 days as I am giving an ODL training session. I will do my best to be connected on irc and emails.
09:05:04 <LindaWang> got it.
09:05:37 <ollivier> Thank you very much. Great work.
09:05:38 <jose_lausuch> ok, thanks
09:05:46 <jose_lausuch> good meeting
09:05:58 <ollivier> #endmeeting