08:00:45 <ollivier> #startmeeting Functest weekly meeting 28 Nov. 2017
08:00:45 <collabot> Meeting started Tue Nov 28 08:00:45 2017 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:45 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:45 <collabot> The meeting name has been set to 'functest_weekly_meeting_28_nov__2017'
08:01:05 <depo_> # info depo (Delia Popescu)
08:01:20 <CristinaPauna> #info Cristina Pauna
08:01:26 <jose_lausuch> #info Jose Lausuch
08:01:30 <ollivier> #info Cédric Ollivier
08:01:33 <juhak> #info Juha Kosonen
08:01:33 <viktor_t> #info Viktor Tikkanen
08:01:51 <LindaWang> #info Linda Wang
08:02:01 <depo_> #info Delia Popescu
08:02:13 <ollivier> #topic action points follow up
08:02:39 <ollivier> AP3 and AP6 done
08:02:54 <ollivier> #info rally db operations were updated and a bug fixed too
08:03:01 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47555/
08:03:07 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47527/
08:03:21 <ollivier> #info prepare_env were removed
08:03:40 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47267/
08:03:56 <ollivier> #info AP1, AP2, AP4 and AP6 in progress
08:04:14 <ollivier> we will detail them in the next topics if required
08:04:29 <ollivier> #topic Euphrates
08:04:37 <ollivier> #info vEPC still under development (the container seems good)
08:04:53 <ollivier> #info the bug related to OS_AUTH_URL were fixed
08:05:01 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-892
08:06:40 <ollivier> Be fre to add info for Euphrates?
08:07:12 <ollivier> depo_: CristinaPauna: What's the current status for ARM containers?
08:07:36 <depo_> whell, functest works with arm containers on euphrates branch
08:07:40 <depo_> and fails on master
08:08:01 <depo_> on euphrates we still have prepare_env
08:08:08 <ollivier> yes.
08:08:45 <depo_> on master even though I see the patch for aarch is being applied the tests fail staring with api_check
08:08:49 <jose_lausuch> what does it fail in master?
08:09:07 <jose_lausuch> oh ok
08:09:16 <jose_lausuch> so, the framework stars correctly
08:09:21 <jose_lausuch> it's the tests...
08:09:22 <depo_> only connection_check from healthcheck is os
08:09:32 <depo_> yes
08:09:34 <ollivier> I haven't checked the jenkins jobs. But now we can rely on the same jjobs for ARM64 and AMD64
08:09:48 <depo_> yes
08:10:04 <depo_> I'm working on fixing it for master
08:10:48 <ollivier> ok. I have seen some incompatible libc issues (mixing architectures) for several ARM runs.
08:11:16 <ollivier> #info functest works with arm containers on euphrates branch
08:12:08 <CristinaPauna> ollivier: do you need help from us for the libc issues>?
08:12:54 <ollivier> CristinaPauna: we have to check if it's not related to manifests.
08:13:47 <CristinaPauna> ok, we can sync on the details after the meeting
08:14:00 <ollivier> ok
08:14:14 <ollivier> #topic Releng
08:14:21 <ollivier> #info we should still implement the container garbage collector
08:14:31 <jose_lausuch> oh yes
08:14:40 <jose_lausuch> will do it this week for sure
08:14:47 <ollivier> Yes. No problem.
08:15:12 <ollivier> The most important is to have switched to multijobs and clean entry points.
08:15:47 <ollivier> cleaning containers will be done by our script.
08:15:58 <ollivier> #topic Fraser
08:16:07 <jose_lausuch> yes, good job with that
08:16:21 <ollivier> Sure thank you all
08:16:22 <serena-zte> how about leveraging --rm, so we don't need to clean containers
08:16:31 <ollivier> Yes but it's not enough
08:16:53 <ollivier> We have to clean the container images locally built.
08:17:08 <jose_lausuch> and sometimes if the job fails or something, there are ugly leftovers
08:17:17 <serena-zte> okey, --rm won't clean images
08:17:19 <ollivier> Regarding the common releng gc, we must take care of the order.
08:17:39 <ollivier> functest-core must be cleant at the end.
08:17:55 <ollivier> (after functest-smoke, etc...)
08:18:22 <ollivier> I would prefer we finish that before refactoring functest-components and functest-smoke.
08:18:42 <ollivier> (functest-smoke could inherit from functest-components)
08:19:03 <ollivier> #info all features are synchronized with OpenStack Pike and are enabled for F-release
08:19:05 <jose_lausuch> yes, we would avoid installing twice the same
08:19:13 <ollivier> yes
08:19:33 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/q/topic:opnfv/requirements
08:20:32 <ollivier> I think we can continue on modifying the copyrights (we haven't received any refusal from legal) as OpenStack has done
08:21:21 <ollivier> I must check milestones but we are more than ready regarding Features integration :)
08:21:39 <serena-zte> is official copyrights guidance from OPNFV issued?
08:22:41 <ollivier> I think there is no issue. I haven't received any answer and we simply applied the same instructions as OpenStack
08:24:18 <ollivier> #info an email was sent about the new repository to host requirements
08:24:43 <ollivier> we are debating about a dedicated project and/or the possible hosting projects
08:24:50 <serena-zte> I would suggest checking with OPNFV first before working on it
08:25:09 <ollivier> serena-zte: Yes. That's what we did.
08:25:16 <serena-zte> in case OPNFV does not follow OpenStack
08:25:19 <serena-zte> that's great
08:26:39 <ollivier> #info rally db operations were updated and a bug fixed too
08:26:45 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47555/
08:26:54 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47527/
08:28:02 <ollivier> juhak: LindaWang: I think there is a warning printing when creating the deployment  (we could update the config?)
08:28:18 <LindaWang> yes, i see
08:28:38 <LindaWang> we need to update the log config
08:29:36 <ollivier> #action Linda,Juha  to update the log config (rally)
08:29:38 <ollivier> Thank you
08:30:00 <ollivier> #info a new class to run any Robot Framework suites is proposed
08:30:22 <ollivier> Could you please review it? Be free to comment if you have any question.
08:30:22 <jose_lausuch> that is a good idea
08:30:28 <jose_lausuch> ok
08:30:47 <LindaWang> ollivier: sure
08:31:31 <ollivier> ODL testcase looks like data conversions only now. But it's great to allow running any robot framework suites.
08:32:03 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/47841/
08:32:15 <LindaWang> those tests: odl, odl_netvirt and fds?
08:33:12 <ollivier> ODL testcase (odl.py) only set variables required for odl robot suites (odl, odl_netvirt and fds)
08:34:01 <LindaWang> any new odl related tests will be joined in future? I remember jose_lausuch  said that
08:34:40 <ollivier> Yes. I remembered as well. And we must check if we can switch to netvirt test suites by default.
08:35:21 <ollivier> (I think I forwarded the discussion with Sam Hague: OVSDB PTL)
08:35:40 <jose_lausuch> yes
08:35:49 <jose_lausuch> I need to connect that discussion with the discussion in sdnvpn
08:35:56 <jose_lausuch> they are also looking into odl tests
08:35:59 <jose_lausuch> let me do that today
08:36:04 <LindaWang> i have not received the email, could you forwad it to me? ollivier
08:36:16 <ollivier> sure.
08:37:13 <serena-zte> about email, could you please send to my gmail from now on?
08:37:19 <ollivier> ok
08:37:40 <serena-zte> now the company make some restrictions, I can not access zmail out side of company :(
08:38:06 <serena-zte> and I can only access it in one computer now
08:38:22 <serena-zte> thank you
08:38:31 <ollivier> #info the next testcases fail: snaps_smoke (partially induced by volume encryption), rally_sanity and tempest_smoke_serial
08:39:27 <ollivier> I think we could disable several tests if no installer can support them. What about volume encryption from the time being?
08:39:52 <LindaWang> volume encryption are not supported by COMPASS yet
08:40:00 <serena-zte> daisy as well
08:40:22 <ollivier> Yes and no useful run for the other installers.
08:40:41 <LindaWang> they passed on Fuel
08:40:56 <ollivier> Ok. I have missed that. Thank you.
08:41:49 <ollivier> #info functest fails with arm containers on master
08:42:44 <ollivier> It would be great if a full run could be successful to validate the updates (OpenStack, rally, etc.)
08:42:53 <ollivier> #topic Functional Gating/Xtesting:
08:43:01 <ollivier> #info Dockerfile proposed to SDNVPN
08:43:45 <ollivier> jose_lausuch: could you please speak about that too during SDNVPN meeting
08:45:21 <jose_lausuch> ollivier: I don´t attend sdnvpn meetings, maybe better per email
08:45:49 <ollivier> jose_lausuch: ok. wait and see. The email has already been sent.
08:45:59 <jose_lausuch> ok
08:46:08 <jose_lausuch> I am sure they will be interested in having xci runs for their gating
08:47:20 <ollivier> I would like now to discuss about the proposal to create new repositories for Functest.
08:48:03 <ollivier> Linda started a discussion on how to host Kubernetes testcases.
08:48:50 <ollivier> Could you please give your feedbacks? I think it's good to split them into multiple python packages then multiple git repositories.
08:49:12 <LindaWang> Konrad has started working on a new container for k8s, https://github.com/djkonro/opnfv_functest_k8s/blob/master/alpine/Dockerfile
08:49:17 <LindaWang> which is based on functest-core
08:49:30 <ollivier> #info Konrad has started working on a new container for k8s
08:49:37 <ollivier> #link https://github.com/djkonro/opnfv_functest_k8s/blob/master/alpine/Dockerfile
08:50:10 <LindaWang> but we are not sure how many tests will be integrated into functest. because there are quite lots of kinds of tests,
08:50:24 <LindaWang> we need to choose some from them
08:50:56 <ollivier> (Ideally we should remove gcc at the end as it's useless from runtime pov).
08:51:21 <LindaWang> and when it comes to trigger and execute those tests, we could just use the binary of kubernetes.
08:51:50 <LindaWang> ollivier: ok
08:52:39 <LindaWang> but i am not sure if we need to create a new repo for k8s tests, cause for the time being, we have not started developing our own k8s tests.
08:52:47 <jose_lausuch> but we agree on having a new k8 container from functrest-core?
08:52:59 <jose_lausuch> functest-core installs a lot of openstack stuff that k8 doesn't need
08:53:00 <LindaWang> jose_lausuch: yes, i agree
08:53:02 <depo_> on what release?
08:53:06 <jose_lausuch> but in the other hand, its our core...
08:53:09 <ollivier> depo_: master
08:53:16 <depo_> ok
08:53:20 <jose_lausuch> k8 container is for fraser
08:53:20 <ollivier> it's should be xtesting
08:53:26 <jose_lausuch> yes
08:53:26 <LindaWang> functest-core will remove those openstack stuff finally i think
08:53:33 <ollivier> from the time being we must inherit from functest core
08:53:47 <jose_lausuch> so, shall we leave it for now functest-core and when xtesting is there we change it?
08:53:51 <jose_lausuch> ok
08:53:56 <jose_lausuch> let's do that then
08:53:57 <ollivier> exact
08:54:53 <ollivier> But it will be great if everyone gives feedbacks. We could ask for creating xtesting repo first then functest-kubernetes
08:55:51 <jose_lausuch> ollivier: we have to ask, to create the repo, it could leave with the functest umbreall
08:55:53 <jose_lausuch> umbrella
08:56:00 <jose_lausuch> since all the repos need a responsible project
08:56:11 <jose_lausuch> acording to the bylaw in opnfv
08:56:13 <ollivier> jose_lausuch: yes. we are working on that too
08:56:46 <ollivier> compared to requirements, Functest is the only choice for xtesting (aka new functest-core) and functest-kubernetes
08:57:13 <jose_lausuch> yes
08:57:22 <jose_lausuch> functest-kubernetes do you mean a new repo?
08:57:26 <ollivier> From the time being I would prefer that requirements is hosted by Functest too  simply because of gerrit rights.
08:58:07 <serena-zte> I think functest-kubernetes is a new container, right?
08:58:12 <ollivier> yes. It's maybe too early more if we select go for this part. That's the best way if we don't want to mix OpenStack and Kubernetes dependencies
08:58:50 <jose_lausuch> but if we host it in functest for now and inherit from xtesting, it won´t have openstack libraries installed
08:59:00 <serena-zte> btw, will we select some test cases from kubernetes like tempest/rally/snaps, or will we develop our own kubernetes test cases?
08:59:00 <ollivier> I think we all agree on the 3 containers. I think we should create 3 python packages simply to manage requirements.
08:59:01 <jose_lausuch> in the end it´s functest tests
08:59:12 <ollivier> jose_lausuch: exact
08:59:38 <ollivier> it's just a proper way to manage python packages (and then the related requirements per package)
08:59:44 <jose_lausuch> serena-zte: the idea is to first integrate and select from upstream, once that works, we will see. it would be interesting to create our own tests for telco deployments
09:00:19 <serena-zte> yeah, agree
09:00:24 <ollivier> regarding xtesting, we could think about a dedicated project as the dev list could differ from functest. But it's too early and useless for this release.
09:00:45 <jose_lausuch> I guess we can change the project ownership later on
09:01:12 <ollivier> yes. It will give us time to write the proposal before G if required.
09:01:28 <jose_lausuch> yes
09:03:11 <ollivier> then it seems that I can ask for the new repo xtesting hosted by Functest (from the time being), no?
09:03:20 <jose_lausuch> I think so
09:03:42 <serena-zte> me too
09:03:56 <serena-zte> new repo first
09:04:24 <ollivier> thank you.
09:04:31 <ollivier> #topic AoB
09:04:40 <LindaWang> could we talk about the long duration test?
09:04:45 <LindaWang> Which tests in Functest suitable for Bottlenecks to call as long during test?
09:05:04 <ollivier> Yes. I'm very sorry to be late.
09:06:47 <ollivier> I think we are more interested in multiplying tests. It could be great to run rally (or tempest/refstack) several times in a row.
09:07:27 <ollivier> It could help detecting remaining resources.
09:07:38 <jose_lausuch> +1 for tempest smoke
09:07:46 <jose_lausuch> and expect same results
09:07:46 <ollivier> full?
09:08:05 <LindaWang> tempest_full contains about 1600+ tests
09:08:07 <jose_lausuch> that would be great, but we don´t test full in CI
09:08:13 <LindaWang> it would take so long
09:08:48 <serena-zte> regarding long duration test, I think we could put tempest_full and rally_full into that area and asking for pod resources?
09:08:59 <serena-zte> I remember morgan had worked on that before
09:09:00 <ollivier> +1
09:09:18 <ollivier> we could reuse functest-components.
09:09:23 <jose_lausuch> yes, but not for bottlenecks
09:09:37 <LindaWang> Bottlenecks has already had the pod resources,
09:09:42 <jose_lausuch> that´s for the functest time slot, where we can run tempest full
09:09:52 <jose_lausuch> but bottlenecks could use tempest smoke for repetability
09:10:08 <LindaWang> what about rally?
09:10:18 <jose_lausuch> rally as well
09:10:34 <ollivier> Why couldn't we use functest-components as "long duration" test ?
09:11:03 <jose_lausuch> yes, that's for Functest long duration tests
09:11:28 <LindaWang> we have not tested tempest_full, maybe lots of them will fail
09:11:31 <jose_lausuch> but for bottlenecks using functest tests repeating them again and again, I would suggest temptes-smoke or other interesting things
09:11:47 <LindaWang> but why donot we run functest-components?
09:12:13 <LindaWang> what is the value of long duation  test for functest?
09:12:14 <serena-zte> As far as Bottleneck long duration is concerned, I think the test case should be stable and mature enough
09:12:25 <jose_lausuch> it´s just my point of view
09:12:25 <LindaWang> serena-zte: agree
09:12:34 <jose_lausuch> serena-zte:  +1, that's why I said tempest_smoke
09:12:39 <serena-zte> because tempest_full and rally_full takes too much time
09:12:46 <serena-zte> we don't have resources to run them
09:13:01 <serena-zte> that's why we require 'long duration' pods for them
09:13:11 <serena-zte> hope we can run them at least once aweek
09:13:25 <ollivier> The topic is long duration testing then 2 or 3 hours shouldn't be a problem.
09:14:08 <ollivier> We could propose running tempest-smoke as you're proposing.
09:14:35 <jose_lausuch> I don´t know how many times bottlenecks runs the tests, but in vping it was several times
09:15:02 <serena-zte> LindaWang, do you know what kind of test cases or which area are Bottleneck looking for?
09:15:02 <ollivier> I close the meeting as we are a little bit late (sorry). We could continue here if you're free or by mails.
09:15:03 <jose_lausuch> sorry, need to leave some minutes
09:15:10 <LindaWang> i think vping is also fine
09:15:34 <ollivier> #endmeeting