08:01:41 <ollivier> #startmeeting Functest weekly meeting 23 Jan. 2018
08:01:41 <collabot> Meeting started Tue Jan 23 08:01:41 2018 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:01:41 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:01:41 <collabot> The meeting name has been set to 'functest_weekly_meeting_23_jan__2018'
08:01:46 <ollivier> #info Cédric
08:01:55 <juhak> #info Juha Kosonen
08:02:12 <xudan> #info xudan
08:02:31 <LindaWang> #info Linda Wang
08:02:38 <jose_lausuch> #info Jose Lausuch
08:03:05 <ollivier> xudan: Thank you for coming. I added a point on dovetail.
08:03:32 <ollivier> #topic action point follow-up
08:03:40 <georgk> #info Georg Kunz
08:03:54 <ollivier> georgk: hello
08:04:02 <ollivier> #info push_results() and get_version() were moved to TestCase
08:04:04 <georgk> ollivier: hi, good morning
08:04:24 <jose_lausuch> ollivier: thanks for completing my patch and sorry for not finishing it, I was in a business trip
08:04:59 <ollivier> No pb. You have done a big part and the last one was not so simple due to falsy unit tests.
08:05:52 <ollivier> now the unit test seems right. Please take pylint errors/warning into account (next topic), it helps detecting issues.
08:06:14 <ollivier> #topic Dovetail
08:06:23 <ollivier> #info a new tag ovp-1.0.0 was published for Dovetail (it's a copy of the former tag cvp.0.5.0)
08:06:57 <ollivier> georgk: xudan: We don't know if we shoud remove the former tags (is it a rename?)
08:07:10 <jose_lausuch> it was asked to me, and I created that tag
08:07:25 <jose_lausuch> not a new docker image though, just a copy of cvp.0.5.0
08:08:13 <ollivier> yes. Thank you. The key point is we are not able to build it automatically as the jenkins jobs have been removed.
08:08:15 <georgk> I believe it was done to align with the new name (OVP) and the official release
08:09:30 <xudan> It's okay for me to remove cvp.0.5.0 and cvp.0.2.0. Please keep ovp.1.0.0
08:09:41 <jose_lausuch> we should be able to do it automatically from jenkins
08:09:48 <xudan> georgk: what do you think?
08:10:14 <georgk> xudan: yes, I think that´s ok. We don´t need to rebuild the container for the release right?
08:10:28 <ollivier> jose_lausuch: I don't think so simply because releng is in rolling release. But I could setup a docker automated builds if needed.
08:10:42 <jose_lausuch> ok
08:10:53 <xudan> yes.
08:11:18 <xudan> ovp.1.0.0 is the final version for OVP first release
08:11:22 <georgk> so, it was just a renaming of the old Danube container to go along with our release
08:11:39 <jose_lausuch> yes, I didn´t build any image
08:11:41 <ollivier> Why don't u select the last Functest danube image?
08:12:29 <ollivier> I haven't checked if it's newer that cvp.0.5.0
08:12:43 <xudan> We try to use danube.3.2 at the very begining. But found there is some bugs.
08:13:22 <xudan> Also we need some new features such as enable sdnvpn in file testcases.yaml and support to edit tempest.conf
08:14:01 <xudan> and all these are merged to master at that time and functest can't build another danube docker image
08:14:28 <ollivier> OK. Then the process is fine. The key point is to reduce now the delay simply because releng and functest only work on the current stable. release.
08:14:50 <xudan> so then build the docker image cvp.0.2.0 and cvp.0.5.0 with the master branch
08:14:51 <ollivier> or to simply use the official Functest container.
08:15:32 <xudan> yes. For OVP second release, I suggest to use opnfv/functest-restapi:euphrates
08:16:26 <xudan> I will discuss this with the Team in the next weekly meeting
08:17:18 <ollivier> Or the other containers.  That's another topic. I think we shouldn't duplicate the containers for the same work and we should fine a better to way .
08:17:32 <djkonro> #info Konrad Djimeli
08:17:32 <ollivier> to select CLI or RESTAPI.
08:17:50 <serena-zte> #info SerenaFeng
08:18:01 <ollivier> #info Dovetail team asks for integrating the tempest plugin to test trunk ports
08:18:16 <depo_> #info Delia Popescu
08:18:30 <ollivier> hello
08:19:21 <georgk> yes, there are a few tests in the neutron repo which we´d like include in our Fraser release
08:20:01 <georgk> I have contacted 3 installers already (Compass, Fuel, Apex), and 2 have enabled the feature (Fuel, Apex)
08:20:25 <serena-zte> georgk which feature?
08:20:33 <georgk> serena-zte: Neutron trunk ports
08:20:38 <serena-zte> sorry I was late, I don't know the context
08:20:49 <georgk> serena-zte: the installers basically just need to load the trunk service plugin
08:21:07 <LindaWang> georgk: do you know which sub test cases exactly are testing this feature?
08:21:30 <LindaWang> georgk: i mean which test cases from tempest?
08:21:34 <georgk> LindaWang: I can send you a list
08:21:37 <ollivier> LindaWang: the purpose is to create a new testcase in Functest leveraging on tempest.
08:21:57 <ollivier> serena-zte: the purpose is to create a new testcase in Functest leveraging on tempest.
08:22:34 <serena-zte> which test case in tempest?
08:22:38 <LindaWang> georgk:  these testacases are selected manually?
08:23:05 <georgk> LindaWang: no. They are implemented in the Neutron repo as a Tempest plugin.
08:23:24 <georgk> LindaWang: and that´s the support we need: make them selectable in Functest
08:23:28 <serena-zte> and are they mandatory or can be selected if are integrated?
08:24:17 <ollivier> it seems here better to create a new test. I don't consider handling exlude lists. Do you agree?
08:24:43 <georgk> yes, a separate tests seems resonable to me, too
08:25:13 <serena-zte> and how to handle it automatically if integrate into tempest_smoke is a big issue, because currently we. don't support it
08:25:32 <serena-zte> a separate tests seems better to manage
08:26:27 <LindaWang> yes, a new test case sounds better
08:27:17 <ollivier> Then we can start working vs Fuel or Apex. I will open a JIRA ticket about that. Be free to assign it to yourself. It would be great that Dovetail helps us for this topic as it's your request.
08:27:28 <LindaWang> georgk  and you are also welcome to work on it, since you are already familiar with it.
08:27:44 <georgk> :-)
08:27:58 <georgk> yes, please create the ticket.
08:28:03 <ollivier> great.
08:28:09 <serena-zte> and I will contact daisy, check if they have the intention to support it
08:28:25 <georgk> serena-zte: yes, please do so
08:28:29 <ollivier> I think it closes the topic. Thank you georgk and xudan.
08:28:36 <georgk> thank you all
08:28:45 <xudan> thanks
08:28:49 <ollivier> #topic opnfv/functest
08:28:57 <ollivier> #info the next wrong tags were removed as they can make confusion: stable, opnvf-5.0.0
08:29:06 <ollivier> #link https://hub.docker.com/r/opnfv/functest/tags/
08:29:17 <ollivier> #info the former Dockerfile in stable/euphrates is being removed
08:29:29 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50931/
08:29:59 <serena-zte> and georgk I may need some help from you to explain the work, is that ok?
08:30:19 <georgk> serena-zte: sure, just include me
08:30:31 <serena-zte> that's great, thank you
08:31:14 <ollivier> Please review it. It aims at avoiding any confusion (and opnfv/functest is no longer built or tested for E)
08:31:27 <LindaWang> got it
08:31:44 <serena-zte> so we fully change to opnfv/functest-xxx?
08:32:57 <ollivier> yes, we did. Here we simply remove the dockerfile as we haven't tested opnfv/functest via ci runs for a while. It should work...
08:33:19 <serena-zte> got it, great work
08:34:06 <ollivier> #topic functest
08:34:17 <ollivier> #info rally output retrieval was fixed
08:34:23 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50703/
08:34:31 <ollivier> #info set of region identity attribute was updated
08:34:38 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50713/
08:34:58 <ollivier> Thank you for that bugfixes.
08:35:03 <ollivier> #info one pending discussion started by Tim Rozet about integrating browbeat/cperf project into functest
08:35:14 <ollivier> #link https://github.com/openstack/browbeat
08:35:26 <ollivier> #link https://hub.docker.com/r/ollivier/functest-browbeat/
08:36:16 <ollivier> I simply built a container to check if we can easily integrate this OpenStack project. For the time being, I'm waiting for technical details.
08:36:44 <LindaWang> Is it expected to run only on Apex?
08:38:46 <ollivier> I don't think so (it's an openstack project). But it's too early. I'm wainting for details. I just installed the python package in a specific container.
08:39:09 <ollivier> I will forward any relevant data.
08:39:19 <ollivier> #info Orange is checking if we can implement a new testcase leveraging on Patrole to check policies
08:39:25 <serena-zte> it looks like kind of stress test from the introduction
08:39:37 <ollivier> serena-zte: yes + optimization.
08:39:52 <ollivier> #link https://github.com/openstack/patrole
08:40:06 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-906
08:40:28 <ollivier> #link https://hub.docker.com/r/ollivier/functest-patrole/
08:40:41 <ollivier> (the href is wrong on the wiki page. sorry)
08:41:33 <ollivier> It mainly adds a tempest plugin to check policy.json. We are finishing the analysis.
08:41:36 <serena-zte> LindaWang looks only on Apex
08:41:36 <serena-zte> In some cases it may not be desired to install Browbeat on the Undercloud (Ex. Limited Resource requirements or Non-Tripleo installed cloud)
08:42:58 <ollivier> serena-zte: I will be careful on that point. I simply check the python packages.
08:43:11 <LindaWang> why do we have to check policy.json files?
08:43:13 <LindaWang> ollivier:
08:43:40 <ollivier> To ensure the roles are properly configured.
08:44:01 <ollivier> It's quite an hot topic here.
08:44:19 <LindaWang> ollivier: i see. the related issue often occurs on commercial cloud os.
08:45:20 <ollivier> I have to find a way to run it on top of any OPNFV installer and to allow overriding the policies on purpose.
08:45:21 <LindaWang> interesting topic
08:45:55 <LindaWang> i used to come across this issue when running tempest or refstack, when i had to reconfigure "tempest_roles" in tempest.conf
08:46:53 <ollivier> Noted. I will give more details soon.
08:47:11 <ollivier> #info most of pylint errors could be easily fixed or disabled: invalid-name, logging-not-lazy, unused-argument, ...
08:47:34 <serena-zte> Look forward for the more details, thank you
08:47:51 <ollivier> please consider fixing them. They could highlight real problems.
08:48:02 <ollivier> #link https://build.opnfv.org/ci/job/functest-verify-unit-tests-and-docs-master/630/console
08:48:19 <LindaWang> ollivier: ok, i will help fix that
08:48:27 <ollivier> Thank you.
08:49:22 <ollivier> I open several JIRA tickets. I will continue. Lots of modules are closed to the perfect rating (rally, ci, ...)
08:49:31 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-906
08:49:36 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-907
08:49:55 <ollivier> #topic functest-kubernetes:
08:50:06 <ollivier> #info the first testcase is still under review
08:50:14 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50591/
08:50:41 <LindaWang> one issue is about openstack.creds file in run_tests.py
08:50:44 <ollivier> please review it it seems closed to be ready. Thank you Konrad
08:51:31 <LindaWang> i am considering moving def source_openrc to testcases.py
08:51:49 <LindaWang> but we have to override it for k8s tests
08:51:50 <ollivier> We can't because it would link the framework to OpenStack.
08:52:23 <LindaWang> can we rename a common name for openstack.creds and k8s.creds?
08:52:26 <ollivier> my feeling is we should force openstack.creds in env.
08:53:31 <ollivier> I do answer the Steven's mail about that. I agree with him for most cases but OpenStack is quite an exception (it deeply leverages on env vars).
08:53:51 <LindaWang> you mean use --env-file when running this container?
08:53:58 <ollivier> Yes.
08:54:12 <LindaWang> he suggested to put all this env vars to config_functest.yaml
08:54:18 <ollivier> Then Fucntest doesn't care about loading them. It simply checks if they are loaded when required
08:55:00 <serena-zte> sounds good
08:55:17 <serena-zte> put to config_functest.yaml is not a good idea
08:55:30 <ollivier> I would have agreed if all calls don't deal with env var. It works fine when calling snaps but not when using openstack binaries.
08:55:33 <serena-zte> openstack.creds are dynamic config
08:55:42 <serena-zte> different from each pods
08:56:10 <ollivier> I think that loading env var should be done by releng not by Functest.
08:56:32 <LindaWang> then it would be better to use --env-file
08:56:35 <ollivier> or the other approach is to remove all env vars as Steven suggested.
08:57:13 <serena-zte> then how to manage openstack.creds and test_db_url?
08:57:28 <ollivier> test_db_url must be set by env var now.
08:57:36 <LindaWang> if we do as Steven suggested, it would be complicated to update openrc
08:57:38 <serena-zte> yes,
08:58:49 <ollivier> If all agree, we could start by loading env vars via jjobs and remove source_openrc
08:59:02 <LindaWang> i am worried how to update the restapi of updating openrc fie
08:59:06 <serena-zte> IMP, config_functest.yaml should only contain the static configuration, no update needed during running
08:59:16 <ollivier> serena-zte: agree
08:59:16 <LindaWang> agree with Serena
08:59:51 <ollivier> agree?
09:00:01 <jose_lausuch> +1
09:00:08 <juhak> agree
09:00:57 <serena-zte> LindaWang, sorry I don't know your concern, I think restapi can also leverage --env-file if we determine to change others, no?
09:01:24 <ollivier> Let's go for that. Then we will be able to publish our first Kubernetes testcase.
09:01:27 <ollivier> thank you.
09:01:32 <ollivier> #topic functest-xtesting:
09:01:44 <ollivier> #info many pylint warnings have been fixed and the coverage of ci/ modules has been improved
09:01:54 <ollivier> #info it remains at least to break the dependencies to functest_utils (e.g. energy) and to fully cover run_tests.py
09:02:00 <serena-zte> another suggest is not to use copy, I will try to help with the error Konrad mentioned
09:02:52 <ollivier> yes. please add any comment in the review. We copy a local testcases.yaml for all Functest containers.
09:03:10 <ollivier> I aleady sent an internal email to Benoit for updating Energy.
09:04:01 <serena-zte> how about putting the configuration file to some common place like /usr/local/share/functest or /etc/functest?
09:04:20 <serena-zte> files like config_functest.yaml/testcases.yaml
09:04:52 <serena-zte> I think that's what python usually do with configurations
09:04:53 <ollivier> Yes. I was thinking about that when rewriting the python packages. That's a good target.
09:05:14 <serena-zte> and that will solve our copy issue
09:05:36 <ollivier> #topic AoB:
09:06:49 <ollivier> It will be great to discuss about offline testing in a next meeting (we are already quite late). That's a good topic even if it seems very difficult.
09:07:17 <LindaWang> which offline testing you mean? ollivier
09:07:41 <LindaWang> cloudify_ims ?
09:07:52 <ollivier> yes as you propose.
09:08:01 <LindaWang> thanks ollivier
09:08:58 <ollivier> At least point2 could be reached. It will add a specific topic next week.But Valentin can't join to the weekly meeting.
09:10:09 <LindaWang> yes, let's discuss it next week.
09:10:11 <ollivier> Thank you for the meeting. 3 new testcase proposals and great work on functest-kubernetes :)
09:10:14 <ollivier> #endmeeting