08:03:10 <ollivier> #startmeeting Functest weekly meeting 13 Feb. 2018
08:03:10 <collabot`> Meeting started Tue Feb 13 08:03:10 2018 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:03:10 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:03:10 <collabot`> The meeting name has been set to 'functest_weekly_meeting_13_feb__2018'
08:03:17 <ollivier> #info Cédric Ollivier
08:03:37 <juhak_> #info Juha Kosonen
08:03:39 <LindaWang> #info Linda Wang
08:05:01 <ollivier> #topic action point follow-up
08:05:06 <jose_lausuch> #info Jose Lausuch
08:05:23 <ollivier> 2 jira tickets were closed
08:05:36 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-929
08:05:43 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-913
08:05:52 <ollivier> #info we should still find the best solution about "Fix the config of fixed_network_name in tempest"
08:06:14 <LindaWang> i updated the patch without modifying the  network
08:06:18 <ollivier> LindaWang: it seems that you publish a new update. Did you change the shared attribute?
08:06:19 <pkaralis> #info pkaralis
08:06:21 <ollivier> ok :)
08:06:48 <ollivier> Then it could be reviewed. I think Morgan already voted +2
08:06:57 <ollivier> #link https://gerrit.opnfv.org/gerrit/51381
08:07:04 <LindaWang> i tested tempest and refstack on compass without configuring fixed_network_name, it worked
08:07:20 <ollivier> #info mainly refstack_defcore(under investigation), rally_sanity(under investigation) cloudify_ims and vyos_vrouter have to fixed
08:07:38 <ollivier> #info juju_epc is now working fine
08:07:50 <ollivier> #info the second Kubernetes testcase is still under review
08:07:58 <ollivier> #link https://gerrit.opnfv.org/gerrit/51635
08:08:16 <ollivier> thank you for all changes.
08:08:35 <LindaWang> still  some issues about results parsing
08:08:45 <ollivier> refstack?
08:08:52 <LindaWang> no, k8s_conformance
08:08:54 <ollivier> k8s?
08:09:15 <LindaWang> yes
08:09:31 <ollivier> Ok. Then be free to vote Verify -1. Have you updated the logging part?
08:09:47 <LindaWang> not yet
08:10:02 <LindaWang> Konrad will update it later.
08:10:05 <depo> #info Delia Popescu
08:11:12 <ollivier> Noted. I haven't checked if Compass deployment is currently working.
08:11:45 <LindaWang> compass has failed to deploy k8s these days
08:11:54 <ollivier> #info https://gerrit.opnfv.org/gerrit/#/c/50213/
08:11:58 <ollivier> #undo
08:11:58 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x235bd90>
08:12:02 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/50213/
08:12:39 <ollivier> XCI support K8s deployment now.
08:12:57 <LindaWang> good to know
08:13:33 <ollivier> #topic Xtesting
08:13:41 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/q/project:functest+branch:master+topic:xtesting
08:13:54 <ollivier> #info all key changes have been published and samples work well locally
08:14:16 <ollivier> I'm leveraging on git sparse checkout to check it (simply cloning part of functest)
08:14:33 <ollivier> #info we are waiting for a full synchonization between functest and functest-xtesting
08:14:40 <ollivier> we can't push all changes due to rules vs authors
08:14:58 <ollivier> I sent an email to Trevor and Aric.
08:15:49 <ollivier> This rule could be removed as it double check the owner. It fails as I was pushing changes done by the whole functest community.
08:16:49 <ollivier> At least we are ready to publish xtesting. I think the log part could be improved in a second step (/home/opnfv/functest/results/functest.log).
08:18:04 <ollivier> #topic Code improvement
08:18:47 <ollivier> Lots of changes have been already published.
08:19:28 <ollivier> The key ideas are to manipulate (read/write) env vars via os.environ instead of CONST and to switch to gettattr/setattr for the remaining configs read operations.
08:19:47 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-931
08:19:54 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-932
08:20:04 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-933
08:20:31 <ollivier> Please let me know if you disagree or if it's not clear.
08:21:25 <LindaWang> do you mean we still keep config.py  to parse config_functest.yaml?
08:22:00 <ollivier> Yes. We may switch to oslo.db but from the time being it's quite fine.
08:22:06 <LindaWang> ok
08:22:57 <ollivier> Using const for env vars simply adds complexity (see CI_LOOP bug as jjobs didn't set it)
08:23:25 <ollivier> Now we can easily removes the former env vars and set the default values.
08:23:52 <ollivier> As already published we can safely remove IS_CI_RUN and CI_DEBUG
08:24:22 <ollivier> (CI_DEBUG is set in jjobs to print to stdout)
08:25:15 <ollivier> I think we could set os-nosdn-nofeature-noha when getting os.environ DEPLOY_SCENARIO.
08:25:30 <ollivier> or os-nosdn-nofeature-noha ?
08:25:37 <ollivier> os-nosdn-nofeature-ha ?
08:25:45 <LindaWang> either is fine
08:25:53 <ollivier> I would prefer noha even if we do modify testcases.yaml
08:27:01 <LindaWang> their main difference lies in running vnf or not locally.
08:27:04 <ollivier> (It seems fine to run vnf when noha scenarios).
08:27:45 <LindaWang> in CI, we do not have to worry as DEPLOY_SCENARIO is provided
08:27:47 <ollivier> I think we should run them locally. The key point is to list (in wiki pages and docs) the mandatory env vars
08:28:18 <LindaWang> from my side, my local deployment is only virtual
08:28:19 <ollivier> I think it's too much to ask a non OPNFV user to set DEPLOY_SCENARIO .
08:29:13 <ollivier> os-nosdn-nofeature-noha could be considered as a default value.
08:30:07 <LindaWang> sounds better.
08:30:16 <ollivier> INSTALLER_IP and INSTALLER_TYPE are required in few testcases and we could skip them in the main configs
08:30:45 <ollivier> https://wiki.opnfv.org/pages/viewpage.action?pageId=13211751
08:31:06 <ollivier> In fact only EXTERNAL_NETWORK could be enough for most of the testcases.
08:31:30 <ollivier> We should precise the added env vars when running ODL, SFC and doctor
08:31:37 <ollivier> in the right sections.
08:32:11 <ollivier> (as for CI_LOOP=weekly in case of components)
08:33:04 <ollivier> I will update the next change https://gerrit.opnfv.org/gerrit/#/c/52047/
08:34:03 <ollivier> About OS_AUTH_URL, every testcase should check a possible KeyError.
08:34:33 <ollivier> NODE_NAME & BUILD_TAG are required only when publishing data to DB or when collecting energy
08:35:23 <LindaWang> another issue is when publishing the data to a local file, NODE_NAME & BUILD_TAG are also required
08:35:37 <LindaWang> it does not make much sense
08:36:28 <ollivier> Yes. In case of a classical run by hand, they are useless.
08:38:01 <ollivier> #action Cédric to update the Alpine wiki pages (configguide) and to update the env vars processing if needed.
08:38:31 <ollivier> Are cli modules still used?
08:40:06 <ollivier> Should we maintain them? It adds an icmp test which could raise side effects (port forwarding over ssh)
08:40:07 <LindaWang> yes
08:40:18 <LindaWang> cli is used by Dovetail
08:40:40 <ollivier> Dovetail doesn't use master ?
08:40:52 <LindaWang> they are using Euphrates now
08:41:36 <LindaWang> and api modules reply on cli
08:41:38 <ollivier> I'm only considering modifying master. I think Euphrates should be updated only in case of bugs.
08:42:57 <LindaWang> what does `icmp test` mean?
08:42:58 <ollivier> I think api should use cli in between. From the time being, we can maintain this modules. We have to take care of different processing when using run_tests or cli (icmp packets, BUILD_TAG regex)
08:43:16 <ollivier> ...api shoudn't use cli in between..
08:43:51 <ollivier> response = os.system("ping -c 1 " + self.endpoint_ip + ">/dev/null")
08:44:27 <ollivier> This doesn't work in case of port forwarding (sshuttle)
08:45:24 <ollivier> I'm fine to add an icmp test but we should test the reachability in both calls.
08:45:31 <ollivier> not in only one.
08:46:07 <LindaWang> we could update the 'functest openstack check' if necessary
08:46:15 <ollivier> We could maintain both run_test and cli but the behavior should be the same.
08:46:18 <ollivier> Yes.
08:46:46 <ollivier> I think there is one diff regarding BUILD_TAG regex as well.
08:47:02 <LindaWang> but cli is easy for users, which would provide some guidance
08:47:56 <ollivier> why not but it duplicated work and only run_test is tested by CI/CD.
08:49:05 <ollivier> LindaWang: could you please update them. I would prefer that Dovetail use the main entry point
08:49:24 <ollivier> #topic Functest
08:49:38 <ollivier> #info tempest has been refactored/improved
08:49:44 <ollivier> #link https://gerrit.opnfv.org/gerrit/51889
08:49:51 <ollivier> #link https://gerrit.opnfv.org/gerrit/51963
08:50:00 <ollivier> #info rebaca team is updating juju_epc to leverage on snaps instead of openstack_utils
08:50:21 <ollivier> only vnfs are currently linked to openstack utils.
08:51:01 <ollivier> jose_lausuch: + maybe SFC and SDNVPN
08:51:52 <jose_lausuch> I haven´t found any other project using our utils
08:52:09 <jose_lausuch> for instance, barometer is based on our test case classes, but without using any other things
08:52:10 <ollivier> thank you. Previous action done.
08:53:22 <ollivier> Then be free to update vyos_router (planed end of February) or orchestra testcase (not clear) which are not updated yet.
08:53:36 <ollivier> #topic Releng
08:53:42 <ollivier> #info the falsy snaps testcase was fixed:
08:53:48 <ollivier> #link https://jira.opnfv.org/browse/SNAPS-263
08:53:59 <ollivier> #info gating is now working except for Apex jobs which are blocked by doctor-notification (timeout)
08:54:25 <ollivier> We should disable it till we haven't published our new jenkins jobs.
08:54:54 <ollivier> #info a proposal rewriting our Jenkins jobs has been published for reviews
08:55:02 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/51853/
08:55:38 <ollivier> Be free to review Delia's proposal. It would avoid one testcase blocking the full suites.
08:56:42 <ollivier> We could also removing former pods to avoid multiplying jobs for nothing.
08:57:01 <ollivier> jose_lausuch: morgan_orange: I will ask your help about History :)
08:57:32 <jose_lausuch> ok
08:58:42 <ollivier> The git commit message is clear. Regarding the last proposal, we should care about verify jobs which could have been removed.
08:59:23 <ollivier> thx
08:59:28 <ollivier> #topic AoB
08:59:43 <ollivier> Do we miss a key point?
08:59:55 <pkaralis> hi ollivier LindaWang all
09:00:02 <pkaralis> I'd like to run the latest smoke-tempest suite. This is related to jira: FUNCTEST-926. Can you provide me any instructions / info / link / document for that?
09:00:40 <ollivier> Yes. We have to work on that too.
09:01:10 <LindaWang> it seems the lists are provided in neutron repo, which uses tempest as a plugin?
09:01:11 <ollivier> The point is about integration trunk port tests (tempest)
09:01:40 <ollivier> We don't clone neutron repo. We simply install the python package.
09:01:42 <pkaralis> LindaWang, yes
09:02:44 <LindaWang> we have to figure out how to install neutron python and install tempest, then generate a test list
09:03:05 <pkaralis> ollivier, my initial problem is how I could run the latest smoke-tempest suite without the neutron plugin. As it is.
09:03:10 <ollivier> I think we can close the meeting. Stay tuned for trunk ports.
09:03:20 <ollivier> #endmeeting