08:08:36 <ollivier> #startmeeting Functest weekly meeting 13 Mar. 2018
08:08:36 <collabot`> Meeting started Tue Mar 13 08:08:36 2018 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:08:36 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:08:36 <collabot`> The meeting name has been set to 'functest_weekly_meeting_13_mar__2018'
08:08:43 <ollivier> #info Cédric Ollivier
08:08:58 <depo> #info Delia Popescu
08:09:02 <viktor_t> #info Viktor Tikkanen
08:09:04 <serena-zte> #info SerenaFeng
08:09:19 <jose_lausuch> #info Jose Lausuch
08:09:24 <LindaWang> #info Linda Wang
08:09:35 <ollivier> #topic Action point follow-up:
08:09:51 <ollivier> #info Patrole testcase is finished and runs well
08:09:59 <ollivier> #link http://artifacts.opnfv.org/logs/functest/lf-pod2/2018-03-10_02-11-14/patrole/tempest-report.html
08:10:08 <ollivier> #link http://artifacts.opnfv.org/logs/functest/lf-pod1/2018-03-10_04-20-36/patrole/tempest-report.html
08:10:20 <ollivier> Do we integrate the testcase in smoke?
08:10:42 <ollivier> From the time being, it's isolated in its own container but it could make sense to add it to smoke
08:10:45 <LindaWang> only glance and neutron is are checked?
08:11:13 <ollivier> yes. I am sending an email to Patrole team. All glance and neutron provide policy.json (Pike)
08:11:35 <LindaWang> i am fine with it included in smoke
08:11:36 <ollivier> and they are badly skipped if the file is missing
08:12:01 <ollivier> volume is listed simply because it works well even if policy.json is missing
08:12:11 <LindaWang> #link http://artifacts.opnfv.org/logs/functest/zte-pod2/2018-03-12_21-44-22/patrole/tempest-report.html
08:12:31 <LindaWang> there are 287 tests in all, only 129 passed
08:13:04 <ollivier> Yes. The prupose is to run Patrole with common policy.json. Else Patrole testcase can be easily configured via testcases.yaml
08:13:40 <ollivier> #link https://git.opnfv.org/functest/tree/docker/patrole/testcases.yaml#n24
08:14:31 <ollivier> The two remaining services can be added and the policy.json could be load via the classical docker volume.
08:14:38 <ollivier> Let's go for smoke?
08:15:08 <LindaWang> ok with me.
08:15:30 <ollivier> I will update that this morning. Thank you.
08:15:41 <ollivier> #info Functest job timeout was increased (temporarily solution due to Compass and Fuel Arm)
08:15:51 <ollivier> #info DNS issues and missing OS_REGION_NAME (Apex) were bypassed in Functest
08:16:23 <ollivier> I think DNS should be addressed by installers but the current proposal is fine and working
08:16:34 <ollivier> #info Refstack defcore are now verified by Rally (report, 1 main tempest configuration, less processing...)
08:17:23 <ollivier> we should double check if there is no side effect. I think we should keep one tempest.conf for all tempest-based testcases
08:18:02 <LindaWang> regarding with this patch https://gerrit.opnfv.org/gerrit/#/c/53613/2
08:18:28 <LindaWang> i do not think a second image and flavor are required for tempest_smoke_serial
08:18:47 <ollivier> Normally we should duplicate the image id as proposed by tempest
08:19:24 <ollivier> Why not registering a second image for tempest as well. It eases maintaining both testcases.
08:20:06 <ollivier> My point is that refstack is simply a specific list of tempest tests.
08:20:27 <ollivier> then both testcases should be handled in the same way.
08:20:51 <LindaWang> ok, that would ease maintaining
08:21:46 <LindaWang> i will update the patch soon.
08:21:49 <ollivier> yes and your improvement about refstack is also fine for tempest.
08:21:51 <ollivier> Thank you
08:22:15 <ollivier> #info Orchestra testcases were removed (we could revert if they are updated on time)
08:23:06 <ollivier> We are closed to the next milestone and they haven't been working for all the dev cycle (OpenBaton would have to be deeply updated).
08:23:26 <ollivier> #info Kubernetes testcases have been declared in TEST DB
08:23:44 <ollivier> #info lots of pylint warnings have been fixed
08:24:27 <ollivier> only vnf testcases have to be fixed now. For G, we would improve the duplicate code.
08:24:42 <ollivier> #topic releng
08:24:52 <ollivier> #info our proposal switching to multijobs would have forced us to increase the number of executors per jenkins nodes
08:25:04 <ollivier> it has been considered as too risky regarding the planning and all possible side-effects
08:25:11 <ollivier> this enhancements are still key and are postponed to the beginning of G-release
08:26:16 <ollivier> there are at least 3 ideas: to stop testing if one key testcase is failed,  to mix sequential and parallel testing when possible and to ease reading the job results ...
08:26:38 <LindaWang> i thought it was Jenkins version issue.
08:26:49 <ollivier> yes regarding //
08:26:59 <LindaWang> you mean more executors are required?
08:27:29 <LindaWang> what does key testcase refer to?
08:27:29 <ollivier> Yes. One due to the multijob even if it only polls the testing job.
08:28:25 <ollivier> If API is not reachable or even tempest is not running, why should we continue? Today it's managed by script. I could have been handled by steps in Jenkins instead.
08:28:53 <ollivier> Regarding all the improvement done over the last releases, we could run tests in //
08:29:34 <ollivier> I only note a lock in xtables which could break that. BYW, I don't understand why we set a forwarding rule.
08:30:20 <ollivier> #link https://git.opnfv.org/releng/tree/jjb/functest/functest-alpine.sh#n124
08:30:34 <ollivier> jose_lausuch: do you know the reason?
08:31:41 <jose_lausuch> let me se
08:32:01 <ollivier> But Delia's patch seems quite fine (minor issue on container name) and could have worked well if we could have increased the magical number.
08:32:03 <jose_lausuch> that was done for apex long time ago
08:32:15 <jose_lausuch> their jumphost contained some firewall rules that blocked some traffic
08:32:21 <jose_lausuch> not sure if it was admin network or similar
08:32:40 <jose_lausuch> maybe that is not needed any more?
08:33:04 <ollivier> Great. I would ping Tim. I think we could also remove the privileged right but that's minor
08:33:30 <jose_lausuch> yes, I think that is not needed either
08:33:42 <jose_lausuch> as we don´t do any operations that requiere privileged mode
08:34:22 <ollivier> yes I think so. The only argument was about wrong permissions on ssh key which sounds false.
08:34:30 <ollivier> I will remove it.
08:34:41 <jose_lausuch> ok
08:35:08 <ollivier> It's much more important in term of security that checking password in variable names in python code.
08:35:50 <ollivier> #info a first proposal has been submitted to build opnfv/xtesting
08:36:21 <ollivier> I noted one possible minor update.
08:36:50 <ollivier> depo: do you agree?
08:37:17 <depo> yes
08:37:33 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/53579/
08:37:44 <ollivier> then releng topic is over for this release.
08:37:59 <ollivier> #ropic Functest
08:38:01 <ollivier> #undo
08:38:01 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Link object at 0x2345550>
08:38:07 <ollivier> #topic Functest
08:38:18 <ollivier> #info neutron-trunk is still under development (it should be merged before the end of the week)
08:38:38 <ollivier> We have to check the first results on Compass before merging it.
08:38:49 <ollivier> The last update was on api_extensions listed in tempest.conf
08:38:55 <LindaWang> a new container or functest-smoke?
08:39:08 <LindaWang> i  have shared the test results on compass
08:39:31 <ollivier> It could be in functest-smoke. I double checked the regex.
08:39:37 <LindaWang> have you checked the results? http://paste.openstack.org/raw/698990/
08:40:19 <ollivier> no so bad. Thank you. 2 tests in failure.
08:40:56 <ollivier> then we could merge it and check if we have to remove subtests. Badly I can't run them here.
08:41:05 <ollivier> agree?
08:41:14 <LindaWang> yes
08:41:31 <viktor_t> ok
08:41:54 <ollivier> thank you. Let's go for that and we are on time for the next milestone :)
08:42:23 <ollivier> Now our priorities are to test the testcases, to identify all remaining  installer bugs and then to check the possible remaining resources at the end
08:42:43 <ollivier> daisy os-nosdn-nofeature-noha is already passing
08:42:51 <ollivier> #link https://build.opnfv.org/ci/view/functest/job/functest-daisy-baremetal-daily-master/2/console
08:43:02 <ollivier> #info JOID is defining non default Keystone v3 domains which may require additional updates
08:43:20 <ollivier> the last runs are not so good...
08:43:52 <ollivier> #link https://build.opnfv.org/ci/view/functest/job/functest-joid-baremetal-daily-master/lastFailedBuild/console
08:44:08 <ollivier> Apex and compass have at least one bug.
08:44:21 <ollivier> https://wiki.opnfv.org/display/functest/Fraser+Release+Gating+via+Functest
08:44:42 <LindaWang> i did not notice juju_epc has passed on Compass?
08:44:55 <LindaWang> the jjb is timeout on compass now
08:45:07 <LindaWang> #link https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-master/10/console
08:45:23 <ollivier> 2018-02-07 06:24:36
08:45:33 <ollivier> It mixes results
08:46:08 <ollivier> Let see the next Arm64 and Compass runs as timeouts were increased.
08:46:27 <ollivier> If you can start a deployment, be free to do so :)
08:46:46 <LindaWang> i cannot actually 😀
08:47:11 <ollivier> I think juju_epc is buggy if region differs from defaut. I will open a new JIRA ticket.
08:47:29 <ollivier> Else the global results are very good.
08:48:07 <ollivier> #info several JIRA tickets have been opened to remove per-installer processing (Apex, Compass, Fuel)
08:48:32 <ollivier> At the end, we should only keep the capability to blacklist tests per installer.
08:48:41 <ollivier> (even if unused)
08:49:09 <ollivier> #topic functest-kubernetes
08:49:23 <ollivier> #link https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-master/22/console
08:50:19 <ollivier> One test is already passing on Compass.
08:50:35 <LindaWang> sorry, i failed to find the jira tickets you mentioned just now. about the blacklist
08:50:51 <ollivier> I can print them here
08:51:01 <ollivier> #link https://jira.opnfv.org/browse/COMPASS-590
08:51:11 <ollivier> #link https://jira.opnfv.org/browse/FUEL-352
08:51:21 <ollivier> #link https://jira.opnfv.org/browse/FUEL-353
08:51:29 <ollivier> #link https://jira.opnfv.org/browse/APEX-570
08:52:20 <LindaWang> thanks, it is about sdn_controller_ip env
08:52:51 <ollivier> Yes. Apex about Region Name, Fuel about volume device name and SDN_CONTROLLER_IP
08:53:15 <ollivier> I think we should set new env vars for ODL ports.
08:54:17 <ollivier> Regarding k8s, we have now to write a new wiki page to explain how to run the new testcases (Fatih asked it yesterday)
08:54:27 <LindaWang> #link https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412
08:54:32 <ollivier> we do check why logs aren't published at the end
08:54:42 <LindaWang> a simple one to be improved
08:55:12 <ollivier> thank you. We may set all vars in env directly.
08:55:56 <ollivier> In case of OpenStack, it's easy to download creds from horizon and to put the file as docker args.
08:56:21 <ollivier> Here it's could be simplified as one file is needed.
08:56:53 <LindaWang> only one k8s_creds file ?
08:56:59 <ollivier> or only env
08:57:41 <ollivier> The wiki page seems fine. Thank you. I would prefer $(pwd)/config instead of /root/.kube/config
08:58:03 <ollivier> #info we are waiting for the first CI runs on Joid and Kubespray
08:58:18 <LindaWang> ok
08:58:26 <LindaWang> where to check the Kubespray job?
08:58:38 <LindaWang> or it is only run locally?
08:59:13 <ollivier> We have to discuss with Fatih about that.
08:59:28 <ollivier> #topic AoB
09:00:30 <ollivier> Eric Debeau has proposed ""Re-using OPNFV framework tests for LFN projects (Eric Debeau, Cedric Ollivier)" (ONAP session). It has been accepted.
09:00:41 <ollivier> I would propose at least an unconference proposal to discuss about G-release (+ another for F-release?)
09:01:11 <ollivier> I dont' know if other OPNFV devs would be interested by our amazing work.
09:01:19 <ollivier> regarding F release
09:02:10 <ollivier> I think we could introduce our new testcases and maybe an hand on about new k8s testcases.
09:02:16 <ollivier> That could be discussed later.
09:02:34 <LindaWang> Konrad will go there
09:03:18 <ollivier> I will check with Ray the different proposals. Be free to propose topics...
09:03:19 <LindaWang> maybe he could give some introduction about k8s tests if possible
09:03:26 <ollivier> yes I think so
09:03:33 <ollivier> maybe a demo
09:04:00 <LindaWang> i will info him to prepare that.
09:04:18 <ollivier> I don't understand clearly the meeting. uncovered, breakouts ...?
09:04:39 <ollivier> let's discuss offline.
09:04:54 <ollivier> then great week. Now we should test and document. We are time. Congrats!
09:05:01 <ollivier> #endmeeting