08:00:25 <ollivier> #startmeeting Functest weekly meeting 5 Feb. 2019
08:00:25 <collabot`> Meeting started Tue Feb  5 08:00:25 2019 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:25 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:25 <collabot`> The meeting name has been set to 'functest_weekly_meeting_5_feb__2019'
08:00:36 <ollivier> #info Cédric Ollivier
08:00:42 <juhak> #info Juha Kosonen
08:00:52 <viktor_t> #info Viktor Tikkanen
08:01:45 <ollivier> #topic Action point follow-up
08:01:59 <ollivier> #info rally_full passed successfully in Fuel master gates
08:02:10 <ollivier> #link https://build.opnfv.org/ci/view/functest%20(Installer%20gates)/job/functest-fuel-baremetal-daily-master/672/console
08:02:27 <ollivier> All our testcases are now executed in all gates. That's great!
08:02:47 <ollivier> #info the request about splitting our POD to several all-in-ones was sent last week
08:03:04 <ollivier> Still waiting... I will re-trigger it today.
08:04:20 <ollivier> #topic OpenStack
08:04:30 <ollivier> #info tempest were updated in Gambia to allow running interfaces_by_fixed_ip if one shared network is already defined
08:04:40 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/66837/
08:04:41 <CristinaPauna> #info Cristina Pauna
08:04:47 <ollivier> Hi Cristina.
08:05:12 <CristinaPauna> Hi guys
08:05:40 <ollivier> #info former cloudify images were removed which broke our gates few days ago.
08:06:06 <ollivier> (functest-alpine.sh asks for downloading images in each run)
08:07:07 <ollivier> our cloudify-based testcases had to be disabled. Their descriptors are obsolete (I'm currently working on updating the testcase).
08:07:16 <ollivier> from gambia to master
08:08:31 <ollivier> #topic Gates
08:08:43 <ollivier> #info it seems that Rally testcases can break few fonctional tests which define short timeouts (60 s)
08:08:54 <ollivier> rally_jobs can break neutron-tempest-plugin-api
08:09:01 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/66801/
08:09:09 <ollivier> #link http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-latest-neutron-tempest-plugin-api-run-35/results/neutron-tempest-plugin-api/tempest-report.html
08:09:37 <CristinaPauna> do you know why?
08:11:03 <ollivier> yes. Routers related workloads will increase the duration before other router interfaces are ACTIVE.
08:11:32 <ollivier> Then test_routers.RoutersIpV6Test.test_router_interface_status failed because timeout is reached.
08:11:40 <CristinaPauna> I see
08:11:42 <ollivier> test_routers.RoutersTest.test_router_interface_status
08:11:47 <ollivier> idem
08:12:02 <ollivier> rally_full (+possibly tempest_scenario) may break few tests in tempest_full
08:12:10 <ollivier> #link https://build.opnfv.org/ci/job/functest-opnfv-functest-components-latest-tempest_full-run/
08:12:20 <ollivier> all tempest testcases seem fine since rally_jobs and rally_full moved to specific Jenkins steps
08:13:01 <ollivier> I would like to ensure that all tests in one tiers can run in //. And I would like to check which tiers can run several times in parallel.
08:14:12 <CristinaPauna> sounds good
08:14:25 <ollivier> I am considering switching testcases from tiers to others (moving rally (job and full) testcases in benchmarking, tempest_smoke in healthcheck, tempest_full in smoke)
08:14:46 <ollivier> From the time it's still under huge analysis.
08:15:30 <ollivier> Be free to run that test on your own. It depends on SUT and I'm considering the results from functest-virtual as default.
08:16:24 <juhak> are there any requirements/goal for max test duration? e.g. smoke?
08:16:25 <ollivier> What do you think about moving tempest_smoke in healthcheck? I know Apex selects it when verifying and I far as I know Fuel is passing it very well.
08:17:15 <CristinaPauna> I think that's fine, especially since snaps is out, but do make sure the change is announced to everybody
08:17:46 <ollivier> No. I would consider smoke as equal to Neutron gate. Then tempest_full should be moved to smoke.
08:18:08 <juhak> ok, fine
08:18:19 <ollivier> Rally sanity could remain in smoke because it only runs few iterations. I consider rally jobs and rally full as benchmarking.
08:18:38 <CristinaPauna> what does rally test extra to tempest?
08:18:38 <juhak> agree
08:18:56 <ollivier> I have to check if rally_jobs, rally full, vmtp and shaker can run in //. If so, we could remove components tiers
08:19:28 <ollivier> healthcheck 10 min. smoke would end after 1 hour. Benchmarking 1-2h. Vnf 1h.
08:19:56 <juhak> are there migration tests in tempest?
08:20:25 <ollivier> I don't think in functest-virtual as there is only one compute.
08:20:34 <ollivier> Shaker and vmtp are also disbled by design.
08:21:42 <ollivier> Any feedback is more that welcome. I will run everything in // in OpenLab as well but I could face with Kolla ansible limitations...
08:24:04 <ollivier> I note that you agree on moving testcases as soon as all tests in  tiers can run in //. We will broadcast to all installers if tempest_smoke moved to healthcheck.
08:24:13 <ollivier> Great!
08:24:34 <ollivier> #topic  VNF must be hardenened
08:25:08 <ollivier> heat_ims sometimes failed which is the only error in our gates.
08:25:42 <ollivier> juju_epc failed once (it seems related to slow download => lf network?)
08:25:50 <ollivier> #link https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-latest-juju_epc-run/24/console
08:28:19 <ollivier> Else regarding the overall results, it seems that we are already ready from Stein (our testcases are running vs last week's OpenStack master)
08:28:53 <ollivier> #topic AoB
08:29:07 <CristinaPauna> nothing from me
08:29:32 <juhak> nothing from me either
08:29:55 <viktor_t> nothing from me
08:30:01 <ollivier> That's all from my side and regarding what I have seen in gerrit. BYW, I made lots of presentations internally to promote  our toolings for any CI/CD toolchain (even unlinked to infra)
08:30:47 <ollivier> Than let's close the meeting. I go fixing cloudify-based testcases :).
08:30:50 <ollivier> #endmeeting