08:00:27 <ollivier> #startmeeting Functest weekly meeting 12 June. 2018
08:00:27 <collabot_> Meeting started Tue Jun 12 08:00:27 2018 UTC.  The chair is ollivier. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:27 <collabot_> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:27 <collabot_> The meeting name has been set to 'functest_weekly_meeting_12_june__2018'
08:00:37 <ollivier> #info Cédric Ollivier
08:00:45 <ollivier> hello
08:00:55 <depo_> #info Delia Popescu
08:00:58 <juhak> #info Juha Kosonen
08:00:59 <depo_> hello
08:01:03 <juhak> hi
08:01:33 <pkaralis> #info Panagiotis Karalis
08:02:07 <depo_> ollivier: Yes, Alexandru is updateing to Queens
08:03:15 <ollivier> depo_: ok. then we won't get any master results. I hope he will finish tonight ;)
08:03:19 <viktor_t> #info Viktor Tikkanen
08:03:43 <ollivier> #topic Action point follow-up
08:03:54 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-975
08:04:03 <ollivier> #info Plugfest
08:04:53 <ollivier> depo_: juhak: could you please report the actions/comments about Functest during the events?
08:06:09 <juhak> this time I was mainly working on plugtests side,
08:06:19 <juhak> but we faced couple of vping issues
08:06:33 <depo_> ollivier: comments: as I mentioned in the email, K8 for healthcheck to be ready
08:06:47 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-976
08:06:52 <depo_> to have more participants on test projects meeting
08:07:27 <depo_> try to reduce/if have any work overlapping between test in all test projects
08:07:51 <ollivier> Yes I have to participate to the test project meeting. It  was difficult with my previous agenda (internal)
08:08:20 <ollivier> Regarding overlap, I think the current model is fine. It seems an overall idea from a release manager
08:08:58 <depo_> yes
08:09:17 <ollivier> The key issue is how to promote test frameworks in the OPNFV context (no resources, no gates)
08:10:55 <depo_> nad there was a discussion if test projects hsould be used outside OPNFV context
08:10:57 <ollivier> About vping, yes we have to check the results in limits: no ip, vlan as overlay, hugepage
08:11:22 <ollivier> It's already the case for Functest
08:11:56 <ollivier> Functest is now independant from OPNFV installers and could work in OpenStack gates. It's already working in ONAP gates.
08:12:30 <ollivier> We have done a great work for Fraser to enforce that.
08:12:53 <depo_> great
08:14:05 <ollivier> It would be great to run functest in OpenStack Gates as third parties. We quickly discussed with Tim Rozet to run Functest in triple-o last week about that (here)
08:14:22 <ollivier> It had been discussed with Fatih lots of times.
08:14:51 <ollivier> (Portland Plugfest)
08:15:33 <juhak> another question was that, in generally, could/should we utilize clouds.yaml instead of bunch of OS_ variables
08:15:52 <ollivier> yes now we should as we switched to Shade
08:16:31 <ollivier> We do first update our vnfs and then the update is trivial.
08:16:50 <juhak> that's good
08:16:58 <ollivier> As we were facing with lots of troubles about cloudify_ims, it forbade the updates to Shade.
08:18:07 <ollivier> Any help for updating vnfs is more than welcome. The key part i to refactor vnf.py to allocate ressources. There are lots of duplicate codes highlighted by pylint.
08:19:55 <ollivier> Thank you for the report.
08:20:31 <ollivier> #topic Functest Fraser
08:20:45 <ollivier> #info APEX os-nosdn-nofeature-ha is still fully verified
08:20:59 <ollivier> #info cloudify_ims has been hugely updated to avoid race conditions, forks, tenancy and role issues:
08:21:15 <ollivier> #info vping results are now taken into account
08:21:36 <ollivier> #info patrole failed once vs APEX os-nosdn-nofeature-noha (APEX?)
08:21:46 <ollivier> #info juju_epc failed once vs APEX due to network outages (Eurecom?)
08:22:54 <ollivier> #info patrole failed once on ARM too https://build.opnfv.org/ci/view/functest/job/functest-fuel-armband-baremetal-daily-fraser/lastFailedBuild/
08:23:15 <ollivier> Globally the results are good.
08:23:56 <ollivier> It mainly remains one enhancement about cloudify_ims (race condition between clearwater and cloudify).
08:24:17 <ollivier> + vping?
08:24:55 <ollivier> All actions are detailed on the wiki page.
08:25:45 <ollivier> #topic Functest Gambia
08:26:04 <ollivier> #info only cloudify_ims is failing vs Fuel
08:26:17 <ollivier> #link https://jira.opnfv.org/browse/FUEL-376
08:27:03 <ollivier> badly Fuel is being updated then Functest is no longer verified...
08:27:32 <ollivier> Daisy is stopped. Apex is facing one bug. Compass is mainly running k8s scenarios.
08:27:41 <ollivier> JIOD is stopped.
08:28:19 <ollivier> I will write an email to Tim and David.
08:29:21 <ollivier> #info refstack client has not been updated yet.
08:29:39 <ollivier> I would propose to normalize tempest files in Functest instead
08:29:53 <ollivier> agree?
08:30:08 <juhak> agree
08:30:16 <depo_> agree
08:30:21 <ollivier> Then refstack is just a list.
08:31:29 <ollivier> Thank you. Linda is not connected. I will check offline if she takes the action.
08:31:40 <ollivier> #info cloudify_ims has to be improved (trustabiliy)
08:31:55 <ollivier> #info vping testcase seem working well in nominal cases but several JIRA tickets have to be taken into account (or simply closed)
08:32:05 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-976
08:32:17 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-970
08:32:23 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-973
08:32:29 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-965
08:32:39 <ollivier> (the last 2 bugs are already closed)
08:32:47 <ollivier> + unit tests (partially done here)
08:33:42 <ollivier> vping is the first testcase updated to shade. We will ensure it works before
08:33:51 <ollivier> updating  vnfs.
08:34:12 <depo_> FUNCTEST-973; the problem was in Fule. I send the JIRA for close. Indeed I do not think we set metadata for image, but it does not impact the result
08:34:58 <depo_> or was in Fuel, it might be resolved already, I have to check. It could not ping VMs from different compute nodes
08:35:12 <ollivier> Yes. Could you please confirm it works now? It has to conform with dpdpk scenarios.
08:35:28 <depo_> I will check and let you know
08:35:57 <ollivier> great. Globally all testcases should work if dpdk.
08:36:32 <ollivier> juhak: thank you for -976
08:36:44 <ollivier> 970 is still unclear.
08:37:23 <ollivier> we could swith to fip instead of vm's public ip if needed.
08:37:50 <ollivier> both data should be equal in nominal cases.
08:38:03 <juhak> ollivier: np, I'll submit a patch -976
08:38:40 <ollivier> #info new testcase proposals
08:39:11 <ollivier> Integrate sdnvpn and sfc tempest plugins
08:39:17 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-979
08:39:25 <ollivier> Integrate OpenStack benchmark tools as Rally
08:39:30 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-980
08:39:50 <ollivier> shaker is already done here. vmtp is finished (an update in Vmtp is required)
08:40:10 <ollivier> Create testcases about cryptography
08:40:16 <ollivier> #link https://jira.opnfv.org/browse/FUNCTEST-981
08:40:28 <ollivier> It's a legal concerns at least in France
08:40:52 <depo_> I will work on it asap
08:41:19 <ollivier> great. It would great to check barbican but I'm not sure one installer supports it
08:41:30 <ollivier> depo_: CristinaPauna: Fuel?
08:42:55 <ollivier> I will send an email to installer teams.
08:43:36 <depo_> I don't know, have to check
08:43:55 <ollivier> #info SNAPS supports specific flavor metadata
08:44:30 <ollivier> #info https://gerrit.opnfv.org/gerrit/#/c/58321/
08:44:37 <ollivier> #undo
08:44:37 <collabot_> Removing item from minutes: <MeetBot.ircmeeting.items.Info object at 0x287dad0>
08:44:40 <ollivier> #link https://gerrit.opnfv.org/gerrit/#/c/58321/
08:45:04 <ollivier> pkaralis: thank you
08:45:29 <pkaralis> ollivier, thank you for support and review!
08:45:36 <ollivier> pkaralis: no pb.
08:45:42 <ollivier> #info we can't declare new testcases (cinder, vtmp, etc...) due to TestAPI changes
08:46:07 <ollivier> Has it been discussed in Nice? It's just a major issue
08:47:08 <depo_> not that I know
08:47:21 <juhak> I'm not aware of either
08:47:27 <ollivier> Could you please confirm that you agree with the new testcase proposals? Be free to suggest other ideas.
08:47:38 <ollivier> depo_: juhak: thank you
08:48:18 <ollivier> georgk: are you aware about htat? It seems related to Dovetail?
08:48:50 <ollivier> And ZTE is not allowed to fix that :)
08:49:13 <ollivier> :(
08:49:37 <georgk> ollivier: Trevor B. mentioned it briefly and that he is looking into this
08:49:55 <georgk> it seems I need to check this, too
08:50:24 <georgk> I am currently not aware of Dovetail-related changed, i.e., why somebody changed the test API for Dovetail
08:50:35 <georgk> this does not make too much sense to me
08:51:07 <ollivier> georgk: thank you. It's unclear. But our testcases have been modified in DB as well.
08:51:45 <ollivier> georgk: I like the KIS model and the previous API was hugely better. And by the way, I don't understand why a client should set an UUID
08:52:03 <georgk> ollivier: interesting, in which way?
08:52:37 <ollivier> georgk: The request is forbidden due to a missing data (id)
08:53:19 <georgk> ollivier: ok
08:53:52 <ollivier> georgk: it's even worse if no project asks for that change :)
08:54:10 <georgk> :-)
08:54:23 <ollivier> #topic AoB
08:55:05 <ollivier> other topics?
08:55:33 <depo_> yes
08:55:54 <depo_> the patch I sent for cleanup the logs. I think the log output is just to big
08:56:00 <depo_> maybe split it?
08:56:17 <depo_> we could have debug log files separate?
08:56:22 <depo_> just a suggestion
08:56:47 <depo_> I agree is good for debuging, but it has a lott of info in one place
08:56:55 <ollivier> 10 Mb is not so big. That's an interesting point.
08:57:29 <depo_> not just the size, but the info for one testcase
08:57:44 <depo_> is a lott.. if it works is redundant
08:57:44 <ollivier> YWe could easify filter one header
08:58:06 <ollivier> agree. But we were not able to debug vnfs (cloudify, juju, etc...)
08:58:42 <ollivier> And worse, it was bad for Features.
08:58:48 <depo_> separate log for vnfs?
09:00:15 <ollivier> Why not? I would say it's easy to filter for one specific test via a sed command.
09:00:36 <ollivier> But we have to discuss about that if we introduce // testing.
09:01:27 <ollivier> We could mute all messages out of Functest but it forces all Features to set __name__ as logger name
09:01:33 <ollivier> out of OPNFV
09:01:40 <ollivier> see barometer
09:01:46 <ollivier> snaps
09:02:14 <ollivier> My main concer is about // testing if 2 testcases write in the same functest.log
09:02:26 <ollivier> then the topic is opened.
09:02:32 <depo_> we don't have to make a decision now. I will look into how other logs are being set up so that I can have a clear image
09:02:47 <depo_> maybe we can discuss it next week?
09:03:49 <ollivier> yes. I can add a specific topic next week. For me the key point is to keep relevant data. The console has to be clear. I consider as fine if functest.log is about 10Mb
09:04:00 <ollivier> (see neutron/nova logs)
09:04:22 <ollivier> Thank you for that topic
09:04:29 <depo_> :)
09:04:40 <ollivier> Then we are a little bit late. Sorry for that.
09:04:46 <ollivier> Thank you very much.
09:05:00 <ollivier> Great Work!
09:05:00 <ollivier> #endmeeting