13:04:54 <morgan_orange> #startmeeting OPNFV weekly_functest 13:04:54 <collabot> Meeting started Tue Aug 18 13:04:54 2015 UTC. The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:04:54 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 13:04:54 <collabot> The meeting name has been set to 'opnfv_weekly_functest' 13:05:10 <morgan_orange> #info Morgan Richomme 13:05:17 <pbandzi> #info Peter Bandzi 13:05:32 <lmcdasm2> #info daniel smith (listenin' ) 13:05:39 <anac1> #info Ana Cunha 13:06:34 <meimei> #info meimei 13:07:02 <morgan_orange> Hi everybody 13:07:09 <anac1> hi 13:07:09 <morgan_orange> agenda for today 13:07:27 <morgan_orange> #link https://wiki.opnfv.org/functest_meeting 13:07:51 <morgan_orange> #topic Feedback from OPNFV Hackfest (end of July) 13:07:52 * lmcdasm2 waves at Morgan! 13:08:36 <morgan_orange> #info results of discussions with Jose available on the wiki 13:08:39 <morgan_orange> #link https://wiki.opnfv.org/functextnexttaks 13:08:52 <morgan_orange> we established a list of tasks 13:08:58 <morgan_orange> there are still some discussions 13:09:12 <morgan_orange> meanwhile a new project has been created to test ODL within OPNFV by d.Farrel 13:09:38 <morgan_orange> pbandzi: did you discuss with him to see if we will reuse what you have done in Arno? 13:09:58 <pbandzi> morgan_orange: not yet 13:10:12 <meimei> no ONOS regarding task? 13:10:16 <pbandzi> I will contact him 13:10:26 <morgan_orange> #action pbandzi discuss with D.Farrel to see ODL extended suites 13:10:45 <morgan_orange> meimei: yes ONOS is under the item 8 13:10:53 <morgan_orange> test other SDN controllers 13:11:22 <morgan_orange> there is a dependcy towards the installers but it is planned... 13:11:39 <morgan_orange> #info ONOS test planned in functest 13:12:00 <morgan_orange> may be in the future it could also be a dedicated test project but for B-release we can do as we did for ODL in A-release 13:12:20 <morgan_orange> please also note that we made the assumption of a B-Release in February.. 13:12:35 <morgan_orange> to avoid the Arno race we experienced in May... 13:12:43 <morgan_orange> but pending discussions 13:13:00 <morgan_orange> do you see missing stuff in the table? 13:13:28 <morgan_orange> #action all review Bramhaputra tasks and roadmap for next week 13:14:28 <morgan_orange> #topic Functest B-Release roadmap 13:15:05 <morgan_orange> #info roadmpa will be discussed next week after review and info from TSC (release date discussions Novemeber 2015 / february 2016) 13:15:22 <morgan_orange> big difference between November and February 13:15:28 <morgan_orange> we will see 13:15:41 <morgan_orange> #topic Status on test collection storage API 13:15:50 <morgan_orange> guy_rodrigue: short status? 13:15:56 <guy_rodrigue> #info guy_rodrigue 13:16:14 <guy_rodrigue> On the collection storage API 13:16:31 <guy_rodrigue> I've pushed some code on gerrit 13:16:50 <guy_rodrigue> to manage tests results 13:17:09 <guy_rodrigue> but we should have a mongo db server for testing 13:17:24 <morgan_orange> #link https://wiki.opnfv.org/collection_of_test_results 13:18:01 <morgan_orange> I saw several +2, but was it merged? 13:18:10 <pbandzi> morgan_orange: quick Q: do you mean Controller performance project from D.Farrel ? 13:18:20 <morgan_orange> yes 13:18:25 <pbandzi> ok thanks 13:18:38 <guy_rodrigue> no 13:18:50 <morgan_orange> #info Storage API code pushed and reviewd 13:19:13 <morgan_orange> ok so we shall ask fatih and aric where we could deploy the Mongo DB and start making the first tests 13:19:42 <morgan_orange> I think we could reuse the existing Arno base to start doing tests (needed for SR1) so we could push the vPing under this format for example 13:20:12 <morgan_orange> #action morgan_orange see with fdegir and aricg for mongo DB deployement 13:20:35 <anac1> yardstick has reviewed the code in gerrit, we had some comments 13:20:36 <morgan_orange> guy_rodrigue: you could modify the vPing to use (optionnaly) your API? 13:21:03 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/1048/ 13:21:22 <morgan_orange> anac1: do not see your comment 13:21:26 <guy_rodrigue> okay 13:21:31 <anac1> from qiliang 13:21:56 <morgan_orange> ok 13:22:07 <morgan_orange> I see then 13:22:43 <anac1> we have a corresponding task to deliver to mongodb, qiliang has the task 13:22:45 <morgan_orange> guy_rodrigue: my understanding is that you took them into account in the last patch 13:23:11 <morgan_orange> #action guy_rodrigue check comment on API 13:23:27 <anac1> great, thanks 13:24:10 <morgan_orange> anac1: I think the easy way is that we install the Mongo / test with vPing to check / give the green light to other test projects (after a presentation during the test slot) 13:24:15 <morgan_orange> what do you think? 13:24:57 <morgan_orange> #action plan demo of the API on week 36 during the test project weekly meeting 13:25:10 <morgan_orange> any other question regarding the API 13:25:50 <morgan_orange> #topic Discussion on dashboard 13:26:05 <morgan_orange> #info discussions started during OPNFV hackfest on test dashboarding 13:26:48 <morgan_orange> before the hackfest I looked at bitergia and I learned during the hackfest that OPNFV already subscribed to bitergia... 13:26:50 <morgan_orange> #link http://projects.bitergia.com/opnfv 13:27:26 <morgan_orange> bitergia is used for dashboarding code, commiters,... 13:27:41 <morgan_orange> but it seems that it could be reasonably used to manage tests 13:28:03 <morgan_orange> per POD, per Test project, per testcases,... 13:28:36 <morgan_orange> I started working on json file that could be elaborate from raw information stored through guy_rodrigue API and that could be used to automatically generate a dashboard 13:28:45 <morgan_orange> I will create a wiki page for that 13:29:02 <morgan_orange> if bitergia is not suitable any open source graph js lib could be used 13:29:08 <anac1> looking forward to seeing the wiki 13:29:28 <morgan_orange> #info presentation of bitergia as a possible candidate to display test results 13:29:47 <morgan_orange> #action morgan_orange complete wiki page for test dashboard 13:30:17 <morgan_orange> anac1: ok I will do that and athat is also a topic we may suggest to the test group 13:30:32 <anac1> ok 13:30:56 <morgan_orange> any question on test dashboard? 13:31:10 <anac1> not yet 13:31:10 <morgan_orange> #topic ONOS testing 13:31:20 <morgan_orange> meimei: it is up to you 13:31:36 <meimei> ok 13:32:45 <meimei> #info regarding ONOS testing , yes? 13:32:52 <morgan_orange> yes 13:33:06 <morgan_orange> you have info on ONOS integration regarding the installers? 13:33:06 <meimei> sorry , my network is unstable 13:33:08 <meimei> we propose that we should integrate TESTON into functest. 13:33:20 <meimei> #info we propose that we should integrate TESTON into functest. 13:33:24 <morgan_orange> yes it could be tested as pbandzi did for ODL on release 1 13:33:38 <meimei> ok 13:33:47 <meimei> if we use TESTON to test ONOS, and use ROBOT to test ODL, we should indendify which type of SDN controller has been deployed automaticly 13:33:54 <meimei> yes? 13:34:07 <morgan_orange> that is a good question, it is not clear for me at the moment 13:34:34 <meimei> so , we can integrate TSETON first 13:35:04 <meimei> I have create a jira regarding this 13:35:06 <meimei> https://jira.opnfv.org/browse/FUNCTEST-25 13:35:12 <morgan_orange> yes, I imagine you can also have a llok at the config_functest that prepares the installation of the tooling on the jumphost before running the tests 13:35:22 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-25 13:35:31 <meimei> ok 13:35:31 <morgan_orange> #info ONOS testing uses TESTON 13:35:46 <morgan_orange> #info TESTOn to be integrated in the functest env before running tests 13:36:16 <morgan_orange> #info quesiton on the way to know which SDN controller is active on which installer => to be discussed with genesis/pharos 13:36:29 <morgan_orange> would we have a global description of the OPNFV system that could be used for that? 13:37:05 <morgan_orange> #action meimei ask genesis/releng where we would retrieve the information on the controller used by the installer 13:37:24 <meimei> ok 13:38:06 <morgan_orange> do you know if ONOS project already indicated their constraint/wish to genesis/apex/fuel/joit/.. 13:38:35 <meimei> sorry , not very clear 13:39:23 <morgan_orange> frankbrockners: you surely has a good view: did you already get requests from controllers (except ODL)? 13:40:15 <morgan_orange> ok we will see..i can see some impact on the roadmap^of the B-Release.. :) 13:41:08 <morgan_orange> meimei: regarding ONOS testing, if you can complete the wiki, I will update it and give you the hand to create a dedicated page 13:41:48 <morgan_orange> do you have already a list of testcases? could be interesting to compare with ODL testcases. 13:42:11 <morgan_orange> As functest is supposed to address OPNFV as a blackbox, we should not be too specific.. 13:42:22 <morgan_orange> any question on ONOS? 13:42:32 <morgan_orange> #topic AoB 13:43:15 <morgan_orange> #info 2 topics to be considered! trafic profile management + testcase declarations 13:43:39 <morgan_orange> #info profile management discussions initiated by Brian 13:43:59 <morgan_orange> #action morgan_orange create a wiki page (explaining arborescence on the wiki + expected doc) 13:44:09 <morgan_orange> #info presentation in the test group to be planned 13:44:40 <morgan_orange> anac1: concerning the template for testcases 13:44:43 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-21 13:44:56 <morgan_orange> any update? 13:45:52 <anac1> sorry, was out for a while. yes, i have a draft, will contact you for first check? 13:46:19 <morgan_orange> #info first draft on testcase template to be shared 13:46:34 <morgan_orange> we can put that popint to the meeting next week? (shared review) 13:46:54 <anac1> yes, it is a wiki page for test case declaration 13:46:59 <morgan_orange> it could also have an influence on the API proposal 13:47:37 <anac1> i have a template i use in yardstick, will also share 13:47:48 <morgan_orange> ok let share it when possible and review it next week during the meeting 13:47:54 <anac1> ok 13:48:05 <guy_rodrigue> I will be interested on this template too 13:48:24 <anac1> ok 13:48:39 <morgan_orange> any other topic? 13:48:46 <anac1> no 13:48:55 <morgan_orange> just one 13:49:11 <morgan_orange> during the Hackfest, we discussed troubleshooting 13:49:34 <morgan_orange> as you know for Arno, we did not really troubleshoot...there are still errors in tempest and in Rally 13:50:09 <morgan_orange> with Jose we thought that it could be a good exercice to keep 5-10 minutes on each functest weekly meeting to discuss an issue 13:50:53 <morgan_orange> we did not create a jira bug for each problem as we do not know if it is due to the test, the tool that tests, the system, the configuration of the system... 13:51:08 <morgan_orange> but there are still errors (see Rally nova or neutron on artifacts) 13:51:26 <morgan_orange> what do you think to suggest in the agenda the "error of the week"? 13:52:22 <meimei> sounds good 13:52:38 <anac1> can you wait one week to fix the fault found? 13:52:47 <anac1> or is it to discuss the cause ? 13:52:53 <morgan_orange> #info morgan_orange suggest to dedicate 5-10 minutes of the functest weekly meeting to a remaining error on Arno (still some mainly on Tempest and Rally - the reason of the 3 ones on ODL are known 13:53:08 <anac1> what about new faults? 13:53:35 <morgan_orange> it is to discuss the cause and try to understand. We understood most of them but not all of them and as we were (and still are not) expert in all the chain, it is a way to share also knowledge 13:53:48 <anac1> ok, understood 13:54:20 <morgan_orange> it could be also the occasion to invite Tempest/Rally/ODL/ expert 13:54:44 <morgan_orange> ok no more questions? 13:54:53 <anac1> not from me 13:55:10 <morgan_orange> when did you suggest the yardstick weekly meeting? 13:55:31 <anac1> mondays utc 7h30 and utc 14 13:55:41 <anac1> to suit all time zones 13:55:50 <morgan_orange> ok 13:56:17 <morgan_orange> have a good week, see you on thursday 13:56:22 <anac1> bye 13:56:23 <morgan_orange> #endmeeting