13:04:54 <morgan_orange> #startmeeting OPNFV weekly_functest
13:04:54 <collabot> Meeting started Tue Aug 18 13:04:54 2015 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:04:54 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:04:54 <collabot> The meeting name has been set to 'opnfv_weekly_functest'
13:05:10 <morgan_orange> #info Morgan Richomme
13:05:17 <pbandzi> #info Peter Bandzi
13:05:32 <lmcdasm2> #info daniel smith (listenin' )
13:05:39 <anac1> #info Ana Cunha
13:06:34 <meimei> #info meimei
13:07:02 <morgan_orange> Hi everybody
13:07:09 <anac1> hi
13:07:09 <morgan_orange> agenda for today
13:07:27 <morgan_orange> #link https://wiki.opnfv.org/functest_meeting
13:07:51 <morgan_orange> #topic  Feedback from OPNFV Hackfest (end of July)
13:07:52 * lmcdasm2 waves at Morgan!
13:08:36 <morgan_orange> #info results of discussions with Jose available on the wiki
13:08:39 <morgan_orange> #link https://wiki.opnfv.org/functextnexttaks
13:08:52 <morgan_orange> we established a list of tasks
13:08:58 <morgan_orange> there are still some discussions
13:09:12 <morgan_orange> meanwhile a new project has been created to test ODL within OPNFV by d.Farrel
13:09:38 <morgan_orange> pbandzi: did you discuss with him to see if we will reuse what you have done in Arno?
13:09:58 <pbandzi> morgan_orange: not yet
13:10:12 <meimei> no ONOS regarding task?
13:10:16 <pbandzi> I will contact him
13:10:26 <morgan_orange> #action pbandzi discuss with D.Farrel to see ODL extended suites
13:10:45 <morgan_orange> meimei: yes ONOS is under the item 8
13:10:53 <morgan_orange> test other SDN controllers
13:11:22 <morgan_orange> there is a dependcy towards the installers but it is planned...
13:11:39 <morgan_orange> #info ONOS test planned in functest
13:12:00 <morgan_orange> may be in the future it could also be a dedicated test project but for B-release we can do as we did for ODL in A-release
13:12:20 <morgan_orange> please also note that we made the assumption of a B-Release in February..
13:12:35 <morgan_orange> to avoid the Arno race we experienced in May...
13:12:43 <morgan_orange> but pending discussions
13:13:00 <morgan_orange> do you see missing stuff in the table?
13:13:28 <morgan_orange> #action all review Bramhaputra tasks and roadmap for next week
13:14:28 <morgan_orange> #topic Functest B-Release roadmap
13:15:05 <morgan_orange> #info roadmpa will be discussed next week after review and info from TSC (release date discussions Novemeber 2015 / february 2016)
13:15:22 <morgan_orange> big difference between November and February
13:15:28 <morgan_orange> we will see
13:15:41 <morgan_orange> #topic Status on test collection storage API
13:15:50 <morgan_orange> guy_rodrigue: short status?
13:15:56 <guy_rodrigue> #info guy_rodrigue
13:16:14 <guy_rodrigue> On the collection storage API
13:16:31 <guy_rodrigue> I've pushed some code on gerrit
13:16:50 <guy_rodrigue> to manage tests results
13:17:09 <guy_rodrigue> but we should have a mongo db server for testing
13:17:24 <morgan_orange> #link https://wiki.opnfv.org/collection_of_test_results
13:18:01 <morgan_orange> I saw several +2, but was it merged?
13:18:10 <pbandzi> morgan_orange: quick Q: do you mean Controller performance project from D.Farrel ?
13:18:20 <morgan_orange> yes
13:18:25 <pbandzi> ok thanks
13:18:38 <guy_rodrigue> no
13:18:50 <morgan_orange> #info Storage API code pushed and reviewd
13:19:13 <morgan_orange> ok so we shall ask fatih and aric where we could deploy the Mongo DB and start making the first tests
13:19:42 <morgan_orange> I think we could reuse the existing Arno base to start doing tests (needed for SR1) so we could push the vPing under this format for example
13:20:12 <morgan_orange> #action morgan_orange see with fdegir and aricg for mongo DB deployement
13:20:35 <anac1> yardstick has reviewed the code in gerrit, we had some comments
13:20:36 <morgan_orange> guy_rodrigue: you could modify the vPing to use (optionnaly) your API?
13:21:03 <morgan_orange> #link https://gerrit.opnfv.org/gerrit/#/c/1048/
13:21:22 <morgan_orange> anac1: do not see your comment
13:21:26 <guy_rodrigue> okay
13:21:31 <anac1> from qiliang
13:21:56 <morgan_orange> ok
13:22:07 <morgan_orange> I see then
13:22:43 <anac1> we have a corresponding task to deliver to mongodb, qiliang has the task
13:22:45 <morgan_orange> guy_rodrigue: my understanding is that you took them into account in the last patch
13:23:11 <morgan_orange> #action guy_rodrigue check comment on API
13:23:27 <anac1> great, thanks
13:24:10 <morgan_orange> anac1: I think the easy way is that we install the Mongo / test with vPing to check / give the green light to other test projects (after a presentation during the test slot)
13:24:15 <morgan_orange> what do you think?
13:24:57 <morgan_orange> #action plan demo of the API on week 36 during the test project weekly meeting
13:25:10 <morgan_orange> any other question regarding the API
13:25:50 <morgan_orange> #topic Discussion on dashboard
13:26:05 <morgan_orange> #info discussions started during OPNFV hackfest on test dashboarding
13:26:48 <morgan_orange> before the hackfest I looked at bitergia and I learned during the hackfest that OPNFV already subscribed to bitergia...
13:26:50 <morgan_orange> #link http://projects.bitergia.com/opnfv
13:27:26 <morgan_orange> bitergia is used for dashboarding code, commiters,...
13:27:41 <morgan_orange> but it seems that it could be reasonably used to manage tests
13:28:03 <morgan_orange> per POD, per Test project, per testcases,...
13:28:36 <morgan_orange> I started working on json file that could be elaborate from raw information stored through guy_rodrigue API and that could be used to automatically generate a dashboard
13:28:45 <morgan_orange> I will create a wiki page for that
13:29:02 <morgan_orange> if bitergia is not suitable any open source graph js lib could be used
13:29:08 <anac1> looking forward to seeing the wiki
13:29:28 <morgan_orange> #info presentation of bitergia as a possible candidate to display test results
13:29:47 <morgan_orange> #action morgan_orange complete wiki page for test dashboard
13:30:17 <morgan_orange> anac1: ok I will do that and athat is also a topic we may suggest to the test group
13:30:32 <anac1> ok
13:30:56 <morgan_orange> any question on test dashboard?
13:31:10 <anac1> not yet
13:31:10 <morgan_orange> #topic ONOS testing
13:31:20 <morgan_orange> meimei: it is up to you
13:31:36 <meimei> ok
13:32:45 <meimei> #info regarding ONOS testing , yes?
13:32:52 <morgan_orange> yes
13:33:06 <morgan_orange> you have info on ONOS integration regarding the installers?
13:33:06 <meimei> sorry , my network is unstable
13:33:08 <meimei> we propose that we should integrate TESTON into functest.
13:33:20 <meimei> #info we propose that we should integrate TESTON into functest.
13:33:24 <morgan_orange> yes it could be tested as pbandzi did for ODL on release 1
13:33:38 <meimei> ok
13:33:47 <meimei> if we use TESTON to test ONOS, and use ROBOT to test ODL, we should indendify which type of SDN controller has been deployed automaticly
13:33:54 <meimei> yes?
13:34:07 <morgan_orange> that is a good question, it is not clear for me at the moment
13:34:34 <meimei> so , we can integrate TSETON first
13:35:04 <meimei> I have create a jira regarding this
13:35:06 <meimei> https://jira.opnfv.org/browse/FUNCTEST-25
13:35:12 <morgan_orange> yes, I imagine you can also have a llok at the config_functest that prepares the installation of the tooling on the jumphost before running the tests
13:35:22 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-25
13:35:31 <meimei> ok
13:35:31 <morgan_orange> #info ONOS testing uses TESTON
13:35:46 <morgan_orange> #info TESTOn to be integrated in the functest env before running tests
13:36:16 <morgan_orange> #info quesiton on the way to know which SDN controller is active on which installer => to be discussed with genesis/pharos
13:36:29 <morgan_orange> would we have a global description of the OPNFV system that could be used for that?
13:37:05 <morgan_orange> #action meimei ask genesis/releng where we would retrieve the information on the controller used by the installer
13:37:24 <meimei> ok
13:38:06 <morgan_orange> do you know if ONOS project already indicated their constraint/wish to genesis/apex/fuel/joit/..
13:38:35 <meimei> sorry , not very clear
13:39:23 <morgan_orange> frankbrockners: you surely has a good view: did you already get requests from controllers (except ODL)?
13:40:15 <morgan_orange> ok we will see..i can see some impact on the roadmap^of the B-Release.. :)
13:41:08 <morgan_orange> meimei: regarding ONOS testing, if you can complete the wiki, I will update it and give you the hand to create a dedicated page
13:41:48 <morgan_orange> do you have already a list of testcases? could be interesting to compare with ODL testcases.
13:42:11 <morgan_orange> As functest is supposed to address OPNFV as a blackbox, we should not be too specific..
13:42:22 <morgan_orange> any question on ONOS?
13:42:32 <morgan_orange> #topic AoB
13:43:15 <morgan_orange> #info 2 topics to be considered! trafic profile management + testcase declarations
13:43:39 <morgan_orange> #info profile management discussions initiated by Brian
13:43:59 <morgan_orange> #action morgan_orange create a wiki page (explaining arborescence on the wiki + expected doc)
13:44:09 <morgan_orange> #info presentation in the test group to be planned
13:44:40 <morgan_orange> anac1: concerning the template for testcases
13:44:43 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-21
13:44:56 <morgan_orange> any update?
13:45:52 <anac1> sorry, was out for a while. yes, i have a draft, will contact you for first check?
13:46:19 <morgan_orange> #info first draft on testcase template to be shared
13:46:34 <morgan_orange> we can put that popint to the meeting next week? (shared review)
13:46:54 <anac1> yes, it is a wiki page for test case declaration
13:46:59 <morgan_orange> it could also have an influence on the API proposal
13:47:37 <anac1> i have a template i use in yardstick, will also share
13:47:48 <morgan_orange> ok let share it when possible and review it next week during the meeting
13:47:54 <anac1> ok
13:48:05 <guy_rodrigue> I will be interested on this template too
13:48:24 <anac1> ok
13:48:39 <morgan_orange> any other topic?
13:48:46 <anac1> no
13:48:55 <morgan_orange> just one
13:49:11 <morgan_orange> during the Hackfest, we discussed troubleshooting
13:49:34 <morgan_orange> as you know for Arno, we did not really troubleshoot...there are still errors in tempest and in Rally
13:50:09 <morgan_orange> with Jose we thought that it could be a good exercice to keep 5-10 minutes on each functest weekly meeting to discuss an issue
13:50:53 <morgan_orange> we did not create a jira bug for each problem as we do not know if it is due to the test, the tool that tests, the system, the configuration of the system...
13:51:08 <morgan_orange> but there are still errors (see Rally nova or neutron on artifacts)
13:51:26 <morgan_orange> what do you think to suggest in the agenda the "error of the week"?
13:52:22 <meimei> sounds good
13:52:38 <anac1> can you wait one week to fix the fault found?
13:52:47 <anac1> or is it to discuss the cause ?
13:52:53 <morgan_orange> #info morgan_orange suggest to dedicate 5-10 minutes of the functest weekly meeting to a remaining error on Arno (still some mainly on Tempest and Rally - the reason of the 3 ones on ODL are known
13:53:08 <anac1> what about new faults?
13:53:35 <morgan_orange> it is to discuss the cause and try to understand. We understood most of them but not all of them and as we were (and still are not) expert in all the chain, it is a way to share also knowledge
13:53:48 <anac1> ok, understood
13:54:20 <morgan_orange> it could be also the occasion to invite Tempest/Rally/ODL/  expert
13:54:44 <morgan_orange> ok no more questions?
13:54:53 <anac1> not from me
13:55:10 <morgan_orange> when did you suggest the yardstick weekly meeting?
13:55:31 <anac1> mondays utc 7h30 and utc 14
13:55:41 <anac1> to suit all time zones
13:55:50 <morgan_orange> ok
13:56:17 <morgan_orange> have a good week, see you on thursday
13:56:22 <anac1> bye
13:56:23 <morgan_orange> #endmeeting