08:00:53 <morgan_orange> #startmeeting Functest weekly meeting September 27th
08:00:53 <collabot`> Meeting started Tue Sep 27 08:00:53 2016 UTC.  The chair is morgan_orange. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:53 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:00:53 <collabot`> The meeting name has been set to 'functest_weekly_meeting_september_27th'
08:00:58 <morgan_orange> #topic call role
08:01:02 <morgan_orange> #info Morgan Richomme
08:01:02 <May-meimei> # info meimei
08:01:05 <jose_lausuch> #info Jose Lausuch
08:01:08 <lhinds> #info Luke Hinds
08:01:09 <ollivier> #info Cédric Ollivier
08:01:09 <juhak> #info Juha Kosonen
08:01:09 <viktor_t> #info Viktor Tikkanen
08:01:22 <morgan_orange> I think we have new Danube contributor today
08:01:31 <jose_lausuch> May-meimei: does it work with a space between "#" and the keyword?
08:01:44 <May-meimei> #meimei
08:01:49 <May-meimei> :P
08:01:56 <May-meimei> #info meimei
08:01:56 <morgan_orange> May-meimei: do you want to introduce your colleague?
08:01:58 <jose_lausuch> May-meimei: and now you missed info
08:02:01 <jose_lausuch> :D
08:02:11 <May-meimei> not gtm?
08:02:15 <hideyasu> #info Hideyasu Hayashi
08:02:23 <morgan_orange> usually no for the weekly meeting, only IRC
08:02:31 <morgan_orange> GTM only for presentation when needed
08:02:36 <morgan_orange> hi hideyasu
08:02:37 <May-meimei> ok!
08:02:48 <hideyasu> Hi! nice to meet you
08:02:51 <Raghav> #info raghavendrachari
08:03:06 <jose_lausuch> nice to meet you, welcome
08:03:20 <hideyasu> hi jose!
08:03:20 <May-meimei> I want to  introduce my colluage hellon here, she is a bueatiful girl, and she is so intrested in functest
08:03:21 <May-meimei> and she is ready to contribute to D release
08:03:26 <morgan_orange> nice to meet you too, for info hideyasu is working in okinawa labs and made a proposal for a new VNF (discussion planned in next topic)
08:03:32 <May-meimei> but she is not on the irc now
08:03:42 <May-meimei> I will send a mail later
08:04:01 <jose_lausuch> ah ok
08:04:19 <morgan_orange> #topic action point follow up
08:04:33 <morgan_orange> #info AP1: morgan_orange answer to fdegir for the tag/master triggering doc issue for "old" projects
08:04:38 <morgan_orange> #info done, trick used to regenerate doc with correct tag
08:04:43 <morgan_orange> #link http://artifacts.opnfv.org/functest/colorado/docs/release-notes/index.html
08:04:48 <morgan_orange> #info AP2: juhak give feedback on 460
08:04:51 <morgan_orange> #info done
08:04:56 <morgan_orange> #info AP3: juhak close JIRA 460
08:04:59 <morgan_orange> #info done
08:05:04 <morgan_orange> #info AP4: morgan_orange update release note / JIRAs
08:05:08 <morgan_orange> #info done
08:05:31 <morgan_orange> be carefull when creating a JIRA to indicate the fix version( planned version with the fix) for the stats of the release manager...
08:05:41 <morgan_orange> #info AP5: SerenaFeng prepare some slide on Kibana dashboard to share with bitergia people and the community
08:05:46 <morgan_orange> #info done
08:05:51 <morgan_orange> thanks SerenaFeng
08:05:58 <morgan_orange> #action morgan_orange push results in the repo
08:06:01 <morgan_orange> #undo
08:06:01 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Action object at 0x30a5990>
08:06:18 <morgan_orange> #action morgan_orange push presentation on dahsboard in the doc repo
08:06:26 <morgan_orange> #info AP6: all Danube review the exit conditions...no more exit -1 out of the main...
08:06:31 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-496
08:06:36 <morgan_orange> #link https://jira.opnfv.org/browse/FUNCTEST-497
08:06:46 <morgan_orange> #info AP7: morgan_orange send mail to initiate PTL election for Functest
08:06:47 <jose_lausuch> I think we can start working on this
08:06:53 <morgan_orange> #info done: 2 (nice) candidates so far, vote planned for the 10th of october, I will vote for Jose
08:07:01 <jose_lausuch> ollivier had some ideas, would you like to share them in jira or wiki?
08:07:02 <morgan_orange> #info remark from jose_lausuch do you prefer an election at every release (every 6 months) or every 2 releases
08:07:27 <jose_lausuch> morgan_orange: let me explain the first option
08:07:36 <ollivier> jose_lausuch: about AP6?
08:07:58 <jose_lausuch> election every release, with the possibility to re-elect the PTL for max 2 times in a row,
08:07:59 <jose_lausuch> for example
08:08:26 <morgan_orange> I am ok with that
08:08:30 <jose_lausuch> release 1) Mr. X
08:08:30 <jose_lausuch> release 2) Mr. X again
08:08:30 <jose_lausuch> release 3) cannot be Mr. X
08:08:30 <jose_lausuch> release 4) Mr. X can be elected again
08:08:58 <ollivier> Poutine style :)
08:09:02 <morgan_orange> hehe
08:09:32 <jose_lausuch> do you think its a good method?
08:09:35 <jose_lausuch> :)
08:09:37 <morgan_orange> any objection for a vote for PTL (IRC vote during weekly meeting) at each release?
08:10:25 <SerenaFeng> haha, sounds like American President voting :)
08:10:28 <morgan_orange> #action morgan_orange formalize PTL election in wiki (vote planned on the 10th so procedure shall be available 1 week before)
08:10:35 <morgan_orange> for AP6
08:10:50 <morgan_orange> ollivier: jose_lausuch you want to share something now?
08:10:58 <jose_lausuch> SerenaFeng: but the releases take 4 years there :p
08:11:10 <ollivier> it seems great too. I am not convinced  we should limit the votes.
08:11:53 <jose_lausuch> ollivier: limit the votes?
08:12:00 <morgan_orange> #info AP8: all brainstorm on D refactoring API definitions, exceptions,...
08:12:07 <morgan_orange> #info we will see...lots of Danube related topic planned for today
08:12:13 <morgan_orange> #info Colorado is over (or almost 2.0 and 3.0 still in the pipe), long life to Danube....
08:12:29 <morgan_orange> the agenda for today is https://wiki.opnfv.org/display/functest/Functest+Meeting
08:12:33 <morgan_orange> do you want to add anything?
08:12:54 <jose_lausuch> nope
08:12:58 <jose_lausuch> your email about the graphs
08:13:05 <jose_lausuch> but that's there
08:13:08 <morgan_orange> it is in the 3rd point
08:13:18 <morgan_orange> #topic Colorado Status
08:13:25 <morgan_orange> #info it is done!!!!!
08:13:32 <morgan_orange> #info thanks everybody for your contributions
08:14:01 <morgan_orange> of course tehre are so many things to improve...but regarding what we planned in Espoo, we are not far from our target
08:14:17 <ollivier> to forbid Mr X in release 3) But I fully agree on turn over of PTLs.
08:14:52 <jose_lausuch> yes, colorado is out, with a lot of new stuff that we wanted
08:14:54 <jose_lausuch> :)
08:14:55 <morgan_orange> Just a question regarding Flash test, jose_lausuch do you have any status, ide for the next release?
08:15:11 <jose_lausuch> ollivier: with the exception if there is no one else who is candidate
08:15:34 <morgan_orange> #topic https://etherpad.opnfv.org/p/Colorado-Testing-postmortem
08:15:35 <jose_lausuch> morgan_orange: I need to ask Nikolas, he didnt make it for this release
08:15:51 <jose_lausuch> morgan_orange: but either flash test or any other way, we need to have it
08:15:52 <morgan_orange> jose_lausuch: could be linked to the feature on an API to access node for Danube
08:15:57 <morgan_orange> yep
08:16:02 <jose_lausuch> I have noticed we need to provide access to the OS nodes in some way
08:16:08 <morgan_orange> so there is a global etherpad for testing
08:16:13 <jose_lausuch> some test cases do ugly fuel node|grep
08:16:16 <morgan_orange> #link https://etherpad.opnfv.org/p/Colorado-Testing-postmortem
08:16:22 <jose_lausuch> or apex overcloud commands
08:16:25 <morgan_orange> #info people are invited to contribute to this etherpad
08:16:31 <jose_lausuch> all that is ugly and unmaintainamble
08:17:03 <morgan_orange> maybe we can ask all the attendees on the IRC to indicate their strong + and strong - for Functest in Colorado
08:17:41 <ollivier> jose_lausuch: I'm not.
08:18:20 <jose_lausuch> morgan_orange: I think we can review it again next week
08:18:26 <morgan_orange> #info morgan_orange +: we were ready in time with a great documentation, -: more feature projects we improved the journey for integration but were not prescriptive enoiugh
08:18:29 <jose_lausuch> to let people fill in more things
08:18:33 <morgan_orange> ok
08:18:34 <jose_lausuch> but I agree with everything
08:18:49 <morgan_orange> I also put on the wiki the graphs I shared by mail with some of you
08:19:39 <morgan_orange> #action jose_lausuch viktor_t lhinds May-meimei ollivier SerenaFeng juhak Raghav complete etherpad postmortem
08:19:55 <morgan_orange> #info some post mortem graphs: https://wiki.opnfv.org/display/functest/Functest+Colorado
08:20:29 <morgan_orange> #info graphs showed only the validated scenarios (ie. 100% successful), in the reporting, when a scenario was successful we store the info in a log file
08:20:48 <Raghav> OK
08:20:51 <morgan_orange> I added a new log with the % now in order to have the daily status to be able to provide a trend
08:21:21 <morgan_orange> the graphs showed that some scenarios were stable and validated for weeks (and could have been removed from CI to save resources if needed)
08:21:24 <jose_lausuch> I have some comments about the graphs
08:21:41 <morgan_orange> jose_lausuch: yes please
08:21:59 <jose_lausuch> I would compare scenarios per installer in terms of numbers
08:22:04 <jose_lausuch> let me explain
08:22:12 <jose_lausuch> some installers have a lot of scenarios
08:22:19 <jose_lausuch> and the graph is missleading
08:22:25 <jose_lausuch> some other have less scenarios
08:22:32 <jose_lausuch> instead of numbers, I'd say a %
08:22:37 <morgan_orange> a graph is always misleading...
08:22:49 <morgan_orange> and here it is due to the input I have
08:23:04 <jose_lausuch> I know
08:23:05 <morgan_orange> I just have a log file indicating wich scenario/installer was 100% OK
08:23:20 <jose_lausuch> those graphs are a perfect representation of the reallity
08:23:23 <morgan_orange> so I do not have the context (nb of overall scenarios) or the scoring
08:23:37 <morgan_orange> so you are right there are misleading
08:23:39 <jose_lausuch> but we can always tweak it to show % instead of num
08:23:56 <jose_lausuch> because I see apex is below all the others
08:23:57 <morgan_orange> which % you want to indicate
08:24:08 <jose_lausuch> for example
08:24:17 <jose_lausuch> for num scenarios versus time
08:24:43 <jose_lausuch> horizontal : % of the total scenarios of that installer
08:25:10 <jose_lausuch> would that be easy?
08:25:10 <morgan_orange> but I do not have this info or I take a snapshot on the 22 on the scenario that have been released
08:25:15 <jose_lausuch> aha
08:25:18 <jose_lausuch> you are right
08:25:19 <jose_lausuch> mmmm
08:25:24 <jose_lausuch> need to think how to capture that
08:25:30 <morgan_orange> yep
08:25:40 <morgan_orange> let's continue ofline, not sure we can do lots of things
08:25:46 <morgan_orange> we need more contextual element
08:25:48 <jose_lausuch> ok
08:25:51 <morgan_orange> we can capture the final one
08:25:58 <morgan_orange> but inputs were incomplete
08:26:07 <morgan_orange> it is no more the case, we now capture the daily scoring...
08:26:17 <SerenaFeng> you can get the all the results from mongodb, and get the pods_to_scenarios information
08:26:30 <SerenaFeng> which is done in kibana now
08:26:50 <morgan_orange> yes a dashbaord represention will be better
08:26:54 <SerenaFeng> but as the number of results increasing, I don't think it is a good idea
08:26:57 <morgan_orange> a scoring view is incomplete
08:27:10 <morgan_orange> ok let's finish this topic
08:27:18 <jose_lausuch> SerenaFeng: right, but there are other ways, for example asking Jenkins directly via rest api
08:27:30 <morgan_orange> #topic Sync with Okinawa labs / new VNF onboarding for Danube
08:28:03 <hideyasu> Hi, I'm Hideyasu Hayashi belogned to Okinawa Open Laboratory("OOL") .http://www.okinawaopenlabs.org/en/
08:28:11 <morgan_orange> #info status updated in Colorado page https://wiki.opnfv.org/display/functest/Functest+Danube
08:28:30 <hideyasu> OOL made test suite on Functest.The test suite can do testing VNF(VyOS) Interoperability for BGP Protocol.
08:28:49 <SerenaFeng> jose_lausuch If this rest api exists, I would like to use it instead of the existed one in kibana, too memory and time consuming
08:29:06 <jose_lausuch> SerenaFeng: let's talk later :)
08:29:11 <jose_lausuch> SerenaFeng: example: https://build.opnfv.org/ci/api/xml?tree=jobs[displayName,url,lastBuild[fullDisplayName,building,builtOn,timestamp,result]]
08:29:13 <morgan_orange> #link https://vyos.io/
08:29:44 <morgan_orange> hideyasu: for teh VNF in Colorado we sliced the test suites in different categories
08:29:50 <morgan_orange> a VNF will be in the VNF category
08:30:10 <morgan_orange> and today it should be run in the weekly jobs (not used to validate a scenario)
08:30:31 <morgan_orange> code should be pushed in https://git.opnfv.org/cgit/functest/tree/testcases/vnf
08:31:05 <morgan_orange> you can use your code or just a pointer to upstream or other oPNFV repo
08:31:23 <zhipeng> hideyasu morgan_orange is it possible we look at Parser integration with Functest for VNF onboarding in D ?
08:31:25 <morgan_orange> the goal is to be able to automate deployement/test/test reporting
08:31:30 <SerenaFeng> jose_lausuch: ok
08:32:06 <morgan_orange> yes, the approach should be similar to vIMS or Parser. zhipeng or SerenaFeng can provide support for the integration
08:32:18 <zhipeng> cool
08:32:19 <OPNFV-Gerrit-Bot> George Paraskevopoulos proposed functest: Move sfc custom flavor creation  https://gerrit.opnfv.org/gerrit/22395
08:32:53 <morgan_orange> it will be great to have a vRouter included in the test suite..
08:32:59 <morgan_orange> any question for the moment?
08:33:29 <hideyasu> Detail https://drive.google.com/open?id=1yjbVY-mFkeLRPWxz0PwnCTAu7IsehuAeecFy50_3Oj0
08:33:57 <hideyasu> Demo  https://drive.google.com/open?id=0Bzw6mxxJof22azlCVzJiZG1DZGM
08:34:41 <morgan_orange> maybe, if it is possible to put the info on the wiki, I know that not everybody has access to google drive (company security restrictions)
08:34:42 <jose_lausuch> vnf_test :)
08:35:05 <morgan_orange> #info details on the VNF https://drive.google.com/open?id=1yjbVY-mFkeLRPWxz0PwnCTAu7IsehuAeecFy50_3Oj0
08:35:11 <morgan_orange> #link https://drive.google.com/open?id=0Bzw6mxxJof22azlCVzJiZG1DZGM
08:35:39 <morgan_orange> #info integration should be similar to vIMS and Parser, code to be integrated into testcases/vnf on functest repo
08:35:43 <jose_lausuch> hideyasu: can you please attend one of the SDNVPN team weekly meetings?
08:35:44 <hideyasu> Ok,  I will move github.
08:35:54 <jose_lausuch> hideyasu: we are doing similar testing
08:36:43 <morgan_orange> #action hideyasu jose_lausuch sync with SDNVPN
08:37:07 <morgan_orange> but in SDNVPN you are working on the use of the bgpvpn api through ODL backend
08:37:15 <morgan_orange> you are not specifically testing a vRouter, are you?
08:37:42 <jose_lausuch> no
08:37:44 <hideyasu> Now vRouter ony
08:37:53 <hideyasu> only
08:37:59 <jose_lausuch> but we could see if it makes sense to converge some test cases in some way maybe
08:38:05 <morgan_orange> sure
08:38:26 <jose_lausuch> backend-agnostic if applies
08:38:29 <jose_lausuch> maybe its crazy
08:38:33 <jose_lausuch> but would be nice to have a discussion
08:38:38 <morgan_orange> so welcome to OOL
08:38:38 <jose_lausuch> but not here :)
08:38:45 <morgan_orange> #topic Danube: Discussion on framework refactoring / SNAPS
08:38:58 <morgan_orange> #info exchange with Steve (Cable labs) the developer of Snaps
08:39:38 <morgan_orange> #link https://nougat.cablelabs.com/SNAPS/provisioning
08:40:02 <morgan_orange> #info 2 options 1) integration of snaps into Functest 2) use of SNAPS at 3rd party library
08:40:20 <morgan_orange> #info to simplify SNAPS is doing what we soemhow do in functest_utils, oepnstack_utils
08:40:41 <morgan_orange> #info 2 framework will remeain in a first step
08:41:21 <morgan_orange> #info it is up to Steve: if he wants Functest to contribute to SNAPS, option 1 is better, if he prefers to keep the hand on the framework option 2 is better
08:41:36 <morgan_orange> I have no strong opinion, and I thing we just can do a recommendation
08:41:58 <jose_lausuch> recommendation depends on desires of contribution
08:42:04 <jose_lausuch> the last #info you did
08:42:11 <jose_lausuch> so its up to them
08:42:41 <morgan_orange> personnaly I think it make sense to integrate it as the framework looks clean (a little bit more than ours...:))
08:42:47 <morgan_orange> but it is Steve decision
08:42:53 <jose_lausuch> the problem of option 2 is that if we need a functionality, we depend on 1 person to implement it
08:43:05 <morgan_orange> we can contribute upstream...
08:43:11 <morgan_orange> but I agree
08:43:44 <morgan_orange> #topic Meetup (Lannion / Huawei Connect EU (Paris) / Barcelona (OpenStack Summit))
08:44:14 <jose_lausuch> https://framadate.org/sK56myIDT6KGeRNv
08:44:17 <morgan_orange> #info several options for the meetup - but I think we need to formalize as we did in Espoo
08:44:29 <morgan_orange> #info another option has been raised
08:45:44 <morgan_orange> #info Huawei organized Huawei connect EU in Paris on the 20th and 21th of October
08:45:57 <morgan_orange> #info would be OK to organize a Functets meetup here
08:46:26 <morgan_orange> #info so 3 options: Lannion/orange (pool), Huawei Connect / Barcelona OpenStack summit
08:46:38 <morgan_orange> we will do something in Barcelona
08:46:59 <jose_lausuch> not sure if I will be able to attend, will try
08:47:04 <jose_lausuch> but is it in the poll?
08:47:16 <morgan_orange> no the poll was just for the Meetup in Lannion
08:47:53 <morgan_orange> one issue on my side I cannot attend Huawei connect in Paris (which is somehow probably more convenient that Lannion)
08:48:15 <jose_lausuch> its complicated for me as well
08:48:36 <morgan_orange> ok so if we both cannot attend, it will be hard
08:48:51 <jose_lausuch> :)
08:49:06 <morgan_orange> so we would maintain the option 1 (in Lannion)
08:49:30 <jose_lausuch> I would prefer so if possible
08:49:36 <morgan_orange> could people who can attend vote https://framadate.org/sK56myIDT6KGeRNv
08:49:48 <morgan_orange> viktor_t: any feedback from Nokia?
08:50:06 <viktor_t> I'm pessimistic at the moment...
08:50:36 <juhak> will there be conference call too?
08:50:52 <morgan_orange> yes we will have a conf bridge
08:51:05 <morgan_orange> maybe you can vote and precise (remote)
08:51:28 <juhak> ok good, I'll attend remotely
08:51:51 <jose_lausuch> face to face is always better, but if there is not possible, conf call should be ok
08:52:07 <SerenaFeng> conf bridge is a good idea, since I cannot attend Lannion
08:52:11 <morgan_orange> in // SerenaFeng or May-meimei do you think it would ne possible to organize an Asian meetup?
08:52:19 <morgan_orange> for Functest
08:52:59 <morgan_orange> I think we should have a global view of Danube roadmap beginning of November
08:53:17 <SerenaFeng> not very sure of that, I will ask the leader and feed back to you later, is that ok?
08:53:20 <morgan_orange> there are already several things listed in teh Danube page but if we can convince new contributors...
08:53:26 <morgan_orange> SerenaFeng: ok
08:54:04 <May-meimei> morgan_orange: there is a meetup in korea
08:54:17 <May-meimei> I think we can add some schedule
08:54:32 <morgan_orange> as a summary, we will probably organize a meetup in Lannion (France), a meeting will be organized during the OpenStack Summit, and hopefully some in Asia when we can add Functest topic
08:55:02 <morgan_orange> I know that there was a meetup in Japan but it was an OPNFV meetup, not specific to Functest
08:55:36 <morgan_orange> ok May-meimei if we can add a topic to collect new tests and invite new contributors to join, it will be great
08:55:49 <morgan_orange> #topic Discuss Dovetail test cases Dovetail test use cases
08:56:20 <morgan_orange> I had a look at dovetail page and was a bit surprise, the first table looks like the Tempest tests we are performing
08:56:39 <jose_lausuch> morgan_orange: there is a docker container also for dovetail already :)
08:56:45 <morgan_orange> How do you understand the wiki page https://wiki.opnfv.org/display/dovetail/Dovetail+test+use+cases
08:57:11 <morgan_orange> I have not a clear view on the differences with our own tests
08:57:21 <morgan_orange> sometimes it is mentioned smoke (functest)
08:57:33 <morgan_orange> but as far as I understand all the first tests are also covered by tempest..
08:57:40 <jose_lausuch> morgan_orange: I'm not sure, but they won't write test case, they'll write specifications
08:57:43 <jose_lausuch> and use the existing tools
08:57:52 <jose_lausuch> like tempest in functest or some other in yardstick
08:58:05 <morgan_orange> ok so it is a sort of ETSI spec...for testing
08:58:15 <jose_lausuch> sort of
08:58:20 <jose_lausuch> some certification specs
08:58:23 <jose_lausuch> or whatever its called
08:58:32 <jose_lausuch> I'm not following 100% all the discussions but sort of that
08:58:42 <morgan_orange> ok I will contact hongbo to be sure that I got the good view
08:58:56 <jose_lausuch> ok
08:59:01 <morgan_orange> #action morgan_orange contact hongbo to see articulation dovetail tests / functest tests (especially in the VIM section)
08:59:05 <jose_lausuch> that's what I got when I had a conversation with them
08:59:09 <jose_lausuch> I hope I got it right
08:59:12 <morgan_orange> does anyone plan to attend the plugfest?
08:59:22 <morgan_orange> #topic AoB
08:59:24 <jose_lausuch> not for now, are you?
08:59:30 <morgan_orange> no
08:59:33 <morgan_orange> not this time
08:59:45 <morgan_orange> but 2 Orange colleagues will be there (but not involved in Functest)
08:59:45 <jose_lausuch> I'll talk to my boss
09:00:00 <jose_lausuch> I think my company is seeing who can/should go or so
09:00:07 <morgan_orange> any other topic you want to share today?
09:00:16 <juhak> still under consideration
09:00:45 <morgan_orange> juhak: and are you ....optimistic?
09:00:55 <jose_lausuch> :)
09:00:56 <juhak> yes :)
09:01:16 <morgan_orange> ok if nothing to add, I will close the meeting
09:01:33 <morgan_orange> thanks for attending, congratulations for Colorado and enjoy Danube
09:01:45 <morgan_orange> #endmeeting