14:00:39 <hongbo5656> #startmeeting dovetail weekly meeting
14:00:39 <collabot> Meeting started Fri Jan 27 14:00:39 2017 UTC.  The chair is hongbo5656. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:39 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:39 <collabot> The meeting name has been set to 'dovetail_weekly_meeting'
14:00:50 <hongbo5656> #rollcall
14:01:08 <hongbo5656> #info Hongbo Tian
14:01:17 <tallgren> #info Tapio Tallgren
14:01:19 <rprakash> #info rprakash
14:01:30 <Wenjing> #info Wenjing Chu
14:01:37 <hongbo5656> I am sorry I need to use the IRC meeting due to the firework
14:01:47 <Mika_R> #info Mika Rautakumpu
14:02:54 <hongbo5656> #info the agenda：1) Action items follow up，2) Test area review3) Open issues
14:03:22 <hongbo5656> let us revew the action item from last meeting
14:03:47 <rprakash> #link https://wiki.opnfv.org/display/meetings/Dovetail
14:04:20 <hongbo5656> it seems there are few people joining todays meeting
14:04:34 <hongbo5656> #topic  Action items follow up
14:04:57 <hongbo5656> #info there are there actionitem from last meeting
14:05:12 <hongbo5656> #link https://etherpad.opnfv.org/p/collabrationofdovetail
14:05:35 <hongbo5656> #info Action item: Chris price will check theboard minutes for dovetail release expectation. Chris donley will follow the ray to do that.
14:05:54 <georgk> #info Georg Kunz
14:05:58 <hongbo5656> anyone has the further information about this？
14:06:23 <georgk> one question: no GTM today?
14:06:27 <dneary> IRC only this week?
14:06:31 <tallgren> No GTM
14:06:32 <hongbo5656> yes
14:06:33 <dneary> #info Dave Neary
14:06:37 <tallgren> Only IRC
14:06:38 <dneary> Thanks
14:06:53 <georgk> ok, thanks
14:07:05 <dneary> hongbo5656, Fireworks sounds cook
14:07:07 <dneary> cool
14:07:20 <hongbo5656> ：）
14:07:42 <jose_lausuch> #info Jose Lausuch (30 min)
14:07:59 <tallgren> Dave, 新年快乐
14:08:03 <bryan_att> #info Bryan Sullivan
14:08:20 <hongbo5656> #info Vpnor l3vpn: unclear if the l3vpn feature is completed fully in C release (action:to collect definitive answer for next review). Commented that it should beexcluded on the ground of not yet mature and have adoption (see test case reqs& cvp description)
14:08:20 <dneary> hongbo5656, I brought up the release expectation on a C&C call this week, but we had other topics and did not have time to discuss it. I do not know when the next board meeting will happen
14:08:50 <hongbo5656> thanks dave
14:08:54 <tallgren> Next board meeting is next week
14:09:07 <s1061123> #info Tomofumi Hayashi
14:09:18 <hongbo5656> ok，maybe we can get the feedback next week
14:09:19 <dneary> tallgren, Same to you!
14:09:24 <Wenjing> I'll follow up on that topic.
14:09:32 <hongbo5656> ok，thanks wenjing
14:10:02 <hongbo5656> I will suspend this action iten to the next week
14:10:20 <Wenjing> board meeting is feb 2, cvp update is on agenda.
14:10:42 <hongbo5656> ok
14:10:57 <hongbo5656> thanks wenjing
14:11:44 <hongbo5656> any other informtion about this board related action item？
14:11:55 <jose_lausuch> hongbo5656: just a clarification: bgpvpn (l3vpn) feature was delivered in C release. But according to the discussions, it seems it does not meeet the release expectation to be considered as part of dovetail. Is this correct?
14:12:28 <hongbo5656> one minute， I will move to this topic next
14:12:36 <hongbo5656> to jose_lausuch
14:13:33 <hongbo5656> if no further information， let move to the next topic.
14:14:01 <hongbo5656> #topic bgpvpn (l3vpn) feature for dovetail test area
14:14:12 <hongbo5656> hongbo5656: just a clarification: bgpvpn (l3vpn) feature was delivered in C release. But according to the discussions, it seems it does not meeet the release expectation to be considered as part of dovetail. Is this correct?
14:14:28 <jose_lausuch> http://artifacts.opnfv.org/functest/colorado/docs/release-notes/functest-release.html#scenario-matrix
14:14:34 <jose_lausuch> this is the Functest matrix for C-release
14:14:42 <hongbo5656> the L3 VPN is also the action item before
14:14:43 <jose_lausuch> bgpvpn was tested on apex/fuel
14:15:04 <jose_lausuch> 1 scenario only (so, specific feature to be activiated at deployment time)
14:15:05 <jose_lausuch> http://artifacts.opnfv.org/functest/colorado/docs/release-notes/functest-release.html#colorado-known-restrictions-issues
14:15:12 <jose_lausuch> this is the table of known issues in Colorado
14:15:13 <tallgren> bgpvpn is only for old?
14:15:29 <jose_lausuch> this is the info I can provide as Functester, then it's up to you to decide :)
14:15:32 <jose_lausuch> tallgren: yes
14:15:43 <tallgren> s/old/odl/
14:15:48 <jose_lausuch> tallgren: or at least it was tested only with odl
14:16:51 <tallgren> Can we mark this feature as being part of ODL tests?
14:17:00 <jose_lausuch> what does it mean?
14:17:19 <rprakash> #mean Berrylium
14:17:21 <jose_lausuch> it wasn't tested by default in odl-nofeature scenarios
14:17:45 <jose_lausuch> it's similar to sfc, it owns a scenario and it's tested only with this scenario
14:18:21 <tallgren> In the test area list, some tests are marked to be ODL specific. I am not sure what this means, but I expect that OPNFV platforms with ODL would be tested for bgpvpn
14:18:42 <hongbo5656> yes, we have the ODL releated area
14:18:56 <tallgren> Do we expect ONOS deployments to pass bgpvpn tests?
14:19:14 <dneary> Does L3VPN depend on ODL for now?
14:19:22 <hongbo5656> #link https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Areas+and+Test+Cases
14:19:25 <jose_lausuch> the plan will be to merge that feature with ODL base scenario, but for sure not in Colorado/Danube
14:19:29 <dneary> tallgren got there first...
14:19:30 <jose_lausuch> tallgren: no
14:19:54 <dneary> jose_lausuch, Does it work with OVS ML2?
14:19:55 <jose_lausuch> dneary: you need to ask ONOS guys, but it was never tested in Colorado
14:20:23 <jose_lausuch> dneary: default OVS as all the other scenarios I think
14:20:46 <dneary> It seems clear then that L3VPN does not meet the "wide availability" criteria for the certification test suite
14:21:05 <jose_lausuch> the only thing the bgpvpn scenario activated was the odl l3vpn extensions and the neutron bgpvpn extensions
14:21:44 <jose_lausuch> ok
14:21:47 <jose_lausuch> fair enough :)
14:22:04 <jose_lausuch> so, by wide avaiability I imagine that a feature is supported by ONOS and ODL, right?
14:22:48 <tallgren> Well, we also have the ODL test area...
14:22:52 <hongbo5656> do we have the consesus:L3VPN does not meet the "wide availability" criteria for the certification test suite
14:23:01 <hongbo5656> this time?
14:23:32 <rprakash> you mean by C-Release time?
14:23:45 <tallgren> I am ok with either (a) dropping "*vpn" or (b) testing "*vpn" only for ODL scenarios
14:23:47 <jose_lausuch> that's up to the dovetailers, I can just provide the feedback from testing part :)
14:24:05 <hongbo5656> other comments?
14:24:15 <jose_lausuch> sorry, need to leave to catch a flight
14:24:22 <hongbo5656> ok
14:24:26 <hongbo5656> thanks jose
14:24:30 <jose_lausuch> npbye
14:24:35 <jose_lausuch> bye
14:25:02 <hongbo5656> there are two option: with either (a) dropping "*vpn" or (b) testing "*vpn" only for ODL scenarios?
14:25:19 <rprakash> side note based on release there will always be disparity in features between odl and onos and will that mean one has to wait for others to get wide availability?
14:25:36 <hongbo5656> other opionion?
14:25:55 <bryan_att> we need to focus on cross-scenario tests
14:26:14 <georgk> i think it makes sense to make it ODL scenario specific
14:26:15 <bryan_att> so yes, if a feature is unique to one SDNC then it should likely not be included
14:26:25 <hongbo5656> the a(dropping) VPN ?
14:26:36 <tallgren> ok
14:27:09 <hongbo5656> #agreed  dropping "*vpn" for the dovetail (based on C release)
14:27:13 <bryan_att> if we don't require multi-component support, we will be forced back into the scenario box
14:27:23 <dneary> tallgren, I am fine with having the VPN tests in the 2nd cut - the plugfest test suite - but not in the compliance suite
14:27:37 <bryan_att> agreed
14:27:41 <georgk> ok
14:28:02 <hongbo5656> thanks, let move to the next aciton item
14:28:19 <hongbo5656> #info Action item: Tapio and bry can help movethe test cases and areas content on the wiki into the jira or gerrit. Pierre also can do some help.
14:28:41 <tallgren> Seems this was done already :-)
14:28:50 <hongbo5656> thanks
14:28:57 <hongbo5656> any other comments?
14:29:02 <bryan_att> we need to have an agreed process for using JIRA for test case reviews
14:29:39 <tallgren> Now we have both Jira and gerrit in use
14:29:45 <bryan_att> we should not be using gerrit for reviews of each test, e.g. per the comments in email
14:30:07 <tallgren> test areas are also in gerrit
14:30:12 <bryan_att> JIRA is a much better tool, esp as we are building a base with a large number of tests to review
14:30:21 <bryan_att> same for the test area
14:30:37 <hongbo5656> ususaly, the jira is a good tool for task tracing
14:30:54 <bryan_att> we should review and agree in JIRA, then update the dovetail config thru gerrit
14:30:55 <hongbo5656> the gerrit is used for +1,+2 and merge
14:31:16 <rprakash> #suggesion why not use opnfv.slack.com and create a Dovetail channel to discuss these?
14:31:23 <dneary> I have to admit being a bit overwhelmed by the scale of what is in Jira now.
14:31:27 <bryan_att> too many tools
14:31:51 <dneary> How should we attack this in a way that ensures multiple people have looked at and agreed each test, when we now have 256 open tickets?
14:31:55 <hongbo5656> sorry, rprakash, for the OPNFV, we already have the jira and gerrit
14:32:14 <tallgren> Here are the jira tickets for test areas: https://jira.opnfv.org/browse/DOVETAIL-214?jql=issuetype%20%3D%20Epic%20AND%20text%20~%20%22for%20test%20area%22
14:33:09 <tallgren> Gerrit for test areas is https://gerrit.opnfv.org/gerrit/#/q/project:dovetail+status:open
14:33:27 <hongbo5656> thanks tallgren
14:33:43 <dneary> tallgren, The issue, I guess, is where a discussion of a test case happens.
14:33:44 <tallgren> So only test areas are in gerrit now. Jira and Gerrit are cross-referenced, you can start from either
14:34:13 <hongbo5656> #chair tallgren
14:34:13 <collabot> Current chairs: hongbo5656 tallgren
14:34:24 <hongbo5656> #chair dneary
14:34:24 <collabot> Current chairs: dneary hongbo5656 tallgren
14:34:33 <dneary> So, is a Jira ticket the philosophical "this is appropriate for certification" discussion, and then Gerrit is only about whether the test is correctly described, integrated into the test tool, etc?
14:34:36 <hongbo5656> #chair bryan_att
14:34:36 <collabot> Current chairs: bryan_att dneary hongbo5656 tallgren
14:34:51 <bryan_att> tallgren: I think that some patches are referring to specific tests in gerrit and asking for review
14:35:16 <bryan_att> dneary: yes I propose that
14:35:32 <bryan_att> evidence of test readiness needs to be discussed in JIRA
14:35:47 <dneary> OK. So we need a way to track "under discussion" and "Agreed" test cases in Jira
14:36:01 <dneary> So that we can focus
14:36:32 <bryan_att> yes, and that is something which needs to be figured out - how to do that.  that's why I think we are moving too fast to get tests into the dovetail suite, before the process has been worked out
14:36:33 <hongbo5656> we can use the label to mark that
14:36:44 <dneary> What I would like to avoid is a situation where people are doing work that is later contested by the community, and someone can say "only 2 people even commented in this report - how can you say there was community agreement?"
14:37:20 <bryan_att> I fully expect tests to be contested later, and think that will help us correct issues with lighter-than-needed reviews
14:37:23 <dneary> This will result in a lot of rework, wasted effort, and will also potentially undermine the process in terms of community confidence
14:37:37 <hongbo5656> yes , we need to get those know by the whole community
14:37:51 <bryan_att> we need a process for test inclusion, and exclusion (at least until issues are addressed)
14:39:24 <tallgren> I agree. I like the gerrit view where you can see the +1's and -1's
14:39:40 <tallgren> Probably the same can be done with Jira
14:40:00 <bryan_att> the big issue I see with dovetail currently is that too much effort is being made to complete the 1st release test suite during the crunch time for projects. this needs to be done after code freeze when we have more time to put into it.
14:40:03 <hongbo5656> basicly, the jira and the gerrit are conbined together.
14:40:22 <rprakash> Suggest have a member subcomitee of two service provisers and two Vendors and a subcomitted chair to whet out and the Jira test entries and then put it in meetings of Dovetail for final review and thus expedite the process
14:41:03 <bryan_att> that's why I don't think this first release will be useful - we don't have the time to setup an adequate review process or conduct reviews until mid-Feb at least.
14:42:01 <hongbo5656> for the first release, it is based on the C
14:42:04 <hongbo5656> release
14:42:07 <bryan_att> we should not be defining the dovetail suite for danube while danube is in development - only after it is stable
14:42:10 <rprakash> Well if folks ntrested are willing o volunteer this can be still done timely
14:42:54 <bryan_att> well then it needs to occur when we have time to do it properly, and not in mid-release
14:43:41 <tallgren> If we release Colorado test set after Danube has been released, people might be less than excited
14:43:55 <hongbo5656> for the future dovetail release, we can improve much. some work can be done withe release and some can be done after the release
14:44:14 <bryan_att> I'm not worried about that - we need to focus on quality and not rush things out
14:45:33 <Wenjing> establishing release cycle is also important for open source
14:45:42 <bryan_att> once we create an effective process and get experience executing it, we will be able to get closer to releases and reduce the gap
14:45:59 <bryan_att> that will take time, and we should not force it
14:46:30 <bryan_att> that is why I think a Colorado based release in Q1 is a bad goal
14:46:51 <tallgren> Sorry I have to leave now. Good rest of the day, weekend and year of the rooster to all!
14:47:47 <rprakash> Good or Bad we need a Beta release for C to logically move to D
14:48:25 <dneary> bryan_att, As Hongbo said, we are not defining anything for D. The intention is to validate only against C
14:48:46 <bryan_att> that is clear, the timing and rush is the issue
14:49:27 <hongbo5656> yes, sometimes, based on the C release, it may help dovetail to create the process  and fundation.
14:49:48 <rprakash> remeber back in Arno & Brhmaputra we chse to alingn for 6 month Cadence to help build market confidence in OPNFV
14:49:57 <dneary> I understand. It will continue to feel like a rush until we have broken this up into manageable chunks. I would like to see us discussing no more than 10 tests at one time - focus, agree, add one more test to the "in discussion", move on
14:50:06 <dneary> It will take many weeks to burn down 256 issues
14:50:58 <rprakash> That sounds good to agree on limited tests in C
14:52:37 <rprakash> question is 10 the right number or may be more?
14:52:41 <bryan_att> I am OK with there being more tests, we just need to manage getting them in effectively and I don't feel we are there yet (at a manageable process)
14:53:33 <bryan_att> but I will look into the JIRA setup and make some suggestions as well as participate in reviews
14:53:45 <hongbo5656> good, thanks bryan
14:54:01 <rprakash> I will too help in this
14:54:01 <hongbo5656> any other good suggestions?
14:54:02 <Wenjing> let's focus on the lira/gerrit process and have some experience with it before jumping to conclusions. the issue list does not appear as daunting because some test areas (the whole area) can be determined without going into each cases.
14:54:04 <bryan_att> I had said earlier that we should use the kanban tool and tag the test issues so we can track their progress and area
14:54:49 <bryan_att> Wenjing: as long as the JIRA issue has all the evidence we need on the group being ready, then fine. let's be efficient where we can
14:55:06 <hongbo5656> yes, there is the kanba showing the whole picture
14:55:13 <bryan_att> what we need is a good/agreed method for voting on readiness in JIRA
14:55:32 <bryan_att> a specific Kanban should be used just for the test case review
14:55:53 <bryan_att> that will be less confusing - we don't need all dovetail issues mixed up with test case reviews
14:56:26 <Wenjing> agree.
14:56:31 <hongbo5656> agree
14:56:48 <bryan_att> I will work on the JIRA setup first for that
14:56:49 <hongbo5656> #agree what we need is a good/agreed method for voting on readiness in JIRA
14:57:39 <hongbo5656> #action bryan: work on the JIRA setup such as: specific Kanban should be used just for the test case review
14:57:44 <hongbo5656> thanks bryan
14:58:30 <rprakash> edit the board's filter. Visit the board > administration (configuration) > general > filter.
14:58:54 <rprakash> project = YourProject and labels = YourLabel ORDER BY Rank ASC
14:59:12 <rprakash> his should help set  it
14:59:43 <hongbo5656> we have one minutes left
15:00:01 <bryan_att> I'll send out a mail when I have updated JIRA
15:00:10 <hongbo5656> ok, thanks bryan.
15:00:30 <Wenjing> Happy new year!
15:00:32 <hongbo5656> I need to endthe meeting
15:00:37 <Wenjing> in an hour
15:00:38 <Mika_R> bye
15:00:40 <hongbo5656> happy new year
15:00:44 <hongbo5656> bye
15:00:48 <s1061123> happy new year and bye.
15:00:50 <rprakash> bye
15:00:58 <hongbo5656> #endmeeting