15:05:11 <yamahata> #startmeeting neutron_northbound
15:05:11 <odl_meetbot> Meeting started Wed Apr 13 15:05:11 2016 UTC.  The chair is yamahata. Information about MeetBot at http://ci.openstack.org/meetbot.html.
15:05:11 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:05:11 <odl_meetbot> The meeting name has been set to 'neutron_northbound'
15:05:19 <yamahata> #chair rcurran mkolesni
15:05:19 <odl_meetbot> Current chairs: mkolesni rcurran yamahata
15:05:27 <yamahata> in case of network problem
15:05:31 <yamahata> #topic agenda bashing and roll call
15:05:33 <yamahata> #info yamahata
15:05:50 <yamahata> #link https://wiki.opendaylight.org/view/NeutronNorthbound:Meetings meeting agenda
15:06:03 <mkolesni> #info mkolesni
15:06:17 <yamahata> we have topics. any other topics today?
15:07:09 <mkolesni> i'd like to talk about the full sync design from the mail if we can
15:07:38 <yamahata> mkolesni: sure. let's discuss at ML2 ODL driver rewrite topic
15:08:08 <mkolesni> sure
15:08:09 <yamahata> I suppose it will take forever. So I'd like discuss boron planning and preparing openstack summit. then ML2 ODL driver
15:08:26 <mkolesni> ok
15:08:55 <yamahata> if we have specific topic in advance, let's update the wiki page so that people can know it in advance
15:09:04 <yamahata> any other agenda items?
15:10:21 <yamahata> seems nothong else. move on.
15:10:25 <yamahata> #topic Announcements
15:10:42 <yamahata> no new topic.
15:10:56 <yamahata> openstack summit planning is underway
15:11:15 <yamahata> Armand posted his draft schedule on neutron devsummit
15:12:12 <yamahata> any other announcement?
15:12:35 <yamahata> As there is openstack summit, April 27, this meeting will be skipped.
15:13:15 <yamahata> #topic action items from last meeting
15:13:41 <yamahata> the patches wrere reviewed and mermged
15:14:22 <oshvartz> #chair oshvartz
15:14:53 <yamahata> next, let's discuss logistics first, then patches/tech discussion.
15:15:02 <yamahata> #topic Boron planning
15:15:42 <yamahata> #link https://wiki.opendaylight.org/view/NeutronNorthbound:Boron_Release_Plan
15:16:15 <yamahata> It's April 21. We will finalize the release plan.
15:17:08 <yamahata> I suppose the requirement and focus is okay. If no new input, I'll finialise it and report it to TSC on time.
15:17:47 <yamahata> move on.
15:17:50 <yamahata> #topic preparing openstack summit austin presentation
15:18:21 <yamahata> we have two items. presentation and devmeeting.
15:18:33 <yamahata> we need to define timeslot.
15:18:59 <yamahata> openstack summit schedule is at https://www.openstack.org/summit/austin-2016/summit-schedule/#day=2016-04-25
15:19:05 <yamahata> #link https://www.openstack.org/summit/austin-2016/summit-schedule
15:19:29 <yamahata> For presentation, how about gather monday morning?
15:19:34 <yamahata> probably during keynote speech.
15:19:48 <vthapar> +1
15:19:58 <asomya> +1
15:20:26 <yamahata> thanks. For details, I'll send a mail offline next week.
15:20:55 <yamahata> #action yamahata offine communication for presentation preparate. On monday morning during keynode
15:21:19 <yamahata> For development discussion, officially we have friday for meetup.
15:21:39 <yamahata> But people may leave friday, It may not work.
15:22:12 <yamahata> monday morning and tuesday morning during keynotes.
15:22:23 <yamahata> comments/thoughts?
15:23:08 <vthapar> +1. I am flying back 2:30pm Friday, though Friday 9am could work for me at least.
15:23:10 <asomya> you're right, attendance on Friday is pretty poor.. we can do it during the keynotes
15:23:56 <yamahata> Let's settle it down for now. Anyway we can be flexible at the venue.
15:24:20 <yamahata> #action yamahata announce f2f dev discussion schedule. (but not too early.)
15:24:43 <yamahata> any other logistic issue?
15:25:01 <yamahata> I guess there would be  neutron gathering and ODL gathering
15:26:02 <yamahata> seems nothing.
15:26:04 <vthapar> there is an ODL booth too, right?
15:26:13 <yamahata> vthapar: yes.
15:26:40 <yamahata> Phil asked demo materials/volunteers?
15:27:00 <yamahata> do we have any?
15:27:01 <vthapar> I plan to, but yet to look at schedule.
15:27:19 <yamahata> Cool.
15:27:20 <vthapar> we have a VPNService slot so may assist Prem with that.
15:27:54 <vthapar> tangential to it, Prem is on vacation this week, once back will get details from him and update the slide deck and if we can have a demo during presentation for BGPVPN.
15:29:11 <yamahata> any other topic?
15:29:31 <yamahata> okay let's discuss on patches/bugs
15:29:33 <yamahata> #topic patches/bugs
15:30:15 <yamahata> we have new attendees. and that's the main reason to change timeslot.
15:30:26 <yamahata> So I hope the discussion be hot
15:30:35 <mkolesni> :)
15:30:44 <asomya> i posted the cleanup thread patch on monday: https://review.openstack.org/#/c/304168/
15:31:09 <yamahata> #link https://review.openstack.org/#/c/304168/
15:31:09 <yamahata> Journal entries can get stuck forever causing busy wait
15:31:50 <mkolesni> i haven't had time to do a thorough review yet, but do you think it can be based on https://review.openstack.org/305132
15:31:52 <mkolesni> ?
15:31:56 <yamahata> #action yamahata review https://review.openstack.org/#/c/304168/
15:32:35 <asomya> just started looking at the patch
15:32:42 <yamahata> #link https://review.openstack.org/#/c/305132/  Add journal maintenance thread
15:32:56 <mkolesni> the patch i sent adds a maintenance thread, after the discussion on the mailing list
15:33:14 <mkolesni> so i think if you base on this your patch will be much more concise
15:33:21 <asomya> there are a few issues with that patch, i'll comment on it
15:33:30 <mkolesni> it is preliminary though and should be reviewed of course
15:34:24 <mkolesni> https://review.openstack.org/302701 has 3 +1s and seems to be in agreement, i think it can be merged
15:34:59 <yamahata> regarding 302701, I'm fine with it. I'll look at it.
15:35:03 <mkolesni> #link https://review.openstack.org/302701 Fix journal row locking
15:35:10 <yamahata> #action yamahata review and merge https://review.openstack.org/302701
15:36:26 <yamahata> Regarding to maintenance thread, is it supposed to run only single thread globally between multiple neutron server instances?
15:36:55 <mkolesni> yes it should only have one process doing maintenance
15:37:27 <mkolesni> basically it's journal maintenance so we don't want several processes stepping on each others toes
15:37:59 <yamahata> that makes sense
15:38:02 <mkolesni> i think currently this simple design makes sense
15:38:18 <mkolesni> if we see any scale issues we can of course think of something more scalable
15:38:43 <mkolesni> but seeing as its not too much real time work there i think we should be fine
15:39:13 <mkolesni> thoguhts?
15:39:31 <oshvartz> Sounds good to me
15:39:46 <rcurran> will the maintenance thead be one per controller or one per deployment (thinking HA)
15:39:55 <yamahata> it sound reasonable to start with simple one.
15:40:16 <mkolesni> what do you mean when you say "controller"
15:40:24 <mkolesni> ODL controller, or Neutron?
15:40:31 <rcurran> neutron
15:40:49 <mkolesni> there is one per Neutron process
15:41:23 <rcurran> per controller (call is made in initialize() so only one will be created - regardless of api_workers)
15:41:49 <mkolesni> in HA there will be the number of processes as the number of nodes
15:41:50 <rcurran> but there will be multiple maintenance threads running in HA ... I think ... looking at code
15:42:19 <mkolesni> yeah thats why i added a lock there to make sure several threads dont do the maintenance ops at the same time
15:42:35 <rcurran> just want to make sure that's the design
15:42:46 <mkolesni> yes thats the design
15:43:46 <mkolesni> i'd also like to mention https://review.openstack.org/#/c/305321/
15:43:57 <mkolesni> #link https://review.openstack.org/#/c/305321/ ODL v2: Fix delay in sync pending rows
15:44:13 <mkolesni> something oshvartz descovered yesterday
15:44:25 <oshvartz> #link https://review.openstack.org/#/c/302053 Reduce update db row code duplication. it had +2 but I had to rebase due to merge conflicts
15:45:11 <yamahata> It's interesting.
15:45:32 <yamahata> There ovsdb/netvirt is sensetive to event order. that would be fixed in Boron.
15:45:54 <oshvartz> Yes. I will work also on fixing it on the OVSDB side
15:46:04 <yamahata> regarding 302053, there is no change except rebase?
15:46:17 <oshvartz> Yes
15:46:20 <mkolesni> i went over it and seems nothing changed
15:46:30 <yamahata> #action yamahata (review and) merge https://review.openstack.org/#/c/302053
15:46:38 <oshvartz> conflict were only on the test_db
15:47:18 <yamahata> oshvartz: okay.
15:47:54 <yamahata> today we have 5 patches pending.
15:48:03 <yamahata> I hope we can settle those down this week.
15:48:06 <oshvartz> regarding the ordering issues. Flavio fixed some of the OVSDB code to add missing rules when on northbound events but still some logic are missing
15:48:38 <yamahata> Do we have any other patches?
15:49:15 <yamahata> For mitaka release, do we want to include any other patches?
15:49:41 <mkolesni> whats the schedule here? i know Mitaka is already on RCs
15:50:11 <yamahata> Openstack mitaka was released. the problem is when to release networking-odl mitaka.
15:50:17 <yamahata> what to be included?
15:50:57 <oshvartz> V2 will be the default for mitaka ?
15:51:14 <yamahata> good question. Are we comfortable with it?
15:51:35 <rcurran> honestly, i'm not sure it's ready for prime time
15:51:47 <mkolesni> i don't feel comfortable with it
15:51:48 <asomya> yamahata: There are still  a few things to be addresses like critical issues, I think Newton is more realistic to make it default
15:51:50 <yamahata> I don't want to get complain.
15:52:04 <mkolesni> it still has a lot of gaps
15:52:13 <rcurran> gaps and lack of testing
15:52:19 <yamahata> We all are on same page. :-)
15:52:20 <mkolesni> however Newton could be a realistic goal
15:52:35 <rcurran> the ODL CI needs to be run against the v2 plugin/drivers
15:52:51 <mkolesni> i think there are many complicated designs there, if we simplify it we can get better quality
15:52:58 <yamahata> Yes. and OPNFV also should run v2. I got complain about the lack of document on how to run v2 driver.
15:53:28 <yamahata> by operators/testers. not by developer.
15:55:01 <mkolesni> so do we have some decision here?
15:55:09 <yamahata> Given that v2 driver won't be default by Mitaka, the realistic plan would be merge current pending bug fix patches and release it.
15:55:29 <yamahata> that's my personal feelings. comments/thoughts?
15:56:05 <yamahata> Regarding to document, I will look into it.
15:56:08 <mkolesni> i think if its not going to be default you can release it now since the pending patches won't fix some other critical issues anyway
15:56:22 <rcurran> does it really matter what v2 code makes it into stable/m vs. when we open up N
15:56:50 <mkolesni> better push good code that we can stand by
15:56:51 <rcurran> no one other than the folks on this IRC are even using it :-)
15:57:10 <mkolesni> ;)
15:57:29 <yamahata> I guess odl ci or opnfv people want to use stable branch, not master.
15:57:38 <mkolesni> is there something making v1 driver unusable in Mitaka?
15:58:16 <mkolesni> besides it being suck, i think theres not much we can do to push v2 in any better stage
15:58:26 <mkolesni> unless we have another month for that :)
15:58:51 <mkolesni> then we can maybe get something stable in as velocity is pretty good atm
15:59:54 <yamahata> So it sounds the quality of v2 driver doesn't matter for mitaka release of networking-odl
16:00:04 <yamahata> so we are ready to release it.
16:00:24 <mkolesni> sounds good to me
16:00:41 <mkolesni> did anything change in the v1 driver this cycle?
16:01:28 <yamahata> except bug fix and refactoring, there is no big change.
16:01:41 <yamahata> compared to v2 driver.
16:02:09 <mkolesni> well if its better than last stable then it should be released
16:02:40 <mkolesni> bacuse v2 driver currently can partially work in a very controlled sandbox..
16:03:23 <yamahata> okay, I'll arrange the release. I need to talk with kyle.
16:03:27 <mkolesni> i opened bug 1569986
16:03:39 <yamahata> #action yamahata arrange networking-odl and talk with kyle.
16:03:50 <mkolesni> #link https://bugs.launchpad.net/networking-odl/+bug/1569986  Race condition in journal between event write and thread processing
16:04:19 <mkolesni> ill sit on it tomorrow to fix it
16:04:49 <yamahata> cool.
16:05:24 <mkolesni> not critical but it could also trigger the incorrect order on ovsdb side
16:06:14 <yamahata> any other patches/bugs to discuss?
16:06:56 * yamahata is very happy to have busy discussions.
16:07:36 <yamahata> okay
16:07:38 <yamahata> #topic open mike
16:07:42 <yamahata> any other topics?
16:07:54 <yamahata> we're over several minutes.
16:08:00 <mkolesni> i wanted to sync on the full sync mail thread
16:08:24 <mkolesni> just to make sure if we're on agrrement on the proposed design or not
16:08:53 <yamahata> Personally high level design is okay. The devels in the details.
16:09:17 <mkolesni> ok cool
16:09:30 <yamahata> I had talked with Yalei and Rui also.
16:09:38 <mkolesni> i think some gaps we already closed on that list..
16:10:25 <mkolesni> im of course awaiting your or anyone else's respinse to the 2 canary proposal
16:10:56 <mkolesni> though its a super corner case
16:11:19 <yamahata> I had though of 2 canaries too.
16:11:29 <yamahata> Agree that it's super corner case.
16:11:58 <mkolesni> i think 2 canaries can probably solve those corener cases involving restart of either neutron or odl
16:12:36 <mkolesni> in that case worst case we will try to recreate some stuff but i think since its a corner case we should worry too much
16:13:19 <mkolesni> shouldn't
16:13:58 <yamahata> Yeah. Anyway in the next phase, healing is planned.
16:14:11 <mkolesni> you mean partial sync?
16:14:33 <yamahata> to find random data breakage and heal it.
16:14:47 <mkolesni> i talked it over with Livnat Peer this morning, she thinks we shouldn't build too much on that mechanism
16:15:15 <yamahata> Due to bug or unintentional/malicious operation, data in odl can be broken random way.
16:15:40 <mkolesni> yea i saw it somwhere in the v2 code/docs
16:15:50 <yamahata> probaby by hash comparison.
16:16:08 <yamahata> actual healing would be erase all and recreate all
16:16:19 <mkolesni> i think we should have some hash capability on ODL side
16:16:37 <mkolesni> i.e. ability to get only hash for any resource or collection of resources
16:17:10 <mkolesni> then we can have a better implementation
16:17:32 <mkolesni> though i hate premature optimization i think here it will be necessary from the start
16:18:08 <yamahata> agree with starting with simplest one. i.e. KISS
16:18:21 <mkolesni> since you probably dont want to fetch whole ODL neutron representation each time you try to see if healing is even needed
16:18:45 <mkolesni> i.e. if healing check is run in maintenance thread its heavy without the hash
16:19:22 <mkolesni> anyway i have to go
16:19:32 <yamahata> we are run over 20min
16:19:33 <yamahata> sorry
16:19:43 <mkolesni> can we continue this next week?
16:19:52 <yamahata> mkolesni: yes off couse.
16:20:00 <yamahata> let's continue next week or on the mailing list.
16:20:00 <mkolesni> cool
16:20:08 <yamahata> thank you every one?
16:20:24 <yamahata> #action yamahata work out discussion agenda at the openstack summit
16:20:30 <yamahata> bye
16:20:37 <yamahata> #topic cookies
16:20:39 <mkolesni> bye!
16:20:42 <yamahata> #endmeeting