16:01:22 <dfarrell07> #startmeeting tsc
16:01:22 <odl_meetbot> Meeting started Thu Nov  1 16:01:22 2018 UTC.  The chair is dfarrell07. Information about MeetBot at http://ci.openstack.org/meetbot.html.
16:01:22 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:22 <odl_meetbot> The meeting name has been set to 'tsc'
16:01:37 <dfarrell07> #info tsc members please #info in, projects please #project in
16:02:04 <abhijitk> #topic Agenda Bashing and Roll Call
16:02:12 <shague> #info shague
16:02:20 <abhijitk> #info abhijitk
16:02:42 <anipbu> #info anipbu
16:02:50 <anipbu> #project usc
16:02:54 <shague> #info shague #project netvirt genius serviceutils odltools ovsdb
16:03:17 <jamoluhrsen> #info jamoluhrsen
16:03:21 <abhijitk> #link https://wiki.opendaylight.org/view/TSC:Meeting#Agenda
16:04:02 <LuisGomez> #info LuisGomez
16:04:24 <gzhao> #info gzhao
16:04:31 <LuisGomez> #project distribution
16:04:41 <vorburger> #info vorburger
16:04:42 <faseelak> #project genius coe serviceutils
16:04:42 <zxiiro> #info Thanh
16:05:48 <abhijitk> #topic Releases
16:06:25 <vorburger> #project infrautils daexim controller genius neutron oldparent serviceutils
16:07:33 <dfarrell07> #info we have fluorine sr2 and oxygen sr4 coming up basically now
16:07:44 <dfarrell07> #info fluorine autorel is passing now
16:08:02 <dfarrell07> #info ariel proposes pushing oxygen back a week to 11/14 so can focus on fl
16:08:16 <abhijitk> #agreed Oxygen SR 4 moved to 11/14
16:08:42 <dfarrell07> #info last chance to get things into oxygen, please cherry-pick now
16:09:38 <lori> #project lispflowmapping
16:09:57 <dfarrell07> #info fluorine was fixed a few days ago, neutron issue worked around and them vorburger properly fixed
16:10:40 <dfarrell07> #agreed autorel 260 will be the version we audit as fluorine sr1
16:11:13 <dfarrell07> #info we will do test sign-offs on fluorine sr1
16:12:18 <jamoluhrsen> #info distro-test job running on fluorine #260 autorelease is https://jenkins.opendaylight.org/releng/view/autorelease/job/integration-distribution-test-fluorine/259/
16:12:22 <zxiiro> #link Fluorine Tracking spreadsheet https://docs.google.com/spreadsheets/d/1wtT78KigRQdRi3Gj--jOJJPI7tC4tIi5brmW01vvCsg/edit#gid=375731420
16:12:28 <abhijitk> #agreed The build  #260 is being considered for being Fluorine SR1
16:13:06 <dfarrell07> #info csit is still running for 260 but we will copy into sheet above and get sign-offs as soon as done
16:17:18 <dfarrell07> #info there is a blocker in mdsal and lots of complexity around getting that verified in the MSI projects
16:18:14 <vorburger> #link https://jira.opendaylight.org/browse/INFRAUTILS-58
16:18:48 <dfarrell07> #info detailed discussion about this bug^^
16:20:50 <dfarrell07> #info this is a problem with the MRI model where it in theory it should be quick to get a patch version fix to blockers like this, but in this case we're being told it could take weeks (with a broken world)
16:22:10 <dfarrell07> #info we don't have tooling that can check proposed mdsal rel version in MSI projects, and there's lots of complexity in making that happen says LuisGomez
16:22:54 <faseelak> #link https://jira.opendaylight.org/browse/MDSAL-379
16:24:03 <dfarrell07> #info the key mess is that projects are using multiple versions of mdsal, so we can't just script find/replace versions
16:24:20 <dfarrell07> #info real solution is to clean up projects to only take one or two versions of mdsal
16:24:56 <dfarrell07> #info we might want to try to push this cleanup as the blocker, says vorburger
16:25:47 <dfarrell07> #info tomp suggests just bumping genius to the proposed-fixed mdsal version to test it, as it's the one that's seeing the problem
16:26:22 <dfarrell07> #info one option is to just release the version of mdsal and bump to it to test
16:26:30 <dfarrell07> #info another option is to back out all the version bump patches
16:27:08 <dfarrell07> #info long-term, we really need to fix this multi-mdsal-version issues if we want to solve this kind of thing in the future
16:27:26 <dfarrell07> #info LuisGomez thinks that might be the best path, even if takes a few days, as it's the more useful thing long-term
16:27:42 <dfarrell07> #info vorburger suggests if we share this load we might be able to do it quickly enough
16:27:56 <dfarrell07> #info shague asks if we can just make mdsal snapshot again
16:28:22 <dfarrell07> #info LuisGomez says that impact of making mdsal mri is huge, and maybe we didn't look at it enough, need to get everyone on the same version first
16:29:59 <dfarrell07> #info when rovarga returns we might want to discuss bringing mdsal back to snapshots until we fix many-mdsal-version problem
16:31:05 <dfarrell07> #info LuisGomez does a better job of explaining this many-mdsal-version thing than I did above
16:31:41 <dfarrell07> #info projects seem to be using internal mdsal versions, different per-bundle
16:32:23 <dfarrell07> #info doing this fixing how we consume mdsal versions solution seems to be a quick-ish way out that would be useful long-term
16:32:42 <dfarrell07> #info that would also make everything for mri mdsal easier
16:33:05 <dfarrell07> #info seems to be consensus that we should clean up how we consume mdsal versions as solution
16:34:02 <dfarrell07> #agreed we will work with all projects to collab on cleaning up mdsal versions in pom files to converge on single mdsal versions
16:34:22 <dfarrell07> #info we need an example, like form skitt, that other projects can follow
16:36:30 <dfarrell07> #info this cleaning up versions would also make it easy to switch back to snapshot mdsal if we wanted to go that route
16:37:04 <dfarrell07> #info and also make it easier to make job that can auto-bump proposed to mdsal version for testing things in the future
16:37:24 <dfarrell07> #info the work of manually changing versions to test one-off now is about the same work as cleaning up versions
16:38:04 <dfarrell07> #info the key is that it needs to be easy to switch mdsal versions, which needs this cleanup
16:39:00 <dfarrell07> #info after we do cleanup, we switch mdsal to 3.0.2 snaphosts and test (rovarga will not like this), iterate until we fix, then make decision about moving mdsal back to mri
16:39:17 <dfarrell07> #info sorry rovarga we know you will not like that but it's the best idea we have
16:40:04 <vorburger> #info Proposed Plan
16:40:27 <vorburger> #info 1. get all projects to switch their mdsal dependencies to mdsal-artifacts, following an example from Stephen
16:40:45 <zxiiro> speaking of super powers to merge. I think it would be a a good idea to have the TSC managed list of community folks who can be part of the group.
16:41:14 <vorburger> #info 2. switch all mdsal versions from 3.0.1 (3.0.0 ?) to 3.0.2-SNAPSHOT on a topic branch, doesn't have to be merged, just so that we can run a multipatch job
16:41:14 <dfarrell07> zxiiro: cool idea
16:41:27 <abhijitk> +1 zxiiro
16:41:35 <dfarrell07> zxiiro: you can likely just send a patch with that list and ask for a vote to approve as the super-committers
16:41:37 <vorburger> #info 3. merge those patches, either for  3.0.2-SNAPSHOT o or  3.0.2 if we have a mdsal release
16:41:42 <zxiiro> Today it's just the release engineering team but I think it would be good to have more folks so that we have merge coverage  around the globe during release bumping and such.
16:41:54 <faseelak> zxiiro, good idea!
16:42:04 <abhijitk> #agreed Proposed Plan above
16:42:24 <zxiiro> dfarrell07: sure, I can send a TSC mailing list email to get the conversation started.
16:42:40 <dfarrell07> zxiiro: I suspect it'll be an easy vote :)
16:42:56 <dfarrell07> #topic stable jobs
16:43:17 <dfarrell07> #info we have a set of jobs (thanks shague!) that we think should be passing
16:43:28 <dfarrell07> #info there are errors in some of the jobs that we want to keep stable
16:44:12 <dfarrell07> #info seems a few jobs broke yesterday, 31st
16:45:28 <dfarrell07> #info LuisGomez points out that it may have broke on various branches, which is odd
16:45:32 <dfarrell07> #info jamoluhrsen is looking at logs
16:46:40 <dfarrell07> #info seeing some logs about binding exception addr in use
16:46:48 <dfarrell07> #info we can't debug this live, need to take to lists
16:47:05 <dfarrell07> #info int/releng call will look at it in a few hours
16:48:43 <abhijitk> #topic KubeCon shared space for ODL
16:49:23 <abhijitk> #link https://lists.opendaylight.org/pipermail/tsc/2018-October/010496.html
16:49:24 <zxiiro> jamoluhrsen:
16:49:25 <zxiiro> 00:07:19.554 looking for "BindException: Address already in use" in log file
16:49:25 <zxiiro> 00:07:19.555 looking for "server is unhealthy" in log file
16:50:18 <jamoluhrsen> zxiiro: that's from a red dot right?
16:50:25 <dfarrell07> #topic kubcon
16:50:26 <zxiiro> jamoluhrsen: no yelllow
16:50:35 <dfarrell07> #info dfarrell07 gives updates about lfn booth at kubecon
16:50:39 <jamoluhrsen> zxiiro: weird. got the link?
16:50:47 <LuisGomez> fyi i see some ssh issues in CSIT jobs probably related to some library update
16:50:47 <zxiiro> jamoluhrsen: https://jenkins.opendaylight.org/releng/view/csit-stable-f/job/openflowplugin-csit-3node-clustering-only-fluorine/259/console
16:50:51 <dfarrell07> #info we have 10 min talk and shared booth
16:50:53 <abhijitk> #topic TSC Elections
16:51:12 <abhijitk> #link https://lists.opendaylight.org/pipermail/tsc/2018-November/010512.html
16:51:25 <abhijitk> #info Call for nominations out
16:51:35 <jamoluhrsen> zxiiro: oh, those messages aren't failures. those are echos for what it's trying to find.
16:51:41 <LuisGomez> 05:26:34.167	FAIL	AttributeError: 'NoneType' object has no attribute 'open_session'
16:51:58 <LuisGomez> robot ssh library is my guess
16:52:06 <jamoluhrsen> LuisGomez: those fialures are mine.
16:52:19 <jamoluhrsen> LuisGomez: I'll fix. but those are only in netvirt/openstack jobs I hope
16:52:28 <LuisGomez> i see in ofp
16:52:40 <dfarrell07> #link https://wiki.opendaylight.org/view/TSC:2018_Election tsc nominations wiki
16:52:42 <jamoluhrsen> LuisGomez: oh. maybe ofp uses tcpdump? got a link?
16:52:51 <LuisGomez> not that i know
16:52:51 <dfarrell07> #info two week window. nominate!
16:52:55 <LuisGomez> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-fluorine/259/robot-plugin/log.html.gz#s1-s1
16:53:28 <vorburger> LuisGomez: are you share the mdsal version bump / change is that hard? It seems to me that, now, it's just a simple search and replace of 3.0.1 by 3.0.2-SNAPSHOT ... what am I missing?
16:53:47 <dfarrell07> maybe let's leave meeting running to record that discussion^
16:53:54 <dfarrell07> #topic cookies
16:54:00 <abhijitk> #info Folks who have not been TSC members - do consider running  for the election
16:54:08 <abhijitk> #endmeeting
16:54:30 <abhijitk> dfarrell07: you will have to do endmeeting
16:55:33 <vorburger> jamoluhrsen: zxiiro: can you share the link to the log file where the ""BindException: Address already in use" appears? I'm just curious.
16:56:00 <LuisGomez> vorburger, depends on the project itself, some projects may have simple mdsal dependency but i saw others consuming multiple artifacts and with different versions
16:56:30 <LuisGomez> thats why the consolidation is required
16:56:32 <zxiiro> vorburger: https://jenkins.opendaylight.org/releng/view/csit-stable-f/job/openflowplugin-csit-3node-clustering-only-fluorine/259/console
16:56:56 <vorburger> LuisGomez: but was that before or after the Neon MRI bump? I just looked at some (controller, aaa, daexim) and only see "3.0.1" so this doesn't seem that hard?
16:58:59 <LuisGomez> vorburger, it is possible that some consolidation has been already done through the MRI work
16:59:08 <LuisGomez> i have not really check after that
16:59:11 <vorburger> zxiiro: I can see "looking for "BindException: Address already in use" in log file" but am not actually finding that in odl_[1-3] on https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-fluorine/259/
17:07:08 <jamoluhrsen> vorburger: zxiiro, I think you are chasing up the wrong tree
17:07:43 <vorburger> dfarrell07: dont' keep the meeting open for me, I'll follow up on email, if there is anything further to say
17:08:15 <dfarrell07> #endmeeting