15:30:02 <anipbu> #startmeeting beryllium weekly sync
15:30:02 <odl_meetbot> Meeting started Wed Feb 17 15:30:02 2016 UTC.  The chair is anipbu. Information about MeetBot at http://ci.openstack.org/meetbot.html.
15:30:02 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:30:02 <odl_meetbot> The meeting name has been set to 'beryllium_weekly_sync'
15:30:07 <anipbu> #topic roll call
15:30:22 <anipbu> #chair zxiiro anipbu
15:30:22 <odl_meetbot> Current chairs: anipbu zxiiro
15:30:37 <AmbikaTripathy> #info AmbikaTripathy representing YANG-PUSH
15:30:38 <hideyuki> #info Hideyuki for VTN project
15:31:05 <anipbu> PTL please #info your projects so we can track and record your attendence
15:31:25 <ttkacik> #info ttkacik for mdsal, controller
15:31:37 <rovarga> #info rovarga yangtools, odlparent
15:31:41 <yamahata> #info yamahata for neutron northbound
15:31:44 <lori> #info lori for lispflowmapping
15:31:51 <zxiiro> #info Thanh releng
15:31:55 <aleckey> #info aleckey for NetIDE
15:32:40 <LuisGomez> #info LuisGomez integration
15:33:07 <Prem_> #info Prem VPNService
15:34:33 <anipbu> #info anipbu for USC, NEMO
15:34:45 <anipbu> Any other folks please info your projects?
15:34:58 <anipbu> #chair LuisGomez anipbu2
15:34:58 <odl_meetbot> Current chairs: LuisGomez anipbu anipbu2 zxiiro
15:35:04 <anipbu> Let's move on to agenda
15:35:08 <anipbu> #topic agenda
15:35:12 <anipbu> #info RC3
15:35:15 <anipbu> #info release review
15:35:19 <anipbu> #info blocking bugs
15:35:24 <anipbu> Any other agenda topic folks would like to discuss in 30 minutes?
15:35:59 <hideyuki> anipbu: No, I don't have.
15:36:25 <anipbu> #topic RC3
15:36:30 <anipbu> #info Projects PTL please cast your vote on Beryllium Go/NOGO
15:36:36 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=1599673811   <-- Vote here
15:36:45 <anipbu> #action need votes from DIDM, GroupBasedPolicy, Integration/Distribution, PacketcablePCMM
15:36:55 <rovarga> anipbu: yeah, I would like to bring up the stability of -autorelease and -distribution jobs
15:37:21 <anipbu> rovarga: okay, i'll add that to the blocking issues discussion
15:37:46 <rovarga> it may not be blockign the release, but it is a world of pain for working on master at the very least
15:37:48 <AmbikaTripathy> anipbu: is there a way to generate ascii doc from project wiki ?
15:38:03 <anipbu> LuisGomez: int/dis still missing the vote
15:38:13 <LuisGomez> i just did
15:38:30 <anipbu> AmbikaTripathy: I don't know of any, but the docs team might be able to point you to the right direction.
15:38:50 <anipbu> #action anipbu and AmbikaTripathy follow up with docs team for any tools to generate ascii doc from project wiki
15:39:05 <anipbu> #info pProjects PTL, please identify if the RC3 issues are "OKAY" to release or "BLOCKING" and need to delay Beryllium.
15:39:17 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=2095652051  <-- Comment on test
15:39:30 <anipbu> #action need comments from ovsdb, sxp, topoprocessing
15:39:41 <anipbu> anyone here from ovsdb, sxp, or topoprocessing?
15:39:50 <abhijitkumbhare> #info abhijitkumbhare for OpenFlow Plugin
15:40:26 <anipbu> #action shague_ still waiting on ovsdb comments on test failures
15:41:11 <anipbu> Do folks have any other RC3 related questions?
15:42:23 <shague_> anipbu: let me check with Jamo and we ill update the clustering test failures.
15:43:01 <anipbu> shague_: thanks!
15:43:08 <anipbu> #topic release review
15:43:16 <anipbu> #info Project PTL, please address the action items during your release review.
15:43:21 <anipbu> #info action items are summarized in the spreadsheet for your convenience
15:43:25 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=45904332
15:43:50 <anipbu> #action ALL PTL to make sure their action items during release review are completed.
15:44:06 <anipbu> Do folks have any other release review related questions?
15:45:13 <anipbu> #topic blocking bugs, red flags, open mic
15:45:36 <anipbu> rovarga: zxiiro: stability of -autorelease and -distribution jobs
15:45:53 <yuling> anipbu:for the follow up action spreadsheet, the entry for tsdr is empty...any idea?
15:46:20 <rovarga> #info it seems we have mismatch between how slaves are configured and what the jobs expect
15:46:23 <anipbu> yuling: let me follow up with you after the meeting
15:46:34 <zxiiro> rovarga: if you're referring to the VMs starting up without xmlstarlet and sshpass that should have been fixed monday
15:46:37 <anipbu> #action anipbu and yuling to follow up on release review action items
15:46:37 <yuling> ok, thanks
15:46:47 <rovarga> zxiiro: unfortunately it is not
15:46:48 <zxiiro> rovarga: however i just noticed this morning that dynamic_merge vms are still spinning up with the old image
15:46:53 <zxiiro> rovarga: I'll fix that this morning
15:47:00 <anipbu> #info rovarga raised some concerns regarding stability of -autorelease and -distribution jobs
15:47:20 <zxiiro> rovarga: on monday we built a new vm image that includes both xmlstarlet adn sshpass so we no longer depend on spinup scripts to start them
15:47:27 <rovarga> zxiiro: is there anything we can  do to make sure we have some QA in place to stop these?
15:48:01 <anipbu> #info zxiiro says that dynamic_merge vms are still spinning up with the old image
15:48:28 <anipbu> #info zxiiro says that on monday we built a new vm image that includes both xmlstarlet adn sshpass so we no longer depend on spinup scripts to start them
15:48:52 <zxiiro> dynamic_merge and dynamic_verify "used" to be separate build slaves but for awhile now we've forced them to be the same but during transition we didn't remove the old merge ones. I'll make sure merge and verify are the same configuration moving forward
15:49:17 <zxiiro> I'll delete the old dynamic_merge slaves to ensure that they no longer spin up even by accident
15:49:29 <rovarga> can we get a heads up along with workarounds?
15:50:04 <zxiiro> rovarga: what kind of heads up? like an email to release that we are modifying build slaves?
15:50:25 <rovarga> or to infrastructure -- if there was a mail about this, I must have missed it
15:50:41 <rovarga> I mean this is equivalent to yangtools merging stuff which make break the world
15:51:00 <zxiiro> sure I can make sure we send emails in the future when we change things.
15:51:09 <zxiiro> This was changed on Monday because ALL builds lsaves were failing
15:51:30 <anipbu> rovarga: agreed, in the future, we can send email to dev/release list so that the community knows we are aware of problems and addressing as soon as possible.
15:51:46 <zxiiro> None of the spinup scripts were working monday so I had to create an emergency image on Monday to bake sshpass and xmlstarlet into the image
15:52:28 <rovarga> zxiiro: any idea about what made them stop working?
15:52:28 <LuisGomez> rovarga, i see yangtools does not have distribution check job, do you mind to add this once these issues are fixed?
15:52:28 <anipbu> #info in the future, we can send email to dev/release list so that the community knows we are aware of problems and addressing as soon as possible.
15:52:57 <rovarga> LuisGomez: as long as it remains stable, sure. but that has not been the experience of the past two weekss
15:53:57 <LuisGomez> agree, zxiiro will work on that so it will be soon good.
15:54:11 <anipbu> #info LuisGomez notes yangtools does not have distribution check job.  asks  rovarga to add this once these issues are fixed.
15:54:17 <zxiiro> rovarga: normally when a VM comes online JClouds (jenkins plugin we use to connect to rackspace) SSH's into the slave and then runs scripts to complete initializing the slave. This was needed as a way to allow proejcts to add things to slaves without requiring Andy or I to rebuild slave images so would allow projects to install dependencies quicker in cases
15:54:17 <zxiiro> where Andy and I were too busy to create new images.
15:54:45 <zxiiro> rovarga: On Monday unfortunately something in Rackspace caused the initial ssh connection to fail so none of the slaves coming online were able to run their spinup scripts
15:55:02 <zxiiro> rovarga: the solution moving forward is to stop using spinup scripts so that we are not affected by Rackspace issues
15:55:24 <zxiiro> rovarga: I want to move everything back into the vagrant images and someone will have to rebuild all our images everytime a project wants to update
15:55:41 <anipbu> #action rovarga to add yangtools distribution check job
15:55:46 <zxiiro> that'll make this part of the slaves less prone to these intermittent issues
15:56:31 <zxiiro> The reason we still see this is because on monday I only updated the dynamic_verify slaves and forgot to switch the dynamic_merge slaves over to tvhe new updated image
15:57:26 <zxiiro> both dynamic_{verify|merge} are supposed to be identical images yet they have separate configuration in jenkisn so thats why it was missed. Moving forward we'll delete the dynamic_merge ones so that there's no confusion when we update images again in the future
15:58:20 <rovarga> zxiiro: thanks
15:58:30 <anipbu> rovarga: thank you for the feedback you have kindly shared with us today.  Do you have any additional comments or concerns you would like to raise?
15:58:30 <zxiiro> So if your autorelease / distribution job happened to run on a dynamic_merge slave this morning that's the cause of the failure
15:58:40 <rovarga> zxiiro: it seems something is broken, still, just got a message from jenkins iwth subject: [controller] $PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!
15:59:02 <zxiiro> rovarga: yes I need to make the update. I'll go do it now
15:59:13 <rovarga> zxiiro: thanks
15:59:22 <rovarga> anipbu: thanks, that is it for now :)
15:59:48 <anipbu> rovarga: very much appreciate your comments and feedback.
16:00:02 <anipbu> OKay folks, any other blocking issues related to Beryllium?
16:00:24 <anipbu> Last I check, there are no blocking items on the spreadsheet.
16:00:35 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=2030147923  <-- List of blocking issues
16:00:58 <LuisGomez> cool :)
16:00:59 <anipbu> any project PTL have anything blocking issues to raise regarding Beryllium?
16:01:39 <anipbu> #action PTL, please verify the blocking issues are no longer appearing and move the bug to the FIXED list in the spreadsheet
16:02:20 <anipbu> These projects need to verify: groupbasedpolicy netconf of-config snmp4sdn lispflowmapping netconf nic unimgr didm reservation sfc
16:02:39 <anipbu> Any projects want to chime in on any blocking issues or red flags to raise?  open mic...
16:03:51 <LuisGomez> so we do not have standing blocking bugs but we still have to verify some fixes, is there an end date for that?
16:04:27 <anipbu> They should be verified against the latest RC before we present to the TSC
16:04:33 <AmbikaTripathy> anipbu: yangtool anyxml bug:  https://bugs.opendaylight.org/show_bug.cgi?id=5099 is not blocking. but if we have tentative fixed data it will be good
16:05:32 <anipbu> LuisGomez: If items are not verified, then TSC will have to decide if we should ship with bugs not verified
16:05:55 <LuisGomez> ok, thanks anipbu
16:06:30 <anipbu> AmbikaTripathy: if the issue is not blocking and does not need to delay Beryllium, I strongly suggest we merge the fix in SR1, which is approximately one month from now.
16:06:56 <ttkacik> anipbu: fix currently does not exist
16:07:22 <AmbikaTripathy> anibhu: not blocking. but good to have during boron release
16:08:29 <anipbu> #info AmbikaTripathy notes there is a bug in yangtools.  However, is NOT BLOCKING and okay to release Beryllium with the known bug.   ttkacik notes there is currently no fix available for this bug.  https://bugs.opendaylight.org/show_bug.cgi?id=5099
16:08:50 <anipbu> ttkacik: thanks for looking into it and giving us an update.
16:09:50 <LuisGomez> #action projects should verify blocking bug fixes and update sheet before tomorrow’s TSC call.
16:10:09 <anipbu> LuisGomez: thanks.
16:10:25 <anipbu> Any last minute topics to raise from folks?
16:10:41 <anipbu> going once
16:11:07 <anipbu> OH QUESTION: do folks still want to continue the weekly sync up?
16:11:21 <anipbu> Maybe I should send out an email
16:12:04 <anipbu> Okay, thanks folks for joining.
16:12:13 <anipbu> #topic cookies
16:12:23 <anipbu> #endmeeting