15:30:02 <anipbu> #startmeeting beryllium weekly sync 15:30:02 <odl_meetbot> Meeting started Wed Feb 17 15:30:02 2016 UTC. The chair is anipbu. Information about MeetBot at http://ci.openstack.org/meetbot.html. 15:30:02 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:30:02 <odl_meetbot> The meeting name has been set to 'beryllium_weekly_sync' 15:30:07 <anipbu> #topic roll call 15:30:22 <anipbu> #chair zxiiro anipbu 15:30:22 <odl_meetbot> Current chairs: anipbu zxiiro 15:30:37 <AmbikaTripathy> #info AmbikaTripathy representing YANG-PUSH 15:30:38 <hideyuki> #info Hideyuki for VTN project 15:31:05 <anipbu> PTL please #info your projects so we can track and record your attendence 15:31:25 <ttkacik> #info ttkacik for mdsal, controller 15:31:37 <rovarga> #info rovarga yangtools, odlparent 15:31:41 <yamahata> #info yamahata for neutron northbound 15:31:44 <lori> #info lori for lispflowmapping 15:31:51 <zxiiro> #info Thanh releng 15:31:55 <aleckey> #info aleckey for NetIDE 15:32:40 <LuisGomez> #info LuisGomez integration 15:33:07 <Prem_> #info Prem VPNService 15:34:33 <anipbu> #info anipbu for USC, NEMO 15:34:45 <anipbu> Any other folks please info your projects? 15:34:58 <anipbu> #chair LuisGomez anipbu2 15:34:58 <odl_meetbot> Current chairs: LuisGomez anipbu anipbu2 zxiiro 15:35:04 <anipbu> Let's move on to agenda 15:35:08 <anipbu> #topic agenda 15:35:12 <anipbu> #info RC3 15:35:15 <anipbu> #info release review 15:35:19 <anipbu> #info blocking bugs 15:35:24 <anipbu> Any other agenda topic folks would like to discuss in 30 minutes? 15:35:59 <hideyuki> anipbu: No, I don't have. 15:36:25 <anipbu> #topic RC3 15:36:30 <anipbu> #info Projects PTL please cast your vote on Beryllium Go/NOGO 15:36:36 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=1599673811 <-- Vote here 15:36:45 <anipbu> #action need votes from DIDM, GroupBasedPolicy, Integration/Distribution, PacketcablePCMM 15:36:55 <rovarga> anipbu: yeah, I would like to bring up the stability of -autorelease and -distribution jobs 15:37:21 <anipbu> rovarga: okay, i'll add that to the blocking issues discussion 15:37:46 <rovarga> it may not be blockign the release, but it is a world of pain for working on master at the very least 15:37:48 <AmbikaTripathy> anipbu: is there a way to generate ascii doc from project wiki ? 15:38:03 <anipbu> LuisGomez: int/dis still missing the vote 15:38:13 <LuisGomez> i just did 15:38:30 <anipbu> AmbikaTripathy: I don't know of any, but the docs team might be able to point you to the right direction. 15:38:50 <anipbu> #action anipbu and AmbikaTripathy follow up with docs team for any tools to generate ascii doc from project wiki 15:39:05 <anipbu> #info pProjects PTL, please identify if the RC3 issues are "OKAY" to release or "BLOCKING" and need to delay Beryllium. 15:39:17 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=2095652051 <-- Comment on test 15:39:30 <anipbu> #action need comments from ovsdb, sxp, topoprocessing 15:39:41 <anipbu> anyone here from ovsdb, sxp, or topoprocessing? 15:39:50 <abhijitkumbhare> #info abhijitkumbhare for OpenFlow Plugin 15:40:26 <anipbu> #action shague_ still waiting on ovsdb comments on test failures 15:41:11 <anipbu> Do folks have any other RC3 related questions? 15:42:23 <shague_> anipbu: let me check with Jamo and we ill update the clustering test failures. 15:43:01 <anipbu> shague_: thanks! 15:43:08 <anipbu> #topic release review 15:43:16 <anipbu> #info Project PTL, please address the action items during your release review. 15:43:21 <anipbu> #info action items are summarized in the spreadsheet for your convenience 15:43:25 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=45904332 15:43:50 <anipbu> #action ALL PTL to make sure their action items during release review are completed. 15:44:06 <anipbu> Do folks have any other release review related questions? 15:45:13 <anipbu> #topic blocking bugs, red flags, open mic 15:45:36 <anipbu> rovarga: zxiiro: stability of -autorelease and -distribution jobs 15:45:53 <yuling> anipbu:for the follow up action spreadsheet, the entry for tsdr is empty...any idea? 15:46:20 <rovarga> #info it seems we have mismatch between how slaves are configured and what the jobs expect 15:46:23 <anipbu> yuling: let me follow up with you after the meeting 15:46:34 <zxiiro> rovarga: if you're referring to the VMs starting up without xmlstarlet and sshpass that should have been fixed monday 15:46:37 <anipbu> #action anipbu and yuling to follow up on release review action items 15:46:37 <yuling> ok, thanks 15:46:47 <rovarga> zxiiro: unfortunately it is not 15:46:48 <zxiiro> rovarga: however i just noticed this morning that dynamic_merge vms are still spinning up with the old image 15:46:53 <zxiiro> rovarga: I'll fix that this morning 15:47:00 <anipbu> #info rovarga raised some concerns regarding stability of -autorelease and -distribution jobs 15:47:20 <zxiiro> rovarga: on monday we built a new vm image that includes both xmlstarlet adn sshpass so we no longer depend on spinup scripts to start them 15:47:27 <rovarga> zxiiro: is there anything we can do to make sure we have some QA in place to stop these? 15:48:01 <anipbu> #info zxiiro says that dynamic_merge vms are still spinning up with the old image 15:48:28 <anipbu> #info zxiiro says that on monday we built a new vm image that includes both xmlstarlet adn sshpass so we no longer depend on spinup scripts to start them 15:48:52 <zxiiro> dynamic_merge and dynamic_verify "used" to be separate build slaves but for awhile now we've forced them to be the same but during transition we didn't remove the old merge ones. I'll make sure merge and verify are the same configuration moving forward 15:49:17 <zxiiro> I'll delete the old dynamic_merge slaves to ensure that they no longer spin up even by accident 15:49:29 <rovarga> can we get a heads up along with workarounds? 15:50:04 <zxiiro> rovarga: what kind of heads up? like an email to release that we are modifying build slaves? 15:50:25 <rovarga> or to infrastructure -- if there was a mail about this, I must have missed it 15:50:41 <rovarga> I mean this is equivalent to yangtools merging stuff which make break the world 15:51:00 <zxiiro> sure I can make sure we send emails in the future when we change things. 15:51:09 <zxiiro> This was changed on Monday because ALL builds lsaves were failing 15:51:30 <anipbu> rovarga: agreed, in the future, we can send email to dev/release list so that the community knows we are aware of problems and addressing as soon as possible. 15:51:46 <zxiiro> None of the spinup scripts were working monday so I had to create an emergency image on Monday to bake sshpass and xmlstarlet into the image 15:52:28 <rovarga> zxiiro: any idea about what made them stop working? 15:52:28 <LuisGomez> rovarga, i see yangtools does not have distribution check job, do you mind to add this once these issues are fixed? 15:52:28 <anipbu> #info in the future, we can send email to dev/release list so that the community knows we are aware of problems and addressing as soon as possible. 15:52:57 <rovarga> LuisGomez: as long as it remains stable, sure. but that has not been the experience of the past two weekss 15:53:57 <LuisGomez> agree, zxiiro will work on that so it will be soon good. 15:54:11 <anipbu> #info LuisGomez notes yangtools does not have distribution check job. asks rovarga to add this once these issues are fixed. 15:54:17 <zxiiro> rovarga: normally when a VM comes online JClouds (jenkins plugin we use to connect to rackspace) SSH's into the slave and then runs scripts to complete initializing the slave. This was needed as a way to allow proejcts to add things to slaves without requiring Andy or I to rebuild slave images so would allow projects to install dependencies quicker in cases 15:54:17 <zxiiro> where Andy and I were too busy to create new images. 15:54:45 <zxiiro> rovarga: On Monday unfortunately something in Rackspace caused the initial ssh connection to fail so none of the slaves coming online were able to run their spinup scripts 15:55:02 <zxiiro> rovarga: the solution moving forward is to stop using spinup scripts so that we are not affected by Rackspace issues 15:55:24 <zxiiro> rovarga: I want to move everything back into the vagrant images and someone will have to rebuild all our images everytime a project wants to update 15:55:41 <anipbu> #action rovarga to add yangtools distribution check job 15:55:46 <zxiiro> that'll make this part of the slaves less prone to these intermittent issues 15:56:31 <zxiiro> The reason we still see this is because on monday I only updated the dynamic_verify slaves and forgot to switch the dynamic_merge slaves over to tvhe new updated image 15:57:26 <zxiiro> both dynamic_{verify|merge} are supposed to be identical images yet they have separate configuration in jenkisn so thats why it was missed. Moving forward we'll delete the dynamic_merge ones so that there's no confusion when we update images again in the future 15:58:20 <rovarga> zxiiro: thanks 15:58:30 <anipbu> rovarga: thank you for the feedback you have kindly shared with us today. Do you have any additional comments or concerns you would like to raise? 15:58:30 <zxiiro> So if your autorelease / distribution job happened to run on a dynamic_merge slave this morning that's the cause of the failure 15:58:40 <rovarga> zxiiro: it seems something is broken, still, just got a message from jenkins iwth subject: [controller] $PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS! 15:59:02 <zxiiro> rovarga: yes I need to make the update. I'll go do it now 15:59:13 <rovarga> zxiiro: thanks 15:59:22 <rovarga> anipbu: thanks, that is it for now :) 15:59:48 <anipbu> rovarga: very much appreciate your comments and feedback. 16:00:02 <anipbu> OKay folks, any other blocking issues related to Beryllium? 16:00:24 <anipbu> Last I check, there are no blocking items on the spreadsheet. 16:00:35 <anipbu> #link https://docs.google.com/spreadsheets/d/1hci5TMUPyB6PX8Al-fwfVqvs5SQVa2wZLja_7rKWN6o/edit#gid=2030147923 <-- List of blocking issues 16:00:58 <LuisGomez> cool :) 16:00:59 <anipbu> any project PTL have anything blocking issues to raise regarding Beryllium? 16:01:39 <anipbu> #action PTL, please verify the blocking issues are no longer appearing and move the bug to the FIXED list in the spreadsheet 16:02:20 <anipbu> These projects need to verify: groupbasedpolicy netconf of-config snmp4sdn lispflowmapping netconf nic unimgr didm reservation sfc 16:02:39 <anipbu> Any projects want to chime in on any blocking issues or red flags to raise? open mic... 16:03:51 <LuisGomez> so we do not have standing blocking bugs but we still have to verify some fixes, is there an end date for that? 16:04:27 <anipbu> They should be verified against the latest RC before we present to the TSC 16:04:33 <AmbikaTripathy> anipbu: yangtool anyxml bug: https://bugs.opendaylight.org/show_bug.cgi?id=5099 is not blocking. but if we have tentative fixed data it will be good 16:05:32 <anipbu> LuisGomez: If items are not verified, then TSC will have to decide if we should ship with bugs not verified 16:05:55 <LuisGomez> ok, thanks anipbu 16:06:30 <anipbu> AmbikaTripathy: if the issue is not blocking and does not need to delay Beryllium, I strongly suggest we merge the fix in SR1, which is approximately one month from now. 16:06:56 <ttkacik> anipbu: fix currently does not exist 16:07:22 <AmbikaTripathy> anibhu: not blocking. but good to have during boron release 16:08:29 <anipbu> #info AmbikaTripathy notes there is a bug in yangtools. However, is NOT BLOCKING and okay to release Beryllium with the known bug. ttkacik notes there is currently no fix available for this bug. https://bugs.opendaylight.org/show_bug.cgi?id=5099 16:08:50 <anipbu> ttkacik: thanks for looking into it and giving us an update. 16:09:50 <LuisGomez> #action projects should verify blocking bug fixes and update sheet before tomorrow’s TSC call. 16:10:09 <anipbu> LuisGomez: thanks. 16:10:25 <anipbu> Any last minute topics to raise from folks? 16:10:41 <anipbu> going once 16:11:07 <anipbu> OH QUESTION: do folks still want to continue the weekly sync up? 16:11:21 <anipbu> Maybe I should send out an email 16:12:04 <anipbu> Okay, thanks folks for joining. 16:12:13 <anipbu> #topic cookies 16:12:23 <anipbu> #endmeeting