13:30:59 <dmcbride> #startmeeting OPNFV release daily
13:30:59 <collabot`> Meeting started Wed Mar 22 13:30:59 2017 UTC.  The chair is dmcbride. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:30:59 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:30:59 <collabot`> The meeting name has been set to 'opnfv_release_daily'
13:31:08 <dmcbride> #topic roll call
13:31:12 <dmcbride> #info David McBride
13:31:21 <trozet> #info Tim Rozet
13:31:29 <jose_lausuch1> #info Jose Lausuch
13:31:40 <mtahhan__> #info Maryam Tahhan
13:31:46 <uli-k> #info Uli Kleber
13:32:44 <TapioT> #info Tapio Talglren
13:33:31 <dmcbride> trozet: I noticed that Apex started failing apex-build-danube
13:33:49 <dmcbride> trozet: which it was passing until recently
13:34:21 <dmcbride> narinder: are you available?
13:34:42 <narinder> dmcbride, yes
13:35:02 <trozet> dmcbride: thanks, looks like #14 was a repo issue, will try again
13:35:26 <dmcbride> narinder: have you identified specific issues with Joid
13:35:29 <narinder> #info Narinder Gupta
13:35:47 <dmcbride> #topic installer status
13:35:49 <narinder> dmcbride, i am not finding issue with deployments. Deployments are getting passed
13:36:02 <narinder> dmcbride, i think issue in color and status from jobs.
13:36:34 <narinder> dmcbride, here is the latest deployment task. https://build.opnfv.org/ci/view/joid/job/joid-deploy-baremetal-daily-danube/
13:36:52 <dmcbride> #info dmcbride notes that apex-build-danube recently started failing.  trozet is looking into it.
13:37:25 <narinder> dmcbride, i knows ODL fails which i already mentioned that there is issue with floating ips and need ODL team help in finding out why it failing for floating ips.
13:38:05 <dmcbride> narinder: when I look at build history:  https://build.opnfv.org/ci/view/joid/builds
13:38:13 <dmcbride> narinder: it looks almost all red
13:38:40 <narinder> dmcbride, thats what i am telling you color is not getting updated even though build passes
13:39:12 <narinder> dmcbride, even though builds are passing. still color is red.
13:39:18 <dmcbride> narinder: are  you saying that Jenkins is falsely reporting a failed build?
13:39:30 <narinder> dmcbride, yes
13:39:44 <dmcbride> fdegir: are you aware of this issue?
13:40:02 <narinder> dmcbride, take this example https://build.opnfv.org/ci/view/joid/job/joid-os-odl_l2-nofeature-ha-baremetal-daily-danube/7/
13:40:14 <jose_lausuch1> I think the red is because it considers deployment+functest+yardstick
13:40:23 <jose_lausuch1> if one of those are red, the overall is red
13:40:42 <narinder> jose_lausuch1, so its not a build issue
13:40:51 <dmcbride> I thought that if deployment passed, but tests failed, then it was yellow
13:41:06 <jose_lausuch1> it could be any: build issue, functest not passing, or yardstick not passing
13:41:16 <narinder> jose_lausuch1, dmcbride also check this scenario everything passed but overall status is red https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-nofeature-noha-baremetal-daily-danube/7/
13:41:48 <jose_lausuch> that's right
13:41:52 <narinder> jose_lausuch, dmcbride so somehow color combination is not working for joid
13:42:11 <dmcbride> not good and very confusing
13:42:35 <dmcbride> fdegir: are  you available?
13:42:51 <trozet> narinder: floating IP wont work correctly with ODL L3 if you are accessing the floating ip from an IP other than the gateway IP from inside the external network
13:44:18 <aricg> dmcbride: hi
13:44:29 <dmcbride> aricg: we seem to have an issue where Jenkins is falsely reporting builds as failed (red) for the Joid project
13:44:53 <dmcbride> aricg: is that a known issue?
13:44:57 <narinder> trozet, i am confused it used to work before is it something changed in ODL?
13:45:06 <aricg> dmcbride: link?
13:45:29 <trozet> narinder: no never worked in new netvirt, it will only work if you are either accessing from gateway or a subnet outside of the external network that routes through the gateway
13:45:32 <dmcbride> https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-nofeature-noha-baremetal-daily-danube/7/
13:45:32 <narinder> trozet, Also in that case how does functest and yardstick will pass the test
13:46:00 <dmcbride> #info narinder says that Jenkins is falsely reporting builds as failed on Joid
13:46:05 <aricg> dmcbride: this error: ERROR: Build aborted. Can't trigger undefined projects. 1 of the below project(s) can't be resolved:
13:46:08 <aricg> > dovetail-joid-baremetal-debug-danube ?
13:46:15 <dmcbride> #link https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-nofeature-noha-baremetal-daily-danube/7/
13:46:22 <trozet> narinder: i think functest uses a docker container in a different network, if that routes through your gateway then it will work
13:46:49 <dmcbride> aricg: try this one
13:46:55 <dmcbride> https://build.opnfv.org/ci/view/joid/job/joid-os-odl_l2-nofeature-ha-baremetal-daily-danube/7/
13:47:18 <dmcbride> #undo
13:47:18 <collabot`> Removing item from minutes: <MeetBot.ircmeeting.items.Link object at 0x215c8d0>
13:47:26 <dmcbride> #link https://build.opnfv.org/ci/view/joid/job/joid-os-odl_l2-nofeature-ha-baremetal-daily-danube/7/
13:48:10 <aricg> dmcbride: I see the same error in the console log https://build.opnfv.org/ci/view/joid/job/joid-os-odl_l2-nofeature-ha-baremetal-daily-danube/7/console
13:49:07 <dmcbride> narinder: are you aware of this error that aric is seeing?
13:49:26 <dmcbride> #info narinder says i knows ODL fails which i already mentioned that there is issue with floating ips and need ODL team help in finding out why it failing for floating ips.
13:49:34 <narinder> dmcbride, for ODL yes but for https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-nofeature-noha-baremetal-daily-danube/7/
13:49:40 <narinder> i am not aware of error
13:50:11 <trozet> narinder: https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-nofeature-noha-baremetal-daily-danube/7/console
13:50:17 <trozet> narinder: dovetail job missing
13:50:20 <dmcbride> #info trozet says floating IP wont work correctly with ODL L3 if you are accessing the floating ip from an IP other than the gateway IP from inside the
13:51:20 <dmcbride> does anyone know why "dovetail-joid" generates unidentified project error in Jenkins?
13:52:24 <aricg> dmcbride: its not defined in joid/joid-daily-jobs.yml
13:52:25 <dmcbride> ERROR: Build aborted. Can't trigger undefined projects. 1 of the below project(s) can't be resolved:
13:52:26 <dmcbride> > dovetail-joid-baremetal-debug-danube
13:52:26 <aricg> properly
13:53:14 <dmcbride> #info examples that seem to be generating "false" failures include the following error message:   ERROR: Build aborted. Can't trigger undefined projects. 1 of the below project(s) can't be resolved: dovetail-joid-baremetal-debug-danube
13:53:32 <dmcbride> aricg: ?
13:53:57 <aricg> its triggered to run, but it has no template
13:54:09 <aricg> no idea how it ever worked. looking at the git log
13:54:32 <dmcbride> #info aricg says  its triggered to run, but it has no template
13:54:52 <dmcbride> aricg: how do we fix it and who needs to fix it?
13:54:53 <aricg> looks like it was put in on 28d20442 jjb/joid/joid-daily-jobs.yml (MatthewLi              2017-03-01 03:11:41 -0500 167)
13:54:58 <aricg> march 1syt
13:55:01 <aricg> /1st
13:55:10 <dmcbride> narinder: are you following this?
13:55:53 <dmcbride> #info looks like it was put in on 28d20442 jjb/joid/joid-daily-jobs.yml (MatthewLi              2017-03-01 03:11:41 -0500 167)
13:55:56 <aricg> and it has never completed. https://build.opnfv.org/ci/view/joid/job/joid-os-odl_l2-nofeature-ha-baremetal-daily-danube/
13:56:30 <dmcbride> narinder: ?
13:56:40 <narinder> dmcbride, i never put dovtail test in joid. It must be releng not me.
13:57:09 <dmcbride> how do we fix?
13:57:33 <dmcbride> aricg: can you reach out to Mathew Li about this?
13:57:55 <dmcbride> I don't know that name.  Is he a DoveTail guy?
13:58:41 <aricg> I dont know, I cant even find a build that worked, so I'm not really sure how to go about showing where it broke
13:58:59 <dmcbride> can we simply remove it?
13:59:01 <aricg> I would reduce the complexity of the build
13:59:07 <aricg> and then add things back as they passed
13:59:38 <dmcbride> who?
14:00:23 <aricg> here is Matt commit history https://gerrit.opnfv.org/gerrit/#/q/owner:+Jun+Li
14:00:29 <fdegir> https://gerrit.opnfv.org/gerrit/#/c/29545/
14:00:40 <fdegir> this the patch narinder reviewed
14:00:44 <aricg> wait no, https://gerrit.opnfv.org/gerrit/#/q/owner:+%22Jun+Li%22
14:00:44 <fdegir> adding dovetail jobs
14:01:00 <fdegir> but this doesn't take away the responsibility from releng committer to ensure the change doesn't break stuff
14:01:07 <fdegir> I'll talk to him
14:01:16 <fdegir> my suggestion is to get rid of what is added by that patch
14:01:17 <dmcbride> fdegir: thanks
14:01:21 <narinder> fdegir, that is today because there was request
14:01:40 <aricg> here is the commit in question https://gerrit.opnfv.org/gerrit/#/c/29543/1/jjb/apex/apex.yml
14:01:43 <dmcbride> Julien-zte: do you know about this?
14:01:54 <dmcbride> Julien-zte: I see your name in the commit history
14:02:00 <fdegir> narinder: I don't understand what you mean
14:02:19 <narinder> fdegir, i reviewed patch today about dovtail inclusion
14:02:31 <fdegir> narinder: I put the gerrit change link there to show when dovetail jobs were added
14:02:49 <fdegir> well the history is on gerrit
14:03:14 <narinder> fdegir, ok i was hoping this is patch https://gerrit.opnfv.org/gerrit/#/c/30367/
14:04:11 <dmcbride> ok - do we have a plan?
14:05:45 <dmcbride> If I understood correctly, fdegir is going to talk to Mathew Li?
14:05:57 <fdegir> narinder: -2d that latest patch
14:06:10 <aricg> Wait, the patchset i linked was the same change to the apex build
14:06:12 <aricg> not joid.
14:06:17 <aricg> I can't actually find the joid one.
14:06:30 <fdegir> and this should remove the dovetail jobs from joid daily master
14:06:31 <fdegir> https://gerrit.opnfv.org/gerrit/#/c/31385/
14:06:35 <aricg> it was cherry picked into joid or something
14:07:46 <dmcbride> #link https://gerrit.opnfv.org/gerrit/#/c/31385/
14:07:47 <aricg> but it doesnt show up in Matt's history, but it does show up as his for git blame
14:08:25 <dmcbride> #info remove dovetail jobs from joid daily master ^^
14:08:59 <fdegir> aricg: this is the patch that added dovetail jobs for joid: https://gerrit.opnfv.org/gerrit/#/c/29545/
14:09:21 <dmcbride> fdegir: so, make that change on daily master, then cherry pick to stable/Danube?
14:09:35 <fdegir> dmcbride: there is no branch in releng
14:09:38 <fdegir> no cherrypicking
14:09:54 <fdegir> the patch I sent for review will be submitted once aricg or someone reviews
14:10:14 <aricg> fdegir: ah, I didn't find it thanks
14:10:26 <fdegir> when it comes to how the job status is set: if deployment succeeds but functest and/or yardstick fails, the parent job turns to yellow (unstable)
14:10:30 <aricg> I will merge 31385
14:10:47 <fdegir> if the deployment fails, then the parent job turns to red (failure)
14:10:57 <fdegir> so in this cae, it was missing dovetail job
14:11:41 <dmcbride> #link https://gerrit.opnfv.org/gerrit/#/c/29545/
14:12:01 <dmcbride> #info patch that added dovetail jobs ^^
14:13:01 <dmcbride> ok - this seems like it was useful
14:13:05 <dmcbride> thanks everyone
14:13:24 <dmcbride> next meeting is tomorrow, same time, same place :)
14:13:28 <dmcbride> #endmeeting