14:59:17 <dmcbride> #startmeeting OPNFV Colorado Release daily
14:59:17 <collabot`> Meeting started Fri Sep  9 14:59:17 2016 UTC.  The chair is dmcbride. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:17 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:59:17 <collabot`> The meeting name has been set to 'opnfv_colorado_release_daily'
14:59:27 <dmcbride> #topic roll call
14:59:33 <dmcbride> #info David McBride
15:01:01 <fdegir> #info Fatih Degirmenci
15:01:26 <narindergupta> #info Narinder Gupta
15:03:17 <Greg_E_> info Greg Elkinbard
15:04:27 <dmcbride> hey guys
15:04:52 <dmcbride> Greg - what's the status of Fuel?  What are the pain points?
15:05:10 <dmcbride> #topic installer reports
15:05:29 <Greg_E_> working on patching ceilometer so that we can meet doctor tests
15:05:31 <dmcbride> Greg_E_: what's the status of Fuel?  What are the pain points?
15:06:06 <dmcbride> Greg_E_: how much time and how difficult?
15:06:41 <Greg_E_> there are 3 or 4 other bugs we are trying to address for mon night
15:06:51 <dmcbride> #info Greg_E_ reports that he is "working on patching ceilometer so that we can meet doctor tests"
15:07:06 <Greg_E_> team was asked to finish by Mon nite
15:07:35 <Greg_E_> if it does not work, we need to ask for doctor to be blacklisted from fuel fucktest
15:07:36 <dmcbride> #info Greg_E_ reports that team will complete by Monday night
15:07:48 <dmcbride> thanks, Greg
15:08:03 <dmcbride> Greg_E_:  anything else you'd like to report?
15:08:39 <dmcbride> narindergupta: any updates on JOID?
15:08:53 <narindergupta> dmcbride, we are good and all scenarios are passing
15:09:12 <narindergupta> dmcbride, currently working on finalizing the installaer and release notes
15:09:46 <narindergupta> dmcbride, ODL scenaio changes required once BORON ODL will be released.
15:09:52 <narindergupta> thats all from me.
15:10:32 <dmcbride> narindergupta:  functest results look promising
15:10:57 <dmcbride> narindergupta:  biggest struggle seems to be with os-nosdn-lxd-ha and os-nosdn-lxd-noha
15:11:01 <narindergupta> dmcbride, yes have blue ball on all scenaio except lxd
15:11:16 <dmcbride> narindergupta: what is the issue with lxd
15:11:22 <narindergupta> lxd is fialing on yardstick because yardstick does not support container based deployment
15:11:40 <narindergupta> functest with lxd is passed now
15:12:04 <Greg_E_> we will try for next rc candidate Monday night, nothing else to report from fuel side
15:12:06 <dmcbride> narindergupta: is it passing on functest?  Functest should show a green ball once 4 passes have been successfully completed?
15:12:31 <dmcbride> narindergupta: btw - excuse my ignorance, but what is lxd?
15:12:41 <narindergupta> dmcbride, it will be green soon as it started passing from yesterday onwards only
15:12:54 <narindergupta> lxd is container based hypervisor
15:13:38 <dmcbride> narindergupta: ok  right, I remember that now
15:13:46 <narindergupta> dmcbride, :)
15:14:16 <narindergupta> dmcbride, i think it is question to yardstick when will they implement contianer based tests.
15:14:19 <narindergupta> ?
15:14:28 <dmcbride> BTW - where are you accessing yardstick results? I'm getting an error when I navigate to the URL I have
15:15:06 <narindergupta> dmcbride, i uses the CI pod status
15:15:22 <narindergupta> dmcbride, https://build.opnfv.org/ci/view/joid/job/joid-os-nosdn-lxd-ha-baremetal-daily-colorado/
15:15:43 <narindergupta> it is yellow because yardstick failed
15:15:57 <dmcbride> narindergupta: unfortunately, it doesn't appear that anyone from the test team joined today
15:16:13 <narindergupta> dmcbride, :(
15:17:03 <dmcbride> fdegir: any issues that you'd like to raise for Colorado 1.0?
15:17:17 <fdegir> dmcbride: not from my side
15:17:49 <dmcbride> #info narindergupta asks "when will they implement contianer based tests."
15:18:08 <dmcbride> #info "they" ==> yardstick
15:18:36 <dmcbride> #topic infra updates
15:19:06 <dmcbride> #info fdegir says no immediate issues related to Colorado 1.0
15:19:26 <dmcbride> jmorgan1: are you with us?
15:19:36 <jmorgan1> dmcbride: yup
15:19:59 <dmcbride> jmorgan1: anything you'd like to add?  Issues, concerns for Col 1.0?
15:20:24 <dmcbride> trozet: any updates on Apex?
15:20:37 <jmorgan1> dmcbride: i'm still working on updating pharos specification and userguide
15:20:46 <trozet> dmcbride: it looks like all functest is failing on lfpod1
15:20:46 <dmcbride> trozet: I saw your email this morning regarding functest error
15:20:51 <jmorgan1> dmcbride: but otherwise, it looks good
15:20:53 <trozet> dmcbride: hope that doesnt give us less points ;)
15:21:21 <trozet> dmcbride: i saw copper passed for nosdn and odl_l2 yesterday, i would expect it to keep passing now
15:21:35 <dmcbride> trozet: good news!
15:21:57 <trozet> dmcbride: btw for odl_l3, there is a bug in opendaylight that causes 5 testscases to fail
15:22:01 <dmcbride> #info trozet reports that copper passed for nosdn and odl_l2 yesterday, i would expect it to keep passing now
15:22:02 <trozet> dmcbride: this is across all installers
15:22:22 <trozet> dmcbride: i believe there was a patch from functest team to disable these tests, i guess to make them pass and document hte bug
15:22:41 <dmcbride> jmorgan1: do you know anything about the issue with lfpod1 reported by trozet?
15:22:47 <trozet> dmcbride: i'm not sure if they are now disabled or not, but it is the reason why odl_l3 tempest cases fail
15:23:23 <dmcbride> trozet: this is the issue we were discussing with bryan_att yesterday?
15:23:39 <jmorgan1> dmcbride: what is the issue with lf-pod1?
15:23:41 <trozet> dmcbride: no the issue we were discussing yesterday was around copper
15:23:47 <trozet> dmcbride: i believe the copper issue is fixed
15:24:10 <dmcbride> trozet: could you describe issue with lf-pod1 to jmorgan1?
15:24:21 <trozet> jmorgan1:https://build.opnfv.org/ci/view/apex/job/functest-apex-apex-daily-colorado-daily-colorado/130/console
15:24:27 <trozet> failed to register layer: devmapper: Thin Pool has 163693 free data blocks which is less than minimum required 163840 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior
15:24:52 <trozet> something wrong with Docker
15:25:20 <jmorgan1> dmcbride: i'm not awear of this issue; this should be addressed with helpdesk i think
15:25:33 <jmorgan1> dmcbride: i they are not already working on it
15:25:49 <trozet> i can look into it in a bit
15:26:04 <trozet> jmorgan1, dmcbride: i am concerned hte failing functest runs will further lower our score on the wiki page
15:26:45 <dmcbride> #info trozet reports issue with lf-pod1.  jmorgan1 suggests filing help desk ticket.  trozet says he will investigate further.
15:27:37 <dmcbride> trozet: I guess you're referring to failures that are out of your control?
15:28:54 <trozet> dmcbride: well it is not a failure of the deployment, it is a infrastructure/functest issue
15:29:02 <trozet> dmcbride: so those should not count against us
15:29:05 <dmcbride> ok - team - does anyone else have any urgent issues related to Col 1.0 that they'd like to raise today?
15:29:34 <dmcbride> trozet: ok - so test artifact, not a "real" failure?
15:29:46 <trozet> dmcbride: right
15:29:56 <trozet> dmcbride: it never really ran the tests
15:30:15 <fdegir> I think I remember seeing similar issue
15:30:19 <fdegir> with docker i mean
15:30:37 <dmcbride> #info trozet raises concern that functest/infra is generating false positives
15:30:43 <fdegir> removing images/containers helped
15:31:31 <jmorgan1> yeah, that errors looks like you are running out of disk space
15:31:57 <trozet> jmorgan1, fdegir: is it disk space, or is it some storage pool for docker
15:32:41 <jmorgan1> my guess is docker storage pool, but not sure
15:33:10 <fdegir> trozet: I think you hit the docker allocation or something
15:33:22 <trozet> ok time to google :)
15:33:35 <fdegir> it's been long time ago since I've seen this error
15:33:44 <fdegir> so don't remember nothing but removing stuff helped
15:34:39 <trozet> /dev/mapper/centos-root  2.2T  328G  1.9T  15% /
15:34:50 <trozet> looks like the host is ok, so let me figure out how to check docker pool
15:34:56 <dmcbride> #info issue possibly related to disk space or docker storage pool
15:35:49 <dmcbride> ok - we're out of time - have a good weekend
15:36:21 <dmcbride> BTW - I believe that trozet is out next week, so contact him today if you need to speak with him!
15:36:55 <dmcbride> #endmeeting