13:58:52 <dmcbride> #startmeeting OPNFV Colorado Release daily
13:58:52 <collabot> Meeting started Fri Sep 16 13:58:52 2016 UTC.  The chair is dmcbride. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:58:52 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:58:52 <collabot> The meeting name has been set to 'opnfv_colorado_release_daily'
13:59:12 <dmcbride> #topic roll call
13:59:17 <dmcbride> #info David McBride
14:01:21 <jose_lausuch> #info Jose Lausuch
14:06:11 <fdegir> aren't you 1 hour early?
14:06:48 <dmcbride> oh wow, you're right!
14:07:06 <dmcbride> jose and I were trying to figure out why no one was joining
14:07:11 <dmcbride> jose_lausuch: one hour early
14:07:13 <dmcbride> bleah
14:07:15 <jose_lausuch> :)
14:07:16 <jose_lausuch> ok
14:07:34 <dmcbride> ok - guys, see you at 8 a.m. PST
14:07:42 <dmcbride> sorry for the confusion
14:07:51 <jose_lausuch> no prob
15:00:40 <dmcbride> ok - shall we try this again?
15:01:49 <kubi001> dmcbride: release meeting?
15:02:01 <dmcbride> kubi001: yes
15:02:21 <dmcbride> kubi001: do you have any updates on the status of yardstick?
15:02:36 <kubi001> due to chinese mid-autumn festival, I missed yesterday meeting, sorry for that
15:03:09 <ChrisPriceAB> #info chris price
15:03:14 <dmcbride> kubi001: no problem, hope you enjoyed the holiday
15:03:16 <kubi001> dmcbride: I checked the minutes of yesterday meeting and send email to you
15:03:32 <dmcbride> ok
15:03:48 <dmcbride> for example, Apex is showing zero results on the dashboard:  http://testresults.opnfv.org/reporting/yardstick/release/master/index-status-apex.html
15:04:29 <dmcbride> also, none of the scenarios seem to report more than about 4 iterations
15:04:56 <dmcbride> any idea why ?
15:05:08 <jose_lausuch> http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
15:05:09 <kubi001> dmcbride: the score show the last 4 iterations status
15:05:28 <kubi001> for the score should be not more than 4
15:05:30 <dmcbride> haha - ok, that would explain it
15:05:52 <rprakash> #info rprakash
15:05:57 <kubi001> the data should be the right, i think
15:06:00 <bryan_att> #info Bryan Sullivan
15:06:02 <morgan_functest> #info Morgan
15:06:03 <dali`> #info Dan Lilliehorn (ARMBand/Enea)'
15:06:06 <dmcbride> kubi001: I think this is a little confusing, given that functest does it differently
15:06:25 <kubi001> there is a little different between the functest and yardstick
15:06:44 <dmcbride> ideally, we would follow a similar methodology for representing results in the dashboards for both functest and yardstick
15:06:53 <kubi001> functest reports the result with the Tier result.
15:07:15 <dmcbride> I think functest shows the results for the past 50 days
15:07:24 <morgan_functest> we reduced to 10
15:07:27 <jose_lausuch> dmcbride: that has changed, now its 10 days
15:07:34 <dmcbride> ok
15:07:41 <jose_lausuch> morgan_functest ? new name? :)
15:07:50 <kubi001> morgan_functest: I think functest 's score also show the status of last 4 interations?
15:08:17 <morgan_functest> yes for the scoring we consider 4 iterations
15:08:21 <dmcbride> I think that's represented by the guage
15:08:44 <morgan_functest> each case can have 0-3 points => never work - works weel 4 consecutive time
15:08:50 <morgan_functest> then we build a global scoring
15:09:14 <morgan_functest> but functest is consdering all the sub testcases and has a success criteria (easy for functional testing)
15:09:20 <morgan_functest> yardstick it si a bit different
15:09:42 <jose_lausuch> the problem with that if a test fails at the beggining (healthcheck) it is not reflected here
15:09:44 <morgan_functest> they consider the ability to run and have no threshold fail/pass criteria per sub test cases
15:09:50 <kubi001> dmcbride:  as morgan said,  their score also for last 4 iteration, and they include the Tier result
15:09:55 <dmcbride> I think there's a natural tendency for users to look at both dashboards and assume that they're following similar methodology for scoring, so its a little confusing
15:10:19 <morgan_functest> yes but are not reporting the same thing...
15:10:36 <dmcbride> I think it would be helpful if the functest and yardstick teams could align on a consistent representation of the results for both
15:10:44 <ChrisPriceAB> +1
15:10:55 <morgan_functest> we can take that for D but as said we do not do the same thing
15:11:02 <morgan_functest> functest case => PASS/FAIL
15:11:08 <morgan_functest> yardstick case => figure
15:11:18 <kubi001> morgan_functest: +1
15:11:30 <morgan_functest> but we can discuss that in the testing working group
15:11:31 <kubi001> but we'd like to align it in D
15:11:43 <morgan_functest> yardstick reporting has been added lately to help the representation
15:11:55 <dmcbride> morgan_functest: figure?
15:12:01 <morgan_functest> numbers
15:12:16 <morgan_functest> average bandwitch, latency, ...
15:12:27 <dmcbride> ok
15:12:36 <morgan_functest> and there are no KPI/Thresholds saying above the figure it is ok or not
15:12:49 <dmcbride> what about Apex?
15:12:51 <morgan_functest> yardstick can qualify a system
15:13:03 <dmcbride> why no yardstick results displayed for Apex?
15:13:26 <kubi001> dmcbride: I just noticed that error, I will try to troubleshoot it ASAP
15:14:00 <dmcbride> sofiawallin: hello
15:14:16 <jose_lausuch> dmcbride: there are, you were looking at master branch
15:14:17 <jose_lausuch> http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
15:14:24 <jose_lausuch> apex is not running master branch jobs any more
15:14:59 <kubi001> jose_lausuch: ha
15:15:02 <morgan_functest> yep so branching has really no sense ...for us here :)
15:15:35 <dmcbride> ah - ok - my bad
15:15:38 <dmcbride> thanks for the correction
15:16:09 <kubi001> jose_lausuch:thanks
15:16:12 <jose_lausuch> np
15:17:03 <kubi001> http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html
15:17:18 <sofiawallin> dmcbride: I'm here
15:17:24 <kubi001> colorado branch is available
15:17:48 <dmcbride> sofiawallin: how is progress with documentation since Tuesday release meeting?
15:18:15 <kubi001> #info kubi
15:19:38 <dmcbride> not sure what happened to Sofia, she was online a minute ago
15:20:07 <dmcbride> there was a thread about disabling Doctor yesterday
15:20:08 <sofiawallin> dmcbride: Still waiting on the Pharos lab specification, and to get all the document  links collected
15:20:21 <jose_lausuch> isn't it disabled already in fuel?
15:20:25 <morgan_functest> dmcbride: di you have info on the date for tagging and/or any comment on my mail. To finalize the release note I wiould like to capture the reality
15:20:25 <dmcbride> ok
15:21:20 <jose_lausuch> dmcbride: doctor running only on apex https://git.opnfv.org/cgit/functest/tree/ci/testcases.yaml#n127
15:21:43 <morgan_functest> and potentially exclude the scenarios that are no more deployable (even if they were some times ago...)
15:21:45 <dmcbride> sofiawallin: I copied you on a mail from dali` regarding Armband documentation
15:22:16 <sofiawallin> dmcbride: Some minor items remaining for 3-4 project. Will follow up before I leave today
15:22:52 <dmcbride> sofiawallin: did you see the email about the Armband documentation?
15:23:12 <sofiawallin> dmcbride: Yes I saw the email, responding as we speak
15:23:22 <dmcbride> ok - great
15:23:34 <dali`> sofiawallin + dmcbride: :-)
15:23:49 <dmcbride> sofiawallin: dali` is online now if you have any questions or comments for him
15:24:47 <dmcbride> morgan_functest: I don't think I've seen your mail.  Was that this morning?  I'll look again.
15:26:08 <dmcbride> morgan_functest: let's tentatively plan on 12:00 p.m. (noon) PST on Sept 22
15:26:21 <morgan_functest> dmcbride: yes it was this morning (for me)
15:26:33 <dmcbride> ok - i will look
15:26:43 <morgan_functest> basically the question is
15:26:49 <sofiawallin> dmcbride: dali`: no questions yet, will have a look in the armband repo just to see what's been done.
15:26:51 <dmcbride> ok - so going back to doctor
15:27:07 <morgan_functest> ok let's have a look we will discuss next time
15:27:08 <dmcbride> was Doctor disabled for all scenarios on Fuel?  Is that accurate?
15:27:36 <morgan_functest> dmcbride: yes it is the decision of the scenario owner
15:28:26 <dmcbride> morgan_functest: ok - I just want to be clear on the scope.  ALL scenarios on Fuel only?
15:28:38 <morgan_functest> yes
15:28:42 <morgan_functest> it works well with apex
15:28:59 <dmcbride> ok - got it - thanks
15:29:29 <dmcbride> ok - we just have a minute or two left, does anyone want to bring up any issues?
15:29:41 <dmcbride> bryan_att: are you a happy man?
15:29:44 <morgan_functest> on apex it works on all scenarios except fdio (specific scenario focusing on fdio)
15:29:59 <bryan_att> don't I seem to be?
15:30:14 <bryan_att> (is it not working?)
15:30:17 <dmcbride> bryan_att: yes, always a cheerful fellow :)
15:30:30 <morgan_functest> bryan_att: could you have a look at functest release note / description and links for copper
15:30:42 <bryan_att> where is it?
15:31:21 <dmcbride> also - if you are a scenario owner and you have not updated your release intentions on the scenario status page, please do so before Monday
15:31:32 <bryan_att> http://artifacts.opnfv.org/functest/colorado/docs/release-notes/functest-release.html?
15:31:50 <bryan_att> no links in that doc
15:32:05 <dmcbride> https://wiki.opnfv.org/display/SWREL/Colorado+Scenario+Status
15:32:55 <bryan_att> dmcbride: was that in answer to my question?
15:32:57 <jose_lausuch> http://artifacts.opnfv.org/functest/colorado/docs/release-notes/index.html
15:33:15 <bryan_att> I see places for links in http://artifacts.opnfv.org/functest/colorado/docs/userguide/index.html but not in the release notes.
15:33:23 <morgan_functest> yes in the userguide
15:33:27 <dmcbride> bryan_att: no - that was the scenario status page that I'm trying to get scenario owners to update if they haven't done so
15:33:32 <bryan_att> I can add links to the userguide. Do you want them by email or as patches?
15:33:44 <morgan_functest> by patch is fine
15:33:54 <bryan_att> OK, I'll work on it
15:33:59 <morgan_functest> thanks
15:34:21 <jose_lausuch> this belongs to the latest release notes
15:34:21 <jose_lausuch> By default, if not precised, the scenarios are HA. HA means OpenStack High Availability (main services). For copper test, the OpenStack congress module is not HA. See the release notes of the installers for details
15:34:24 <jose_lausuch> is it ok?
15:34:44 <bryan_att> I'll check but that is the plan. Congress is not HA.
15:34:55 <jose_lausuch> ok
15:35:10 <bryan_att> If I can deliver HA in a point release I will.
15:35:26 <morgan_functest> and network controllers are not HA ...HA scenarios are sometimes a bit over optimistic :)
15:35:41 <bryan_att> I consider the lack of HA for Congress a bug (maybe optimistic)
15:36:17 <morgan_functest> +1 and I think that also this kind of feedback OpenStack community may be interested
15:36:44 <bryan_att> We are working that.
15:37:02 <morgan_functest> sofiawallin: we have still Arno reference in the doc http://artifacts.opnfv.org/functest/colorado/docs/userguide/ but it seems it is due to the genration of the doc
15:37:54 <morgan_functest> shall we open a JIRA?
15:38:13 <dmcbride> morgan_functest: I think that would be a good idea
15:38:33 <morgan_functest> I do
15:38:40 <jose_lausuch> I have used grep in our repo and there is no reference to arno, so I dont know where that is coming from
15:39:23 <sofiawallin> morgran_functest: I thought that was updated after we labeled the release
15:40:13 <jose_lausuch> could be, but labeled bahmaputra
15:40:18 <jose_lausuch> why still arno? :)
15:40:21 <jose_lausuch> well, I dont kno
15:40:24 <jose_lausuch> w
15:40:42 <jose_lausuch> maybe a JIRA ticket would make sesnse
15:41:08 <morgan_functest> https://jira.opnfv.org/browse/DOCS-160
15:41:34 <sofiawallin> morgan_functest: thanks!!
15:41:36 <jose_lausuch> so quick
15:41:47 <morgan_functest> so maybe it will be fixed once we have tagged
15:41:53 <jose_lausuch> maybe ryota can help
15:42:06 <dali`> I need to go, but contact me on email if you need any updates from ARMBand on docs and we'll try to get them in during the weekend.
15:42:59 <morgan_functest> we are already a bit late...what are the plans for the release, still on track for the 22nd?
15:44:42 <dmcbride> morgan_functest: yes
15:46:53 <dmcbride> #endmeeting