08:30:28 <anac1> #startmeeting Yardstick work meeting
08:30:28 <collabot> Meeting started Wed Jan 20 08:30:28 2016 UTC.  The chair is anac1. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:30:28 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
08:30:28 <collabot> The meeting name has been set to 'yardstick_work_meeting'
08:30:35 <anac1> #info Ana Cunha
08:30:48 <kubi1> #info kubi
08:30:49 <QiLiang> #info QiLiang
08:30:57 <DanSmithEricsson> #info Daniel Smith
08:31:12 <vincenzo_riccobe> #info Vincenzo Riccobene
08:31:15 <anac1> #topic release-B
08:31:58 <anac1> #info code freeze has been moved to thursday Jan 21st
08:32:36 <kubi1> ok
08:32:39 <anac1> #info documentation, result visualization and bug fixes will continue after that data
08:32:49 <anac1> i have informed release team
08:33:06 <anac1> #info test cases for SFC are OK to continue this week
08:33:48 <anac1> questions ?
08:34:22 <anac1> #topic scenarios for release-B
08:34:54 <anac1> #info yardstick fails for ODL scenarios
08:35:20 <anac1> jnon has found there are problems with the yardstick generated template
08:35:28 <akis> #info Akis Kourtis
08:35:33 <QiLiang> fail also for onos scenarios
08:36:13 <anac1> QiLiang: have you tried a manually created template ?
08:36:35 <QiLiang> not yet, i can try that
08:37:05 <anac1> please try and let us know, jnon tried a manually created template for ODL and it works
08:37:11 <QiLiang> for onos failure is floatingip create/delete related
08:37:24 <patrick2> #info patrick
08:37:30 <anac1> ok, for ODL floating ip works
08:38:42 <anac1> #info team will continue troulbeshooting ODL, ONOS scenarios
08:39:02 <QiLiang> if onos cannot fix floating ip issue, and manually create template works, can the finnall test use manually created template ?
08:39:29 <anac1> #info criteria for release-B scenarios have to run 4 successfull times
08:40:38 <anac1> QiLiang: as a workaround, but we need to document the limitation and fix the automatic template
08:41:06 <QiLiang> got it, thanks
08:42:50 <anac1> i suggest we try to fix the automatic template as main track
08:44:11 <QiLiang> i agree
08:44:41 <patrick2> anac1: is the code frozen?
08:44:47 <anac1> jnon will continue today for odl, we keep each other informed and check again in tomorrow's meeting
08:44:56 <anac1> patrick2: moved to thursday 21st
08:44:57 <CNlucius> QiLiang>:when I start the ping, the log show that the stack already exist, what is the meaning ?
08:45:12 <patrick2> anac1: ok
08:45:13 <QiLiang> the problem is time is limited onos may have not enough to fix bugs
08:45:44 <QiLiang> CNlucius: heat stack-delete demo
08:46:10 <jnon> just to point out the manual template i used is very similar to yardstick generated template
08:46:26 <CNlucius> ok
08:46:29 <jnon> there are a few small differences
08:47:17 <QiLiang> ok, i think onos and odl 's isssue are not the same
08:47:21 <jnon> yardstick template has a servicegroup and sshkey resources defined, the manual template did not have that and
08:47:54 <jnon> yardstick template includes several 'depends_on' keys in the resources
08:47:54 <QiLiang> with CNlucius from onos try to find workaround
08:48:10 <kubi1> and i think the reason of failure in odl scenario and onos which is deployed by fuel may different with the reason which is deployed by compass
08:48:53 <anac1> kubi1: so there are different faults caused by the different installers?
08:49:23 <QiLiang> yes, change depends_on can change resource create sequence, may be can also fix onos floatingip issues
08:50:40 <kubi1> i'm not sure, but i think there are some network problem in  odl scenario which is deployed by compass.
08:51:04 <jnon> i also suspects depends_on changes resource creation sequence
08:51:47 <anac1> do we know that resource creation sequence in os, odl and onos are different?
08:55:45 <anac1> jnon: suggested way forward ?
08:56:19 <jnon> i will test odl without depends_on in template
08:57:06 <anac1> QiLiang, kubi1: maybe you can try the same for onos ?
08:57:19 <QiLiang> i will test if any heat template work for onos
08:57:28 <QiLiang> ok
08:57:28 <anac1> ok, great
08:57:38 <jnon> if it works we can make a patch and maybe exclude depends_on in our generated templates for odl scenarios
08:58:43 <jnon> or onos
08:58:59 <anac1> ok, we continue today and check again tomorrow
08:59:02 <jnon> its a workaround patch
08:59:30 <anac1> jnon: and fix the generic way as a target ?
08:59:51 <anac1> #info license missing in several files in yardstick repo
09:00:10 <jnon> yes if it is possible to have generic templates for all scenarios
09:00:35 <anac1> #action all - check https://etherpad.opnfv.org/p/yardstick_release_b_troubleshooting and fix the files you are author
09:01:07 <anac1> #info the license header MUST be there this week
09:01:27 <anac1> #info requirement from legal OPNFV
09:01:43 <anac1> anything else?
09:01:57 <QiLiang> i have a question from compass team: what’s the percentage of yardstick sla testcase is acceptable?
09:01:58 <vincenzo_riccobe> I was looking at THE LSIT
09:02:04 <vincenzo_riccobe> I was looking at the list
09:02:18 <vincenzo_riccobe> and some of the files are strange format, like pcap
09:02:30 <vincenzo_riccobe> there is no way to include a license there
09:03:03 <QiLiang> does sla failure matters？
09:03:48 <anac1> vincenzo_riccobe: OPNFV legal have done a screening on all repos, they don't know about format...
09:03:56 <anac1> QiLiang: what sla ?
09:04:15 <QiLiang> SLA
09:04:32 <vincenzo_riccobe> I can fix all the others but there is no actual way to fix them. Any suggestion?
09:05:05 <anac1> vincenzo_riccobe: i will let legal know, i'll come back to you
09:05:14 <vincenzo_riccobe> thanks
09:05:21 <anac1> QiLiang: what test case?
09:05:32 <QiLiang> [01/19/16 21:22:31] yardstick.benchmark.runners.iteration iteration.py:68 WARNING SLA validation failed: latency 31.382000 > sla:max_latency(30.000000); latency 30.330000 > sla:max_latency(30.000000); latency 31.437000 > sla:max_latency(30.000000);
09:05:40 <QiLiang> https://build.opnfv.org/ci/job/yardstick-compass-huawei-us-deploy-bare-1-daily-brahmaputra/20/consoleFull
09:06:10 <QiLiang> there are many sla failure
09:06:29 <kubi1> WARNING SLA validation failed: rtt 43.300000 > sla:max_rtt(10.000000);
09:06:36 <anac1> ok, we need to document the results - right now there isn't a specific SLA decision
09:07:20 <QiLiang> ok
09:07:35 <anac1> we expalin in the report that we set SLA = 30 and for scenario X in lab y it is actually 31.5 the average, that gives an overview of the platform behavior
09:08:06 <kubi1> ok
09:08:07 <anac1> and also a view that yardstick actually verifies the SLA
09:08:29 <anac1> usefull for next releases
09:08:38 <QiLiang> hahaha
09:08:50 <kubi1> baseline
09:09:01 <anac1> baseline, i like that !
09:09:22 <anac1> #topic results visualization
09:10:03 <anac1> when you have time, check whether the server is reachable, i can get to the landing page
09:10:48 <QiLiang> ok
09:10:57 <anac1> thanks !
09:11:01 <akis> Ok
09:11:02 <anac1> anything else ?
09:11:23 <DanSmithEricsson> can the nodes inside montreal be taken down now?
09:11:45 <anac1> Dan: could you wait until we veify it works ?
09:11:58 <DanSmithEricsson> sure - im referring to TrafClass, Influx and Graph
09:12:06 <DanSmithEricsson> for the YS setup i am assuming that stays?
09:12:19 <anac1> this is for the influx+grafana, the other nodes for TrafClass will stay in Montreal lab
09:12:28 <anac1> also YS will stay
09:12:35 <anac1> it's a dev lab
09:12:55 <DanSmithEricsson> ok - so the TraffClass-Packet1 machine and the YS blades
09:13:10 <DanSmithEricsson> and you will notify me when to Influx server can be returned to the pool.
09:13:12 <DanSmithEricsson> thank you
09:13:20 <anac1> there was a decision by testing sub-group to collect all results in a common server, that's why
09:13:50 <DanSmithEricsson> fair enough - with your results, there is no IP or other informatoin availablke right?
09:13:56 <anac1> TraffClass and YS stay, i'll let you know about the influx server asap
09:13:59 <DanSmithEricsson> that people can use to get into the lab and call trouble?
09:14:09 <DanSmithEricsson> cause*
09:14:12 <anac1> Dan: right
09:14:17 <DanSmithEricsson> great.. thank ou
09:14:18 <DanSmithEricsson> you*
09:14:23 <anac1> thank you
09:14:36 <anac1> anything else?
09:15:26 <vincenzo_riccobe> jnon: can we try to sync today on the vTC integration?
09:15:32 <anac1> ok, thanks all for today - continue odl + onos and check again tomorrow
09:15:38 <anac1> #endmeeting