14:02:02 <mackonstan1> #startmeeting CSIT meeting
14:02:02 <collabot`> Meeting started Wed May 11 14:02:02 2016 UTC.  The chair is mackonstan1. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:02 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:02:02 <collabot`> The meeting name has been set to 'csit_meeting'
14:02:39 <mackonstan1> #chair skobza
14:02:39 <collabot`> Current chairs: mackonstan1 skobza
14:03:05 <skobza> #topic agenda bashing
14:04:23 <skobza> #topic operational status
14:06:41 <skobza> #info func tests jenkins jobs: no issues spotted. One problem is still with vlan sub-interfaces, and some packets were not making it thru to VPP. Damjan asked for more information, Carsten working on it. Issue exists for more than 3 weeks
14:07:51 <skobza> #info it was unknown whether the problem was due to VPP problem, CSIT, or VIRL
14:08:41 <skobza> #info Carsten: at this stage it is a VPP bug, and CSIT is providing data as requested
14:08:55 <edwarnicke> Is there a VPP bug in Jira we can link here?
14:09:19 <skobza> no clue, would have to search
14:09:47 <edwarnicke> skobza: So without a bug... do we know its on the vpp guys radar?
14:10:02 <edwarnicke> skobza: And are we simply looking at needing to do a binary search here?
14:10:08 <skobza> Carsten had e-mail thread with Damjan
14:10:24 <skobza> nope, this test was failing from the start
14:10:26 <edwarnicke> skobza: OK... but without a bug, it doesn't show up in the vpp scrub
14:10:39 <skobza> there might be one, I just don't know
14:10:51 <edwarnicke> skobza: A bug also gives a good place to accumulate info about the bug and work on its resolution
14:11:01 <edwarnicke> skobza: Who might know?
14:11:02 <skobza> #info csit-verify is still not working
14:11:25 <skobza> edwarnicke: anyone who goes to VPP jira and does a quick glance :)
14:11:29 <skobza> i'm writing notes, no time
14:12:31 <skobza> #info csit-vpp-verify-weekly patch is in review
14:13:53 <skobza> #info perf test cases update: vpp csit jobs are running, some are in review. Work on tb1 and tb3 in progress.
14:14:43 <skobza> #info perf jobs are not heavily used by vpp devs.
14:14:55 <skobza> #info no feedback from vpp-devs why they are not using them.
14:16:37 <skobza> #topic jira tasks and issues tracking
14:16:57 <skobza> #info CSIT review queue had been reduced.
14:17:33 <skobza> #info iacl, SHG tests are in
14:18:06 <skobza> #info FDS team has sent us their test plan. We are working with that test plan to capture the scenarios.
14:18:31 <skobza> #info Zdenko looking into the FDS test cases.
14:19:13 <skobza> #info FDS: we have to add VXLAN and VLAN tests w/ and w/out vhost-user test cases.
14:20:04 <skobza> #info 2 more items in pipeline, the rest of the Jira issues will get prioritized later
14:21:38 <skobza> #action mackonstan1 Jan: walk thru the list of Jira CSIT issues, and mark down items that are VPP code not ready
14:22:08 <skobza> edwarnicke: https://jira.fd.io/projects/VPP/issues/VPP-40
14:22:21 <skobza> #link https://jira.fd.io/projects/VPP/issues/VPP-40 VLAN-tagged ingress packets discarded by VPP on VIRL
14:22:38 <edwarnicke> skobza: Thank you :)
14:23:12 <skobza> yw
14:23:36 <skobza> #topic VPP performance results - analysis
14:24:51 <skobza> #info On vpp call some people believed to be low, but also there were some inconsistency in the results.
14:25:48 <skobza> #info some vpp devs commented that the perf numbers are low, becuase of external factors - setup, traff gen, and so on
14:26:33 <skobza> #info CSIT perf has to work out what telemetry data should we collect and report while test run.
14:27:53 <skobza> #info Peter states that CSIT should add host environment data to the perf results. Our LF based testing infra is a little bit different from vpp-dev folks.
14:28:26 <skobza> #info ... we should consider how to normalize test environment to share and compare test results
14:30:06 <skobza> #info ... CSIT should publish the test bed setup doc.
14:32:01 <skobza> #info ... NDR testing for zero packet loss vs. non-zero loss tolerance difference.
14:33:25 <skobza> #info ... Peter to look into implement one place for zero packet loss and non-zero loss tolerance
14:36:13 <skobza> #info ... request to track the perf test results anomalies in Jira
14:36:30 <skobza> #action mackonstan1 to create Jira issues to track the perf test anomalies (multi-core)
14:38:05 <skobza> #topic test results readability
14:38:24 <skobza> #info log and report files are still not viewable - have to be downloaded
14:38:56 <edwarnicke> #link https://wiki.fd.io/view/Infra/tickets <- prioritization of tickets
14:39:08 <skobza> #chair edwarnicke
14:39:08 <collabot`> Current chairs: edwarnicke mackonstan1 skobza
14:39:22 <mackonstan1> #info still can't read in-browser RF log.html and report.html
14:39:30 <mackonstan1> #info example
14:39:35 <mackonstan1> #link https://jenkins.fd.io/view/csit/job/csit-vpp-perf-hw-all/78/robot/report/log.html
14:47:54 <skobza> #info perf team to write down perf TB configs: HW, bios, grub, vpp startup config
14:49:00 <skobza> #info all perf doc going to go to wiki
14:50:50 <skobza> #topic dev work plan
14:51:04 <skobza> #info Michal confirmed he updated the links at wiki.fd.io
14:51:54 <skobza> #info vnet virl - two virl machines are operational since weekend. The only thing that needs to be resolved, is some configs have to be committed to git. Next item is test the nested virtualization env in LF
14:52:26 <skobza> #info VIRL monitoring to be looked at too.
14:54:59 <skobza> #topic CSIT usability
14:55:25 <skobza> #info LISP the only project to submit test cases to CSIT
14:57:01 <skobza> #info LISP willing to write perf tests, if not really different than func tests.
14:57:19 <skobza> #action mackonstan1 Florin: Look at potential LISP perf tests
14:58:12 <skobza> #topic documentation
14:58:26 <skobza> #info additions to wiki: writing a test case, near finish. Some comments exists.
15:02:33 <skobza> #link https://atlas.hashicorp.com/bento/boxes/ubuntu-16.04
15:02:35 <skobza> #endmeeting