14:11:54 <mackonstan> #startmeeting FD.io CSIT Project Meeting
14:11:54 <collab-meetbot> Meeting started Wed Jul  7 14:11:54 2021 UTC and is due to finish in 60 minutes.  The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:11:54 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:11:54 <collab-meetbot> The meeting name has been set to 'fd_io_csit_project_meeting'
14:12:03 <vrpolak> #info Vratko Polak.
14:12:05 <mackonstan> #info
14:12:06 <jlinkes> #info Juraj Linkeš
14:12:07 <tifrank> #info Tibor Frank
14:12:15 <mackonstan> #info Maciek Konstantynowicz
14:12:18 <mackonstan> #chair
14:12:18 <collab-meetbot> Current chairs: mackonstan
14:12:25 <dwallacelf> #info Dave Wallace
14:12:26 <fan_zhang> #info Fan Zhang
14:13:49 <mackonstan> #topic Agenda bashing
14:13:54 <mackonstan> #topic Inputs from projects
14:14:01 <mackonstan> #topic Inputs from projects - VPP
14:14:12 <mackonstan> #topic Inputs from projects - VSAP
14:14:25 <mackonstan> #topic Inputs from projects - TSC
14:15:28 <mackonstan> #topic New performance tests - gtpu
14:16:16 <mackonstan> #info Vratko, tests are merged.
14:16:53 <mackonstan> #info Tibor, covered in report, will add to trending. Can be removed from the agenda.
14:18:03 <mackonstan> #topic New functional Tests - flow "offload" patch
14:18:55 <mackonstan> #info Juraj, comments addressed, one comment from Vratko outstanding.
14:20:49 <mackonstan> #info ignore two posts listed above
14:21:17 <mackonstan> #topic New functional Tests - IPsec SPD outbound flow cache patch
14:21:28 <mackonstan> #info Juraj, comments addressed, one comment from Vratko outstanding.
14:22:02 <mackonstan> #info Vratko, 32308, review almost complete, can be merged soon.
14:22:14 <mackonstan> #info Vratko, 32895, waiting for VPP.
14:22:33 <mackonstan> #topic New functional Tests - flow "offload" patch
14:23:52 <mackonstan> #info Fan, waiting to have the CVL fully onboarded in vpp_device testbed.
14:26:47 <mackonstan> #info Peter, missing QSFP28 cable for the 2nd testbed, awaiting new cable shipment (in-flight). Will be finishing the coding part later this week.
14:27:44 <mackonstan> #info Fan, the benchmarking is TBD.
14:28:16 <mackonstan> #topic Release - CSIT-2106
14:29:06 <mackonstan> #info Tibor, rls2106 testing going well, iterative tests, reconfig tests are almost done.
14:29:22 <mackonstan> #info Tibor, will be starting coverage tests
14:29:36 <mackonstan> #info Tibor, draft report ver in master
14:29:59 <mackonstan> Tibor, same test failures as in rc2
14:33:56 <mackonstan> #info Tibor, intermittent failures: 1) rdma-core on one testbed, switching driver problem; 2) create avf interface, moving thru NIC rxq configurations; 3) nat44ed 16M sessions not be established in allotted time for some tests; 4) deterministic for specific (frame_size, core config): ipsec sw asyn tests, no traffic forwarded; 5) VSAP tests with MLX
14:33:56 <mackonstan> NIC, no traffic forwarded;
14:34:28 <mackonstan> #info Vratko, Peter re 5) ARP resolution issue.
14:34:42 <vrpolak> #link https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-skx/476/  <-- Ipsec async scheduler results for 3n-skx
14:34:58 <vrpolak> #link https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-dnv/109/  <-- for 3n-dnv
14:35:31 <vrpolak> #info The links are for master, but we expect 2106 to look the same.
14:36:16 <vrpolak> #info The performance change happened after we switched CSIT code globally to use one rxq per dataplane worker and interface.
14:37:30 <mackonstan> #info Vratko, multiple progressions observed.
14:38:40 <mackonstan> #info Vratko, 2 regressions on Xeons: 1) ipsec, 2) nat44ed minimum scale cps.
14:39:20 <mackonstan> #info Vratko, 1 regressions tsh: 1) vhost with vpp in vm.
14:39:40 <mackonstan> #info correction for above line: with testpmd in vm.
14:41:14 <mackonstan> #info Tibor, to complete report, the main work is static content for methodology sections per report TODO email sent last Tue
14:42:16 <mackonstan> #Dave, re data comparison tables for tsh and for tx2 retest of v2101 in current CSIT env, getting failures in build jobs, working on it.
14:42:30 <mackonstan> #info Dave, re data comparison tables for tsh and for tx2 retest of v2101 in current CSIT env, getting failures in build jobs, working on it.
14:43:55 <mackonstan> #topic Releases CSIT-2101
14:44:40 <mackonstan> #info Dave, VPP v21.01.1 maintenance release artefacts to be produced next week
14:45:02 <mackonstan> #topic Physical Infrastructure
14:51:04 <mackonstan> #info Maciek, lab extension 2021 is going ahead.
14:52:16 <mackonstan> #info Maciek, vendor equipment funding agreement - WIP, vendors' reps are gathering every 2 weeks to progress this, I'm coordinating.
14:52:28 <mackonstan> #topic Virtual Infrastructure, CI/CD
14:52:54 <mackonstan> #info Dave, no issues (apart from fdio-yul vpn outage for EU folks)
14:55:49 <mackonstan> #info Juraj, new issue in vpp_Device, not all interfaces are seen in vpp after vpp start, symptom similar to previous issue we had, but root cause likely different. Detail in jira "vfio is busy when initializing VPP with multiple VPP instances using different VFs"
14:55:56 <mackonstan> #link https://jira.fd.io/browse/VPP-1986
14:57:34 <mackonstan> #info Peter, would like to go about it differently compared to previous issue we had. Don't wait too long for resolution from Intel side, instead of switch to AVF.
14:57:48 <mackonstan> #info Fan, would like to see logs from Xeon platform.
14:58:04 <mackonstan> #info Juraj, also observed this issue with AVF.
14:59:24 <mackonstan> #endmeeting