14:11:54 <mackonstan> #startmeeting FD.io CSIT Project Meeting 14:11:54 <collab-meetbot> Meeting started Wed Jul 7 14:11:54 2021 UTC and is due to finish in 60 minutes. The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:11:54 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:11:54 <collab-meetbot> The meeting name has been set to 'fd_io_csit_project_meeting' 14:12:03 <vrpolak> #info Vratko Polak. 14:12:05 <mackonstan> #info 14:12:06 <jlinkes> #info Juraj Linkeš 14:12:07 <tifrank> #info Tibor Frank 14:12:15 <mackonstan> #info Maciek Konstantynowicz 14:12:18 <mackonstan> #chair 14:12:18 <collab-meetbot> Current chairs: mackonstan 14:12:25 <dwallacelf> #info Dave Wallace 14:12:26 <fan_zhang> #info Fan Zhang 14:13:49 <mackonstan> #topic Agenda bashing 14:13:54 <mackonstan> #topic Inputs from projects 14:14:01 <mackonstan> #topic Inputs from projects - VPP 14:14:12 <mackonstan> #topic Inputs from projects - VSAP 14:14:25 <mackonstan> #topic Inputs from projects - TSC 14:15:28 <mackonstan> #topic New performance tests - gtpu 14:16:16 <mackonstan> #info Vratko, tests are merged. 14:16:53 <mackonstan> #info Tibor, covered in report, will add to trending. Can be removed from the agenda. 14:18:03 <mackonstan> #topic New functional Tests - flow "offload" patch 14:18:55 <mackonstan> #info Juraj, comments addressed, one comment from Vratko outstanding. 14:20:49 <mackonstan> #info ignore two posts listed above 14:21:17 <mackonstan> #topic New functional Tests - IPsec SPD outbound flow cache patch 14:21:28 <mackonstan> #info Juraj, comments addressed, one comment from Vratko outstanding. 14:22:02 <mackonstan> #info Vratko, 32308, review almost complete, can be merged soon. 14:22:14 <mackonstan> #info Vratko, 32895, waiting for VPP. 14:22:33 <mackonstan> #topic New functional Tests - flow "offload" patch 14:23:52 <mackonstan> #info Fan, waiting to have the CVL fully onboarded in vpp_device testbed. 14:26:47 <mackonstan> #info Peter, missing QSFP28 cable for the 2nd testbed, awaiting new cable shipment (in-flight). Will be finishing the coding part later this week. 14:27:44 <mackonstan> #info Fan, the benchmarking is TBD. 14:28:16 <mackonstan> #topic Release - CSIT-2106 14:29:06 <mackonstan> #info Tibor, rls2106 testing going well, iterative tests, reconfig tests are almost done. 14:29:22 <mackonstan> #info Tibor, will be starting coverage tests 14:29:36 <mackonstan> #info Tibor, draft report ver in master 14:29:59 <mackonstan> Tibor, same test failures as in rc2 14:33:56 <mackonstan> #info Tibor, intermittent failures: 1) rdma-core on one testbed, switching driver problem; 2) create avf interface, moving thru NIC rxq configurations; 3) nat44ed 16M sessions not be established in allotted time for some tests; 4) deterministic for specific (frame_size, core config): ipsec sw asyn tests, no traffic forwarded; 5) VSAP tests with MLX 14:33:56 <mackonstan> NIC, no traffic forwarded; 14:34:28 <mackonstan> #info Vratko, Peter re 5) ARP resolution issue. 14:34:42 <vrpolak> #link https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-skx/476/ <-- Ipsec async scheduler results for 3n-skx 14:34:58 <vrpolak> #link https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-3n-dnv/109/ <-- for 3n-dnv 14:35:31 <vrpolak> #info The links are for master, but we expect 2106 to look the same. 14:36:16 <vrpolak> #info The performance change happened after we switched CSIT code globally to use one rxq per dataplane worker and interface. 14:37:30 <mackonstan> #info Vratko, multiple progressions observed. 14:38:40 <mackonstan> #info Vratko, 2 regressions on Xeons: 1) ipsec, 2) nat44ed minimum scale cps. 14:39:20 <mackonstan> #info Vratko, 1 regressions tsh: 1) vhost with vpp in vm. 14:39:40 <mackonstan> #info correction for above line: with testpmd in vm. 14:41:14 <mackonstan> #info Tibor, to complete report, the main work is static content for methodology sections per report TODO email sent last Tue 14:42:16 <mackonstan> #Dave, re data comparison tables for tsh and for tx2 retest of v2101 in current CSIT env, getting failures in build jobs, working on it. 14:42:30 <mackonstan> #info Dave, re data comparison tables for tsh and for tx2 retest of v2101 in current CSIT env, getting failures in build jobs, working on it. 14:43:55 <mackonstan> #topic Releases CSIT-2101 14:44:40 <mackonstan> #info Dave, VPP v21.01.1 maintenance release artefacts to be produced next week 14:45:02 <mackonstan> #topic Physical Infrastructure 14:51:04 <mackonstan> #info Maciek, lab extension 2021 is going ahead. 14:52:16 <mackonstan> #info Maciek, vendor equipment funding agreement - WIP, vendors' reps are gathering every 2 weeks to progress this, I'm coordinating. 14:52:28 <mackonstan> #topic Virtual Infrastructure, CI/CD 14:52:54 <mackonstan> #info Dave, no issues (apart from fdio-yul vpn outage for EU folks) 14:55:49 <mackonstan> #info Juraj, new issue in vpp_Device, not all interfaces are seen in vpp after vpp start, symptom similar to previous issue we had, but root cause likely different. Detail in jira "vfio is busy when initializing VPP with multiple VPP instances using different VFs" 14:55:56 <mackonstan> #link https://jira.fd.io/browse/VPP-1986 14:57:34 <mackonstan> #info Peter, would like to go about it differently compared to previous issue we had. Don't wait too long for resolution from Intel side, instead of switch to AVF. 14:57:48 <mackonstan> #info Fan, would like to see logs from Xeon platform. 14:58:04 <mackonstan> #info Juraj, also observed this issue with AVF. 14:59:24 <mackonstan> #endmeeting