========================================= #fdio-meeting: FD.io CSIT Project Meeting ========================================= Meeting started by mackonstan at 13:56:44 UTC. The full logs are available at http://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2021/fd_io_csit_project_meeting/fdio-meeting-fd_io_csit_project_meeting.2021-07-21-13.56.log.html . Meeting summary --------------- * Peter Mikus (pmikus, 13:58:25) * mackonstan (mackonstan, 14:01:15) * Vratko Polak. (vrpolak, 14:01:57) * Dave Wallace (dwallacelf, 14:02:06) * Agenda bashing (mackonstan, 14:02:53) * Inputs from projects - VPP (mackonstan, 14:03:43) * Inputs from projects - TSC (mackonstan, 14:04:07) * Juraj Linkeš (jlinkes, 14:04:09) * Dave, Peter Mikus nominated as ci-management committer, all committers voted in favour, TSC approval this week. (mackonstan, 14:05:42) * New tests (mackonstan, 14:06:09) * Fan Zhang (fan_zhang, 14:06:51) * Fan, re flow "offload" patch functional test, waiting for vpp_device testbed/framework update. (mackonstan, 14:08:08) * Fan, re flow "offload" patch performance test, testbeds/framework ready, will ask Yulong to go ahead and create a patch for that. (mackonstan, 14:09:20) * Maciek, re TRex support for DPDK CVL, will send a summary note re status, will need help to verify any remaining gaps. (mackonstan, 14:11:42) * Juraj, re IPsec SPD flow cache patches, 3 patches open, found one issues, working on a fix. Vratko wants to run perf tests to see any collateral impact of those patches. (mackonstan, 14:13:41) * Releases (mackonstan, 14:14:14) * CSIT-2110, will follow VPP schedule (mackonstan, 14:15:27) * LINK: https://wiki.fd.io/view/Projects/vpp/Release_Plans/Release_Plan_21.10 (mackonstan, 14:15:29) * CSIT-2106, Tibor, still testing (delayed by jenkins issues, talked about later), 2n-zn2 1 cvg job, af_xdp driver tests (late arrival to job_specs). (mackonstan, 14:17:30) * CSIT-2106, Tibor, methodology sections under review, ipsec async patch, uni-dir crypto, af_xdp content to be added (Peter, Maciek). (mackonstan, 14:19:54) * CSIT-2106, Peter, MLX issue fixed for good. (mackonstan, 14:21:51) * CSIT-2106, Vratko, re IPsec async broken test, still open; Fan, vpp patch may need to be re-worked. (mackonstan, 14:23:22) * CSIT-2106, high latency issue reported by TRex, Vratko, one very clear symptom: latency running high to 5msec, some NICs affected some are not, same for testbeds. (mackonstan, 14:25:01) * Vratko, issue appeared when testbeds got upgraded to Ubuntu 20.04 and TRex v2.88. Unclear what is the root cause. One similar symptom reported in public domain for TRex. (mackonstan, 14:25:51) * LINK: https://github.com/cisco-system-traffic-generator/trex-core/issues/508#issuecomment-662363667 <-- Known TRex issue, comment about impact on latency measurements, but for different firmware and TRex versions. (vrpolak, 14:30:50) * Peter, also tried TRex v2.90, also observing high latency. (mackonstan, 14:31:43) * Peter, another suspicion is PCIe bus management, checking ... (mackonstan, 14:34:20) * CSIT-2101, vpp artefacts ready, will test once CSIT-2106 is complete, after latency issue is fixed, and one iterative job per testbed is run with latency fixed. Then CSIT-2101.1 maintenance rls tests will get scheduled. (mackonstan, 14:37:35) * Physical Infrastructure (mackonstan, 14:37:53) * Peter, going ahead with 3n-hsw decommissioning, tickets open. Waiting for Vexxhost engineer on-site engineer allocation. (mackonstan, 14:38:36) * Virtual Infrastructure, CI/CD (mackonstan, 14:38:51) * Dave, Maciek, escalated Jenkins outage issue (mackonstan, 14:39:44) * Dave, trigger: last Wednesday Jenkins VM and Jenkins ingress got moved to different location(s), some other changes on Saturday; symptoms: as of Saturday, very high latency between jenkins ingress and nomad; resolution: migrated jenkins ingress somewhere else; Peter cleaned vpp_device jobs, Dave cleared the vpp job queue. Now back to normal (mackonstan, 14:44:14) * Dave, jenkins maintenance not needed (mackonstan, 14:46:22) * Dave, Vanessa applied Jenkins JCasC changes during the outage. (mackonstan, 14:47:11) * Peter, Dave, Maciek (mackonstan, 14:47:20) * Peter, Dave, Maciek, changes of this nature MUST BE communicated over the slack channel. (mackonstan, 14:47:53) * Dave, Maciek: will request post mortem readout from Vexxhost/LFIT team. (mackonstan, 14:51:04) * Trending (mackonstan, 14:51:54) * Vratko, CSIT l2, ip4, ip6 scale tests are failing due to VPP removing support for VAT APIs used. (mackonstan, 14:52:24) * Vratko, solution is still WIP. (mackonstan, 14:53:58) Meeting ended at 14:54:59 UTC. People present (lines said) --------------------------- * mackonstan (40) * collab-meetbot` (4) * vrpolak (2) * pmikus (1) * dwallacelf (1) * jlinkes (1) * fan_zhang (1) Generated by `MeetBot`_ 0.1.4