14:58:46 <mackonstan> #startmeeting FD.io CSIT Project Meeting 14:58:46 <collab-meetbot> Meeting started Wed Dec 8 14:58:46 2021 UTC and is due to finish in 60 minutes. The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:58:46 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:58:46 <collab-meetbot> The meeting name has been set to 'fd_io_csit_project_meeting' 14:59:51 <pmikus> #info Peter Mikus 15:00:03 <tifrank> #info Tibor Frank 15:03:20 <mackonstan> #topic Agenda bashing. 15:03:35 <mackonstan> #topic Inputs from projects - VPP 15:03:51 <vrpolak> #info Vratko Polak. 15:04:01 <mackonstan> #topic Inputs from projects - VSAP 15:04:07 <mackonstan> #topic Inputs from projects - TSC 15:05:11 <mackonstan> #info Maciek - got a ping from Trishan re submitting FD.io CSIT talk, and I did submit one, see later in the agenda 15:05:26 <mackonstan> #topic New or refactored tests 15:05:43 <dwallacelf> #info Dave Wallace 15:05:55 <mackonstan> #info E810 flow "offload" tests - perf: work-in-progress - TBC 15:06:44 <mackonstan> #info Scale tests - Any remaining perf test jobs with excessive execution time? - Tibor - most jobs are under the limit 15:08:07 <mackonstan> #info Tibor - 2n-clx-mlr and 2n-skx-mlr weekly jobs take 35hrs and ~29hrs, respectively. Not a big issue as we have 3 and 4 testbed instances respectively. 15:09:22 <mackonstan> #info Maciek - let's revise those two, and see what needs to be done to bring it down below the target of 24hrs. 15:09:49 <mackonstan> #info Maciek, Tibor - the good news is that iterative daily jobs are all <12hrs in duration 15:13:35 <mackonstan> #info E810 flow "offload" tests - perf: work-in-progress - Fan reporting - team has been discussing this, need to revise which of the functional tests make sense to port to performance tests 15:14:44 <mackonstan> #info Fan, Maciek will follow up on this, and see what perf tests make sense. It is unclear if CSIT-2202 milestones times can be met, we only have few working weeks left due to extended Xmas and year-end breaks and shutdown. 15:15:34 <mackonstan> #topic Releases 2202 15:16:59 <mackonstan> #link https://wiki.fd.io/view/CSIT/csit2202_plan 15:17:04 <mackonstan> #topic Releases 2110 15:17:32 <mackonstan> #info Tibor, got new data from Intel for icx, will add to report CSIT-2110 maintenance ver 15:18:13 <mackonstan> #info Maciek, intel also provided updated BIOS and bare-metal config info, will add to repo and update report accordingly 15:18:41 <mackonstan> #topic Physical Infrastructure 15:19:45 <mackonstan> #info Lab extension - Peter reporting - Initial 5 Intel ICX servers arrived and being installed and commissioned 15:20:13 <mackonstan> #info Peter, rack layout designed, upstreamed to gerrrit, sent to Vexxhost through ticket and slack 15:20:42 <mackonstan> #info Peter, received update that servers are assembled and (to be?) racked and cabled 15:22:59 <mackonstan> #info Peter, going ahead with 1 x 2n-icx, 1 x 3n-icx. 15:25:34 <mackonstan> #info Juraj, Maciek, Ampere Ultra servers are expected in January, but no hard commitmment 15:25:55 <mackonstan> #info Maciek, talking with Vexxhost this Friday, hoping for the remaining supermicro parts to arrive in January, so this fits. 15:27:07 <mackonstan> #info Peter, stretch-goal to get 1 x 2n-icx, 1 x 3n-icx up-and-running and in production before Xmas / year-end break, but time is quickly running out. 15:27:38 <mackonstan> #topic Virtual Infrastructure, CI/CD 15:27:50 <mackonstan> #chair dwallacelf 15:27:50 <collab-meetbot> Warning: Nick not in channel: dwallacelf 15:27:50 <collab-meetbot> Current chairs: dwallacelf mackonstan 15:31:05 <dwallacelf> #info New Docker images for CI which fixed VPP rdma plugin missing 15:32:26 <dwallacelf> #info Docs job clean up for all prjects were merged & LF ticket opened to manage data migration from nexus to amazon s3 15:32:30 <dwallacelf> #link https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-23352 15:33:03 <dwallacelf> #info vpp device job on arm enabled per patch, non-voting 15:36:38 <dwallacelf> #info Topic Environment, Framework 15:36:59 <dwallacelf> #topic Environment, Framework 15:38:30 <dwallacelf> #info Fan reported VPP upgrade of dpdk to 21.11 testing in progress, email to csit-dev & patch to vpp related to this upgrade in next day or two. 15:39:19 <dwallacelf> #info Vratko noted dpdk testpmd issues are showing up in csit testing with dpdk latest 15:40:03 <dwallacelf> #topic Trending 15:40:05 <vrpolak> #info testpmd error message: /tmp/openvpp-testing/dpdk/build/app/dpdk-testpmd: invalid option -- 'w' 15:40:51 <vrpolak> #link https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-dpdk-perf-mrr-weekly-master-2n-skx/174/log.html.gz#s1-s1-s1-s1-t2-k1-k4-k4-k1-k4 <-- testpmd message visible here. 15:43:26 <dwallacelf> #info VPP-1995: Update reported to Ray K @ Intel 15:44:42 <dwallacelf> #info Fan reports Intel feature dev team now understands issue and will work with VPP & DPDK team to resolve the issue. 15:46:04 <dwallacelf> #info VPP-1986: Resolved previously, removed from agenda 15:48:54 <dwallacelf> CSIT Breakages for week 48: DET44 failures: Vratko applied fix but failures still occur intermittently 15:50:11 <dwallacelf> DET44 failures: hard to reproduce, so verification of potential fixes is hard. Vratko thinks it may be a race condition in VPP, but unable to confirm. 15:57:07 <dwallacelf> #info Vratko to loop in VPP developers, but only occurs on zen platform 15:58:25 <dwallacelf> #info error 2 has fix waiting for review 15:58:41 <fan_zhang> https://gerrit.fd.io/r/c/vpp/+/30386 15:59:00 <dwallacelf> #info error 3 has fix in progress 15:59:11 <dwallacelf> #link https://gerrit.fd.io/r/c/vpp/+/30386 15:59:13 <dwallacelf> #link https://gerrit.fd.io/r/c/vpp/+/30386 15:59:59 <dwallacelf> #info error 4: needs rca, no update 16:00:58 <dwallacelf> #info outstanding fixes should be confirmed next week and moved to fixed column 16:02:11 <dwallacelf> #endmeeting