14:58:46 <mackonstan> #startmeeting FD.io CSIT Project Meeting
14:58:46 <collab-meetbot> Meeting started Wed Dec  8 14:58:46 2021 UTC and is due to finish in 60 minutes.  The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:58:46 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:58:46 <collab-meetbot> The meeting name has been set to 'fd_io_csit_project_meeting'
14:59:51 <pmikus> #info Peter Mikus
15:00:03 <tifrank> #info Tibor Frank
15:03:20 <mackonstan> #topic Agenda bashing.
15:03:35 <mackonstan> #topic Inputs from projects - VPP
15:03:51 <vrpolak> #info Vratko Polak.
15:04:01 <mackonstan> #topic Inputs from projects - VSAP
15:04:07 <mackonstan> #topic Inputs from projects - TSC
15:05:11 <mackonstan> #info Maciek - got a ping from Trishan re submitting FD.io CSIT talk, and I did submit one, see later in the agenda
15:05:26 <mackonstan> #topic New or refactored tests
15:05:43 <dwallacelf> #info Dave Wallace
15:05:55 <mackonstan> #info E810 flow "offload" tests - perf: work-in-progress - TBC
15:06:44 <mackonstan> #info Scale tests - Any remaining perf test jobs with excessive execution time? - Tibor - most jobs are under the limit
15:08:07 <mackonstan> #info Tibor - 2n-clx-mlr and 2n-skx-mlr weekly jobs take 35hrs and ~29hrs, respectively. Not a big issue as we have 3 and 4 testbed instances respectively.
15:09:22 <mackonstan> #info Maciek - let's revise those two, and see what needs to be done to bring it down below the target of 24hrs.
15:09:49 <mackonstan> #info Maciek, Tibor - the good news is that iterative daily jobs are all <12hrs in duration
15:13:35 <mackonstan> #info E810 flow "offload" tests - perf: work-in-progress - Fan reporting - team has been discussing this, need to revise which of the functional tests make sense to port to performance tests
15:14:44 <mackonstan> #info Fan, Maciek will follow up on this, and see what perf tests make sense. It is unclear if CSIT-2202 milestones times can be met, we only have few working weeks left due to extended Xmas and year-end breaks and shutdown.
15:15:34 <mackonstan> #topic Releases 2202
15:16:59 <mackonstan> #link https://wiki.fd.io/view/CSIT/csit2202_plan
15:17:04 <mackonstan> #topic Releases 2110
15:17:32 <mackonstan> #info Tibor, got new data from Intel for icx, will add to report CSIT-2110 maintenance ver
15:18:13 <mackonstan> #info Maciek, intel also provided updated BIOS and bare-metal config info, will add to repo and update report accordingly
15:18:41 <mackonstan> #topic Physical Infrastructure
15:19:45 <mackonstan> #info Lab extension - Peter reporting - Initial 5 Intel ICX servers arrived and being installed and commissioned
15:20:13 <mackonstan> #info Peter, rack layout designed, upstreamed to gerrrit, sent to Vexxhost through ticket and slack
15:20:42 <mackonstan> #info Peter, received update that servers are assembled and (to be?) racked and cabled
15:22:59 <mackonstan> #info Peter, going ahead with 1 x 2n-icx, 1 x 3n-icx.
15:25:34 <mackonstan> #info Juraj, Maciek, Ampere Ultra servers are expected in January, but no hard commitmment
15:25:55 <mackonstan> #info Maciek, talking with Vexxhost this Friday, hoping for the remaining supermicro parts to arrive in January, so this fits.
15:27:07 <mackonstan> #info Peter, stretch-goal to get 1 x 2n-icx, 1 x 3n-icx up-and-running and in production before Xmas / year-end break, but time is quickly running out.
15:27:38 <mackonstan> #topic Virtual Infrastructure, CI/CD
15:27:50 <mackonstan> #chair dwallacelf
15:27:50 <collab-meetbot> Warning: Nick not in channel: dwallacelf
15:27:50 <collab-meetbot> Current chairs: dwallacelf mackonstan
15:31:05 <dwallacelf> #info New Docker images for CI which fixed VPP rdma plugin missing
15:32:26 <dwallacelf> #info Docs job clean up for all prjects were merged & LF ticket opened to manage data migration from nexus to amazon s3
15:32:30 <dwallacelf> #link https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-23352
15:33:03 <dwallacelf> #info vpp device job on arm enabled per patch, non-voting
15:36:38 <dwallacelf> #info Topic Environment, Framework
15:36:59 <dwallacelf> #topic Environment, Framework
15:38:30 <dwallacelf> #info Fan reported VPP upgrade of dpdk to 21.11 testing in progress, email to csit-dev & patch to vpp related to this upgrade in next day or two.
15:39:19 <dwallacelf> #info Vratko noted dpdk testpmd issues are showing up in csit testing with dpdk latest
15:40:03 <dwallacelf> #topic Trending
15:40:05 <vrpolak> #info testpmd error message: /tmp/openvpp-testing/dpdk/build/app/dpdk-testpmd: invalid option -- 'w'
15:40:51 <vrpolak> #link https://s3-logs.fd.io/vex-yul-rot-jenkins-1/csit-dpdk-perf-mrr-weekly-master-2n-skx/174/log.html.gz#s1-s1-s1-s1-t2-k1-k4-k4-k1-k4  <-- testpmd message visible here.
15:43:26 <dwallacelf> #info VPP-1995: Update reported to Ray K @ Intel
15:44:42 <dwallacelf> #info Fan reports Intel feature dev team now understands issue and will work with VPP & DPDK team to resolve the issue.
15:46:04 <dwallacelf> #info VPP-1986: Resolved previously, removed from agenda
15:48:54 <dwallacelf> CSIT Breakages for week 48:  DET44 failures: Vratko applied fix but failures still occur intermittently
15:50:11 <dwallacelf> DET44 failures: hard to reproduce, so verification of potential fixes is hard.  Vratko thinks it may be a race condition in VPP, but unable to confirm.
15:57:07 <dwallacelf> #info Vratko to loop in VPP developers, but only occurs on zen platform
15:58:25 <dwallacelf> #info error 2 has fix waiting for review
15:58:41 <fan_zhang> https://gerrit.fd.io/r/c/vpp/+/30386
15:59:00 <dwallacelf> #info error 3 has fix in progress
15:59:11 <dwallacelf> #link https://gerrit.fd.io/r/c/vpp/+/30386
15:59:13 <dwallacelf> #link https://gerrit.fd.io/r/c/vpp/+/30386
15:59:59 <dwallacelf> #info error 4: needs rca, no update
16:00:58 <dwallacelf> #info outstanding fixes should be confirmed next week and moved to fixed column
16:02:11 <dwallacelf> #endmeeting