15:03:27 <mackonstan> #startmeeting FD.io CSIT Project Meeting
15:03:27 <collab-meetbot> Meeting started Wed Jan 20 15:03:27 2021 UTC and is due to finish in 60 minutes.  The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:03:27 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:03:27 <collab-meetbot> The meeting name has been set to 'fd_io_csit_project_meeting'
15:03:35 <mackonstan> #chair
15:03:35 <collab-meetbot> Current chairs: mackonstan
15:03:39 <jgelety> #info Jan Gelety
15:03:43 <vrpolak> #info Vratko Polak.
15:03:48 <jlinkes> #info Juraj Linkeš
15:04:12 <tifrank> #info Tibor Frank
15:04:36 <mackonstan> #topic Agenda bashing
15:07:14 <mackonstan> #topic Inputs from projects
15:07:25 <mackonstan> #info VPP v21.01 rls on schedule
15:07:57 <mackonstan> #topic Releases CSIT-2101
15:08:58 <mackonstan> #info Tibor, >5 runs completed with rc2, added now also geneve tests.
15:09:11 <mackonstan> #info Tibor, latency graphs updated, see draft report in master
15:09:24 <mackonstan> #info Tibor, added 2n-tx2 perf data
15:10:50 <mackonstan> #info Tibor, did not observe any surprises, comparison current vs previous looking okay, no big changes seen.
15:11:13 <mackonstan> #info Tibor, issue with reconfig tests, all tests reporting zero loss, Vratko looking into it
15:13:35 <mackonstan> #info Maciek, methodology sections need to be added - ip4/ip6 randomized, latency.
15:14:02 <mackonstan> #info Vratko, nat44ed throughput tests still in review.
15:14:59 <mackonstan> #info Vratko, ip4-rnd and ip6-rnd need to be added to job_specs for iterative/report and for full coverage.
15:18:03 <mackonstan> #info Maciek, async crypto tests patch, is still in review (thx to Peter, Vratko for comments), Yulong to address, https://gerrit.fd.io/r/c/csit/+/30039
15:18:39 <mackonstan> #topic Physical Infrastructure
15:19:26 <mackonstan> #info Peter, issues with 1n-skx infra:
15:20:43 <mackonstan> #info 1. DPDK FVL driver issue, causing sporadic failures when multiple VF owners, reported into Intel IPS systems. Resolution unlikely to arrive soon.
15:23:58 <mackonstan> #info 2. Running out of memory, applied a fix, monitoring.
15:24:00 <mackonstan> #info 3. Jenkins killing the test introducing a racing condition related to VF allocation, and when it happens, it's putting the testbed out of service.
15:33:02 <mackonstan> #info re 1. - current expectation is that we will need to upgrade the driver/firmware stack as part of move to ubuntu 20.04.
15:33:19 <mackonstan> #info re 2. - high confidence this is now fixed, will keep monitoring.
15:38:00 <mackonstan> #info re 3. - working on the automated garbage collection after jenkins cut-out. Followed by root causing the racing condition.
15:40:49 <mackonstan> #info Peter, Dave, Juraj, collaborating on the fix, led by Peter. Keeping the vpp-csit-device-verify job voting for now.
15:44:23 <mackonstan> #topic FD.io DC move in Jan/Feb 2021 and lab extension in 2021.
15:48:14 <mackonstan> #info Maciek, Vexxhost okay with the plan as per patch: https://gerrit.fd.io/r/c/csit/+/30662
15:50:15 <mackonstan> #info Maciek, Vexxhost confirmed the new site YUL1 should be coming live this week, awaiting updates ...
15:51:34 <mackonstan> #info Dave, seeing more frequent errors, events of >20% pkt loss, between the sites.
15:52:17 <mackonstan> #info Dave, will provide a recent weekly report showing that, Maciek to forward to Vexxhost ask for opinion
15:52:54 <mackonstan> #Peter, observing latency spikes, will provide report to be forwarded to Vexxhost.
15:53:13 <mackonstan> #info Peter, observing latency spikes, will provide report to be forwarded to Vexxhost.
15:53:58 <mackonstan> #topic Virtual Infrastructure, CI/CD
15:54:13 <mackonstan> #info Dave, still working on upgrading executors.
15:55:45 <mackonstan> #info Dave, sandbox offline most of last week, should be back now.
15:57:32 <mackonstan> #endmeeting