13:59:06 <mackonstan> #startmeeting FD.io CSIT project meeting
13:59:06 <collabot`> Meeting started Wed Jul  3 13:59:06 2019 UTC.  The chair is mackonstan. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:59:06 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:59:06 <collabot`> The meeting name has been set to 'fd_io_csit_project_meeting'
13:59:09 <mackonstan> #chair
13:59:09 <collabot`> Current chairs: mackonstan
13:59:54 <jlinkes> #info Juraj Linkeš
14:00:43 <mackonstan> #topic Agenda bashing
14:00:51 <vrpolak> #info Vratko Polak.
14:00:58 <jgelety> #info Jan Gelety
14:02:37 <mackonstan> #topic FD.io CSIT physical labs
14:02:50 <pmikus> #info Peter Mikus
14:02:57 <dwallacelf> #info Dave Wallace
14:04:03 <mackonstan> #info Maciek, Peter: Denverton testbeds are finally working, test jobs are running, Ray and Luyong confirmed.
14:08:41 <mackonstan> #topic FD.io lab extension
14:08:55 <mackonstan> #info Maciek: provided update on the spec of lab extension
14:09:14 <mackonstan> #info Following new servers are expected to arrive in Jul/Aug:
14:09:23 <mackonstan> #info Servers: 7x SuperMicro 7049GP-TRT (4U), 3x SuperMicro 1029P-WTRT (1U)
14:09:32 <mackonstan> #info Processors: 14x Cascade Lake Platinum 8280, 6x Cascade Lake Gold 6252N (with priority based frequency, will arrive later)
14:09:42 <mackonstan> #info NICs: 7x XXV710-DA2, 7x X710DA4
14:10:09 <mackonstan> #info Proposed setup: i) Performance testbeds: 2x 2n-clx plus 1x 3n-clx, or 3x 2n-clx plus 1x 1n-clx, ii) Backends: 3x SuperMicro 1029P-WTRT
14:16:36 <mackonstan> #info Propose to evaluate the pros/cons on the csit-dev list
14:16:57 <mackonstan> #topic Inputs from VPP
14:17:12 <mackonstan> #info Dave: v19.08 continues, no calls for maintenance releases
14:17:29 <mackonstan> #topic Inputs from HC
14:18:19 <mackonstan> #info it's quiet, no changes for CSIT
14:19:03 <mackonstan> #info Peter: 1908 plan for HC is unclear
14:19:18 <mackonstan> #topic Inputs from TSC
14:21:38 <mackonstan> #info Vratko: week before last there was discussion about ROI/efficiency of FD.io minisummits co-located with major events. Calls for proposals for better, more efficient ways for FD.io community get togethers
14:22:09 <mackonstan> #topic Operational status
14:23:27 <mackonstan> #info Maciek: Jenkins maintenance last night UTC, impacted perf-mrr-daily jobs
14:23:40 <mackonstan> #info Peter: jobs seem to be running
14:24:48 <mackonstan> #info Vratko: looks like there is still some memory leak in the system, compile jobs are failing, merge jobs issues, other symptoms, so expecting more maintenance
14:26:28 <mackonstan> #info Vratko, Ed: admins are adding more logs to track down the memory leak issue
14:31:53 <mackonstan> #info above issues were happening before the maintenance
14:32:03 <mackonstan> #info so far no issues observed after the maintenance
14:33:40 <mackonstan> #topic Centos CI
14:36:30 <mackonstan> #info Peter: need a separate job for Centos7, need to enable it in Nomad env on 1n-skx. Should not require any changes to Nomad env, job should just work, will report if issues.
14:40:02 <mackonstan> #info Jan: two issues in trending: i) ip6 fib scale tests failing, bug in VPP VAT fixed by Neale; ii) ip route api parameter name change was missed in CSIT code, now fixed, some side fix required in VPP code too; iii) vxlan scale test are failing due to PAPI not good with handling scale, moving back to VAT as for all other scale tests
14:40:14 <mackonstan> #topic VPP code performance
14:41:50 <mackonstan> #info patchy coverage due fib api issue per above, and jenkins issues
14:42:49 <mackonstan> #topic Framework: VAT to PAPI migration
14:43:08 <mackonstan> #info Vratko: see email with implementation steps
14:44:38 <mackonstan> #info Vratko: already know that CSIT scale tests do not work with PAPI, ip4 fib 200k takes >30min  with synch PAPI and 10min with remote socket, show stopper for usage of PAPI
14:47:38 <mackonstan> #info Vratko: async mode may help to bring the time down, but expecting issues so unlikely to get it working in 1908
14:49:29 <mackonstan> #info Vratko: another issue is specific to Centos - remote socket PAPI uses ssh tunnel, and this seems to be not supported in Centos7
14:50:36 <jgelety> #info VAT-to-PAPI migration on track to be finished this release cycle
14:51:47 <mackonstan> #topic Framework: VPP-API "Flag Day" gate-keeping
14:52:06 <mackonstan> #info Vratko: will be able to make progress on it once remote socket PAPI is in production
14:52:25 <mackonstan> #info Dave: still need to update the proposed process doc
14:52:50 <mackonstan> #topic Test environments - VPP_Device environment
14:53:51 <mackonstan> #info Juraj: Arm onboarding, having issues with understanding how the pipeline works, as the process is unclear, lacking documentation
14:57:00 <mackonstan> #info Juraj: posted a pull request to add Arm to VPP_device environment
14:58:47 <mackonstan> #info Peter: need to complete review of Juraj's pull request, run tests, and if okay merge
14:59:07 <mackonstan> #info Peter: ongoing issue is lack of documentation
14:59:48 <mackonstan> #info let's drive the conclusion over email, if not converging then separate TWS call
15:01:34 <mackonstan> #info Vratko: discussion on slack with Denverton team want to remove tests that are running daily on Denverton
15:01:52 <mackonstan> #info Vratko: awaiting patch from Denverton team side
15:02:51 <mackonstan> #info Vratko: e.g. some scale tests can not run on some platforms
15:03:41 <mackonstan> #endmeeting