12:00:54 #startmeeting CIP IRC weekly meeting 12:00:54 Meeting started Thu Nov 12 12:00:54 2026 UTC and is due to finish in 60 minutes. The chair is jki. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:00:54 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 12:00:54 The meeting name has been set to 'cip_irc_weekly_meeting' 12:01:07 #topic AI review 12:01:22 - draft of improved patch review process [all maintainers] 12:01:39 hello 12:01:46 pavel: I think you wanted to start with that 12:02:07 if he is already around.. 12:02:27 otherwise postpone that to AOB 12:02:36 nothing else recorded 12:02:41 5 12:02:43 4 12:02:44 3 12:02:46 2 12:02:47 1 12:02:49 #topic Kernel maintenance updates 12:03:06 This week reported 21 new CVEs and 50 updated CVEs. 12:03:09 4.4 request for reviews is out, 63 patches 12:03:20 I reviewed 6.12.110 and 111. 12:03:24 pave1 has so far reviewed ~50/90 of 4.19 12:05:02 anything to add? 12:05:05 Sorry for being late :-). 12:05:21 Yep, I was reviewing 4.19. 12:05:54 At same point, we should clarify if / how stable kernel rules apply to -cip. 12:06:53 we have something in the wiki, intentionally vague - IIRC according to your wishes 12:07:57 but maybe keep that topic in mind for AOB 12:08:03 Yep. 12:08:04 anything else on maintenance? 12:08:18 5 12:08:20 4 12:08:22 3 12:08:23 2 12:08:25 1 12:08:28 #topic Kernel release status 12:09:13 so, 4.4 and 4.19 are still waiting for the second reviews, right? 12:09:20 correct 12:09:31 what can be done to accelerate that? 12:10:33 am also reviewing those patches, but since there are so many, it is taking some time... 12:11:06 That's the aob topic... should we be doing complex backports? 12:11:25 Stable rules say: - It must be obviously correct and tested. 12:11:35 and - It cannot be bigger than 100 lines, with context. 12:11:39 is 4.19 late because of complex feature backports? 12:11:48 would be surprising 12:11:55 or are the bug fixes too complex? 12:12:03 Not feature backports, but bug fixes are getting complex. 12:12:05 we don't backport any features. but some fixes get more complex the older the kernel gets 12:12:41 but that is nothing new: fix or document as "known bug", no? 12:12:57 was there a discussion on that choice for the current queue already? 12:13:33 Well, rules are more than that. It also says "obviously correct", which, frankly, some really are not. 12:13:54 They also say "tested", but we are backporting even patches we are not testing, because it seems like lesser evil. 12:14:05 the problem is that some of the tricky fixes have rather high CVSS scores. if not, i would drop them anyway. 12:14:08 And it gives limit of 100 lines we are not enforcing. 12:14:54 so, problem is now missing bandwidth to perform and validate security fix backports 12:15:04 Yes. 12:15:15 that would mean, we should ramp our our staffing for this purpose (as well) 12:15:35 i guess so. i don't see an obvious easy solution. 12:15:37 and look for some more kernel engineer(s) to support? 12:16:03 Or rather, bandwidth to fix backports, some of which are likely security fixes, some of which are not. 12:16:42 in any case, senior kernel engineer profile needed - or good LLMs... 12:17:06 we still have codex tokens... 12:17:20 There's tradeoff between "all fixes to mainline are backported to xx-cip" and "no regressions in xx-cip". 12:17:31 yes, but that needs discussion 12:17:37 not simply "rules" 12:17:39 `kUsing multiple LLMs for analysis might be a good idea. 12:17:58 And we don't yet see regressions in xx-cip, so maybe I worry too much and it would be enough to document that 12:18:02 a) we don't test all fixes 12:18:14 b) we do take non-obvious fixes 12:18:22 c) we take fixes longer than 100 lines. 12:18:41 "100 lines" is a pointless rule 12:18:56 Well, it is there in stable-kernel-rules.rst :-). 12:19:13 Plus you can hardly call 200 lines patch "obviously correct". 12:19:23 sure, but that is not the point 12:19:36 you can split, "just" takes more effort 12:19:52 and if staffolding is needed for old kernels, that write it 12:20:18 all doable, but that takes time and resources to do and validate 12:20:38 but "simply" droping a CVE fix because of "too complex by rule" is not an option 12:21:00 it needs to be discussed at least, reasoned and documented 12:21:08 Well, due to way CVEs work for kernel, everything is "CVE fix" now. 12:21:20 but not everyting is critical 12:21:38 Yes. 12:21:49 can we make this a bit more concrete? which backports are under discussion now? 12:23:06 Concrete example is e37dae2a4576c5 12:23:26 4.19-st tree? 12:23:38 -rc 12:23:39 Yes. 12:24:02 already tried different LLMs as well? 12:24:22 for review or backport? 12:24:23 That's a locking fix, and it would be good to have. 12:24:28 happy to ask claude or codex later on for an opinion 12:25:25 I recently had very good results with Opus and Fable on Xenomai trees, so rather "special" and "complex" as well 12:25:52 Well... it is probable LLM did quite a good job. 12:26:08 the problem is not this specific patch, it's that we have to look at all of them, unless we simply say that LLM review is enough... 12:26:33 ...but now we have patch touching locking in ax25 protocol, and patch looks plausible (as LLM patches do), but we certainly did not test it. 12:26:44 yes, understood - engineering capacity again 12:26:46 ...plus its over 100 lines at so. 12:27:24 To reiterate, LLM is probably doing good job there. 12:27:43 folks, then prioritize: complex, critical backport go to a backlog, get documented as unfixed for now, but we move forward with the rest 12:27:43 If I had to cram that patch into 4.19 at all costs, I may have done the same job. 12:27:49 and we process the complex queue later 12:28:06 holding back releases because of some complex corners is also not good 12:29:11 my preference would be to keep the complex unless there is a concrete reason not to 12:29:41 should we introduce voting? :) 12:29:48 I believe "put this into backlog" is not good option. 12:30:15 Voting is not really solution here :-). 12:30:20 then, what to do now, for the current queue and for the future? 12:30:34 I guess we should introduce transparency: we are backporting complex patches we can not test. 12:31:01 yes, i think that's better than holding potentially important fixes back 12:31:14 We are trying to keep regressions to the minimum, but due to the need to backport some complex patches, we may have some. 12:31:37 We are not following "obviously correct", "tested" and "less than 100 lines" rules any more. 12:31:59 is upstream still doing that...? 12:32:32 Lets say that upstream was always flexible with that :-). 12:32:44 see 12:32:52 [It was probably enforced for patches submitted by people, but not for patches backported by stable team]. 12:33:46 ok, what's next then for the current queue? 12:34:15 pave1: how long do you think you'll still need for 4.19? 12:34:47 I believe the current queue should be just released. I went through most patches that looked relevant. 12:35:07 ok 12:36:13 what about 4.4? 12:36:29 I think I’ll be able to finish reviewing everything by tomorrow as well. 12:36:52 thanks, i'll wait until tomorrow then. 12:37:31 good, that sounds like some plan - thanks all already! 12:37:34 4.4: Lets say we do the release at tuesday? 12:37:48 ok for me 12:38:02 ok for me too. 12:38:25 4.4, 4.19: If patch for minor resource leak requires llm work, I'd say don't dot it. 12:38:40 that's already implemented for 4.4 12:39:03 that's why there are "only" 63 patches :) 12:39:08 There's ton of those, there will be more and noone cares. 12:39:10 Aha, ok. 12:39:45 Another possible rules might be "if bug requires GFP_KERNEL allocation failing we don't care" because we are already OOM and system is already unusuable. 12:40:18 are there a lot of those? 12:40:59 And "error paths in probing don't really require fixes" ... because if your driver fails to probe, your hardware is unusable, anyway. 12:41:10 ...and you have already caught that during development. 12:41:45 I have seen at least one of those in 4.19-st review. 12:42:07 I was saying to myself "hmm, probably does not break anything, but hmm, I would not backport this". 12:42:30 ok, i'll add those and see how it goes 12:43:49 Thanks! 12:44:03 IGNORE now includes: - it fixes an issue caused by a GFP_KERNEL memory allocation failure 12:44:03 - it fixes an issue in the error path during device probing 12:44:23 actually, i'll have to add "exclusively" to that 12:45:39 ok, are we done with release topics? 12:45:41 Do we prevent LLM from working on stuff we don't enable in our configs? 12:46:00 i have a script sorting those out before the llm sees it 12:46:32 Ok, good. That is one more reason to prune our configs. 12:47:00 of course, as well as to maintain them according to real demands 12:47:15 so, move on? 12:47:40 yes. 12:47:44 5 12:47:45 4 12:47:47 3 12:47:48 2 12:47:50 1 12:47:52 #topic Kernel testing 12:48:42 anyone with testing topics today? 12:48:44 If someone could take a look at https://gitlab.com/cip-project/cip-kernel/linux-cip/-/jobs/17875327047 12:48:52 Several labs are down.cybertrust and siemens-prag. 12:48:59 It is new kind of failure and does not go away with retry. 12:50:37 we can not use all x86-siemens-mcom device now. I think that is the cause. 12:51:05 the M-Com in Munich does not run kernel tests, only image tests 12:51:09 no update here, other than some people are starting to use kci-dev with the mcp 12:51:22 Aha, so that's known. It says "There are currently 2 "x86-siemens-mcom" devices online.", which confused me. 12:51:25 https://kernelci.org/blog/2026/09/12/talk-to-kernelci-kci-dev-now-speaks-mcp/ 12:51:26 the one in Prague should do the other way around, but... 12:51:49 clarifying Prague lab state right now 12:52:36 uh, we even have 3 mcoms in Prague now, just no online lab 12:53:39 I suppose we had no chance (time) yet to play with kci-mcp in our context :( 12:55:16 more on testing? 12:55:29 5 12:55:31 4 12:55:33 3 12:55:35 2 12:55:36 1 12:55:39 #topic AOB 12:56:04 SECURITY.md: can send a v3 with the minor adjustments unless there are more comments 12:56:30 Looks good. 12:56:45 further topics? 12:56:48 I wanted to ask : if there's SPARC problem in 4.4, we don't care, right? 12:57:03 Does CRA allow us to do that? 12:57:04 right, per scope we define in "Security Context" 12:57:33 sparc is out. 12:57:46 Ok, being more explicit about that would not be bad. 12:57:58 I haven't read anything different about the CRA 12:58:20 pavel: we cannot write down the details of the configs in the SECURITY.md, they are not defined there 12:58:38 that's why we pointed to that repo - please read again 13:00:04 Something like "if the issue is in unsupported parts of 4.4 or 4.19 kernel, please send report to dev null, we don't care". 13:00:35 Ok, maybe not that explicit. 13:00:50 you really need to read the whole documents 13:01:03 Ok. 13:01:04 LLM normally does today ;) 13:01:29 ok more topics for today? 13:02:11 5 13:02:14 4 13:02:16 3 13:02:18 2 13:02:21 1 13:02:23 #endmeeting