12:00:54 <jki> #startmeeting CIP IRC weekly meeting
12:00:54 <collab-meetbot> Meeting started Thu Nov 12 12:00:54 2026 UTC and is due to finish in 60 minutes.  The chair is jki. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00:54 <collab-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
12:00:54 <collab-meetbot> The meeting name has been set to 'cip_irc_weekly_meeting'
12:01:07 <jki> #topic AI review
12:01:22 <jki> - draft of improved patch review process [all maintainers]
12:01:39 <iwamatsu> hello
12:01:46 <jki> pavel: I think you wanted to start with that
12:02:07 <jki> if he is already around..
12:02:27 <jki> otherwise postpone that to AOB
12:02:36 <jki> nothing else recorded
12:02:41 <jki> 5
12:02:43 <jki> 4
12:02:44 <jki> 3
12:02:46 <jki> 2
12:02:47 <jki> 1
12:02:49 <jki> #topic Kernel maintenance updates
12:03:06 <masami> This week reported 21 new CVEs and 50 updated CVEs.
12:03:09 <uli> 4.4 request for reviews is out, 63 patches
12:03:20 <iwamatsu> I reviewed 6.12.110 and 111.
12:03:24 <uli> pave1 has so far reviewed ~50/90 of 4.19
12:05:02 <jki> anything to add?
12:05:05 <pave1> Sorry for being late :-).
12:05:21 <pave1> Yep, I was reviewing 4.19.
12:05:54 <pave1> At same point, we should clarify if / how stable kernel rules apply to -cip.
12:06:53 <jki> we have something in the wiki, intentionally vague - IIRC according to your wishes
12:07:57 <jki> but maybe keep that topic in mind for AOB
12:08:03 <pave1> Yep.
12:08:04 <jki> anything else on maintenance?
12:08:18 <jki> 5
12:08:20 <jki> 4
12:08:22 <jki> 3
12:08:23 <jki> 2
12:08:25 <jki> 1
12:08:28 <jki> #topic Kernel release status
12:09:13 <jki> so, 4.4 and 4.19 are still waiting for the second reviews, right?
12:09:20 <uli> correct
12:09:31 <jki> what can be done to accelerate that?
12:10:33 <iwamatsu> am also reviewing those patches, but since there are so many, it is taking some time...
12:11:06 <pave1> That's the aob topic... should we be doing complex backports?
12:11:25 <pave1> Stable rules say: - It must be obviously correct and tested.
12:11:35 <pave1> and -  It cannot be bigger than 100 lines, with context.
12:11:39 <jki> is 4.19 late because of complex feature backports?
12:11:48 <jki> would be surprising
12:11:55 <jki> or are the bug fixes too complex?
12:12:03 <pave1> Not feature backports, but bug fixes are getting complex.
12:12:05 <uli> we don't backport any features. but some fixes get more complex the older the kernel gets
12:12:41 <jki> but that is nothing new: fix or document as "known bug", no?
12:12:57 <jki> was there a discussion on that choice for the current queue already?
12:13:33 <pave1> Well, rules are more than that. It also says "obviously correct", which, frankly, some really are not.
12:13:54 <pave1> They also say "tested", but we are backporting even patches we are not testing, because it seems like lesser evil.
12:14:05 <uli> the problem is that some of the tricky fixes have rather high CVSS scores. if not, i would drop them anyway.
12:14:08 <pave1> And it gives limit of 100 lines we are not enforcing.
12:14:54 <jki> so, problem is now missing bandwidth to perform and validate security fix backports
12:15:04 <pave1> Yes.
12:15:15 <jki> that would mean, we should ramp our our staffing for this purpose (as well)
12:15:35 <uli> i guess so. i don't see an obvious easy solution.
12:15:37 <jki> and look for some more kernel engineer(s) to support?
12:16:03 <pave1> Or rather, bandwidth to fix backports, some of which are likely security fixes, some of which are not.
12:16:42 <jki> in any case, senior kernel engineer profile needed - or good LLMs...
12:17:06 <jki> we still have codex tokens...
12:17:20 <pave1> There's tradeoff between "all fixes to mainline are backported to xx-cip" and "no regressions in xx-cip".
12:17:31 <jki> yes, but that needs discussion
12:17:37 <jki> not simply "rules"
12:17:39 <iwamatsu> `kUsing multiple LLMs for analysis might be a good idea.
12:17:58 <pave1> And we don't yet see regressions in xx-cip, so maybe I worry too much and it would be enough to document that
12:18:02 <pave1> a) we don't test all fixes
12:18:14 <pave1> b) we do take non-obvious fixes
12:18:22 <pave1> c) we take fixes longer than 100 lines.
12:18:41 <jki> "100 lines" is a pointless rule
12:18:56 <pave1> Well, it is there in stable-kernel-rules.rst :-).
12:19:13 <pave1> Plus you can hardly call 200 lines patch "obviously correct".
12:19:23 <jki> sure, but that is not the point
12:19:36 <jki> you can split, "just" takes more effort
12:19:52 <jki> and if staffolding is needed for old kernels, that write it
12:20:18 <jki> all doable, but that takes time and resources to do and validate
12:20:38 <jki> but "simply" droping a CVE fix because of "too complex by rule" is not an option
12:21:00 <jki> it needs to be discussed at least, reasoned and documented
12:21:08 <pave1> Well, due to way CVEs work for kernel, everything is "CVE fix" now.
12:21:20 <jki> but not everyting is critical
12:21:38 <pave1> Yes.
12:21:49 <jki> can we make this a bit more concrete? which backports are under discussion now?
12:23:06 <pave1> Concrete example is e37dae2a4576c5
12:23:26 <jki> 4.19-st tree?
12:23:38 <uli> -rc
12:23:39 <pave1> Yes.
12:24:02 <jki> already tried different LLMs as well?
12:24:22 <uli> for review or backport?
12:24:23 <pave1> That's a locking fix, and it would be good to have.
12:24:28 <jki> happy to ask claude or codex later on for an opinion
12:25:25 <jki> I recently had very good results with Opus and Fable on Xenomai trees, so rather "special" and "complex" as well
12:25:52 <pave1> Well... it is probable LLM did quite a good job.
12:26:08 <uli> the problem is not this specific patch, it's that we have to look at all of them, unless we simply say that LLM review is enough...
12:26:33 <pave1> ...but now we have patch touching locking in ax25 protocol, and patch looks plausible (as LLM patches do), but we certainly did not test it.
12:26:44 <jki> yes, understood - engineering capacity again
12:26:46 <pave1> ...plus its over 100 lines at so.
12:27:24 <pave1> To reiterate, LLM is probably doing good job there.
12:27:43 <jki> folks, then prioritize: complex, critical backport go to a backlog, get documented as unfixed for now, but we move forward with the rest
12:27:43 <pave1> If I had to cram that patch into 4.19 at all costs, I may have done the same job.
12:27:49 <jki> and we process the complex queue later
12:28:06 <jki> holding back releases because of some complex corners is also not good
12:29:11 <uli> my preference would be to keep the complex unless there is a concrete reason not to
12:29:41 <jki> should we introduce voting? :)
12:29:48 <pave1> I believe "put this into backlog" is not good option.
12:30:15 <pave1> Voting is not really solution here :-).
12:30:20 <jki> then, what to do now, for the current queue and for the future?
12:30:34 <pave1> I guess we should introduce transparency: we are backporting complex patches we can not test.
12:31:01 <uli> yes, i think that's better than holding potentially important fixes back
12:31:14 <pave1> We are trying to keep regressions to the minimum, but due to the need to backport some complex patches, we may have some.
12:31:37 <pave1> We are not following "obviously correct", "tested" and "less than 100 lines" rules any more.
12:31:59 <jki> is upstream still doing that...?
12:32:32 <pave1> Lets say that upstream was always flexible with that :-).
12:32:44 <jki> see
12:32:52 <pave1> [It was probably enforced for patches submitted by people, but not for patches backported by stable team].
12:33:46 <jki> ok, what's next then for the current queue?
12:34:15 <uli> pave1: how long do you think you'll still need for 4.19?
12:34:47 <pave1> I believe the current queue should be just released. I went through most patches that looked relevant.
12:35:07 <uli> ok
12:36:13 <uli> what about 4.4?
12:36:29 <iwamatsu> I think I’ll be able to finish reviewing everything by tomorrow as well.
12:36:52 <uli> thanks, i'll wait until tomorrow then.
12:37:31 <jki> good, that sounds like some plan - thanks all already!
12:37:34 <pave1> 4.4: Lets say we do the release at tuesday?
12:37:48 <uli> ok for me
12:38:02 <iwamatsu> ok for me too.
12:38:25 <pave1> 4.4, 4.19: If patch for minor resource leak requires llm work, I'd say don't dot it.
12:38:40 <uli> that's already implemented for 4.4
12:39:03 <uli> that's why there are "only" 63 patches :)
12:39:08 <pave1> There's ton of those, there will be more and noone cares.
12:39:10 <pave1> Aha, ok.
12:39:45 <pave1> Another possible rules might be "if bug requires GFP_KERNEL allocation failing we don't care" because we are already OOM and system is already unusuable.
12:40:18 <uli> are there a lot of those?
12:40:59 <pave1> And "error paths in probing don't really require fixes" ... because if your driver fails to probe, your hardware is unusable, anyway.
12:41:10 <pave1> ...and you have already caught that during development.
12:41:45 <pave1> I have seen at least one of those in 4.19-st review.
12:42:07 <pave1> I was saying to myself "hmm, probably does not break anything, but hmm, I would not backport this".
12:42:30 <uli> ok, i'll add those and see how it goes
12:43:49 <pave1> Thanks!
12:44:03 <uli> IGNORE now includes: - it fixes an issue caused by a GFP_KERNEL memory allocation failure
12:44:03 <uli> - it fixes an issue in the error path during device probing
12:44:23 <uli> actually, i'll have to add "exclusively" to that
12:45:39 <jki> ok, are we done with release topics?
12:45:41 <pave1> Do we prevent LLM from working on stuff we don't enable in our configs?
12:46:00 <uli> i have a script sorting those out before the llm sees it
12:46:32 <pave1> Ok, good. That is one more reason to prune our configs.
12:47:00 <jki> of course, as well as to maintain them according to real demands
12:47:15 <jki> so, move on?
12:47:40 <pave1> yes.
12:47:44 <jki> 5
12:47:45 <jki> 4
12:47:47 <jki> 3
12:47:48 <jki> 2
12:47:50 <jki> 1
12:47:52 <jki> #topic Kernel testing
12:48:42 <jki> anyone with testing topics today?
12:48:44 <pave1> If someone could take a look at https://gitlab.com/cip-project/cip-kernel/linux-cip/-/jobs/17875327047
12:48:52 <iwamatsu> Several labs are down.cybertrust and siemens-prag.
12:48:59 <pave1> It is new kind of failure and does not go away with retry.
12:50:37 <iwamatsu> we can not use all x86-siemens-mcom device now. I think that is the cause.
12:51:05 <jki> the M-Com in Munich does not run kernel tests, only image tests
12:51:09 <arisut> no update here, other than some people are starting to use kci-dev with the mcp
12:51:22 <pave1> Aha, so that's known. It says "There are currently 2 "x86-siemens-mcom" devices online.", which confused me.
12:51:25 <arisut> https://kernelci.org/blog/2026/09/12/talk-to-kernelci-kci-dev-now-speaks-mcp/
12:51:26 <jki> the one in Prague should do the other way around, but...
12:51:49 <jki> clarifying Prague lab state right now
12:52:36 <jki> uh, we even have 3 mcoms in Prague now, just no online lab
12:53:39 <jki> I suppose we had no chance (time) yet to play with kci-mcp in our context :(
12:55:16 <jki> more on testing?
12:55:29 <jki> 5
12:55:31 <jki> 4
12:55:33 <jki> 3
12:55:35 <jki> 2
12:55:36 <jki> 1
12:55:39 <jki> #topic AOB
12:56:04 <jki> SECURITY.md: can send a v3 with the minor adjustments unless there are more comments
12:56:30 <pave1> Looks good.
12:56:45 <jki> further topics?
12:56:48 <pave1> I wanted to ask : if there's SPARC problem in 4.4, we don't care, right?
12:57:03 <pave1> Does CRA allow us to do that?
12:57:04 <jki> right, per scope we define in "Security Context"
12:57:33 <uli> sparc is out.
12:57:46 <pave1> Ok, being more explicit about that would not be bad.
12:57:58 <jki> I haven't read anything different about the CRA
12:58:20 <jki> pavel: we cannot write down the details of the configs in the SECURITY.md, they are not defined there
12:58:38 <jki> that's why we pointed to that repo - please read again
13:00:04 <pave1> Something like "if the issue is in unsupported parts of 4.4 or 4.19 kernel, please send report to dev null, we don't care".
13:00:35 <pave1> Ok, maybe not that explicit.
13:00:50 <jki> you really need to read the whole documents
13:01:03 <pave1> Ok.
13:01:04 <jki> LLM normally does today ;)
13:01:29 <jki> ok more topics for today?
13:02:11 <jki> 5
13:02:14 <jki> 4
13:02:16 <jki> 3
13:02:18 <jki> 2
13:02:21 <jki> 1
13:02:23 <jki> #endmeeting