12:01:11 <jki> #startmeeting CIP IRC weekly meeting
12:01:11 <collab-meetbot`> Meeting started Thu Jun 23 12:01:11 2022 UTC and is due to finish in 60 minutes.  The chair is jki. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:01:11 <collab-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
12:01:11 <collab-meetbot`> The meeting name has been set to 'cip_irc_weekly_meeting'
12:01:14 <jki> Hi all!
12:01:29 <patersonc[m]> Hello
12:01:30 <uli> hello
12:01:34 <masami> hi
12:02:26 <alicef> hi
12:02:45 <pave1> hi
12:08:06 <jki> got distracted - let's start
12:08:11 <jki> #topic AI review
12:08:15 <jki> 1. Resolve/filter irrelevant failures of KernelCI for 4.4-cip - patersonc & alicefm
12:08:42 <patersonc[m]> Nope
12:08:46 <alicef> there is a issue open or a pull request about this?
12:08:58 <alicef> ah found
12:10:23 <jki> 2. Check cip devices on kernelci old pull request - patersonc
12:10:42 <patersonc[m]> I'm looking into that now
12:10:54 <patersonc[m]> Nothing to report yet though
12:11:04 <alicef> jki: why are you asking to drop 4.4.cip checks ? looks like the problem is only on few boards
12:11:42 <alicef> I cannot see the kernelci log as too much time passed by the way
12:12:13 <alicef> looks like is failing only on mips boards
12:12:34 <jki> alicef: we discussed long ago that the current 4.4-cip kernelci results contain too much noise for our scope
12:12:43 <jki> pavel remarked this e.g.
12:13:07 <alicef> too much noise ? the last time it run was last month
12:13:07 <jki> therefore the idea was born to look into make the report more focused
12:13:16 <alicef> 2022-05-18
12:13:31 <jki> pavel: can you comment on that?
12:14:04 <alicef> also the link you are linking on the bug issue is about stable-rc and not cip
12:14:14 <alicef> See https://linux.kernelci.org/build/stable-rc/branch/linux-4.4.y/kernel/v4.4.302/
12:14:22 <alicef> this is stable-rc not cip
12:15:12 <alicef> I think the solution is solving the issues not suppress the tests
12:15:49 <jki> the primary goal is to have reports that quickly point our new regressions
12:16:08 <pave1> alicef: if someone is able to say which issues are real
12:16:09 <jki> the secondary would be finding and resolving potential existing ones
12:16:34 <pave1> alicef: we should solve them.
12:17:26 <pave1> alicef: take a look, tell me what to solve and I ll solve it.
12:17:31 <alicef> we should make a kanboard like the kernelci is doing https://github.com/orgs/kernelci/projects/6
12:17:39 <alicef> for each issue
12:18:13 <alicef> pave1: that's not the a good way of solving issues
12:18:57 <jki> well, we likely have no time spend on mips
12:19:17 <pave1> alicef: so suggest something else. Last time I looked, most issues seemed "not real"
12:19:43 <alicef> "not real" means on machine not related to cip i suppose
12:20:40 <pave1> Not real meaning it was not kernel that was failing.
12:21:31 <alicef> so it was a kernelci problem ? is it fixed yet?
12:22:06 <jki> it seems the first to-do remains to start looking into the issue, so a board can be step forward
12:22:20 <jki> but that alone will not change the picture
12:22:29 <pave1> It is long tine since I investigated.
12:22:30 <alicef> jki: the issue is too much ambiguos for expecting us to do something
12:23:05 <alicef> if you can pinpoint what is broken is better
12:23:25 <pave1> Go through reports, point me at kernel problem.
12:23:48 <alicef> pave1: that's not just reading kernelci reports?
12:24:03 <pave1> I could not find single one so I gave up.
12:25:16 <jki> ...and that's where the "filtering" idea came from
12:25:32 <jki> maybe not the best but a pragmatic reaction
12:25:59 <jki> anyway, someone needs to drive this who is familiar with the setup, I'm afraid
12:26:20 <jki> that's why the AI landed on your table
12:27:00 <jki> better concrete suggestions always welcome!
12:27:10 <alicef> I'm still not sure of what you want kernelci to do
12:28:30 <patersonc[m]> Only run CIP configs and arches on CIP kernel builds?
12:28:33 <alicef> if you a concrete solution is more than welcome
12:28:56 <patersonc[m]> So in effect reduce build and testing coverage?
12:29:37 <jki> pavel is your main "user" here, he should define the final result
12:29:51 <pave1> I dont think it is too much coverage problem.
12:30:15 <jki> I'm happy to moderate and help defining what needs to be done between kernelci and that result
12:30:30 <pave1> Begining would be "show me single report I should work on".
12:31:09 <alicef> that's not how kernelci work
12:31:24 <pave1> I see.
12:32:39 <pave1> If you cant show me single good report, how do you expect me to find one?
12:33:16 <alicef> you expect to get a summary for all cip kernels with only bugs you are interested on?
12:33:39 <patersonc[m]> https://linux.kernelci.org/job/cip/
12:33:39 <alicef> based on what decisions?
12:34:08 <pave1> I expect to be able to find single good report in all the noise.
12:34:28 <pave1> Based on any reasonable decisions.
12:34:42 <alicef> my proposal would be that you start a issues board and write down issues that you are not iterested and one that you are interested
12:34:54 <alicef> instead of tagging everything as noise
12:36:05 <pave1> Give me one good report to start with.
12:36:35 <pave1> If it is not noise, it should not be hard.
12:36:39 <alicef> good is not a objective definition of something
12:38:37 <jki> maybe start with one concrete error of that report?
12:38:49 <alicef> I have no idea of your definition of good
12:39:14 <jki> https://linux.kernelci.org/build/id/61fbabe0cb6f27a6ce5d6f03/logs/?
12:39:16 <alicef> and you are getting mail for each report that came from kernelci about cip
12:39:57 <alicef> you should start make a filter list and me and patersonc[m] could help filtering out upstream
12:40:44 <alicef> also usually kernelci accept only working kernels
12:40:59 <alicef> so if you get errors is usually a regression
12:41:14 <alicef> maybe not relevant for cip but still a regression
12:42:08 <alicef> for exemple mips error from what I can see as I cannot see the logs. there was no error on the previous kernel cip 4.4 build
12:43:29 <pave1> that is not what I remembered. I can look at jki's link.
12:43:36 <alicef> if is a board failure, it happen. is nothing that we can do about
12:43:53 <alicef> I cannot prevent a board failure before it fail
12:44:19 <pave1> What you can do is pointing me issues you believe should be fixed.
12:44:41 <pave1> Then I can look for similar issues.
12:44:53 <jki> pavel: ideally, you should once be able to see them as well, I think
12:45:05 <jki> but it seems we are playing ping-pong here
12:45:13 <jki> who needs to do a first step
12:45:18 <alicef> pave1: I have no idea what issues you want to fix
12:46:23 <jki> pavel, alicef: please list one issue each about which you think it should be filtered or fixed
12:46:33 <jki> then let us discuss about these two issues next time
12:46:38 <jki> want do you think?
12:46:47 <pave1> Ok, lets do that over email?
12:46:56 <jki> also fine for me
12:47:30 <alicef> we already have the kernelci issue that is frozen
12:47:48 <alicef> i'm still not sure what you expect me to do
12:48:47 <alicef> https://github.com/kernelci/kernelci-core/issues/1053#issuecomment-1050679725 also gtucker replayed to your request
12:50:39 <jki> ok, so someone needs to come up with a reasonable blocklist for 4.4 first
12:51:02 <jki> for that, pavel could start with a proposal of one (or more) entries?
12:51:25 <jki> alicef: would that help you then?
12:51:36 <pave1> Ok, let me take a look.
12:52:17 <jki> thanks a lot in advance
12:52:23 <jki> move on?
12:52:50 <alicef> yes
12:53:01 <jki> 3. Update https://wiki.linuxfoundation.org/civilinfrastructureplatform/ciptesting/cipreferencehardware - iwamatsu-san
12:53:10 <jki> any news on the qemu entries?
12:55:06 <jki> ah, iwamatsu is not participating :)
12:55:10 <jki> then move on
12:55:22 <jki> #topic Kernel maintenance updates
12:55:40 <uli> done with reviewing 5.10.121
12:55:53 <pave1> Reviews -- 5.10.123 and 124.
12:55:56 <masami> This week reported 6 new CVEs and 4 updated CVEs.
12:56:04 <masami> Most of new CVEs were fixed in mainline and  stable kernels.
12:58:28 <pave1> Applied Uli's backports after some tweaking. May still break bisectability.
12:59:11 <pave1> I would not mind someone looking at 4.4-cip to double-check.
13:00:52 <jki> uli? or better even a third person?
13:01:29 <pave1> Whatever review I can get :-)
13:02:11 <jki> ok, other topics here?
13:02:24 <jki> 3
13:02:26 <jki> 2
13:02:28 <jki> 1
13:02:30 <jki> #topic Kernel testing
13:03:07 <alicef> jki: can you confirm that isar-cip-core kernelci deploy script is working again?
13:03:39 <alicef> I remember you told that was not working
13:03:39 <jki> hmm, honestly, I only checked if it does not fail during the execution
13:03:50 <jki> didn't check the pushed artifacts
13:04:02 <alicef> looks like is starting to send artifacts again
13:04:08 <jki> good
13:04:30 <jki> let me know if they do not look like as they should
13:04:45 <alicef> last artifact is of 20220622
13:05:08 <alicef> ok
13:06:57 <jki> anything else?
13:07:07 <jki> 3
13:07:09 <jki> 2
13:07:11 <jki> 1
13:07:12 <patersonc[m]> Iwamatsu-san got the kernel config we needed for nfs on the openblocks board, and we've tested that it works in LAVA. I've fixed the healthcheck and one of the boards is online now. When Minda-san creates an MR on cip-kernel-config we'll be able to start including testing of the openblocks board in our CI.
13:07:22 <alicef> nice !
13:07:29 <jki> cool
13:08:08 <patersonc[m]> Nothing else from me
13:08:30 <jki> then let's move on
13:08:35 <jki> 3
13:08:37 <jki> 2
13:08:38 <jki> 1
13:08:40 <jki> #topic AOB
13:10:16 <patersonc[m]> I've got a Q about cip-core, if it's not completely the wrong place?
13:10:33 <patersonc[m]> Are the package versions from cip-core-isar and deby kept in sync?
13:10:49 <jki> I don't think so
13:10:59 <jki> for deby, best ask kazu
13:11:03 <patersonc[m]> Should they be?
13:11:18 <jki> and for cip-core, we are pulling from upstream debian of the day
13:11:46 <jki> we don't do any extra work on top of the current debian packages to justify a gating or freezing at this point
13:12:06 <jki> but that would indeed be a better topic for the cip-core WG
13:12:28 <jki> maybe ask your question on the list, providing more details on why things should be sync'ed?
13:13:12 <patersonc[m]> Sure
13:13:16 <patersonc[m]> Thanks Jan
13:13:30 <jki> anything else?
13:14:53 <patersonc[m]> Not from me
13:14:54 <jki> 3
13:14:55 <jki> 2
13:14:58 <jki> 1
13:15:01 <jki> #endmeeting