17:02:08 <rgoulding> #startmeeting kernel projects
17:02:08 <odl_meetbot> Meeting started Tue Feb  6 17:02:08 2018 UTC.  The chair is rgoulding. Information about MeetBot at http://ci.openstack.org/meetbot.html.
17:02:08 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:08 <odl_meetbot> The meeting name has been set to 'kernel_projects'
17:02:18 <rgoulding> #topic agenda bashing
17:02:36 <rgoulding> #info Infra issues
17:04:59 <vorburger> skitt: we could discuss https://git.opendaylight.org/gerrit/#/c/67833/
17:05:27 <rgoulding> #info re-enable archetype self testing
17:05:39 <rgoulding> #topic infra issues
17:06:17 <rgoulding> #info three updates
17:06:32 <rgoulding> #info 1) last night there was severe disk i/o and network slowdowns
17:06:41 <rgoulding> #info vexhost trying to give new hardware, and they did this last night
17:07:01 <rgoulding> #info when they did this, the upgrade slowed everything down due to sync costs
17:07:18 <rgoulding> #info hopefully will not see the very slow transfers that were happening last night
17:07:25 <rgoulding> #info 2) filesystem ran out of disk inodes
17:07:39 <rgoulding> #info the ext4 partition ran out of inodes for nexus
17:07:54 <rgoulding> #info ran out of the ability to index the disk
17:08:04 <rgoulding> #info xfs partitioned disk has been created
17:08:08 <rgoulding> #info which can support this
17:08:15 <rgoulding> #info the sync will take a very long time to sync
17:08:26 <rgoulding> #info some things have been deleted to helpf accelerate the transition
17:08:39 <rgoulding> #info includes a lot of logs that are tiny, which take up a ton of inodes
17:08:50 <rgoulding> #info should hold us over for the week or so until we have the XFS system in place
17:09:01 <rgoulding> #info 3) issues with maven merge jobs
17:09:26 <rgoulding> #info globaljjb maven merge jobs is different than what we used in ODL originally (freestyle)
17:09:53 <rgoulding> #info CLI tool is producing invalid timestamp for files it is producing locally
17:10:15 <rgoulding> #info a bug has been discovered in maven 3.5.0 which is similar to this issue, even though we aren’t using it (we use 3.3.9)
17:10:23 <rgoulding> #info reverted to ODL - specific merge jobs
17:10:41 <rgoulding> #info these jobs were around for a month, which is why it is confusing that we suddenly see this issue
17:11:31 <rgoulding> #info rovarga has a theory that it was probably due to the fact that we had a limited number of patches actually getting merged during the odlparent/yangtools migration
17:11:53 <rgoulding> #info now we have a huge backlog of patches, so we are more likely to run into this issue
17:13:31 <rgoulding> #info in the meantime we can possibly detect and correct the maven metadata
17:13:59 <rgoulding> #info we are going to wait on switching this until the release is out the door
17:14:29 <rgoulding> #info if merge jobs continue to fail, please contact zxiiro and he will look into it
17:14:51 <rgoulding> #info vorburger brings up a 4th possible error;  there are memory issues with validate-autorelease
17:16:05 <rgoulding> #info the VM template was changed, which could cause this
17:21:41 <rgoulding> #info reverting validate-autorelease to use 2 core / 8 gig RAM templates will result in slower jobs, but they won’t crash due to OOM at least
17:23:16 <rovarga_> #link https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/mdsal-validate-autorelease-oxygen/181/console.log.gz
17:24:13 <rgoulding> #info the above link was provided to show cpu/memory analysis for validate-autorelease process
17:24:29 <rgoulding> #info it would appear that validate-autorelease is very memory sensitive, but less CPU sensitive
17:24:55 <rgoulding> #info zxiiro will revert the template for validate-autorelease, but not touch the others since they don’t require the same resources
17:26:36 <rgoulding> #info further analysis of this and broader messaging could help in the future
17:28:12 <rgoulding> #topic archetype self-testing
17:28:15 <rgoulding> #link https://git.opendaylight.org/gerrit/#/c/67833/
17:31:08 <rgoulding> #info 14:49:16  networkx.exception.NetworkXUnfeasible: Graph contains a cycle or graph changed during iteration
17:31:20 <rgoulding> #info it appears there is a cycle
17:32:35 <rgoulding> #link https://github.com/opendaylight/controller/blob/master/opendaylight/archetypes/opendaylight-startup/src/main/resources/archetype-resources/features/odl-__artifactId__-rest/pom.xml#L35-L38
17:32:42 <rgoulding> #info this causes the cyclic dependency loop
17:34:20 <rgoulding> #info so this can’t actually work (with self-testing) in controller
17:34:31 <rgoulding> #info since then controller depends on netconf, which is not correct
17:35:31 <rgoulding> #info this self-test helps detect errors before releasing
17:35:41 <rgoulding> #info and it is used a bunch more than what we had imagined
17:36:35 <rgoulding> #info coretutorials may be a good landing spot for this, since it is outside of autorelease
17:36:50 <rgoulding> #info i.e., this is how you generate the code etc.
17:37:14 <rgoulding> #info coretutorials is currently broken (behind nitrogen and oxygen currently) but Robert has some patches to fix this
17:37:58 <rgoulding> #info tpantelis is a committer / PTL for it
17:39:27 <rgoulding> #info rovarga is suggesting to move the archetype to coretutorials
17:39:32 <rgoulding> #info with the tests and everything
17:39:51 <rgoulding> #info you can then evolve the archetype outside the normal release process since you can release the artifacts outside of autorelease
17:40:26 <rgoulding> #info would it make sense to keep in autorelease? that would allow us to see it breaking live
17:41:06 <rgoulding> #info skitt brings up the fact that we could do it in other ways
17:41:46 <rovarga_> #link https://git.opendaylight.org/gerrit/#/q/status:open+project:coretutorials+branch:stable/nitrogen
17:44:17 <rgoulding> #info if it lives in autorelease, you cannot provide bugfixes etc. outside of autorelease
17:44:39 <rgoulding> #info there are benefits and drawbacks associated with all of this
17:45:54 <rgoulding> #action vorburger to move archetype to coretutorials
17:47:48 <rgoulding> #info propose a separate project for just the archetype
17:48:24 <rgoulding> #info why don’t we just delete it?
17:51:47 <rovarga_> #link https://wiki.opendaylight.org/view/Project_Proposals:Main
17:52:04 <rovarga_> #info wiki page describing what needs to do to propose a new project
17:56:21 <rgoulding> #info this page will likely need to be reverted as someone screwed the template
18:00:03 <rgoulding> #info tpantelis to look at https://git.opendaylight.org/gerrit/#/c/67662/
18:02:17 <rgoulding> #endmeeting