17:01:45 #startmeeting OCI 3/30 17:01:45 Meeting started Wed Mar 30 17:01:45 2016 UTC. The chair is mrunalp. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:45 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:01:45 The meeting name has been set to 'oci_3_30' 17:01:57 RobDolinMS: priorities!! 17:01:57 I am at Linux Collab summit so #conferencewifi 17:02:04 no worries :) 17:03:01 wking: do you have a url to the start/create split email? 17:03:18 #topic https://github.com/opencontainers/specs/issues/357 17:03:23 duglin: I'll look it up 17:04:18 On the question of renaming the spec, I don't have a strong preference as to the exact name, but good to rename "spec" to foo-spec 17:04:35 #action Brandon to email LF to go ahead with the rename 17:04:55 #topic Create/start split 17:05:02 #link https://groups.google.com/a/opencontainers.org/forum/#!msg/dev/qWHoKs8Fsrk/k55FQrBzBgAJ 17:07:35 duglin: what if the initial PID 1 dies, can you run another start? 17:07:46 mrunalp: no, you'd be starting another container with another container ID 17:08:15 julz_: one think that's nice about proposal 3 is that it the PID / container relationship is clear 17:08:48 duglin: does having a separate stop make sense? Or separate stop/delete? It sounds like the container process dies, and everything gets stopped and deleted 17:09:12 julz_: if you want to keep the namespaces around, you could (via bind mounts or whatever), but there wouldn't be an explicit stop/delete 17:10:17 the process namespace is going away when it's PID 1 dies, regardless of anything we do in OCI 17:10:45 mrunalp: if you start a new container reusing the mount namespace, there may be leftover files, etc., from the previous container 17:11:18 julz_: if you're just creating a container to preserve the namespaces, you don't have to ever start it, just leave the idling init and launch sub-containers 17:11:31 mrunalp: that's supported now via namespace paths 17:11:49 julz_: yes, and that's what we all want for exec implementations. 17:12:46 duglin: if we go back to a single stop/delete, do we need a hook? Or push all cleanup to after the namespaces are gone 17:12:59 julz_: we already run post-stop hooks after the container process is dead 17:13:23 crosbymichael: this seems super complex 17:13:26 here 17:13:51 (is anyone able to participate in these meetings?) 17:14:02 yep 17:14:13 uberconference.com/ssaul 17:14:48 julz_: this is less complicated than proposal 2 (from the list), because we don't need any bind mounting of namespaces, etc 17:14:56 Flexibility is running hooks is the main use case.. 17:15:04 *in running 17:15:32 julz_: and if you want a combined start you can skip the named socket (https://groups.google.com/a/opencontainers.org/d/msg/dev/qWHoKs8Fsrk/ug4PNraWBwAJ) 17:15:44 crosbymichael: proposal 3 talks about a Unix socket... 17:16:06 julz_: that's so a later 'start' call can tell the init process (launched from 'create') what user-code it should execute 17:16:50 duglin: this is basically what we're doing now (with a pipe between the runtime and container process) 17:17:16 julz_: yeah, we already have this dance going on, and if you know at 'create' time what you want to run, in which case this devolves to the current behaviour 17:17:35 mrunalp: and after 'start' is called, we follow the current lifecycle? 17:17:37 julz_: yes 17:17:52 vishh (I think): what about preserving the current namespaces? 17:18:12 julz_: with this proposal we don't have to distinguish between the container" 17:18:21 * "the sandbox" and "the container process" 17:18:45 julz_: which means any namespace, including the process namespace, are consistently alive or dead 17:19:14 vishh: but in reality you setup the sandbox, and then invoke the user process, and after that process dies you want to do stuff with the sandbox 17:20:08 duglin: vishh, what does "sandbox" mean to you? Does it include process namespaces? 17:20:19 vishh: no, it doesn't include the process namespace 17:21:07 julz_: if you want to preserve other namespaces, create a container to hold the namespaces open, and then use sub-containers inside that persistent wrapping container 17:21:24 vishh: we want simple concepts that multiple users can bend to their wishes 17:22:11 julz_: I think proposal 3 is that simple concept, because the lifecycle of the sandbox (whatever you want it to be) are all tied to a single container process 17:22:51 julz_: it's more complicated if you start special-casing sandboxes. Sometimes it's fine without a process namespace in the sandbox, and sometimes you need the process namespace to be part of the sandbox 17:23:11 mrunalp: the problem I see with putting this into runC is that it only works in a particular way, but runC works fine right now 17:23:32 mrunalp: this would force runC to implement the lifecycle of namespaces in a particular way 17:23:59 julz_: right now runC doesn't support a distinction between "sandbox" and "container process", and I'm not sure we want it to 17:24:30 julz_: with proposal 3, runC looks almost identical to it's current state. The only cost is idling the init process 17:24:56 julz_: but that seems like a small cost, especially compared to bind-mounting namespaces and the associated cleanup handling 17:25:19 mrunalp: we want a separate "sandbox" concept that's not tied to a container ID, then you can start a container in that sandbox 17:25:34 mrunalp: that way you can separate the lifecycle of the container and the lifecycle of the sandbox 17:26:07 julz_: that's my problem, I *don't* think we want to separate the sandbox from the container process, because we can't agree on what "the sandbox" means (e.g. if it includes the process namespace) 17:26:39 julz_: But propsal 3 seems to cover all of our use cases, and proposal 2 does not 17:27:11 vishh: so the main issue is the complication in runC? 17:27:21 mrunalp: yes, but there are also some unclear corner cases 17:27:44 julz_: we know the edge cases, because this is the same as what we do now, just with an idling pause in the init process 17:28:22 mrunalp: vishh might disagree with you on that, because he cares about the bind mounts 17:28:45 vishh: the split between create and start is awesome, but I want a split between the death of the container process and the sandbox cleanup 17:29:07 julz_: you can do that with sub-containers or bind mounts; either will keep the namespaces alive 17:29:53 vishh: create would create all the namespaces, bind mount them, and keep PID 1 running. I haven't seen a case for preserving process namespaces, but I do want to preserve the mount namespaces 17:30:24 julz_: you could setup bind mounts after 'create' finishes, before running 'start' if you wanted 17:31:19 julz_: crosbymichael is already concerned about complexity, so we don't need to build in help for bind-mounting namespaces 17:31:34 julz_: but you could build that into the spec if you wanted to 17:32:15 duglin: vishh, so you want two separate steps for 'stop' and 'delete', preserving all namespaces except for the process namespace until 'delete' 17:32:34 julz_: and you can still do that with sub-containers and not kill the wrapping container. No need for bind mounts 17:32:44 julz_: killing the wrapping container's PID 1 is like 'delete' 17:33:18 vishh: you can do all of this in many ways. This seems too invasive/low-level for going into the spec 17:34:06 julz_: we've talked about punting 'exec' to a higher level, and it's not hard to build 'exec' on top of this 17:34:27 vishh: I'm not disagreeing. This is a step in the right direction. 17:34:48 vishh: we should also consider more separation between the sandbox and the process at a later stage 17:35:11 duglin: vishh, would you be ok with proposal 3 with a split stop/delete? 17:35:32 vishh: I'm ok. I'd rather not expose cgroups and namespaces, but if you have to leak them out, I'm ok with that for now 17:35:43 duglin: I'm also worried about requiring consumers to bind-mount thing 17:35:58 julz_ keeps pointing out that user's don't need to bind-mount anything, and should use sub-containers ;) 17:37:39 we have something else besides Linux? :-) 17:37:54 julz_: with option 3 we can express all of these use-cases. You can always wright higher-level wrappers. With option 2, there are some workflows you can't express, and that's harder to work around 17:38:04 vishh: how to express this on other operating systems? 17:38:19 duglin: heh 17:38:55 julz_: it's hard to predict what's hard/easy on other OSes. I suspect creating everything except the user process seems easier with VMs 17:39:40 vishh: I just want to avoid making life overly difficult for other OSes 17:40:06 julz_: maybe make the create/start split optional, so OSes where that's a problem can expose only a unified create/start command 17:40:22 vishh: I'd rather keep the spec more consistent across OSes 17:40:23 julz_: agreed 17:40:48 vishh: I want an explicit create/delete for the sandbox and explicit start/stop for the container process. I don't care about the process namespace 17:41:13 vishh: you'd still have to bind-mount to separate stop from delete, but I'm not sure how this translates to other OSes 17:41:25 What's the definition of sandbox that we're going with here? 17:41:45 julz_: so there's a create/start split and a stop/delete split. I think we both agree that create/start is a good split with all the namespaces. 17:41:55 mainly all NSs - whether it includes PID varies 17:42:02 JakeWarner|Work: ^^ 17:42:14 julz_: it may also be useful to split stop/delete and preserve namespaces after the container process dies, and you can do that with bind-mounts, but that seems orthogonal 17:42:26 Got it. 17:42:29 Thanks 17:42:49 philips: isn't the goal to have some pre-start and post-stop things for a pod? It could be an init-system, or bind-mounts. Why be specific for the spec? 17:43:30 julz_: I agree. These implementation differences don't matter, but the bind-mount approach has a user-visible difference (no process namespace or container PID) 17:44:16 vishh: the spec is currently 1:1 with runC, so we don't consider spec changes until we see if something is implementable in runC 17:44:25 vishh: the question is "do we want to implement this in runC" 17:44:59 philips: if I was implementing this myself, I would use an init system to spawn processes. There are options that don't require bind mounts or application processes running as PID 1 17:45:13 julz_: I suspect we should define this in a way that you can implement things like that 17:46:01 vishh: for hooks, you can have multiple ways of running the hooks, but it should be tied to the container lifecycle clearly enough to make the changes you need 17:46:42 mrunalp: there is an argument about breaking down runC into smaller steps, so you don't have to build everything in. But there are also arguments for building things in. 17:47:06 thanks for taking notes wking 17:47:11 np 17:47:24 duglin: but if you make it easy to have a runC create join an existing namespace, you can do that without exposing all the lower-level details 17:47:31 mrunalp: so what do we do next? 17:47:39 crosbymichael: I dunno 17:47:49 mrunalp: maybe a concrete proposal that covers everything? 17:48:00 vishh: we should PR a spec change and argue there 17:48:14 +1 to a concrete spec change PR so we can see the exact proposal 17:48:24 vishh: this has stalled because there wasn't an implementation in runC, but now we have a runC implementation. 17:48:44 vishh: a concrete spec change would give us something more concrete to argue about 17:48:51 +1 to spec as well 17:49:01 +1 to spec 17:49:25 #link https://github.com/opencontainers/specs/issues/299 17:49:25 I'm always +1 to a concrete PR-form proposal 17:49:39 ^ existing create/start proposal (proposal 2 in my summary email) 17:49:56 sorry, need to run off folks! 17:50:24 #action julz/duglin to create a new PR 17:50:29 duglin: I'll be off for the next two weeks, but I can work on a PR with julz_ 17:50:41 mrunalp: other topics? 17:50:41 s/off/traveling/ not vacation :-) 17:50:47 duglin: ah well ;) 17:50:59 #endmeeting