17:03:02 #startmeeting 2016-05-11 discussion 17:03:02 Meeting started Wed May 11 17:03:02 2016 UTC. The chair is wking. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:03:02 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:03:02 The meeting name has been set to '2016_05_11_discussion' 17:03:23 #chair mrunalp 17:03:23 Current chairs: mrunalp wking 17:03:35 https://github.com/opencontainers/runtime-spec/pull/384 17:03:38 #topic create/start split 17:03:46 duglin: this just needs reviews 17:04:14 mrunalp: autoremove might make more sense as a runtime option, because it might not apply to other implementations 17:04:42 duglin: maybe the runtime should be responsible for populating part of the config.json? 17:04:54 mrunalp: I was thinking of not putting it in the spec and making it a runC option 17:05:35 duglin: for example, you would say: runc create --autoremove PATH ID 17:05:44 mrunalp: yeah, I'm just putting that up for discussion 17:06:00 duglin: I don't really care, but I'm not sure where you draw the line once you open that door 17:06:14 crosbymichael: it shouldn't be in the config 17:07:23 Why not part of the config? 17:07:37 duglin: I can see this if you consider the config a portable thing. But I see the config as "what you need to run the container", whether or not it's portable 17:08:01 julz_: I think hooks also fall under this runtime-option, not-in-the-config position 17:08:10 (sorry, don't have a mic) 17:08:38 mrunalp: do you mean that different Linux implementations can handle the idling container process in different ways? 17:08:51 duglin: you have to implement it, but the mechanism is optional 17:09:17 mrunalp: but we need to make sure hooks can be written in a portable manner, so some of the implementation has to be standardized 17:09:32 duglin: agreed, but I don't want the wording to be too perscriptive 17:09:57 ty wking 17:10:14 duglin: config.json is supposed to be portable 17:10:58 It scares me for the sake of the future to not have an atomic config for runtime 17:11:16 mrunalp: the other argument is that this is a lifecycle option, and the same config can have different lifecycles 17:11:33 Like, wouldn't some containers by nature ask to be autoremove? 17:11:59 If we introduce —autoremove as a flag, what other flags will there be in the future? 17:12:21 duglin: what's the rationale for whether something is in config.json vs. a command-line option? 17:12:36 mrunalp: good question. In this case, it feels like it doesn't belong in the config 17:13:30 duglin: we were moving down the "everything the runtime needs goes into the config" path, and the runtime option isn't following that approach 17:13:59 julz_: as an orchestrator, autoremove is important from a cleanup perspective. I'd rather not have to inspect the container to figure out whether I need to clean it up 17:14:27 duglin: I think someone *does* have to look at the config, e.g. "are they asking for too much memory". Why is autoremove different from that? 17:15:28 maybe we do need a runtime.json again. I like the idea of being able to pull down a config.json and never have to take a look at it but then also have the runtime.json that allows me as an orchestrator to specify what needs to happen 17:15:48 vishh: do we have to worry about this now? Can we punt and talk about autoremove later? 17:16:15 mrunalp: can we go with a runtime option now, and then talk about it again later if we have portability issue 17:16:42 duglin: I'm ok with it this time, but I don't want to set a precedent to have a portable vs. non-portable discussion all over again 17:16:58 duglin: I don't want an annoying overlay process 17:17:07 i'd agree with duglin there 17:17:14 me too 17:17:46 duglin: how would portability-time changes work with extensions? 17:18:06 julz_: analogy: I think --autoremove is an operation 17:18:31 julz_: create-with-pinning vs. create-without-pinning 17:18:39 julz_: but I don't want the user to be able to make that call 17:18:46 duglin: but what about memory options? 17:19:22 julz_: changing memory limits changes the container. This is the exact same container, just started with/without pinning. The container doesn't care whether those namespaces are pinned 17:19:39 I think the config.json would be treated as immutable — it'd come with the container and you wouldn't need to tweak it. It's specific to the container. The runtime.json would be things you'd like to customize about actually running the container 17:20:05 JakeWarner: problem with that^ is if a user has a malicious hook or some such 17:20:22 wking: but if hook is part of orchestration 17:20:26 wouldn't that be in runtime.json? 17:20:38 but removing hooks and such is a job for the runtime-caller, not a job for the runtime itself 17:20:46 https://github.com/opencontainers/runtime-spec/pull/412/files 17:20:53 defining which bits are part of "orchestration" is a slippery slope ;) 17:21:05 lol true true. 17:21:11 #topic Explicit container namespace for uid, gid 17:21:18 #url https://github.com/opencontainers/runtime-spec/pull/412 17:24:23 mrunalp: the glossary docs don't seem super clear to me. I don't have suggestions now, but will take another look after the call 17:24:41 #topic post-stop hooks 17:24:45 https://github.com/opencontainers/runtime-spec/pull/395 17:24:50 url https://github.com/opencontainers/runtime-spec/pull/395 17:24:53 #url https://github.com/opencontainers/runtime-spec/pull/395 17:25:02 mrunalp: duglin, is this getting rolled into #384? 17:25:09 I was under the belief that hooks would go away with 384 17:25:12 as well 17:25:20 duglin: I think post-stop hooks are gone after #384 17:25:37 I thought so too, which is why we were adding autoremove^ 17:26:02 but the all-in-one operation will still need hooks 17:26:06 mrunalp: ^ 17:26:23 duglin: I was under the impression that folks who need hooks would not use 'run' 17:26:42 mrunalp: there are users of hooks today that will not need the split 17:26:56 I'm not sure I agree with that 17:27:02 if we haven't hit 1.0 yet 17:27:04 mrunalp: we don't want to change the all-in-one command because of this split 17:27:22 vishh: if the new model supports all the hook use cases, why keep them as a first-class items? 17:27:33 mrunalp: why force people to change? 17:27:51 vishh: we can have ocitools map the 'run' workflow into the new workflow 17:29:36 duglin: it's not so much "the usecase isn't valid" as "there's a simpler bottom layer" 17:29:50 vbatts|work: runc could support this on top of create/start/stop/delete 17:30:03 Maybe the hooks only come into play if someone specifies 'run' vs 'create/start' ? 17:30:13 mrunalp: but the previous approach was to keep the old 'run', and now folks want to delete it 17:30:24 julz_: not "delete it", maybe "not specify it" 17:30:49 mrunalp: the old approach was "we'll add the new approach", not "we'll also remove the proven-today workflow" 17:31:01 I'd agree with julz_ 17:31:04 julz_: I'm not suggesting removing it from runC, but I don't think we need it in the spec 17:31:23 julz_: if we put it in 1.0 and it turns out that it's not necessary, we're stuck supporting it forever 17:31:37 duglin: I don't like having multiple ways to do the same thing in the spec 17:31:44 Maybe we support hooks up until RunC 1.0? Deprecate it 17:32:15 vishh: create/start/stop/delete is a clean base layer. Combining that to build in hooks sounds like a layer on top 17:32:28 vishh: as julz_ said, runtimes are free to support this 17:32:54 mrunalp: but you can do everything you do now with hooks? Why not drop create/start? 17:33:09 vishh: because with create/start you don't need binary hooks, although they are still possible 17:33:27 mrunalp: I acknowledge that, but don't like throwing out workflows that are used in production 17:33:49 I had attached a usecase to the PR regarding how removing the hooks benefits the CaaS I'm working on: https://github.com/opencontainers/runtime-spec/pull/384#issuecomment-218347771 17:33:51 vishh: It's ok to say "until this is proven, we won't make the switch" 17:34:32 mrunalp: if you want access to your host mount namespace, there's a problem with create/start. 17:34:44 #action mrunalp to comment on #384 about host mount namespace access 17:34:54 vishh: so punt until we understand all the use cases 17:35:18 mrunalp: yeah. And I don't like dropping the current all-in-one approach. But that can be visited later 17:36:59 mrunalp: implementers have a choice between all-in-one (like it is now), or to use create/start/... 17:37:09 duglin: I want to understand use-cases that aren't supported by the split 17:37:38 mrunalp: mount namespace issue: before a pivot, you have access to the host mount namespace until the pivot 17:38:16 crosbymichael: there's a time in the lifecycle where we're in the container mount namespace before the pivot 17:38:28 that's when you bind mount from the host into the container 17:38:43 crosbymichael: then you pivot and it locks the mount namespace into the new root 17:39:07 mrunalp: the current hooks get called before the pivot happens 17:39:33 vishh: why not handle the bind mounts through the config? 17:39:56 vishh: if the config doesn't require mount ordering, we should fix that 17:40:17 I think the current spec already requires config-specified mounts happen in order 17:40:32 mrunalp: previously we had to make the container privileged, but with hooks we don't need to 17:40:43 vishh: but you can achieve this by modifying the config... 17:41:01 mrunalp: but it's a good feature for the runtime-caller to modify the mount namespace 17:41:13 vishh: but you can inspect the command via the config. Why do you need hooks? 17:41:31 mrunalp: this is something that you don't want users to specify every time that should happen automatically 17:41:44 mrunalp: you can see "the user is trying to run systemd, so I want to setup these mounts..." 17:41:53 what's the difference between a custom edit of the config vs a custom tool on a hook? 17:41:56 vishh: that sounds like a runtime-caller concern 17:42:16 mrunalp: yeah, we could do that with some command line options, but now we get it easily with a hook 17:42:37 vishh: I don't see why we need spec support for this, when you could put in a higher wrapper that does this 17:42:52 I agree with vishh 17:42:55 vishh: there are use cases that we don't want to support in the spec, and you need hook-like things for that 17:43:32 crosbymichael: hooks aren't for specific use-cases anyway. They're lifecycle-extention points where you can do anything 17:43:57 crosbymichael: we've been talking about the same 1% use cases for two months, and aren't getting anything done 17:44:20 crosbymichael: the config is the I/O for a container. If you want to do more stuff, make your own runtime that does create/start today 17:44:47 crosbymichael: there's no reason to put that in the spec. 17:47:07 (relevant again: https://github.com/opencontainers/runtime-spec/pull/384#issuecomment-218347771) 17:47:16 crosbymichael: if networking is not in the spec, then it's a higher-level decision to set that up 17:47:39 crosbymichael: orchestrators will have their own runtimes, so they can call their network-setup script however they want 17:47:47 crosbymichael: it's not something that needs to be exposed to the users 17:48:19 ^ in response to me asking "when to I call my network-setup code if we don't specify either hooks or create/start?" 17:48:51 duglin: to me it comes down to interop. If we go with crosbymichael's really low level, then that really limits interop between runtimes 17:49:26 crosbymichael: there's not interop anyway with the create/start split, because there's too much flexibility (whether you pin namespaces or not, etc.) 17:49:51 julz_: I think this is the exact reason that we want create/start *or* an all-in-one, and not "the runtime can do whatever it wants" 17:50:12 crosbymichael: right now the contract is, "I give you the config and the root filesystem, please make a container" 17:50:43 ^ pick one... 17:51:04 julz_: I still think we need to pick one basic layer which covers everyone, and folks can put 99% solutions on top of that 17:51:26 dqminh_: we can achieve the same things by having the upper layer create namespaces ahead of time 17:52:53 crosbymichael: for all these use cases, you can setup the namespaces ahead of time. But people don't want to write all that 17:53:08 The thing I like about the create/start split is that I don't have to call another binary to make changes 17:53:18 julz_: what's the point of a spec if I have to rewrite the runtime? 17:54:05 julz_: if we can make everyone happy, why not do that? 17:55:19 crosbymichael: create/start is a 1% use case that is the orchestrator's problem 17:55:43 I disagree with that :\ 17:56:03 julz_: if you buy that argument, why have hooks? Just fork the runtime 17:56:16 crosbymichael: because hooks are a small change that adds the most flexibility 17:57:30 crosbymichael: even if it does pause during create, the runtime is still jumping in and out of container namespaces 17:57:46 Completely agree with julz_ 17:57:55 (though super applicable to my use case) 17:57:57 julz_: but the orchestrator can keep the hook-like-things in memory and doesn't have to write hooks to disk 17:57:57 …so biased. 17:58:18 crosbymichael: hooks live on tmpfs, and can talk to the orchestrator as they run 17:58:39 this does not sound appealing to me^ ;) 17:58:50 wking: which? 17:59:12 hooks in a tmpfs talking to the orchestrator over sockets ;) 17:59:19 yeah 17:59:25 crosbymichael: hooks are a minimal change to the current orchestrators 18:00:13 julz_: there's a danger to doing the expedient thing in a spec, because once we cut 1.0 we're stuck with it 18:00:44 RobDolinMS: we're at 11 18:00:54 RobDolinMS: can julz_ write up his approach? 18:01:15 julz_: it's just duglin's #384. We still need an implementation PR in runC, but the spec side is there and waiting for comments 18:01:34 I'd agree with that action (384) 18:01:40 RobDolinMS: so the action is "everyone review #384 so we can make an up/down decision on this next week"? 18:01:43 julz_: yeah 18:02:11 RobDolinMS: I don't know that we need to wait for a quorum, but it would be good to move forward on this 18:02:35 #topic dockercon meetup 18:02:41 Actually, I think we SHOULD wait for quorum as we have a few folks missing 18:02:44 duglin: we're meeting Wednesday from 9 to 12. Not sure where yet 18:02:49 I'll meet you all there! :D 18:02:59 Thanks Doug! 18:03:00 RoDolinMS, ah, sorry. Quorum sounds good to me too ;) 18:03:13 #endmeeting