17:01:26 #startmeeting OCI 6/8 17:01:26 Meeting started Wed Jun 8 17:01:26 2016 UTC. The chair is mrunalp. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:26 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:01:26 The meeting name has been set to 'oci_6_8' 17:01:52 https://github.com/opencontainers/image-spec/issues/87 17:03:10 related: https://github.com/opencontainers/runtime-spec/issues/492 17:04:29 philips: the runtime doesn't handle any networking 17:04:43 mrunalp: it happens outside the container context on the host 17:05:02 stevvooe: so this is happening on the runtime annotations, not the image-spec annotations? 17:05:05 mrunalp: yeah 17:05:17 #chair wking 17:05:17 Current chairs: mrunalp wking 17:05:29 stevvooe: while runC doesn't implement exposed ports, it is part of the container experience for Docker 17:05:45 philips: that is the suggestion, that we add exposed ports to annotations 17:05:59 stevvooe: networking is complicated; one annotation is not sufficient 17:06:13 philips: this is the first key/value we've encountered 17:06:22 stevvooe: what if the rest of the data is more structured? 17:06:43 philips: I'm not suggesting we copy all the image-spec stuff in. This is the one bit that the runtime-spec doesn't already handle 17:06:54 philips: we can put JSON blobs into annotations 17:06:56 ^yuck ;) 17:07:18 philips: ExposedPorts is a single value, which is why I think it should go into annotations 17:07:43 duglin: wouldn't it make sense to make it more generic, because there will likely be more things that some runtimes can't handle 17:07:55 duglin: can't we copy the whole image config to a known location? 17:08:16 philips: that's the discussion we're having. I'm not opposed to it, but if it's just the one value that seems like overkill 17:08:41 Brendan: independent of the image-spec, there will be a need to pass opaque information down to the runtime 17:09:15 Brendan: if we copy the whole thing in, folks consuming it have to have a deeper understanding of the image-spec config 17:09:53 mrunalp: how do we tell people about semantically-meaningful annotations? 17:10:12 Brendan: we have this in Kubernetes, where all annotations use domain-name prefixes 17:10:48 stevvooe: classically, are annotations used to configure runtime behavior? I thought they were for marking orthogonal-to-running information 17:11:08 Brendan: they can be used for lighting up experimental features that aren't stable enough for a formal API yet 17:11:26 stevvooe: so you're using it as a parameter bag? 17:11:36 Brendan: annotations are just meta-information, so it's fine 17:11:54 Joe Beda: labels and annotations are distinct in Kubernetes 17:12:00 For context here is the spec on annotations in OCI: https://github.com/opencontainers/runtime-spec/blob/master/config.md#annotations 17:12:09 labels vs annotations - its a very fine line 17:12:14 Brendan: labels are used for idenfitication, and annotations are used for arbitrary metadata 17:12:43 philips: rkt + Kubernetes annotates whether the container runs in a VM or not 17:13:16 philips: Would the "plugins" structure that I suggested here make any sense? https://github.com/opencontainers/runtime-spec/issues/492#issuecomment-224498950 17:13:16 stevvooe: so there are two subtly-different things: labels and annotations, but runtime-spec uses annotations for both 17:13:51 stevvooe: labelings aren't as binding for core Docker concepts, but they're usually used for plugin features and such 17:14:11 mrunalp: originally labels and annotations were proposed as a pair, but annotations covers both cases 17:14:38 Brendan: we split them, because we only support querying on labels, and you don't want to index binary blobs 17:15:01 vishh: we were not able to implement label selectors in runC, so we just have annotations 17:15:13 philips: the runtime spec doesn't have a concept of enumerating running containers 17:15:37 Joe Beda: labels as a term of identity starts to bump up against naming and discovery, although that's a bit out of scope for the current discussion 17:16:04 philips: all we're talking about is flattening anonymous metadata (ExposedPorts) down into the config 17:16:21 Brendan: we don't have structured data in annotations, and I wish we did 17:16:40 philips: the general consensus was that folks can flatten the JSON to a string if they want 17:16:49 Brendan: but we want to keep it native JSON 17:17:19 Brendan: it makes parsing a lot more complicated to keep it a string (e.g. you can validate the whole thing with JSON Schema) 17:17:38 crosbymichael: the point of annotations is we aren't acting on them. Why add structure to opaque data? 17:17:53 I say, "why require a string->string map for opaque data" ;) 17:18:21 Brendan: when the thing gets created, you want to know that it's valid 17:18:55 proposal: all runtime-unknown image properties get copied into an annotation with a well-defined naming pattern. 17:19:03 Brendan: it's not a big deal. We just wish we had the ability to put structured JSON in annotations 17:19:28 crosbymichael: is networking the only thing you see so far? 17:19:47 philips: yeah, this is the only thing in Docker's v2 image format that doesn't map directly to a runtime-known field 17:20:14 crosbymichael: if we use Docker as an example, it's going to do all of the networking setup, and you don't have to pass this down to the bundle filesystem at all 17:20:41 philips: the network-setup code should look at the bundle to find the network namespace. It should also be looking for ExposedPorts 17:20:54 crosbymichael: that can happen in the orchestration layer 17:21:37 philips: this is coming out of a desire to have the image -> bundle unpacking be separate from the runtime/network-setup/etc. 17:21:55 philips: it could be that nobody is interested in that use case 17:22:27 Brendan: there's a difference between having a daemon running and not 17:24:34 It's hard to have the image-downloading tool handle network setup on its own 17:25:00 mrunalp: even if we're not clear about who handles the networking setup, it's still ok to put the information in annotations 17:25:12 Brendan: definitely put in the networking namespace stuff 17:25:37 #action philips to PR a discussion of ExposedPorts with assorted use-cases from the meeting discussion 17:25:51 #topic image-spec 1.0 procedures 17:26:16 philips: to a large degree, concerns are due to unmet expectations from a lack of documentation 17:26:39 philips: the TDC (i.e. maintainers) are supposed to "approve" releases (in the charter) 17:26:46 philips: but there's no definition of "approve" 17:27:29 philips: we need a policy where we talk about "here's what we want to accomplish in the next milestone" or "we want to cut a release", and have rules for maintainers accepting those proposals 17:27:39 AFAIK, TDC is not necessarily just maintainers 17:27:43 stevvooe: the timeline was set by the TOB, but really it should be up to the TDC 17:27:59 RobDolinMS: the current charter points to MAINTAINERS for a list of TDC members 17:28:09 ^ that is wking talking to RobDolinMS ;) 17:28:47 section 5.e of https://www.opencontainers.org/about/governance 17:29:15 philips: in none of our projects have we defined the necessary threshold 17:29:37 wking - There are a number of references to the TDC that may not all be consistent 17:29:38 philips: if we document the process, than nobody gets surprised 17:29:59 agreed on TDC inconsistency ;) 17:30:27 https://github.com/opencontainers/tob/issues/2 would help clarify the TDC definition 17:30:51 stevvooe: I'm not sure which forum is correct. I go to the mailing list, and folks point me at GitHub issues 17:31:05 Brendan: I like GitHub issues for multiple technical concerns 17:31:20 Brendan: The GitHub milestone for the 1.0 release has been around for months 17:31:51 Brendan: I agree that we need to figure out the process and document that. 1.0 means "we're ready", and we want a process to determine if the TDC agrees with that 17:32:11 stevvooe: maybe I should have spoken up earlier, but do you not acknowledge the concerns I'm raising 17:32:33 Brendan: I think we need to have those discussions, and the issues haven't been filed yet 17:33:29 RobDolinMS: I'm encouraged by the open conversation around open image-spec issues 17:34:34 RobDolinMS: trying to get a lot of people who spend a portion of their time working on containers, it does take some time to get folks together around a minor release, gather feedback, and post back to the repositories 17:34:51 RobDolinMS: so it's good to give folks time to go out and gather that feedback 17:35:20 philips: agreed. And the release policy should set a bar 17:35:49 philips: in runtime-spec, we had a 6 month target for 1.0, and now we're a year out, and we're still not sure if we're there or if all the stakeholders are on board 17:36:56 mrunalp: so do we wait on this process before 1.0? 17:37:07 mrunalp: or do we go ahead with 1.0 as this process is developed? 17:37:49 philips: feedback from RobDolinMS is that "a week isn't long enough to review", and the list discussion shows lots of people who aren't all on the same page with respect to the release process 17:38:06 philips: so I think we need to pause for a week and change and get a policy in place 17:38:56 #action philips to write up a policy proposal for handling release stamping to apply to all OCI projects 17:39:46 Brendan: I think there's a distinction between releasing a binary (e.g. runC) and releasing a spec (e.g. runtime-spec) 17:40:22 philips: if the procedure is "get maintainer agreement", then the runC maintainers can pick their own threshold 17:41:48 RobDolinMS: as you draft the policy, I'd encourage a goal and a minimum. Or split between major/minor/patch? 17:42:24 giants! :-) 17:42:35 RobDolinMS: I mailed the list with notes about other organizations. The W3C and IETF may be heavyweight, but we shouldn't be reinventing the wheel 17:43:03 #topic ocitools as an OCI Project 17:43:20 philips: we just created ocitools, but it hasn't been formally approved by the TOB 17:43:53 #link https://github.com/opencontainers/ocitools/issues/83 17:44:01 philips: do we just move it back in? 17:44:22 philips: if we want it to stay a separate project, we need the TOB to bless it 17:44:30 mrunalp: ocitools predates the TOB 17:44:48 stevvooe: does this mean that generation and verification stay in the image-spec repository? 17:45:20 philips: yeah, I think we want oci-runtime-tool (like the current oci-image-tool) and we close down opencontainers/ocitools 17:45:43 stevvooe: I think spec maintainers should be overseeing test-suite development, so I'm in favor to bring them in 17:46:01 mrunalp: there's always going to be a lag to adding code (it's more expensive than adding spec language) 17:46:17 philips: even though they live in the same repo, we could have them tagged differently 17:46:29 please no multiple tags for a single repo^ :p 17:47:14 what's the benefit to squashing to a single repo? 17:47:36 RobDolinMS: in Microsoft we've had to generate a new build because of a security issue. Always being able to ship is important 17:47:49 RobDolinMS: the offset between the spec and test cases makes me scratch my head 17:48:10 RobDolinMS: if we have to cut 1.0.1 to fix a spec security hole, I'd rather not have the tools tying my hands 17:48:35 RobDolinMS: I'm not sure what the distinctions are between multiple tag sets in one repo vs. two repos with the same set of maintainers 17:49:02 RobDolinMS: we can probably make this a TOB meeting right now ;) 17:49:20 ^ Not necessarily what I said, but it's possible ;) 17:49:45 philips: the main think is practical. It makes sense to release the JSON Schema with the prose spec. This avoids a chain of PRs 17:49:54 philips: it's nicer to fix everything at once 17:50:16 philips: I don't have a strong optinion (although multiple PRs is annoying) 17:50:31 mrunalp: another issue is that we want the tools to be able to validate multiple versions, not just the head 17:50:50 stevvooe: shouldn't the structure we release in the specs be able to reflect all past versions? 17:52:07 mrunalp: for example, we currently switch back and forth between pointers. Maybe it will be easier after 1.0, but it would have been bad until now 17:52:23 RobDolinMS: what if we kept the schema in the spec but put the code in separate repos 17:52:51 stevvooe: I think either solution is fine, as long as maintainers are reviewing test code 17:53:09 stevvooe: And we want downstream issues to get pushed back up to the spec 17:53:41 mrunalp: not all maintainers are interested in the tool 17:55:41 the specification should stand on its own (without the test suite informing the interpretation) 17:56:14 stevvooe: as a maintainer, I feel a responsibility to transmit the intended meaning, and monitoring the test suite is a good way to double-check the spec language 17:56:27 philips: I don't feel like we're any closer to a conclusion 17:56:48 RobDolinMS: would it be useful to get TOB approval for optional test repositories for each spec if the TDC wants to 17:57:09 RobDolinMS: then it's not a TOB issue, and it's just a question of "what makes the most sense to the spec maintainers" 17:57:34 philips: I'm overallocated for one day. Can another TOB member make that proposal? 17:58:07 RobDolinMS: I'm not on the TOB, but I can help draft the proposal (maybe with mrunalp or crosbymichael?) and have John Grossman submit it 17:58:18 philips: sounds good 17:58:22 philips: I'm happy to review 17:58:28 Jason: I'm happy to review too 17:58:45 #action RobDolinMS to write up a TOB proposal for optional test repositories 17:58:56 #endmeeting