#opendaylight-group-policy log

15:00:18 <alagalah> #startmeeting GBP 9/9
15:00:18 <odl_meetbot> Meeting started Wed Sep  9 15:00:18 2015 UTC.  The chair is alagalah. Information about MeetBot at http://ci.openstack.org/meetbot.html.
15:00:18 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:18 <odl_meetbot> The meeting name has been set to 'gbp_9_9'
15:00:23 <alagalah> #info alagalah
15:01:37 <alagalah> #topic Agenda bashing
15:01:46 <alagalah> #link meeting held on hangouts: https://talkgadget.google.com/hangouts/_/gz4g7iazv6vburqjnrtkyjknmea
15:02:00 <alagalah> #link https://wiki.opendaylight.org/view/Group_Based_Policy_(GBP)/Team_Meeting Agenda
15:02:46 <alagalah> #topic M2 milestone delivered
15:03:25 <alagalah> #info M2 was delivered last Thu, the main point was the final RP
15:03:46 <martin-sunal> #info martin-sunal
15:03:54 <alagalah> #topic NEutron NPE bug
15:04:08 <alagalah> #info dileep_r on Hangout says he was able to reproduce it?
15:04:29 <alagalah> #info alagalah: patch merged to stable lithium yesterday and master too I believe
15:05:24 <alagalah> #info please note there is an issue OVSDB currently that causes NPE in InventoryHelper. alagalah logged a bug, martin-sunal was unable to reproduce but dileep_r can....
15:05:39 <alagalah> #info fix is to: 1. Pull OVSDB stable-lithium-SR1 (important: SR1)
15:06:14 <alagalah> #info 2. build by either removing OVSDB from .m2/repository/org/opendaylight OR you could try building with -U ... is mvn clean install -U -DskipTests
15:06:37 <alagalah> #info 3. Pull GBP stable/lithium and build: mvn clean install -nsu -DskipTests
15:07:41 <alagalah> martin-sunal, It seems dileep can reproduce the issue I was seeing
15:08:13 <alagalah> martin-sunal, He is using openJDK1.7, I am using openJDK1.8
15:08:26 <martin-sunal> alagalah: I removed m2 and I will try again
15:08:27 <alagalah> martin-sunal, Not that it should matter....
15:08:32 <alagalah> martin-sunal, Thanks sir!
15:08:59 <alagalah> #info Note this is not the NPE bug I was talking about in the topic
15:10:36 <alagalah> #info NPE bug I was referring to which was patched in GBP stable and master, was when we add the chain action to neutron generated policy, we would NPE due to VXLANGPE port not getting added to augmentation. This has been resolved!
15:10:56 <alagalah> #topic Be Release plan - multirenderer SDK
15:11:02 <alagalah> #info khaldoon you around ?
15:11:10 <khaldoon> yes
15:11:47 <alagalah> #info khaldoon mentioned that he had a policyInfo YANG model that he will push... khaldoon can you expand ?
15:12:56 <khaldoon> Yes, I have the Yang model
15:13:27 <alagalah> khaldoon, Excellent, excited to see it.
15:13:28 <khaldoon> I can convert PolicyInfo to the YANG and store it in Datastore
15:13:47 <khaldoon> of-overlay can also listen to changes
15:13:48 <alagalah> khaldoon, Any ETA on when a patch (either draft or published) may be able to be pushed for review ?
15:14:23 <khaldoon> Also, the existing solution should still work until we happy with the new solution
15:14:41 <khaldoon> This week
15:14:42 <alagalah> khaldoon, Thats great. This ties in nicely with the next topic
15:15:09 <alagalah> #topic Multirenderer SDK - IOVisor agent
15:15:31 <alagalah> #info khaldoon's work ties in nicely with the timing of the next item, which is an IOVisor agent.
15:16:46 <alagalah> #info The IOVisor project (iovisor.org) leverages eBPF. I am on the TSC for the IOVisor project, and as a POC, will start work on an IOVisor Module for taking policy from ODL GBP and applying it to the linux kernel networking pipeline via IOVisor
15:17:38 <alagalah> #info this will be "another" renderer but will be very lightweight from the ODL GBP side I hope, in that we need to potentially serialize policy as YAML to the agent running on the linux host. Hence khaldoon's work on making policyInfo available in the datastore is key
15:17:52 <alagalah> Any questions on that ?
15:18:27 <alagalah> #info This is meant as a POC to showcase the capabilities of IOVisor and eBPF
15:19:20 <alagalah> Ok if there are no questions I'll move onto the next topic
15:19:42 <alagalah> #topic Instrumentation
15:20:59 <alagalah> #info The 3 things we want to achieve with Instrumentation:
15:21:21 <alagalah> #info 1. OAM 2. Time-series data a) immediate b) historical/trend
15:22:27 <alagalah> #info #2 is proving to be a tricksy hobbit. Just using EP data is not going to yield the results we want, and the alternative is a bunch of interfaces for every object.. Ideally we could have leveraged simple data and derive stats, but thats a bit of a fools errand...
15:22:39 <alagalah> #info as discussed last week, the best way forward appears to be sFlow
15:22:58 <alagalah> #info since this is an open standard, this *should* be ok with FaaS as well as OVS.
15:23:07 <alagalah> #info that was Yapeng's opinion on the last call.
15:23:15 <alagalah> #info The big issue here is the collector
15:24:09 <alagalah> #info where would it live? Would it be external? It maybe something someone wants to take a look at as there are existing sFlow collectors available we could play with while we see what sort of data it can pull, and how we map that to policy constructs, before we start worrying about whether we want an sFlow collector in the ODL datastore.
15:24:30 <alagalah> #info unless someone wants to take that on, it seems the next sensible step is OAM.
15:25:02 <alagalah> #info I have some thoughts on leveraging our existing packet Out/In capability we use for neutron external gateway ARP to achieve this. Is anyone interested in working on this with me ?
15:26:45 <alagalah> #topic Stable lithium update - CSIT
15:27:50 <alagalah> #info There appears to have been some confusion about how we progress with Ubuntu in CSIT. We were under the impression that Vagrantfiles would be provided, but it appears we need to provide our own, which makes sense. We have them. Trying to organise a meeting today with "rel-eng" to get some pointers on how to start pushing our Vagrantfiles for GBP-neutron and GBPSFC demo environments up into CSIT.
15:28:51 <alagalah> #info dileep wants to work on CSIT, so hopefully he and I can get some help this morning to kick this off from edwarnicke  et al
15:31:03 <alagalah> #info we could leverage the GBPSFC demo branch that uses a pre-built BOX so we aren't trying to provision everything from SHELL (40+min per VM) ....
15:31:17 <alagalah> #link https://github.com/alagalah/gbpsfc-env/tree/box GBP SFC demo that leverages prebuilt box
15:33:52 <alagalah> #info note that this box has the OVS pre-installed. There were issues where the install required more RAM but with the BOX I believe since OVS NSH is already installed, we can drop this to 512M RAM which will be nicer for CSIT
15:34:32 <alagalah> #info I have tested successfully the same approach for neutron integration ... ie I have a box for devstack-control and a box for devstack-compute with the right repos etc already installed.
15:35:28 <alagalah> #info this means you can (from ~/devstack) do "restack.sh" and 4min (control) 1min (compute) have the system up and ready then run through the scripts already in the path (ie step01.sh etc) /vagrant/devstack-scripts/tutorial/
15:36:27 <alagalah> #info The issue with PBR requirements with devstack stable/kilo has been resolved, so we have a couple of avenues to pursue....
15:37:58 <alagalah> #info 1. A fixed BOX configuration for DevStack. This will have the downside that it won't have all the latest updates for stable/kilo but for base neutron testing/development will be ideal. I believe there is a branch for GBP devstack called "newbox" BUT PLEASE DO NOT USE THIS YET
15:38:04 <alagalah> #link https://github.com/alagalah/gbp-devstack/tree/newbox new box branch
15:38:16 <alagalah> #info this is still under development and should be ready next week.
15:38:49 <alagalah> #info with this in place, we can have a locked down devstack that we can use not only for development but also for CSIT since it should remain invariant.
15:39:21 <alagalah> #info the next thing we need to progress on is on pulling both the latest stable/kilo and latest master liberty and testing those for upstream compatibility / breakages.
15:39:53 <alagalah> #info our current thinking is that the OPNFV project already has test environments that include openstack Tempest testing so we are in the works of leveraging those environments.
15:40:49 <alagalah> #topic Stable/lithium SFC+neutron working
15:41:06 <alagalah> #info yesterday with the tunnel NPE bug fixed I was able to get two NOVA instances talking over SFC!!!!
15:41:10 <alagalah> #info woot!!!
15:41:31 <alagalah> #info The NPE patch has been merged but that leads onto the next topic for martin-sunal ...
15:41:40 <alagalah> #topic neutron-mapper redesign
15:41:56 <martin-sunal> #info draft was created
15:42:02 <alagalah> #info as some of you may know, we have mapped neutron security groups to EPGs
15:42:07 <alagalah> martin-sunal, hold on mate
15:42:14 <martin-sunal> alagalah: ok :)
15:42:35 <alagalah> #info A Security Group rule is more like an ACL, ie one sided... but an EPG and contract needs an EPG to provide and an EPG to consume
15:43:08 <alagalah> #info to make policy resolution happen, each SecurityGroup mapped to an EPG and its Security Group Rules mapped to a contract that the SecurityGroup/EPG PROVIDED.
15:43:28 <alagalah> #info this contract was CONSUMED by an EPG known as the ANY group.
15:44:30 <alagalah> #info this worked reasonably well in the simple case, but as we tested more complex scenarios with SecurityGroupRules that used either ip-prefix or "remove-security-group" we found instances where we could get asymmetric policy resolution. This has been an outstanding bug since before Li release.
15:44:48 <alagalah> #info to that end, martin-sunal has been working ona solution and has a draft that is almost ready for review
15:44:56 <alagalah> martin-sunal, Could you perhaps expand on the design here?
15:45:28 <martin-sunal> #info mapping between neutron and GBP entities is much easier
15:46:04 <martin-sunal> #info each security group rule is represented as a contract with one subject
15:46:50 <martin-sunal> #info a subject has one rule which contains classifier and action reference
15:47:25 <martin-sunal> #info classifier ref points to classifier-instance created based on attributes of security group rule
15:47:44 <martin-sunal> #info action-instance is always Allow b/c neutron has only this action
15:48:27 <martin-sunal> #info a contract is always provided by EPG which represents security-group from security-group-rule
15:48:57 <martin-sunal> #info we are testing this implementation now
15:49:18 <martin-sunal> #info I expect to have it done and publish at the end of this week?
15:49:23 <alagalah> martin-sunal, Great stuff mate, so the result is that the automatically generated policy is much easier to read in the UX ?
15:50:13 <martin-sunal> #info yes, all attributes from classifier-instance are used in naming convention of rule, clause, etc.
15:51:50 <alagalah> #info cool so instead of seeing a UUID, we see the actual components of the classifier in the name ?
15:52:01 <martin-sunal> #info ip-prefix from security-group-rule is mapped as endpoint-identification-constraint in clause
15:53:16 <martin-sunal> #info UUID is still there - contract does not have name but ID
15:53:53 <martin-sunal> #info everything what has name as key contains text representing classifier
15:55:07 <alagalah> #info Ok thats very cool!
15:55:22 <alagalah> #info we are having a side conversation on the google hangout, and dileep asked about conflicts
15:55:23 <martin-sunal> #info even name of contract-named-selector is composed from classifer which is inside contract
15:55:57 <alagalah> #info ie if we have SG1 and SG2 with each having a rule TCP 80, would there be one or two classifiers in that case ?
15:56:47 <martin-sunal> #info there would be 2 classifiers
15:57:07 <martin-sunal> #info sorry, there is one classifier-instance
15:57:26 <martin-sunal> #info but 2 contracts using that classifier-instance
15:57:26 <alagalah> #info I think thats great
15:57:37 <alagalah> #info ack, thats what I thought... so in the case where we have that
15:58:14 <alagalah> #info two contracts sharing a classifier, if SG2 were to say change its rule from TCP 80 to TCP 80-100 would we create a new classifier and associate it with the contract and hence have two classifiers ?
15:58:38 <martin-sunal> #info exactly
15:59:11 <alagalah> #info Very cool, and how do you know a classifier is still in use? ie do you do "garbage collection" or is this an item of optimisation for later ?
15:59:55 <martin-sunal> #info classifier-instance is removed immidietely after it is not used anymore
16:00:01 <alagalah> #info dileep Asks on hangout, do we expect these things to only be changed via neutron or via the ODL GBP GUI ... alagalah prepares a long answer :)
16:00:09 <alagalah> #info nice answer martin-sunal
16:01:07 <alagalah> #info 1. Neutron expects to be single-writer
16:01:09 <martin-sunal> #info changing of entities which were created based on neutron state is considered as illegal state
16:01:49 <alagalah> #info 2. As Martin points out, we would consider this illegal state but since we don't have AuthZ/AuthN on parts of the datastore (YET), this is something you can do
16:02:19 <alagalah> #info 3. In order to support SFC, we DO modify the ACTION from ALLOW to CHAIN-"foo" (tested yesterday) in order to make this work since Neutron doesn't support SFC (yet)
16:02:28 <alagalah> #info martin-sunal Anything to add ?
16:03:00 <martin-sunal> #info I think that basicly I told everything
16:03:47 <alagalah> #info Ok thats great... so I best wrap this up since its top of the hour
16:04:04 <alagalah> #info we will be presenting more on this in the next meeting
16:04:35 <alagalah> #info ok with that, lets wrpa this up... thanks everyone for joining
16:04:43 <alagalah> #endmeeting