13:59:51 <farheen_cefalu_a> #startmeeting Architecture Committee 13:59:51 <collabot> Meeting started Thu May 9 13:59:51 2019 UTC. The chair is farheen_cefalu_a. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:59:51 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 13:59:51 <collabot> The meeting name has been set to 'architecture_committee' 14:04:02 <farheen_cefalu_a> #topic New Agenda Items 14:04:38 <farheen_cefalu_a> #info Bryan - Please add License Renewal to the agenda 14:05:48 <farheen_cefalu_a> #link https://lists.deeplearningfoundation.org/g/acumosai-comm 14:06:11 <farheen_cefalu_a> #info Michelle - Justin (Ericsson) put together the licensing demo. 14:06:35 <farheen_cefalu_a> #info Before we get into the demo please describe the POC or demo. 14:06:49 <farheen_cefalu_a> #topic 14:07:04 <farheen_cefalu_a> #topic Licensing demo 14:07:38 <farheen_cefalu_a> #info Justing Model runner publishes to the subscriber. We wanted to get usage metrics on how many times a model is consumed and report elk stack. 14:09:45 <farheen_cefalu_a> #info we used the face detect model from the soup to nuts. We enabled one feature to track the model and activity off of ELK stack off of the Acumos platform. Be able to report not just on the consumer instance of Acumos but report back to the supplier through federation gateway 14:09:50 <farheen_cefalu_a> #link https://wiki.acumos.org/display/LM/%5BDRAFT%5D+Model+Usage+Tracking+%28Federation%29+Proof+of+Concept+-+Boreas 14:11:02 <farheen_cefalu_a> #info Justin and Parag demonstrated the components involved in the link above. 14:13:49 <farheen_cefalu_a> #info demonstrating two instances a publisher and a subscriber. 14:18:35 <farheen_cefalu_a> #info created a kiban dashboard acumos-model-usage-logs. Based on the index you can go to Visualize and Dashboard to see the usage count "subscriber ->/detect - Count. Check the wiki page to get the definitions. 14:21:18 <farheen_cefalu_a> #info aio2 (att) is subscriber aio1 is publisher (ericsson). Model runner between the two instances. I am going to call the detect function. I am going to select the face detect image and will see the usage count change in the kibana dashboard. It has a subscriber count of 1 and subscriber count of 1. The All in one is supplier or publisher aio2 is the subscriber. ai02 has federated has pulled from the ai01. 14:22:55 <farheen_cefalu_a> will be a component that will track all the models or per model? Currently available as part of Acumos. In Admin interface Login to Dashboard. 14:25:39 <farheen_cefalu_a> #info Bryan: Don't assume that all users will have access to dashboard only the admin. Will this be per model or all the models? This will be one component that will show all the models. Yes, when you deploy acumos you get a configuration that you can import into your kibana. We can export the configuration. The configuration has mapping to all the platforms. The admin has to manually set up the dashboard and 14:26:36 <farheen_cefalu_a> #info we should let the developer should be able to assign a configuration. The admin should be able to configure per model. 14:28:26 <farheen_cefalu_a> #info Michelle - It does not set this for all the model. It would be based on the agreement between the supplier and subscriber. If they choose to agree to have the model tracking between mode subscriber and supplier then they have the option to use. Manoop - There is a trigger if tracking of a model is needed then all the files will be created for that model. Currently we are configuring files where model solut 14:30:53 <farheen_cefalu_a> #info generating wrt kibana it's the reference to the elastic index needs a set of user components and each component has an identifier. Prag did you want to look at the fields? kibana/Discover being reviewed: you see in the table view and pure json format. This is the entry from elastic index and putting that log into... timestamp are being used by filebeat but response remoteInfo we added as a part of log gener 14:33:01 <farheen_cefalu_a> #info response request size and model meta data solution id, version, etch. There is different data that one can use to see where that model has been generated. Different set of information that we collect is captured in the wiki page. Right now across two instances supplier/subscriber but if we have many peers there are certain issues that we noticed with the CVS data entry. Please see the open questions isn t 14:33:36 <farheen_cefalu_a> #info What if we collect information from he 5th peer when subscribed to peer1? 14:34:29 <farheen_cefalu_a> #info We want to handle log entries to address any failures. Should we package the usage logs to CDS? Do we need to include the license data in the usage log? 14:35:31 <farheen_cefalu_a> #info currently, admin becomes tht owner for subscribed model - does it need to keep the original/publisher info -for reporting/filter. 14:37:43 <farheen_cefalu_a> #info referring to the data flow from model runner to model supplier to the model publisher. the federation gateway. We added an api to forward the log from the acumos subscriber instance to the supplier instance. So when I make a call to the model runner the log goes to the supplier. 14:38:26 <farheen_cefalu_a> #info Questions? Michelle - for your open questions we can address at our meeting. The usage manager is going to need the logging data. 14:39:21 <farheen_cefalu_a> #info from the overall flow the model runner filebeat and the persisted volume we are adding additional model usage tracking. What level of overhead are we adding on top of a raw model. How much size of the docker image has increased. 14:42:51 <farheen_cefalu_a> #action Prag please provide resource overhead. Expectations are in ONAP some policy related we want to run a model. Be specific about the overhead requirements with profiles that you are expecting. Then we can give the overall and the percentage of overhead that we are expected to support. Kafka and flink will add more overhead. Someone needs to think ahead and give us the requirements. Irrespective we can pro 14:45:18 <farheen_cefalu_a> #info of the docker file? We are not changing the model we are using a proxy. So the docker image is not modified at all. Manoop - But do we deploy the model and nothing else? There are 2 other njex proxy and filebeat. It is a cron job. It is out of the cycle. Would we deploy services for every model? In deploy to local it is optional. On by default but can turn off. Lets say k8 and I have another model wi 14:46:46 <farheen_cefalu_a> #info per model. Models today contain 3 or 4 pods. Therefore a model we are adding overhead. We should put some details around overhead usage. 14:48:34 <farheen_cefalu_a> #info Guy- It is still possible to deploy a bare usage model. It still reacts the same way so if you have a quick application and you want to reduce latency and dont care about logging you can do it. If it is licensed model it is up to the user to adhere to the supplier. Acumos does not enforce. Making it available to the suppliers and subscribers. Are you using CDS to store this tracking information? No it 14:51:09 <farheen_cefalu_a> #info We push to logstash in the elk stack. Why are we not storing in CDS? What are the critical usage tracking to see the counts. Then you may need CDS to store this information. We don't have to use an out of box. It is up to the user. If you want an out of box solution you can keep the logs and send it to the end point. It keeps track of the last usage entry it has posted to the endpoint. It helps with fai 14:51:47 <farheen_cefalu_a> #info and efficient and out of box kibana to query data. Is there value in storing CDS. 14:53:08 <farheen_cefalu_a> #info Bryan- some summarized items that would add processed information can be discussed for the future release. 14:55:58 <farheen_cefalu_a> #info Doesn't need to happen but something to consider. Manish- sales are trying to have a marketplace with subscribed customer. Assuming one model is up and running and getting the logs. Scalability of the logs will scale up and down so when we design it we have to address. This was developed on an assessment environment. 14:57:12 <farheen_cefalu_a> #info I noticed that you are subscribing per model vs. subscribing per catalog. Michelle this was a proof of concept for Boreas we will address in future Clio release. 15:00:19 <farheen_cefalu_a> #topic license renewal. Please read email. If we don't have a reasonable way to update a license of a model. There is no way to upload additional artifacts. If this was a document yes, but there was a request to change it to an artifact. There is no way to update an artifact. Mukesh - if the license is not onboarded then you have the ability to upload the license with the artifact. 15:01:14 <farheen_cefalu_a> #info this is a new key. Do we stop the workflow all together? This was something the portal implemented. And its told do not proceed. So if license.txt is not availble will security verification stop? 15:04:09 <farheen_cefalu_a> #info What is required when is completely configurable. Nobody can stop anything. Admin can configure no models except license. And currently there is no way to onboard the license with. It's already designed to turn flags on or off. Mukesh - has to be a model artifact because I have not added license.txt. Bryan security verification makes the decision. Licens json will occur part of onboarding and publicatio 15:06:16 <farheen_cefalu_a> #info to remove the add a license. Bryan don't remove that but be sure to save as an artifact. Separately in manage my model i have the ability to upload an artifact. If I want to add I use manage my model screen. If that's the place where you have already implemented an add a license dialogue. Where we view the license that is where we take care of it there. Bryan and Mukesh have an understanding. 15:06:42 <farheen_cefalu_a> #action Mukesh update the email about this resolution. 15:06:45 <farheen_cefalu_a> #endmeeting