14:02:33 <farheen_cefalu> #startmeeting Architecture Committee 14:02:33 <collabot> Meeting started Thu Mar 21 14:02:33 2019 UTC. The chair is farheen_cefalu. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:02:33 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 14:02:33 <collabot> The meeting name has been set to 'architecture_committee' 14:14:17 <farheen_cefalu> #topic Licensing Architecture review 14:15:51 <farheen_cefalu> #info Mukesh: Wireframes are a work in process for Licensing. Had a design discussion with Michelle and Bryan. Licensing onboarding is planned to be delivered in Sprint 4. 14:17:51 <farheen_cefalu> #info Michelle: We have to discuss adding a new tab on the model screen so you can view the model license where the signature tab is. Another tab will be added for license so you can view the artifact. We did use the license.txt in dev challenge. The design has changed and it is now a json file (artifact). 14:18:56 <farheen_cefalu> #topic Bryan reviewing overall Architecture Roadmap for Clio 14:20:25 <farheen_cefalu> #info Bryan reviewed roadmap 14:20:29 <farheen_cefalu> #link https://wiki.acumos.org/display/AR/Boreas+Architecture 14:21:39 <farheen_cefalu> #info Bryan: We have three deployments which will collapse into templates of how models are built and deployed. #action Team update the description field of #link https://wiki.acumos.org/display/AR/Boreas+Architecture 14:22:48 <farheen_cefalu> #info Bryan: We have an opportunity in Clio to hand jobs off to jenkins (CI process). #action Team we need to start having discussions about these. 14:25:42 <farheen_cefalu> #info Bryan: I recommend people start putting your components as they evolve. Anwar: Concern that the team will have bandwidth to get started with Clio discussions. What are the main top features in Clio? Bryan: For AT&T it's to merge with CMLP. Deploying a run time view and much more CI/CD design. We use jenkins to manage the pipelines. That is the number one feature for the deployment project and I recommend i 14:28:24 <farheen_cefalu> #info effort is large and the ci will improve. Anwar: CMLP is the internal Ai platform that is merging with Acumos. This will bring many more features to Acumos that will improve the user ability to create and share models. There are some other things that were targeted for Clio. The extensions of how we are going to do the data, deployment pipelines. We have to review our original plans and see if we should rev 14:29:14 <farheen_cefalu> #info Design studio which is a java component. Interactions will be over the network vs. inside the box. These features are coming and we need to start having those discussions. 14:29:25 <farheen_cefalu> #topic Feedback from PTLs 14:30:09 <farheen_cefalu> #info Anwar: What would PTLs like to see in terms of gaps and improvements. 14:30:38 <farheen_cefalu> #info Phillippe: No major gaps I can see. Perhaps next week. 14:33:08 <farheen_cefalu> #info Guy: I'm happy with the common services. I prefer we use jenkins. I would make ms generation tightly coupled with the onboarding app. Bryan: There's a dependency in their spring configuration that's why. Guy: The biggest most costly thing we do processing. The bottleneck needs to be cleared up by jenkins. I am in favor of jenkins and we have to make this work. We don't pull things out of the source repo t 14:35:09 <farheen_cefalu> #info Anwar: Enhancements besides jenkins? Guy: we need to expand to different kind of ms to build. Separating the ms build environment into more sophisticated. We still haven't gotten the test harness that will do code generation for you in using ms if they exists. Between deployment of ms generation we need to be better coordinate. Bryan: That will be addressed in rapid miner and nifi in Boreas and Clio. 14:38:00 <farheen_cefalu> #info Kazi: Design Studio. We will have nifi cluster that will give us a nifi environment for a single user. Do we have a nifi cluster? Bryan: Not clearly defined. How we are going to manage the single user container for nifi is not clear. Also we don't have the back end integration. In Boreas we have a lite integration. User is involved in onboarding of artifacts. Kazi: Kubeflow. In my humble opinion we shoul 14:40:48 <farheen_cefalu> #info chance. Sayee Training. We should do a good CI/CD. With respect to workbench we will have all he assets. We don't have a running model in Acumos today. That will help users connect nifi. This will help developers. Bryan: The run time view of models running will be a key feature in Clio. They will help developers to accomplish their goals. Nat: Sayee and Bryan do you mean run time embedded in Acumos or 14:41:15 <farheen_cefalu> #info run time extends beyond design time. Nat: So it is going to be connecting to a run time environment? Correct. 14:42:58 <farheen_cefalu> #info Nat: The point I was making is that every company has their own unique run environment. The model run time is unique case by case. We should give an option to connect to any of their run times then fine. Sayee: If I run a predictor in my enterprise I want to connect it. Both are feasible. The open source is contribution driven so we will focus on the common ground kubernetes. 14:44:52 <farheen_cefalu> #info Nat: We have azure openstack and k8. What is the intent for others? The process engine is jenkins and the recipe is a see template. Rather than encoding in java we develop it using jenkins job builder scripts. Will you duplicate that feature? We will go thru a deprication cycle. 14:45:51 <farheen_cefalu> #info Licensing Michelle: Major upgrades for Clio. Data Services Chris Lott: CMLP integration work. 14:46:40 <farheen_cefalu> #info Anwar: Let's finish Boreas strong. 14:46:47 <farheen_cefalu> #topic PTL Score card 14:48:14 <farheen_cefalu> #info Portal Marketplace: Mukesh: We are on track. We have taken one story out and replaced it with Catalog Management. Anwar: In terms of completion for sprint 4 and 5. any gaps? 14:48:50 <farheen_cefalu> #info Mukesh: Instead of global search we've replaced it with global search. 14:49:37 <farheen_cefalu> #info Anwar: the demos for sprint 4 should cover 80% or more in Sprint 4. If we don't show that we will be behind. 14:51:08 <farheen_cefalu> #info Phillippe Onboarding: We met with Bryan, Mukesh, and Vinayak. We agreed to use what Bryan suggested. I have created jira onboarding for docker proxy. 14:53:23 <farheen_cefalu> #info CDS Chris Lott: I put out a draft version of CDS for workbench and training team. Please look at it and give me feedback now. Since I started asking two and half weeks ago I haven't heard anything. Anwar: Does any PTL anticipate change in their database layer? No. Anwar: For any changes that come up set up a deadline and share with the PTLs. 14:55:25 <farheen_cefalu> #info Training Kayee: I am working on use case. Mukesh is working on the nifi integration with the ML workbench. We have Bryan working on Jupyter integration. I also did a little POC on the pipeline and workbench. That will help in deploy in pipeline. I am working on the deploy pipeline. Anwar: Where are we with finishing up the design? Kazi: Mukesh 14:57:44 <farheen_cefalu> #info Mukesh's team has not defined how the nifi will be integrated with the workbench. Bryan: How do we manage single user containers for nifi and zeppelin. Mukesh: We will give a demo tomorrow. We have not chalked out about nifi instances per user. Are we going to show one notebook? It is in the critical path. Sayee: We want to loosly integrate. Anwar: At least the team knows the bare minimum way. Sayee: We 15:00:36 <farheen_cefalu> #info for Clio. Anwar: Concern that we are still at 60%. We are at 80%. Kazi: On design studio is 90 - 95% but training we will keep at 60% because we don't know how to integrate with nifi and jupyter. Bryan: We have an option to have these services work on the workstation. Anwar: At least bare minimum we should do what the CMLP team has done for nifi and zeppelin. If we can make it better than that should be t 15:01:41 <farheen_cefalu> #info Anwar: Keeping it at 60%. Mukesh: issue with proxy. 15:02:00 <farheen_cefalu> #action Nat: Focus on Training deliverables. 15:03:49 <farheen_cefalu> #info Bryan: Security Kazi did a POC of a program in a nifi container. That is to be worked out as a design for the kubernetes client. We have the framework we have to make the template modifications. The other is flinc kafka etc including filebeat. There are at least 3 significant thing that has to be done. That is an open area for deployment. Security verification is coming together. We have the design persp 15:04:28 <farheen_cefalu> #info but we have something. Licensing: Michelle: We should keep it as is. 15:04:33 <farheen_cefalu> #endmeeting