14:03:39 <farheen_cefalu_a> #startmeeting Architecture Committee
14:03:39 <collabot> Meeting started Thu Apr 18 14:03:39 2019 UTC.  The chair is farheen_cefalu_a. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:39 <collabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:03:39 <collabot> The meeting name has been set to 'architecture_committee'
14:04:39 <farheen_cefalu_a> #topic Challenges the team is facing for integrating jupyter and nifi
14:10:41 <farheen_cefalu_a> #info Mukesh: Based on discussions I talked to Vaibhav what can be done.  We discussed docker compose.  Next 7 days to deploy certificates.  Login is still required.  Password is arbitrary and has nothing to do with the acumos account.  Brian: Avoid hacking jupyter hub code.  That won't be supported in the k8 version.  I am giving an option.  Brian: You don't need to do the user authentication.  It is not essential
14:11:55 <farheen_cefalu_a> #info Do we want to do a work around that will take time and will be throw away or do we ask the user to login separately.
14:16:27 <farheen_cefalu_a> #info We need a self-signed certificate to import either automated or ask the user to install manually.  Per instance it will not happen every time.  The password is selected by the user.  Out of the box the user will be able to come back as long as that container is still running.  Mukesh: Maximum we can put python code to authenticate the user.  Token creation is a third option.  Discuss with Vaibhav.  Copy/paste
14:18:45 <farheen_cefalu_a> #info Mukesh: It is not feasible to pass the request to the jupyter hub without a login screen.  Lack of expertise to do this with Docker compose.  Even though we succeed in jupyter hub.  It will not have nifi integration.  The team is working on the k8 style of deployment that can use both jupyter and nifi.  What is the effort that will take time?
14:19:47 <farheen_cefalu_a> #info No one has done the analysis that gets nifi to work under docker compose.  The style or requirements of the platform needs nifi to be scalable.  It will take time to figure out how to deploy nifi under docker.  The CMLP has proven it as a deployment of k8.
14:22:08 <farheen_cefalu_a> #info Manoop: With docker style of deployment we will have a jupyter notebook.  Minimumally we are going to get jupyter hub integrated with docker second plan is to have nifi and jupyter.  Mukesh: There is a login screen will have to happen for the jupyter hub.
14:24:00 <farheen_cefalu_a> #info if it comes down to the user having to enter credentials twice then that is acceptable.
14:25:24 <farheen_cefalu_a> #info we need somebody to look into the single user authenticated user for the nifi.
14:27:37 <farheen_cefalu_a> #info Bryan: We have the ability to use nifi under the acumos platforms without credentials in a nifi registry.  The first being able to authenticate the user.  CMLP has brought somebody in who did a proof of concept.  It won't work with Kong and then there is the backend question.  How do we prepare the user in the nifi registry.  It is deployable and can be used but not at the desired end state.
14:29:10 <farheen_cefalu_a> #info CMLP has an experiment that is running on a laptop.  It seems to be working.  There are a lot of adaption issues so I am not going to commit to a time frame.
14:30:42 <farheen_cefalu_a> #info Our plan or goal is to get jupyter and nifi for both installation.  Nifi thru docker compose?  Jupyter hub on docker based and nifi mvp for boreas.
14:31:13 <farheen_cefalu_a> #info Kazi: You will be able to deploy jupyter hub using docker compose.  Mukesh: yes, it will take at least a week.
14:32:11 <farheen_cefalu_a> #action Mukesh: Talk to Nat and bring it up on the TSC call an we will vote.  We have to be careful that we balance end to end testing and include as much as we can with the feature list.  This will require time from everyone to bring it into shape.
14:33:01 <farheen_cefalu_a> #action my preference is to get features in with the timeline given.
14:33:36 <farheen_cefalu_a> #topic AI4EU topic will be postponed to next week.
14:34:10 <farheen_cefalu_a> #action Manoop: Add AI4U project to next week's architecture committee agenda.
14:34:42 <farheen_cefalu_a> #action Manoop: Bring AI4EU to the TSC committee call.
14:35:14 <farheen_cefalu_a> #info Bryan has brought in minimum design and Clio related architecture.  Can PTLs give me a heads up or planning for the next week?
14:35:27 <farheen_cefalu_a> #topic Clio design
14:37:13 <farheen_cefalu_a> #info Anwar: Nothing gets moved to Clio without TSC approval.  Skip this topic for this week.  We should be at a level where Ofer has to collect the partners requirements.  Let that process work itself out.
14:37:46 <farheen_cefalu_a> #info Manoop: We are aware of some that will be moved to Clio and the team are in green and can pick up work.
14:39:56 <farheen_cefalu_a> #action Manoop have the items ready for next weeks call.
14:42:18 <farheen_cefalu_a> #info CMLP integration aspects are in conflict.  These discussions should happen sooner or later.
14:42:34 <farheen_cefalu_a> #link https://wiki.acumos.org/display/AR/Clio+Architecture
14:43:01 <farheen_cefalu_a> #info For the deployment project there is a lot of work for Clio.
14:45:51 <farheen_cefalu_a> #info Anwar: Chris when can you start design Clio CDS.  Acumos has adopted a shim to a certain kind of dbase.  CMLP is using a licensed dbase.  Data layers are not agreed upon.  We are a significantly apart.  Steve, Mukesh, Sayee, and Chris can present.  We are at a time when we should start an agreed on final plan of the database design implementation so we know what our target will be.  This will include changes
14:48:54 <farheen_cefalu_a> #action Steve take the lead on bringing together the architecture for the integrated platform.  Steve: Sayee was lining up to have the CDS abstraction used for the ML Workbench.  What scratch pad dbase they want to use.  Let's put a timeline and give both CMLP and Acumos side one week to converge.
14:50:16 <farheen_cefalu_a> #topic Acumos ONAP integration
14:50:54 <farheen_cefalu_a> #info Model runners and client libraries in Acumos can start on that.
14:51:08 <farheen_cefalu_a> #topic Scorecard review
14:54:37 <farheen_cefalu_a> #info ML Workbench: Rapid miner was not in the original scope and it's not integrated with Acumos which is out in the cloud vs. the users workstation.  This was not on the radar.  Zeppelin and Tensorboard were held over for deployed model architecture.  This was related to kubeflow.
14:55:32 <farheen_cefalu_a> #action Anwar: Discuss rapid miner offline.
14:57:01 <farheen_cefalu_a> #info We want Zeppelin from CMLP as a code base.  Bryan: We never had an idea of what CMLP was going to deliver until February and then we had to write the code ourselves.
14:59:19 <farheen_cefalu_a> #info Manoop suggesting to not keep the design at levels below 100% so late in the development.  There are still design and implementations that still have to be designed.
15:02:34 <farheen_cefalu_a> #info Deployment we didn't have the bandwidth to deliver.  The deployment of the nifi pipeline is gated until we see what types of artifacts that we will deploying.  The deployment of nifi pipelines will not be delivered this week.
15:03:15 <farheen_cefalu_a> #endmeeting