#acumos-meeting: Architecture Committee

Meeting started by farheen_cefalu_a at 14:03:39 UTC (full logs).

Meeting summary

  1. Challenges the team is facing for integrating jupyter and nifi (farheen_cefalu_a, 14:04:39)
    1. Mukesh: Based on discussions I talked to Vaibhav what can be done. We discussed docker compose. Next 7 days to deploy certificates. Login is still required. Password is arbitrary and has nothing to do with the acumos account. Brian: Avoid hacking jupyter hub code. That won't be supported in the k8 version. I am giving an option. Brian: You don't need to do the user authentication. It is not essential (farheen_cefalu_a, 14:10:41)
    2. Do we want to do a work around that will take time and will be throw away or do we ask the user to login separately. (farheen_cefalu_a, 14:11:55)
    3. We need a self-signed certificate to import either automated or ask the user to install manually. Per instance it will not happen every time. The password is selected by the user. Out of the box the user will be able to come back as long as that container is still running. Mukesh: Maximum we can put python code to authenticate the user. Token creation is a third option. Discuss with Vaibhav. Copy/paste (farheen_cefalu_a, 14:16:27)
    4. Mukesh: It is not feasible to pass the request to the jupyter hub without a login screen. Lack of expertise to do this with Docker compose. Even though we succeed in jupyter hub. It will not have nifi integration. The team is working on the k8 style of deployment that can use both jupyter and nifi. What is the effort that will take time? (farheen_cefalu_a, 14:18:45)
    5. No one has done the analysis that gets nifi to work under docker compose. The style or requirements of the platform needs nifi to be scalable. It will take time to figure out how to deploy nifi under docker. The CMLP has proven it as a deployment of k8. (farheen_cefalu_a, 14:19:47)
    6. Manoop: With docker style of deployment we will have a jupyter notebook. Minimumally we are going to get jupyter hub integrated with docker second plan is to have nifi and jupyter. Mukesh: There is a login screen will have to happen for the jupyter hub. (farheen_cefalu_a, 14:22:08)
    7. if it comes down to the user having to enter credentials twice then that is acceptable. (farheen_cefalu_a, 14:24:00)
    8. we need somebody to look into the single user authenticated user for the nifi. (farheen_cefalu_a, 14:25:24)
    9. Bryan: We have the ability to use nifi under the acumos platforms without credentials in a nifi registry. The first being able to authenticate the user. CMLP has brought somebody in who did a proof of concept. It won't work with Kong and then there is the backend question. How do we prepare the user in the nifi registry. It is deployable and can be used but not at the desired end state. (farheen_cefalu_a, 14:27:37)
    10. CMLP has an experiment that is running on a laptop. It seems to be working. There are a lot of adaption issues so I am not going to commit to a time frame. (farheen_cefalu_a, 14:29:10)
    11. Our plan or goal is to get jupyter and nifi for both installation. Nifi thru docker compose? Jupyter hub on docker based and nifi mvp for boreas. (farheen_cefalu_a, 14:30:42)
    12. Kazi: You will be able to deploy jupyter hub using docker compose. Mukesh: yes, it will take at least a week. (farheen_cefalu_a, 14:31:13)
    13. ACTION: Mukesh: Talk to Nat and bring it up on the TSC call an we will vote. We have to be careful that we balance end to end testing and include as much as we can with the feature list. This will require time from everyone to bring it into shape. (farheen_cefalu_a, 14:32:11)
    14. ACTION: my preference is to get features in with the timeline given. (farheen_cefalu_a, 14:33:01)

  2. AI4EU topic will be postponed to next week. (farheen_cefalu_a, 14:33:36)
    1. ACTION: Manoop: Add AI4U project to next week's architecture committee agenda. (farheen_cefalu_a, 14:34:10)
    2. ACTION: Manoop: Bring AI4EU to the TSC committee call. (farheen_cefalu_a, 14:34:42)
    3. Bryan has brought in minimum design and Clio related architecture. Can PTLs give me a heads up or planning for the next week? (farheen_cefalu_a, 14:35:14)

  3. Clio design (farheen_cefalu_a, 14:35:27)
    1. Anwar: Nothing gets moved to Clio without TSC approval. Skip this topic for this week. We should be at a level where Ofer has to collect the partners requirements. Let that process work itself out. (farheen_cefalu_a, 14:37:13)
    2. Manoop: We are aware of some that will be moved to Clio and the team are in green and can pick up work. (farheen_cefalu_a, 14:37:46)
    3. ACTION: Manoop have the items ready for next weeks call. (farheen_cefalu_a, 14:39:56)
    4. CMLP integration aspects are in conflict. These discussions should happen sooner or later. (farheen_cefalu_a, 14:42:18)
    5. https://wiki.acumos.org/display/AR/Clio+Architecture (farheen_cefalu_a, 14:42:34)
    6. For the deployment project there is a lot of work for Clio. (farheen_cefalu_a, 14:43:01)
    7. Anwar: Chris when can you start design Clio CDS. Acumos has adopted a shim to a certain kind of dbase. CMLP is using a licensed dbase. Data layers are not agreed upon. We are a significantly apart. Steve, Mukesh, Sayee, and Chris can present. We are at a time when we should start an agreed on final plan of the database design implementation so we know what our target will be. This will include changes (farheen_cefalu_a, 14:45:51)
    8. ACTION: Steve take the lead on bringing together the architecture for the integrated platform. Steve: Sayee was lining up to have the CDS abstraction used for the ML Workbench. What scratch pad dbase they want to use. Let's put a timeline and give both CMLP and Acumos side one week to converge. (farheen_cefalu_a, 14:48:54)

  4. Acumos ONAP integration (farheen_cefalu_a, 14:50:16)
    1. Model runners and client libraries in Acumos can start on that. (farheen_cefalu_a, 14:50:54)

  5. Scorecard review (farheen_cefalu_a, 14:51:08)
    1. ML Workbench: Rapid miner was not in the original scope and it's not integrated with Acumos which is out in the cloud vs. the users workstation. This was not on the radar. Zeppelin and Tensorboard were held over for deployed model architecture. This was related to kubeflow. (farheen_cefalu_a, 14:54:37)
    2. ACTION: Anwar: Discuss rapid miner offline. (farheen_cefalu_a, 14:55:32)
    3. We want Zeppelin from CMLP as a code base. Bryan: We never had an idea of what CMLP was going to deliver until February and then we had to write the code ourselves. (farheen_cefalu_a, 14:57:01)
    4. Manoop suggesting to not keep the design at levels below 100% so late in the development. There are still design and implementations that still have to be designed. (farheen_cefalu_a, 14:59:19)
    5. Deployment we didn't have the bandwidth to deliver. The deployment of the nifi pipeline is gated until we see what types of artifacts that we will deploying. The deployment of nifi pipelines will not be delivered this week. (farheen_cefalu_a, 15:02:34)


Meeting ended at 15:03:15 UTC (full logs).

Action items

  1. Mukesh: Talk to Nat and bring it up on the TSC call an we will vote. We have to be careful that we balance end to end testing and include as much as we can with the feature list. This will require time from everyone to bring it into shape.
  2. my preference is to get features in with the timeline given.
  3. Manoop: Add AI4U project to next week's architecture committee agenda.
  4. Manoop: Bring AI4EU to the TSC committee call.
  5. Manoop have the items ready for next weeks call.
  6. Steve take the lead on bringing together the architecture for the integrated platform. Steve: Sayee was lining up to have the CDS abstraction used for the ML Workbench. What scratch pad dbase they want to use. Let's put a timeline and give both CMLP and Acumos side one week to converge.
  7. Anwar: Discuss rapid miner offline.


People present (lines said)

  1. farheen_cefalu_a (38)
  2. collabot (3)


Generated by MeetBot 0.1.4.