14:03:36 <farheen_cefalu_a> #startmeeting Architecture Committee
14:03:36 <collabot`> Meeting started Wed Jul 24 14:03:36 2019 UTC.  The chair is farheen_cefalu_a. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:36 <collabot`> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:03:36 <collabot`> The meeting name has been set to 'architecture_committee'
14:08:19 <farheen_cefalu_a> #info Agenda - Starting next week Manoop will send the agenda one day in advance with the agenda topics for that week. Any other topics on the call?
14:09:50 <farheen_cefalu_a> #info Bryan - at some point we are going to bring together the jenkins / camunda approach.  #action Bryan Sullivan bring proposal to this meeting in the weeks ahead.
14:10:24 <farheen_cefalu_a> #topic OAM - Parichay regarding Platform deployment as Jenkins as a Service.
14:15:10 <farheen_cefalu_a> #info Overall platform from two approaches.  One approach is Docker compose and Kubernetes with Helm.  Docker compose pre-requisites deploy core components portal fe, be, ms, and the configurations for non core components like mariadb and elk stack.  And optional installation jupyterhub and Nifi. Both approaches are using Docker containers they dif
14:15:11 <farheen_cefalu_a> fer are the containers using exclusive docker compose.  Using Helm charts for K8.  We will continue to support Docker compose.  For Clio Helm charts K8 needs to be running.
14:22:45 <farheen_cefalu_a> #info Sayee: When you deploy core component using helm chart.  Does every component run on a single container or multiple containers.  Parichay - All this will be in a single pod or you can define it in a separate pod.  Bryan we have to design this so every service can be distributed across a k8 cluster.  It would benefit to have everything as a po
14:22:46 <farheen_cefalu_a> d.  Single container is for a single pod.  Sayee - A certain service will be running one container.  Onboarding uses less traffic.  When you have multiple components in one container you will have port issues.  Bryan - under k8 can use exactly the same port externally because they are fronted by a service.  In K8 we don't have to worry about port c
14:22:46 <farheen_cefalu_a> onflicts.  As long as the service name is different?  Manoop - We are agreeing to having a single pod per service.  There will be no node ports.  Bryan is working on it for Clio.  The one component I'm having a problem is Federation.  How can federation be front ended by an ingress controller.  Once the entire platform is deployed what will be port
14:22:47 <farheen_cefalu_a> s in bytes by command line?  There will be no ports.  You access the platform through https at the domain name.  Every service has to have a context path with the URL.
14:23:58 <farheen_cefalu_a> #info Questions in chat from Justin asking what version of HELM?  Latest version of HELM.  Using Jenkins to deploy the HELM charts.
14:24:26 <farheen_cefalu_a> #info Chat questions: question- what version of helm is being targeted?  there are risks of having tiller installed in helm? In helm v3 it appears that tiller is being removed (v3 is in alpha right now)
question - is there any CI testing of helm being considered? such as - https://github.com/helm/chart-testing.
14:33:31 <farheen_cefalu_a> #info bryan - I use CI testing to deploy HELM.  #action Parichay post the links to the HELM.  Justin - there are issues with tiller.  You are a super user that is being addressed.  For CI testing those tests are not available in open source.  Can you post those tests for the open community.  Bryan - I have recognized that jenkins and built jobs wil
14:33:32 <farheen_cefalu_a> l be published.  I will publish them as a part of Boreas release.  the charts are stored under system integration and the all in one.  #action Bryan Sullivan post the test scripts on the wiki.  Bryan - They are lightly parameterized.  Justin - I discovered the workflow isn't clear.  If a new version is release of a docker image then we are unclear
14:33:32 <farheen_cefalu_a> on which version to use.  Bryan - There are cross component dependencies and configuration file.  The intent is you should be able to redeploy every component individually.  The lifecycle management of the platform using these tools is something we could improve.  Justin - example we publish a new docker image.  #action Bryan update the documentati
14:33:33 <farheen_cefalu_a> on for the workflow of the AIO.
14:33:42 <farheen_cefalu_a> #topic Archival Process
14:35:36 <farheen_cefalu_a> #info Mukesh - The API should be asynchronous and synchronous responses.  Second we need to have a notification process.  These apis from the backend should run both asynchronously as well as synchronously.
14:37:18 <farheen_cefalu_a> #info we are archiving the Elastic search database logs.  The tables from the logs grow and have to be archived to clear up disc space.  Presentation of logs and dashboards through kibana.
14:39:26 <farheen_cefalu_a> #info For that archival api you can do it in batch or real time.  The notification tells the user that the job is complete.
14:42:43 <farheen_cefalu_a> #info Byran who will be our developer for the ELK client?  Amit is still there but Sumit will be taking over.  We had a wireframe review.  #action Parichay make sure that synchronous turned on.  This is similar to what onboarding does to asynchronously do a call.  Manoop - you need a user story to track this item.  #action Parichay create a user st
14:42:44 <farheen_cefalu_a> ory to enable both asynch and synch processes on the back end.  Mukesh - We have the user stories.  This is critical.
14:44:51 <farheen_cefalu_a> #info design is fair.  Tausif - this should be at the API level.  Will be send a flag?  Elastic search has a true / false flag.  You can decide using logstash server.  There are two approaches.  You can put an array list.  #action Parichay discuss with Tausif.
14:47:15 <farheen_cefalu_a> #info Sayee - We are going to write a log locally and then locally we push to ELK stack?  When a model is created is it written directly or is there a cron job?  Bryan - How do the logs get into elastic search.  filebeat monitors persistent volume and synchs to logs.  Parichay - archival backup and cleanup.
14:48:41 <farheen_cefalu_a> #info Mukesh - security of archival process?  Swagger is not exposed to anyone in Acumos environment.  You will not be able to access it in a production environment.  Bryan - I am working on tools that will expose the Swagger interface.  You shouldn't need it in a production environment.
14:49:13 <farheen_cefalu_a> #topic Michelle License Artifacts
14:50:45 <farheen_cefalu_a> #info Debate of license should be stored in Nexus Acumos or LUM?  We will continue to have the artifacts stored in Nexus as it is today.  Have the scan results provided to the LUM.  We will be discussing further.  Keep the artfact in Acumos Nexus.  We will store the scan results in the LUM.
14:52:50 <farheen_cefalu_a> #info It represents values in an api.  SV will provide the key attributes that the LUM needs.  We are going to provide the results from he security verification.  Justin - we have a draft of the sequence of where it is stored.  It will go through the License manager library that will go to LUM.
14:54:51 <farheen_cefalu_a> #action Justin - we are still working on it.  We also have to cover model runner and profile.  We need to vet it with the team and we also are having discussions now.  We definitely want to be on the agenda.  We are also talking about run time models need to check with LUM for rite to use off the platform.  We need to get approval off the platfom.
14:54:52 <farheen_cefalu_a> We are discussing with Guy.
14:55:09 <farheen_cefalu_a> #topic Docker Compose mismatch
15:01:10 <farheen_cefalu_a> #info Ken and Bryan were working together.  Ken - I updated the version number in my template and it wouldn't start.  I am comparing YMLs.  Bottom line is it needs to be communicated well to everyone.  Bryan it is fairly simple to remove the extra hosts field and run with what you have with version 2.  The only way to do it is to run the scripts yo
15:01:11 <farheen_cefalu_a> u installed.  Ken is struggling with All in One installation.  It is very complicated.  Manoop - Do we need Bryan to update anything in gerret?  Bryan - Until we decide to align the environment variables we can use the AIO.
15:02:57 <farheen_cefalu_a> #info Ken - The All In One is using Docker Compose 3.2.  If you deploy the Platform using DNS resolving names you don't need Docker Compose 3.2.  I use an environment without hosts I use Docker compose.  Compose 3.2
15:03:37 <farheen_cefalu_a> #topic Sayee with ML Workbench
15:04:44 <farheen_cefalu_a> #info Sayee in one or two weeks we will show something.  Camunda and we can show something next week for ML Workbench.  #action Sayee provide ML Workbench design reviews next week.
15:04:55 <farheen_cefalu_a> #endmeeting