#acumos-meeting: Architecture Committee

Meeting started by farheen_att at 15:04:47 UTC (full logs).

Meeting summary

  1. Agenda (farheen_att, 15:05:53)
    1. Bryan - we are going to have to tweek the deployment design for Demeter. (farheen_att, 15:08:01)
    2. Bryan - We can't ship he logs around. I created an epic under deployment project we talked about this morning. We are going to have to deploy the solution the nginx and side beat side cars as a single pod. The resource overhead in solutions are going to be higher. we can optimize. We either choose the expensive design approach or conform (farheen_att, 15:09:35)
    3. Bryan the model, nginx, and filebeat. (farheen_att, 15:09:48)

  2. Loose ends for the Clio maintenance release (farheen_att, 15:10:12)
    1. R models are not building ms correctly. Phillippe, Guy, and team are looking into that. (farheen_att, 15:10:33)

  3. Decide on Data description languages used inside acumos. When we describe the services by a model we use protobuf. It is efficient and puts a barrier to entry for people running their models. Understand the difference of protobuf to serialize and protobuf to describe what the data types of the ms used are. (farheen_att, 15:12:31)
    1. ONAP integration revealed protobuf is not as proficient as others. Nested lists are not supported in protobuf. We added a raw mode. Instead of understanding the datatypes we simply say it is a raw bytes. It is up to the model for inputs and outputs. That was sufficient for ONAP integration. (farheen_att, 15:14:49)
    2. Bryan - If you include a schema you can discover. (farheen_att, 15:15:02)
    3. Bryan - the concept of a schema is a representation of how the attributes are going to be structured. Protobuf is a medium. (farheen_att, 15:16:12)
    4. Reuben - we're using protobuf as a schema definition and also as a serialization tool. (farheen_att, 15:17:32)
    5. Guy - We use an alternative API or completely throw away schema. I will have a more detailed discussion next week. (farheen_att, 15:18:15)
    6. If we want to connect two JSON schema you can not. You can connect protobuf to protobuf. (farheen_att, 15:19:47)
    7. Reuben does that mean the whole library is not available to me? (farheen_att, 15:20:00)
    8. Guy yes, there could be either. JSON and protobuf can't talk to each other. The problem with ONAP is they use JSON. (farheen_att, 15:20:43)
    9. JSON schema and OpenAPI are widely accepted standards. We wanted efficiency with binary objects which we get from protobuf. (farheen_att, 15:21:27)
    10. Raw mode is better than etiher of the schemas we have now. Having raw mode is useful even outside of the discussion of incorporating JSON schema. (farheen_att, 15:22:42)
    11. DCAE from ONAP has a philopsophy that all data types are named titles in a catalog. In acumos data types are anonymous. 3 floats are 3 floats and can be connected with 3 floats. (farheen_att, 15:23:46)
    12. Reuben - We found a lot of different translators for JSON. (farheen_att, 15:34:15)
    13. Justin - There are hurdles with protobuf using acumos vs. server to server communication. (farheen_att, 15:34:58)
    14. Guy - We talked about building a language specific tool that would generate an interface for the protobuf file. You can say here is protobuf. Give me the tool I need to call the Acumos model like a simple function call. If we could generate language by language. (farheen_att, 15:36:07)
    15. Guy for example build a protobuf and call an existing model that will generate the classes or the code for me then you would not need protobuf. (farheen_att, 15:36:40)
    16. if you had a tool that said download this interface to this class. In python you would connect to the acumos instance with credentials and it would create a class which had methods that had simple method calls that handled the serialization of protobuf and deserialized and gave it back in native format. (farheen_att, 15:37:58)
    17. Bryan that is a fairly common problem. (farheen_att, 15:38:14)
    18. Sayee - You want a simple JSON file that you can train your model with. (farheen_att, 15:38:41)
    19. Guy - During the training the part you don't need protobuf for example, csv file. No need to mention protobuf. (farheen_att, 15:39:09)
    20. The model updates daily. Are we complicating it for the user? (farheen_att, 15:39:34)
    21. Why are we restricting users? (farheen_att, 15:39:42)
    22. Justin - CSV input is used in the running the model. Conversion is required out of order, then you have to find the right library. This is the burden to the app developer. (farheen_att, 15:40:39)
    23. it was intended to download the protobuf in the language. Then the csv files are reasonably straight forward? (farheen_att, 15:41:11)
    24. Sayee - That is additional latency. Why don't was support the format that developers want. (farheen_att, 15:41:45)
    25. we don't restrict everyone to protobuf. you can send data in JSON as well. (farheen_att, 15:42:12)
    26. but if you want to isolate the data transformation from the model part there are tools that will read from a file, table, and prepare them to feed into the acumos model. (farheen_att, 15:43:00)
    27. Why can't I use the data directly. (farheen_att, 15:43:09)
    28. What does latency mean in your context? (farheen_att, 15:43:20)
    29. For a real time onap network prediction in milliseconds and respond whether it's a fraud call or not are we going to take time. (farheen_att, 15:43:55)
    30. That needs to experimented. Protobuf is very efficient. Parsing the CSV file will take longer than Protobuf. (farheen_att, 15:44:30)
    31. For predicting the data marshalling. (farheen_att, 15:44:58)
    32. Sayee - Mojo and Pojo don't support protobuf. Are we going to convert them from protobuf to CSV. (farheen_att, 15:45:28)
    33. Guy - I would not worry about the data marshaling time. (farheen_att, 15:46:13)
    34. Bryan - A model can take in any data type. You hav to discover what that model is expecting or have a contract in advance such as meta data so the client knows. You can send a request "what are you configured as?" response is I am JSON or protobuf. The model runner could support different types of input to the model. (farheen_att, 15:47:41)
    35. Do we want to implement as a separate piece or the modelrunner? (farheen_att, 15:48:10)
    36. modelrunner (farheen_att, 15:48:16)
    37. every modelrunner (farheen_att, 15:48:24)
    38. there are modeler runners for each language such as python model runner, r model runner, etc... (farheen_att, 15:49:03)
    39. That work has to be done for all of the model runner. (farheen_att, 15:49:27)
    40. Sayee - Maintenance of code becomes challenging. (farheen_att, 15:49:46)
    41. Yes, so we only have to write one model runner per language. (farheen_att, 15:50:18)
    42. It is possible we will defer this to E release. (farheen_att, 15:50:33)
    43. Justin - When I did the UI it was without he possibility of doing a JSON. then the botteneck would be the conversion of the CSV file. (farheen_att, 15:51:56)
    44. Guy - Data frame type of work we do in protobuf is weird. It's a transposed version of the way I like to think of the data. You could conceive something that is malformed. Needs work. (farheen_att, 15:53:12)
    45. ACTION: Team post suggestions to the wiki. (farheen_att, 15:53:36)
    46. next week Bryan agreed to talk about namespace. (farheen_att, 15:53:59)
    47. Bryan - I did put my thoughts on the developer guide. (farheen_att, 15:54:19)
    48. Sayee next weeks topic I would like to add Glowroot to the agenda. (farheen_att, 15:55:21)
    49. Glowroot is a jar file that monitors your service account. It gives memory dumps for data analysis. It's a service. It is for monitoring both platform. (farheen_att, 15:56:40)
    50. YOu can add it to a system as a common repo or to invidual component. (farheen_att, 15:57:13)
    51. we have Prometheus with our OS using ELK. Bring the argument for that comparison. (farheen_att, 15:57:43)
    52. Sayee - How is training planned by Ericsson. What is the model training approach taken. Can we take into account Nifi. (farheen_att, 15:58:47)
    53. Bryan - I put an epic for deploying pipelines. Training use cases and deploying pipelines needs to be discussed. (farheen_att, 15:59:23)
    54. Justin - For training we should be on the list. We are not ready for next week. We are not focused on Nifi at the moment. We do have some ethics in plance under the training project. Feedback will be helpful. (farheen_att, 16:00:14)


Meeting ended at 16:00:36 UTC (full logs).

Action items

  1. Team post suggestions to the wiki.


People present (lines said)

  1. farheen_att (64)
  2. collabot` (3)


Generated by MeetBot 0.1.4.