#opendaylight-clustering: “streaming design”

Meeting started by moizer at 17:18:35 UTC (full logs).

Meeting summary

    1. all protocol buffer messages that send across a normalized node should now be migrated to a java serializable which implements custom readObject/writeObject (moizer, 17:21:09)
    2. Saving snapshot in the replicated log uses up too much memory (moizer, 17:30:32)
    3. maybe we need to create a snapshot when we need to Install the snapshot (moizer, 17:30:54)
    4. when shard is not persistent - there is more chance of install snapshot being required (moizer, 17:31:46)
    5. AGREED: We will capture shot when we need to install it (moizer, 17:44:00)
    6. we want to keep serialization of instance identifiers on WriteData/MergeData with the new streaming (moizer, 17:47:43)
    7. create a bug for instance identifier serialization (moizer, 17:48:48)
    8. ACTION: create a bug for snapshotting/install snapshot to use the new streaming classes (moizer, 17:50:33)
    9. Parallel Task 1 - Install snapshot changes with ReadDataReply (moizer, 17:52:24)
    10. Parallel Task 2 - WriteData/MergeData - change streamwriter to write instance identifiers (moizer, 17:53:09)
    11. Parallel Task 3 - Add more unit tests for the stream writer (moizer, 17:53:42)
    12. Task 4 - Optimize the stream writer to reuse the builder - definitely the leafnode/leafsetentry builders can be reused (moizer, 17:54:55)
    13. Task 5 - ReplicatedLogEntry also needs to use a java serializable (moizer, 17:56:28)
    14. We need to take care of versioning in the java serializable (moizer, 17:57:08)
    15. ACTION: evaluate the work required to support backward compatibility (moizer, 18:03:59)
    16. by backward compatibility I mean each message should be backward compatibility (moizer, 18:05:43)
    17. What should we use as the version? (moizer, 18:09:03)
    18. A static field in the clustering-messages could be enough (moizer, 18:09:23)
    19. 1. NormalizedNode , instead of bytestring , will be stored as part of the snapshot. (moizer, 18:10:21)
    20. 2. Installsnapshot will use the NormalizedStreamWrite (moizer, 18:10:35)
    21. 3. All messages will have a version number. An actor only accepts messages with version >= its version. (moizer, 18:10:46)
    22. 4. Its possible for a leader to have an higher version than the follower. In a 3 member cluster, when the first member is upgraded, its not possible for the leader > follower. But when the second member is upraded, it can happen that the one of the upgraded ones become the leader. (moizer, 18:10:57)
    23. 5. Backward compatibility is supported. If a V1.1 node gets V1.0 message , it should honor it. (moizer, 18:11:05)
    24. 6. If a V1.0 node gets V1.1 message, it would ignore the message. The V1.0 replica, if follower, will be behind the Leader, till upgraded. The V1.0 replica , if Leader, will not honor messages from upgraded V1.1 follower. (moizer, 18:11:13)
    25. 7. Heartbeats should be counted , even if messages are not honored. (moizer, 18:11:27)
    26. 8. VoteRequest should also comply (version >= its version) rule, there by ensuring that the leader is elected from the majority with same version (moizer, 18:11:38)
    27. These are rules which Kamal proposes (moizer, 18:11:53)
    28. When we get a message with an older version that can be handled the reply needs to be of the same version as the request (moizer, 18:15:33)
    29. all fields in our java serializable should be transient (moizer, 18:23:37)
    30. we may need to create message classes to cleanly migrate from v1 -> v2 messages (moizer, 18:26:57)
    31. Send message here asking about backward compatibility here -> https://lists.opendaylight.org/mailman/listinfo/opendaylight-users (moizer, 18:33:22)
    32. Message to users (moizer, 18:36:14)
    33. We are changing the serialization of clustering messages to improve performance for persistence and communication. (moizer, 18:37:34)
    34. Persisted data will also not be recovered due to this incompatibility (moizer, 18:39:25)
    35. Is this acceptable? (moizer, 18:39:36)
    36. Title : MD-SAL Clustering for Helium (moizer, 18:41:42)
    37. MD-SAL Clustering for Helium R1 (moizer, 18:41:54)
    38. Final title : MD-SAL Clustering changes for Helium R1 (moizer, 18:42:16)
    39. Response requested by COB Friday (moizer, 18:42:31)
    40. ACTION: Moiz to send the message (moizer, 18:42:56)
    41. We will proceed with the assumption that backward compatibility is not to be supported (moizer, 18:43:30)


Meeting ended at 19:19:18 UTC (full logs).

Action items

  1. create a bug for snapshotting/install snapshot to use the new streaming classes
  2. evaluate the work required to support backward compatibility
  3. Moiz to send the message


People present (lines said)

  1. moizer (44)
  2. odl_meetbot (3)


Generated by MeetBot 0.1.4.