17:18:35 <moizer> #startmeeting “streaming design”
17:18:35 <odl_meetbot> Meeting started Tue Oct 28 17:18:35 2014 UTC.  The chair is moizer. Information about MeetBot at http://ci.openstack.org/meetbot.html.
17:18:35 <odl_meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:18:35 <odl_meetbot> The meeting name has been set to '_streaming_design_'
17:21:09 <moizer> #info all protocol buffer messages that send across a normalized node should now be migrated to a java serializable which implements custom readObject/writeObject
17:30:32 <moizer> #info Saving snapshot in the replicated log uses up too much memory
17:30:54 <moizer> #info maybe we need to create a snapshot when we need to Install the snapshot
17:31:46 <moizer> #info when shard is not persistent - there is more chance of install snapshot being required
17:44:00 <moizer> #agreed We will capture shot when we need to install it
17:47:43 <moizer> #info we want to keep serialization of instance identifiers on WriteData/MergeData with the new streaming
17:48:48 <moizer> #info create a bug for instance identifier serialization
17:50:33 <moizer> #action create a bug for snapshotting/install snapshot to use the new streaming classes
17:52:24 <moizer> #info Parallel Task 1 - Install snapshot changes with ReadDataReply
17:53:09 <moizer> #info Parallel Task 2 - WriteData/MergeData - change streamwriter to write instance identifiers
17:53:42 <moizer> #info Parallel Task 3 - Add more unit tests for the stream writer
17:54:55 <moizer> #info Task 4 - Optimize the stream writer to reuse the builder - definitely the leafnode/leafsetentry builders can be reused
17:56:28 <moizer> #info Task 5 - ReplicatedLogEntry also needs to use a java serializable
17:57:08 <moizer> #info We need to take care of versioning in the java serializable
18:03:59 <moizer> #action evaluate the work required to support backward compatibility
18:05:43 <moizer> #info by backward compatibility I mean each message should be backward compatibility
18:09:03 <moizer> #info What should we use as the version?
18:09:23 <moizer> #info  A static field in the clustering-messages could be enough
18:10:21 <moizer> #info 1. NormalizedNode , instead of bytestring , will be stored as part of the snapshot.
18:10:35 <moizer> #info 2. Installsnapshot will use the NormalizedStreamWrite
18:10:46 <moizer> #info 3. All messages will have a version number. An actor only accepts messages with version >= its version.
18:10:57 <moizer> #info 4. Its possible for a leader to have an higher version than the follower. In a 3 member cluster, when the first member is upgraded, its not possible for the leader > follower. But when the second member is upraded, it can happen that the one of the upgraded ones become the leader.
18:11:05 <moizer> #info 5. Backward compatibility is supported. If a V1.1 node gets V1.0 message , it should honor it.
18:11:13 <moizer> #info 6. If a V1.0 node gets V1.1 message, it would ignore the message. The V1.0 replica, if follower, will be behind the Leader, till upgraded. The V1.0 replica , if Leader, will not honor messages from upgraded V1.1 follower.
18:11:27 <moizer> #info 7. Heartbeats should be counted , even if messages are not honored.
18:11:38 <moizer> #info 8. VoteRequest should also comply (version >= its version) rule, there by ensuring that the leader is elected from the majority with same version
18:11:53 <moizer> #info These are rules which Kamal proposes
18:15:33 <moizer> #info When we get a message with an older version that can be handled the reply needs to be of the same version as the request
18:23:37 <moizer> #info all fields in our java serializable should be transient
18:26:57 <moizer> #info we may need to create message classes to cleanly migrate from v1 -> v2 messages
18:33:22 <moizer> #info Send message here asking about backward compatibility here -> https://lists.opendaylight.org/mailman/listinfo/opendaylight-users
18:36:14 <moizer> #info Message to users
18:37:34 <moizer> #info We are changing the serialization of clustering messages to improve performance for persistence and communication.
18:38:50 <moizer> As a result the new clustering messages will be incompatible with base Helium and thus will require users to stop and upgrade all cluster nodes together
18:39:25 <moizer> #info Persisted data will also not be recovered due to this incompatibility
18:39:36 <moizer> #info Is this acceptable?
18:41:42 <moizer> #info Title : MD-SAL Clustering for Helium
18:41:54 <moizer> #info MD-SAL Clustering for Helium R1
18:42:16 <moizer> #info Final title : MD-SAL Clustering changes for Helium R1
18:42:31 <moizer> #info Response requested by COB Friday
18:42:56 <moizer> #action Moiz to send the message
18:43:30 <moizer> #info We will proceed with the assumption that backward compatibility is not to be supported
19:19:18 <moizer> #endmeeting