Cumulo is not a library, I was trying to build one but didn't make it. My code is coupling heavily and I have to copy/paste to reuse the code in current stage.
React uses DOM diff
To tell the whole story, it's first talk about how React works. It's very simple in concept. You have DOM tree A represented in virtual DOM VA, which is the wrote to the real DOM by React. Then you need to update to DOM tree B so you create virtual DOM VB let handle it to React. React is smart. It diffs VA and VB and generate a patch of A and B, and apply the patch to A. And the job is finished. Very simple.
The tricky part is the diff algorithm. It has to be fast enough. It should learn to skip unchanged branches. It need to handle special cases of complicated DOM trees. I was very curious and tried to play with LCS algorithm but never being good at it, especially for the dynamic programming part. For the DOM, there's also other solution like traversing the whole DOM tree and fix up all the changes. I know deku traverses, I'm not sure how React is diffing. But the most important thing is, smart people in the open source community already wrote it, that's cool!
Similarity between DOM updates and Store updates
Let's talk about Backbone for a bit since I got my first experiences from it. In the early times I was writing in Backbone, which was 2013 in Hangzhou, quite often we need to write some code.
To update DOM, like this(stayed away from Backbone for long, excuse me if there's bugs):
Backbone.View.extend
init: ->
@model.on 'change:title', @updateTitle
@model.on 'change:countVotes', @updateVotes
updateTitle: (title) ->
@$el.querySelector('.title').text(title)
updateVotes: (votes) ->
@$el.querySelector('.votes').text(votes)
And if you've ever program with server pushed event via WebSocket, such code might be familiar, no matter for Backbone or React you use. I use React with global Store and Immutable data, so it's like this:
socket.on 'message:change', (message) ->
id = message.id
teamId = message.teamId
channelId = message.channelId
immutableMessage = Immutable.fromJS message
store.updateIn ['messages', teamId, channelId], (messages) ->
messages.map (cursor) ->
if cursor.get('id') is id
message
else cursor
socket.on 'message:remove', (message) ->
id = message.id
teamId = message.teamId
channelId = message.channelId
immutableMessage = Immutable.fromJS message
store.updateIn ['messages', teamId, channelId], (messages) ->
messages.filterNot (cursor) ->
cursor.get('id') is id
In Backbone, store could be messageCollection or something. You can already see what it's doing. It's receive individual change events and update local copy of data which was also sent from server.
Here's the thing, we are always doing the boring job: manually syncing data from server to client, and from store to view inside client. We talked about React and we already knew React is doing the second step updating view for us, then who can help us to do the first step?
Answer can be Meteor. That's right, except for I don't like Meteor. It's working like MongoDB and it't not in functional style. See? React is declarative since it functional idea and became very expressive and suitable for abstractions. Meteor's guide is very like expressjs, rather than React.
Speaking of MongoDB, there's query language, here's an example from the docs:
var updateRestaurants = function(db, callback) {
db.collection('restaurants').updateOne(
{ "name" : "Juni" },
{
$set: { "cuisine": "American (New)" },
$currentDate: { "lastModified": true }
}, function(err, results) {
console.log(results);
callback();
});
};
It's very like the way we update the DOM. And here's my conclusion: among database, server, store, DOM, people are always repeating themselves, trading for performance! When we are updating a message, we have to do 1) send query to database to update, 2) push event to client store to update, 3) update DOM tree(now done by React).
Meteor it still cool even I'm not in favor of it.
Learn it and do it for server/clients
That's why I started my project called Cumulo and make experiments. First, I need to admit that Cumulo is very slow. And it's does not use a database so it's not capable of holding large amount of data. Here's some details:
- Cumulo is putting data in memory in language runtime
- it uses diff to generate patches, in JSON and EDN(Clojure)
- it patches store in client and let React to update the DOM
Here's the overall data flows in Cumulo's idea:
I'm not going to talk about the detail, anyone reading this diagram can build his how implementation of Cumulo as long as he finds a good enough diff library. And definitely they will be better than mine, I just programmed for about four years.
Synchronization JSON data is common problem and there's JSON Patch Spec for it. I'm lucky. I have tried several libraries:
- github.com/intelie/immutable-js-diff
- github.com/benjamine/jsondiffpatch
- github.com/Skinney/differ (for Clojure EDN)
My implementations
If you check Cumulo on GitHub there're already lots of repos. All of them are my attempts to see how Cumulo will be. There is not a stable Cumulo implementation. It's evolving and far from matureness.
Good news is I already have some basic achievements, first on JavaScript side:
- Combined WebSocket, Webpack, React, Immutable Data as a demo
- Hot replace JavaScript in browser, also on server behind WebSocket
- Diff/patch immutable JSON to sync data
I also rewrote code in ClojureScript(actually Sepal.clj in favor of Cirru syntax). By using Figwheel, hot replacing code in browser and on server is very smooth. Clojure is designed carefully and more reliable, the most important aspect, it's good for abstraction thanks to the Lisp internals and persistent collections.
I'm still exploring.