An Experiment in Gutting JInterface for core.async
February 13, 2016
The zhang project aims to create a very minimal library in Clojure, built on core.async, that, as closely as possible, implements the Erlang process model. It does not aim to provide any opionions about (and thus even less, any implementations for) either message structure or infrastructure necessary for supporting remote processes
Today yet another Clojang project was created:
From the project's README, here is a (partial) list of desired features:
- the ability to quickly create and destroy processes, nodes, and mboxes
- ensure the safe "crashing" of a process
- very fast message-passing between processes
- the ability support very large numbers of processes
- shared nothing
- low-overhead function-level mailboxes
- ordered message queues
- pattern matching (
core.match) and selective
- process registration hooks (in order to support arbitrary publishing mechanisms)
A great deal of work (and time in production deployments) has gone into Erlang's JInterface Java package. Using it in Clojure has been such a pain, however, that two projects were created to mitigate two major issues around that (Clojure idioms and sane OTP defaults). Even with these libraries under active development, their ultimate goals (and areas of focus) are beyond the scope of addressing the JInterface internals of threads, sockets, and queues. As such, zhang was created to find a good solution for replacing those bits with
The project is experimental in nature and will likely be developed only very slowly, especially while jiface and clojang are in development. Once those projects hit a level of stability, though, I expect to be spending more time on the interesting problems to be solved in zhang.
Additionally, I expect that zhang, jiface, and clojang will end up sharing a certain amount of code. Only once jiface and clojang stabilize will it become clear what common bits for zhang can be split out into supporting library projects. I expect that one of these will be the type conversions developed in clojang (i.e., there will likely eventually be a clojang/types library and corresponding repo).
Since zhang aims to be messaging and deployment agnostic, there could also be a set of message implementation libraries created – the first of which would be the default: OTP process messaging. However, there's no reason other message formats couldn't be implemented or integrated. This would allow zhang (and thus Clojang in general) to transcend the current domain of OTP messaging and take fault-tolerant, soft real-time programming into other areas of the Clojure ecosystem.
On a fun side note, the zhang project takes it's name from Zhang Heng, who approximated pi early in the first millennium (Han Dynasty). This is an obscure pun, since the Erlang process model could be, in some ways, interpreted as an approximation of the process calculus. In addition to being a mathematician, Zhang Heng was also a poet, astronomer, and engineer – a wonderful patron for a software project :-)