Refactor cluster based dispatch and enable customizable routers #353
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #246 and #344 and #345 :)
This PR introduces customizable routing modules for distributed caches and will also include documentation updates based on the comments in #246 (namely #246 (comment)). It should be noted that the next version of Cachex will be v4.0.0 so we are free to make breaking changes here.
There is a lot in here, but the gist is that you can now provide an implementation of
Cachex.Router
to dictate exactly how your keys are assigned to nodes in your cluster. Routers can either be synchronous state, or they can delegate to other linked processes. There are four routers included in this PR:Cachex.Router.Local
Cachex.Router.Mod
hash(key) % len(nodes)
)Cachex.Router.Jump
Cachex.Router.Ring
libring
Each of these routers (except
Cachex.Router.Local
) can be configured to use the currently connected nodes, rather than providing them. The way to do this differs per router, so make sure to visit them for docs (although I'll be writing a new "distribution" guide based on these prior to v4.x).I previously wrote that I would add two functions to the Cachex API to allow addition/removal of nodes, but I don't think this makes sense now that we can configure routers independently. If we need additional APIs for this in future, happy to add them but they don't seem necessary.
The old
Cachex.Router
module was moved into a service inCachex.Services.Conductor
to better fit the new role, and to allowCachex.Router
to focus on behaviour.The
:nodes
option has also been removed from Cachex's main API, because now it's an option on (some of) the Router implementations. This is also much cleaner, IMO.