Architecture Overview

Dshackle is built using/based on:

  • Java Virtual Machine, Java 20+

  • Kotlin

  • Spring Framework, Boot and Reactor

Its own communication protocol is based on Protobuf and gRPC, though it connects to other nodes using different protocols, such as JSON RPC or Websockets.

Dshackle connects to actual blockchain nodes and provides an aggregated API, automatically directing requests to currently healthy nodes. It’s built using reactive and non-blocking code and provides low latency and high efficient API.

Dshackle servers can be connected to each other as well, or work in parallel, to provide scalable and fault tolerant connection to blockchains. All connections can be secured and authenticated with client/server certificates.

Upstream selection

Dshackle tries to find an optimal upstream for each particular request, to do so it consider following factors of a node:

  • Current height

  • Is it fully synchronized, missing few (less than 6) blocks, or in the process of initial sync?

  • How many peers it has?

And for a request:

  • Is it for concrete data (block #100) or the latest (get balance)?

  • Is result a static value (just block) or may vary depending on network and node (latest nonce)?

  • Does it need to be repeated over multiple nodes (broadcast transaction)?

  • Request can also specify which subset of nodes should be able to execute the request by selecting node Labels (see "Quorum and Selectors")

Based on these factors, Dshackle executes the request on a most optimal node. For most of the simple requests, it just gets a node that is synchronized to the point that must have a response for that particular request. If node failed, returned an invalid response, returned an empty response when shouldn’t then Dshackle tries again on another node, or on the same node after awhile (default is 200ms between failover repeats)

Proxy

Dshackle provides access with standard JSON RPC protocol, functioning as a proxy to upstreams. It provides:

  • Routes only realy/alive upstreams, i.e., synchronized and with enough peers

  • Load Balancing

  • Request Retry on upstream errors

  • Local Caching (memory and Redis, see Caching)

  • Broadcasting and Quorum for requests

gRPC protocol

Dshackle native protocol is based on gRPC, which provides many additional features:

  • provides additional parameters on top of upstreams JSON RPC requests

  • based on HTTP/2 with low latency, compression, server push, pipelining and multiplexing

  • gRPC has binding and code generators for most of the languages and frameworks

  • easy to support TLS encryption and authentication

gRPC protocol is more flexible, and many of the Dshackle features are accessible mostly through that native protocol.

Dshackle has extra methods and functionality on top of standard APIs of upstreams, and it’s more flexible to wrap original APIs, such as JSON, into gRPC, and Protobuf + has additional data provided by Dshackle. It also allowed to push new data from the server or send responses asynchronously, immediately after it gets executed on an upstream.

Distributed Load Balancing

Dshackle servers can connect to each other through a secure encrypted and authenticated connection. It allows to build a network of nodes deployed to different regions or run blockchain nodes outside of the main network.

Later is especially important for blockchain nodes, as they require an open firewall with incoming connections for P2P, and to execute untrusted code ("smart contracts") at the same time. With Dshackle, it’s possible to separate insecure nodes from the business application.

But in general, Dshackle allows running a fault tolerant load balancing to build scalable and fail safe systems on blockchain.