Enhanced Methods

Important
 Dshackle provides an enhanced API based on gRPC and Protobuf, in addition to JSON RPC proxy

Dshackle provides an enhanced and unified API based on HTTP2, gRPC and Protobuf. It works in addition to JSON RPC proxy and can be used separately. Clients can be generated for all major programming languages, and there’re official libraries for Java and Javascript.

Note
 It’s not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.

gRPC definition

The source code for the Protobuf definitions could be found in /proto.

Service API
service Blockchain {
    rpc SubscribeHead (Chain) returns (stream ChainHead) {}
    rpc SubscribeBalance (BalanceRequest) returns (stream AddressBalance) {}
    rpc SubscribeTxStatus (TxStatusRequest) returns (stream TxStatus) {}

    rpc GetBalance (BalanceRequest) returns (stream AddressBalance) {}

    rpc NativeCall (NativeCallRequest) returns (stream NativeCallReplyItem) {}
    rpc NativeSubscribe (NativeSubscribeRequest) returns (stream NativeSubscribeReplyItem) {}

    rpc Describe (DescribeRequest) returns (DescribeResponse) {}
    rpc SubscribeStatus (StatusRequest) returns (stream ChainStatus) {}
}

Wrapped JSON RPC methods

To call standard JSON RPC methods provided by Ethereum/Bitcoin you use NativeCall wrapping method, which provides additional flexibility and configuration for the calls.

NativeCallRequest
message NativeCallRequest {
    ChainRef chain = 1;
    repeated NativeCallItem items = 2;
    Selector selector = 3;
    int32 quorum = 4;
    AvailabilityEnum min_availability = 5;
}

message NativeCallItem {
    uint32 id = 1;
    string method = 3;
    bytes payload = 4;
    uint64 nonce = 5;
}

Where:

  • chain target chain (see reference for ids)

  • items as a list of independent requests, which may be executed in different nodes in parallels or in different order, with:

    • method - a JSON RPC standard name, ex: eth_getBlockByHash

    • payload - list of parameters for the methods, encoded as JSON string, ex. ["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]

  • Selector and AvailabilityEnum are described in reference, in short they allow to specify which nodes must be selected to execute the reques (i.e. "execute only on an archive node")

NativeCallReplyItem
message NativeCallReplyItem {
    uint32 id = 1;
    bool succeed = 2;
    bytes payload = 3;
    bytes error = 4;
    NativeCallReplySignature signature = 5;
}

message NativeCallReplySignature {
    uint64 nonce = 1;
    bytes signature = 2;
    uint64 key_id = 3;
    string upstream_id = 4;
}

Where:

  • payload is JSON response for a particular call (result field), encoded into a string (succeed is true)

  • or error if request failed (succeed is false)

Note
 Reply Items comes right after their execution on an upstream, therefore streaming response. It allows building non-blocking queries

Signed JSON RPC Responses

Dshackle can sign the responses it received from an upstream. It can be enabled on server by configuring a path to a Secp256K1 Key used for signing. And passing a nonce as part of the call request. When both of them are set Dshackle adds a Signature to the response, which can travel through multiple levels of Dshackle-Dshackle connections.

Warning
 Caching is disabled for signed requests, and you always touch an actual node even if there is a ready to use data in local cache.

The signed message looks like:

DSHACKLESIG/$nonce/$upstreamId/hex(sha256($response))
Where

  • nonce is a 64-bit number provided with the request

  • upstreamId id of an upstream which produced the result

  • response is a part of the original JSON RPC message, i.e. it’s what you have in payload field of NativeCallReplyItem

A signed response includes:

  • nonce original nonce used for the call

  • signature signature bytes (of the message above)

  • key_id identifier of a key used for the signing

  • upstream_id id of upstream which produce the response

How to generate a key

Here we generate a pair of Secret and Public keys using openssl.

export KEYNAME=mykey

openssl ecparam -name secp256k1 -out $KEYNAME_param.pem
openssl ecparam -in $KEYNAME_param.pem -genkey -noout -out $KEYNAME.pem

openssl ec -in $KEYNAME.pem -text
openssl ec -in $KEYNAME.pem -out ${KEYNAME}_pub.pem -pubout

rm $KEYNAME_param.pem
cat ${KEYNAME}_pub.pem

As a result you get mykey.pem with secret key to use on server, and mykey_pub.pem with public key to use on client to verify signatures.

How to verify a signature

Here is the example how to verify the signature with command line, and it can be easily adapted for your language of choice.

First, let’s prepare all the values:
export PUBKEY=testing/dshackle/test_key.pub
export NONCE=10

export UPSTREAM=drpc
export PAYLOAD='["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]'

export SIGNATURE=3045022100be1d730e0e381e25bff64f0fc598d19e31688a01db751098d0ed21847ca785b0022002f5651a0e8d447b0815aeb7b48738cb470cf46d60ee4e2f5bc9c0dc4e072dc3
Then rebuild the signed message:
echo -n "DSHACKLESIG/$NONCE/$UPSTREAM/" > msg.txt
echo -n $PAYLOAD | shasum -a 256 - | awk '{ printf $1 }' >> msg.txt
And save signature as a binary file:
rm -f msg.sig && echo $SIGNATURE | xxd -r -p - msg.sig
Now you can verify the payload with the following:
openssl dgst -sha256 -verify $PUBKEY -signature msg.sig msg.txt
Which should print:
Verified OK

What is the Key Identifier?

Key Id is the first 64 bits of SHA-256 hash of the x509 encoded Public Key. It’s provided with the Signed Response for a reference.

You can get it with:

cat $PUBKEY | sed -e '$ d' | awk '(NR>1)' | base64 -d | shasum -a 256 - | head -c 16

Wrapped JSON RPC subscriptions

Most of Ethereum APIs provides subscription to events usually accessed through WebSocket connection. Dshackle gives access to same events through gRPC protocol via the NativeSubscribe method.

Note
 Dshackle doesn’t actually wrap existing subscription or dispatch request to an upstream. It rather generates same events based on the available data, i.e., aggregates it from multiple upstreams.

Supported subscriptions:

  • newHeads

  • logs

  • syncing

Method data:

message NativeSubscribeRequest {
    ChainRef chain = 1;
    string method = 2;
    bytes payload = 3;
}

message NativeSubscribeReplyItem {
    bytes payload = 1;
}

Where:

  • method is a subscriptions method (one of newHeads, logs or syncing)

  • payload in request is optional subscription params object, which exists only for logs methods. In that case it may be address or topics. Both address and topics can be a string or array of strings. Empty payload for logs accepted as subscription to all events.

  • payload in reply item is as subscription response encoded as JSON

For example to subscribe to USDC ERC-20 coin Approval events on Ethereum mainnet the request would be:

  • chain=100

  • method=logs

  • payload={"address": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", "topics": ["0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925"]}

SubscribeHead

This methods provides subscription to the new blocks on the specified chain. Returns stream of blocks right after it was accepted (and verified by Dshackle) by any of the upstreams.

ChainHead
message ChainHead {
    ChainRef chain = 1;
    uint64 height = 2;
    string block_id = 3;
    uint64 timestamp = 4;
    bytes weight = 5;
    uint64 reorg = 6;
}

Where:

  • chain - chain id

  • height - block number

  • block_id - block hash, as a string (please note that it doesn’t have 0x prefix)

  • timestamp - timestamp of that block

  • weight - total network difficulty on that block, as raw bytes

  • reorg - number of reorganized blocks, if reorg happened