Upstreams Configuration

Dshackle can connect to multiple independent APIs ("upstreams") and provides an unified API on top of it.

Supported upstream protocols:

  • JSON RPC

  • Websockets

  • gRPC (i.e. can connect to another Dshackle)

Those protocols can be configures with additional security, TLS and authentication.

Notes on upstream configuration

Generic

  • Most common way to connect upstream. Supported for Ethereum, Starknet, Solana and Varanet upstreams. It uses rpc and ws connection to provide access via jsonprc calls.

Solana

In case of Solana nodes configuration with WS connection it is important to use --rpc-pubsub-enable-block-subscription on solana node to enable head subscription through ws

Bitcoin

  • Bitcoind needs to be configured to index/track addresses that you’re going to request. Make sure you configure it to index your addresses with importaddress. If you request balance for an address that is not indexed then it returns 0 balance.

  • To track all transactions you need to setup index for transactions, which is disabled by default. Run it with -reindex option, or set txindex=1 in the config.

Example Configuration

upstreams.yaml
version: v1

cluster:
  upstreams:
    - id: us-nodes
      node-id: 1
      chain: auto
      method-groups:
       enabled:
         - trace
      connection:
        grpc:
          host: 35.226.252.117
          port: 443
          tls:
            ca: ca.crt
            certificate: client.crt
            key: client.p8.key
    - id: drpc-eth
      node-id: 2
      chain: ethereum
      role: fallback
      labels:
        provider: infura
      options:
        disable-validation: true
      connection:
        ethereum-pos:
            execution:
              rpc:
                url: "https://lb.drpc.org/ogrpc?network=ethereum&dkey=${DRPC_KEY}"
              ws:
                url: "wss://lb.drpc.org/ogws?network=ethereum&dkey=${DRPC_KEY}"
    - id: solana
      node-id: 3
      chain: solana
      connection:
        generic:
          rpc:
            url: ${SOLANA_NODE_RPC_URL}
          ws:
            url: ${SOLANA_NODE_WS_URL}

There are two main segments for upstreams configuration:

  • upstreams - a list of API to connect to, with all configuration specific for upstream and chain

  • and default options as common configuration options applied to all nodes in that group

In the example above we have:

  • as upstreams it has 2 configurations - for ethereum and solana

  • balancer connects to another Dshackle/another machine by using gRPC protocol

    • accepts (i.e. proxies) any blockchain available on that remote

    • verifies TLS certificate of the server

    • uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a certificate

  • connects to DRPC provided Ethereum Mainnet

    • as a fallback upstream, which means that it’s used only if us-nodes fails

    • configuration is using placeholders for ${DRPC_KEY} which will be replaced with corresponding environment variables values

    • label [provider: drpc] is set for that particular upstream, which can be selected during a request.For example for some requests you may want to use nodes with that label only, i.e. "send that tx to drpc nodes only", or "read only from archive node, with label [archive: true]"

    • upstream validation (peers, sync status, etc) is disabled for that particular upstream

Nodes

[node-id: 1] is numeric node identifier defined in a range [1..255] and used to forward eth_getFilterChanges request to the node where one of eth_newFilter, eth_newBlockFilter or eth_newPendingTransactionFilter methods was executed (because filter is s stateful method).

It’s kindly recommended to strictly associate node-id parameter with a physical node and keep it during any configuration changes

Supported chains

List of supported chains could be found here

Roles and Fallback upstream

By default, the Dshackle connects to each upstream in a Round-Robin basis, i.e. sequentially one by one. If you need more gradual control over the order of which upstream is used and when you can assign following roles:

  • primary (default role if nothing specified)

  • secondary

  • fallback

Where primary and secondary are considered here a standard upstreams, and fallback is used on failure of standard upstreams. I.e. the Dshackle always starts with making requests to standard upstreams. If all of them failed, if responses are inconsistent (ex. for eth_getTransactionCount), or when it needs to broadcast to a wider network (sendrawtransaction), then upstreams with role fallback cames to use.

The internal request order is (goes to next only if all upstreams on current step a not available or failed):

  1. tries with primary upstreams

  2. tries with secondary upstream

  3. …​ delay (100ms at first, increased each iteration)

  4. tries with primary upstreams

  5. tries with secondary upstream

  6. tries with fallback upstreams

  7. …​ go to step 3

Steps 3-6 are repeated until a valid response received, or a timeout for the original request is reached.

In general: - you set role secondary for upstream in another cluster/datacenter - you set role fallback for an external upstream which may be provided by a third party, and you want to use it as a last resort

Configuration options

Options (default or as part of upstream config):

Option Default Description

disable-validation

false

if true then Dshackle will not try to verify status of the upstream (could be useful for a trusted cloud provider such as Infura, but disabling it is not recommended for a normal node)

min-peers

3

specify minimum amount of connected peers, Dshackle will not use upstream with less than specified number

timeout

60

timeout in seconds after which request to the upstream will be discarded (and may be retried on an another upstream)

balance

true for ethereum, false for bitcoin

specify if this node should be used to fetch balance for an address

Connection type

Dshackle currently supports

  • rpc a standard Ethereum JSON RPC

  • ws websocket connection (supposed to be used in addition to rpc connection)

  • grpc connects to another Dshackle instance

Connection mixture modes

In case of rpc and ws connection we can specify different modes of works together:

Type Description

WS_ONLY

Default mode in case WS endpoint specified. In this mode WS connection is used for all requests and subscriptions.

RPC_ONLY

Default in case WS endpoint not specified. In this mode RPC connection is used for all requests, subscriptions doesn’t work, head subscription works through scheduled RPC head request.

RPC_REQUESTS_WITH_MIXED_HEAD

All requests are sent through RPC connection, eth_subscribe is sent through WS connection, head subscription works through scheduled RPC head request mixed with WS subscription.

RPC_REQUESTS_WITH_WS_HEAD

All requests are sent through RPC connection, all subscriptions works through WS connection.

You can specify this modes through connector-mode parameter in connection config.

Bitcoin Methods

By default an ethereum upstream allows call to the following JSON RPC methods:

  • getbestblockhash

  • getblock

  • getblocknumber

  • getblockcount

  • gettransaction

  • getrawtransaction

  • gettxout

  • getreceivedbyaddress

  • listunspent

  • sendrawtransaction

Plus following methods are answered directly by Dshackle

  • getmemorypool

  • getconnectioncount

  • getnetworkinfo

Ethereum Methods

By default, an ethereum upstream allows calls to the following JSON RPC methods:

  • eth_gasPrice

  • eth_call

  • eth_estimateGas

  • eth_getBlockTransactionCountByHash

  • eth_getUncleCountByBlockHash

  • eth_getBlockByHash

  • eth_getTransactionByHash

  • eth_getTransactionByBlockHashAndIndex

  • eth_getStorageAt

  • eth_getCode

  • eth_getUncleByBlockHashAndIndex

  • eth_getTransactionCount

  • eth_blockNumber

  • eth_getBalance

  • eth_sendRawTransaction

  • eth_getBlockTransactionCountByNumber

  • eth_getUncleCountByBlockNumber

  • eth_getBlockByNumber

  • eth_getTransactionByBlockNumberAndIndex

  • eth_getTransactionReceipt

  • eth_getUncleByBlockNumberAndIndex

  • eth_feeHistory

  • eth_getLogs

Plus following methods are answered directly by Dshackle

  • net_version

  • net_peerCount

  • net_listening

  • web3_clientVersion

  • eth_protocolVersion

  • eth_syncing

  • eth_coinbase

  • eth_mining

  • eth_hashrate

  • eth_accounts

It’s possible to enable additional methods that are available on upstream, or disable an existing method. For that purpose there is methods configuration:

upstreams:
  - id: my-node
    chain: ethereum
    labels:
      archive: true
    methods:
      enabled:
        - name: trace_transaction
      disabled:
        - name: eth_getBlockByNumber

Such configuration option allows executing method trace_transaction and also disables eth_getBlockByNumber on that particular upstream. If a client requests to execute method trace_transaction then it will be scheduled to that upstream (or any upstream with such method enabled).

There are also method groups to more easily manage methods in batches, currently there are several groups:

upstreams:
  - id: my-node
    chain: ethereum
    labels:
      archive: true
    method-groups:
      enabled:
        - trace
      disabled:
        - default

This config allows you to ONLY enable trace methods and disable everything else.

Note
 It’s especially useful when used together with upstream labels.If an archive upstream has label archive: true it’s possible to specify that the client wants to execute method trace_transaction only on an archive node(s), which has complete historical data for tracing.

eth_subscribe ethereum methods

It is possible to control ethereum subscription types like regular methods. For now there are only one option - newPendingTransactions - you need to allow this method if you want to activate this type of subscription for Dshackle upstreams (disabled by default)

Static Methods

You can overwrite existing methods or add new ones using a static response:

upstreams:
  - id: my-node
    chain: ethereum
    methods:
      enabled:
        - name: net_version
          static: "\"100000\""
        - name: eth_chainId
          static: "0x186a0"
        - name: eth_custom_array
          static: '["custom_array_response"]'
        - name: eth_custom_bool
          static: "false"

Authentication

TLS

All connection types can use TLS secured connection, with optional client certificate authentication:

  • ca path to certificate required from remote server

  • optional certificate and key for client authentication.

Note
 Please note that key must be encoded with PKCS 8

Basic Authentication

For JSON RPC and Websockets a Basic Authentication can be used:

  • username - username

  • password - password

Chains specific configuration

We can use chain settings to specify chain specific behavior, for example rules for dshackle to work with upstream statuses

chains.yaml
chain-settings:
  default:
    lags:
      syncing: 6
      lagging: 1
  chains:
    - id: eth
      lags:
        syncing: 6
        lagging: 1
    - id: polygon
      lags:
        syncing: 20
        lagging: 10

Options

Option Description

lags.syncing

the size of the lag after which the upstream is determined to be syncing

lags.lagging

the size of the lag after which the upstream is determined to be lagging