Building an application specific blockchain using Cosmos SDK Part-5

Photo by Markus Spiske on Unsplash

In last part we covered handling the msg transactions. Hope you have tested handlers locally :) . In current part we will learn about indexing custom events & how users can subscribe to them via tendermint rpc websocket. Also we will see, how one can handle the state migrations during chain upgrade.

Indexing & Subscribing to custom events:

Tendermint allows us to index transactions and blocks and later query or subscribe to their results. By default tendermint uses the KV storage for indexing and tx.height, tx.hash are always indexed by default. Let’s see the custom event indexing in action.

One can index the custom events based on composite key. For suppose, we emit a custom event when creating a deal -

ctx.EventManager().EmitEvent(
 sdk.NewEvent(sdk.EventTypeMessage,
 sdk.NewAttribute(sdk.AttributeKeyModule, types.ModuleName),
 sdk.NewAttribute(types.IDVALUE, dealId),
 sdk.NewAttribute(types.OWNER, newDeal.Owner),
 sdk.NewAttribute(types.VENDOR, newDeal.Vendor),
 ),
)

Here the composite keys for above custom events are -

• message.module=’deal’
• message.action=’create_deal’
• message.idvalue={dealId}

etc…

Lets index the composite key message.action=’create_deal’

Configuration, genesis, private keys and storage files are generally stored under home directory in a hidden folder (.appName). In our case hidden folder name would be .deal You can find the same on your local machine. Among various configuration files,config.toml is the one that contains different parameters related to tendermint consensus engine, whereas app.toml contains app-specific parameters. Genesis.json contains the genesis state for each of the module.

Open the config.toml file and add the following tags field under indexer=kv.

tags="message.action='create_deal'"

Save the file and restart the chain. Before creating any deal let us subscribe to the composite key “message.action=’create_deal’” via web socket. You can use any web socket cli client or install the one- https://github.com/oliver006/ws-client for testing.

Open another terminal and run the command-

ws-client ws://localhost:26657/websocket
> { "jsonrpc": "2.0", "method": "subscribe", "params": ["message.action='create_deal'"], "id": 1 }

Now we are subscribed to the /subscribe tendermint RPC endpoint for custom event type create_deal . Let’s open a new instance of terminal to issue the create deal tx -

deald tx deal create-deal cosmos19aurpre8j3zd5gkngupz830vz5ta8ye8deuyzh 50 --from owner

As soon as execution gets completed, tendermint indexes the event and we get back the response via web-socket. For instance, I got back the response as below:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "query": "message.action='create_deal'",
    "data": {
      "type": "tendermint/event/Tx",
      "value": {
        "TxResult": {
          "height": "38235",
          "tx": "CogBCoUBCiEvSGFycnkwMjcuZGVhbC5kZWFsLk1zZ0NyZWF0ZURlYWwSYAotY29zbW9zMXh5cG4zZWw4bW1wOHA4Y3YyanR2ZXE1ODdwaHZ5eHl6OThsaHNnEi1jb3Ntb3MxOWF1cnByZThqM3pkNWdrbmd1cHo4MzB2ejV0YTh5ZThkZXV5emgYMhJYClAKRgofL2Nvc21vcy5jcnlwdG8uc2VjcDI1NmsxLlB1YktleRIjCiEDKxIThp2LiM3qZnARSu1Ve1E+mRsEks2ebAJYI+AvgskSBAoCCAEYEBIEEMCaDBpAdbvWanujmmFUVf8Nm0IM8V/8q5nxqykEHINi20tzRP8rd+x0SAu04yjW93LOldF1zvAqxISDL/fA9ytE+didFA==",
          "result": {
            "data": "CigKIS9IYXJyeTAyNy5kZWFsLmRlYWwuTXNnQ3JlYXRlRGVhbBIDCgE3",
            "log": "[{\"events\":[{\"type\":\"message\",\"attributes\":[{\"key\":\"action\",\"value\":\"create_deal\"},{\"key\":\"module\",\"value\":\"deal\"},{\"key\":\"IdValue\",\"value\":\"7\"},{\"key\":\"Owner\",\"value\":\"cosmos1xypn3el8mmp8p8cv2jtveq587phvyxyz98lhsg\"},{\"key\":\"Vendor\",\"value\":\"cosmos19aurpre8j3zd5gkngupz830vz5ta8ye8deuyzh\"}]}]}]",
            "gas_wanted": "200000",
            "gas_used": "48707",
            "events": [
              {
                "type": "tx",
                "attributes": [
                  {
                    "key": "ZmVl",
                    "value": "",
                    "index": true
                  }
                ]
              },
              {
                "type": "tx",
                "attributes": [
                  {
                    "key": "YWNjX3NlcQ==",
                    "value": "Y29zbW9zMXh5cG4zZWw4bW1wOHA4Y3YyanR2ZXE1ODdwaHZ5eHl6OThsaHNnLzE2",
                    "index": true
                  }
                ]
              },
              {
                "type": "tx",
                "attributes": [
                  {
                    "key": "c2lnbmF0dXJl",
                    "value": "ZGJ2V2FudWptbUZVVmY4Tm0wSU04Vi84cTVueHF5a0VISU5pMjB0elJQOHJkK3gwU0F1MDR5alc5M0xPbGRGMXp2QXF4SVNETC9mQTl5dEUrZGlkRkE9PQ==",
                    "index": true
                  }
                ]
              },
              {
                "type": "message",
                "attributes": [
                  {
                    "key": "YWN0aW9u",
                    "value": "Y3JlYXRlX2RlYWw=",
                    "index": true
                  }
                ]
              },
              {
                "type": "message",
                "attributes": [
                  {
                    "key": "bW9kdWxl",
                    "value": "ZGVhbA==",
                    "index": true
                  },
                  {
                    "key": "SWRWYWx1ZQ==",
                    "value": "Nw==",
                    "index": true
                  },
                  {
                    "key": "T3duZXI=",
                    "value": "Y29zbW9zMXh5cG4zZWw4bW1wOHA4Y3YyanR2ZXE1ODdwaHZ5eHl6OThsaHNn",
                    "index": true
                  },
                  {
                    "key": "VmVuZG9y",
                    "value": "Y29zbW9zMTlhdXJwcmU4ajN6ZDVna25ndXB6ODMwdno1dGE4eWU4ZGV1eXpo",
                    "index": true
                  }
                ]
              }
            ]
          }
        }
      }
    },
    "events": {
      "message.Owner": [
        "cosmos1xypn3el8mmp8p8cv2jtveq587phvyxyz98lhsg"
      ],
      "tx.hash": [
        "84264CAE7C5A99A495BDA5DECD2CC40562EFC524BF2CF6128522C1362D8254A9"
      ],
      "tx.acc_seq": [
        "cosmos1xypn3el8mmp8p8cv2jtveq587phvyxyz98lhsg/16"
      ],
      "tx.signature": [
        "dbvWanujmmFUVf8Nm0IM8V/8q5nxqykEHINi20tzRP8rd+x0SAu04yjW93LOldF1zvAqxISDL/fA9ytE+didFA=="
      ],
      "message.IdValue": [
        "7"
      ],
      "message.Vendor": [
        "cosmos19aurpre8j3zd5gkngupz830vz5ta8ye8deuyzh"
      ],
      "tm.event": [
        "Tx"
      ],
      "tx.height": [
        "38235"
      ],
      "tx.fee": [
        ""
      ],
      "message.action": [
        "create_deal"
      ],
      "message.module": [
        "deal"
      ]
    }
  }
}

You can also index multiple-events using tags field in config.toml-

tags="message.action='create_deal',message.action='create_contract'"

Note that tendermint will be implementing psql indexer & kv will be deprecated in future as the query syntax is limited for the kv indexer type whereas psql indexer will be able to proxy events to an external configured Postgresql instance. This will leverage SQL to perform a series of rich and complex queries which are not supported by the kv indexer type. Good to see more interesting features soon :)

Handling chain upgrades via in-place store migrations:

Suppose our chain is live in production and we want to upgrade the same. For instance to add the new feature, we might want to change the state structure. Or let us consider a simple task of removing expired contracts from the store. We do have an upgrade chain guide available here as part of cosmos sdk documentation - doc . We will refer the same for our migration example.

Let’s create a migrator concrete type inside our keeper package.

//migrator.go
package keeper
type Migrator struct {
 keeper Keeper
}
// NewMigrator returns a new Migrator.
func NewMigrator(keeper Keeper) Migrator {
 return Migrator{keeper: keeper}
}

The migrator wraps around the keeper which will be useful to make state transitions in migration script. We will put our migration script inside legacy folder under v01 package as suggested in the guide.

//v01.go
package v01
import (
 "fmt"
 "github.com/Harry-027/deal/x/deal/types"
 "github.com/cosmos/cosmos-sdk/codec"
 "github.com/cosmos/cosmos-sdk/store/prefix"
 sdk "github.com/cosmos/cosmos-sdk/types"
)
func MigrateDealStore(ctx sdk.Context, storeKey sdk.StoreKey, cdc codec.BinaryCodec) error {
 store := ctx.KVStore(storeKey)
  return pruneOldContracts(store, cdc)
}
func pruneOldContracts(store sdk.KVStore, cdc codec.BinaryCodec) error {
  dealStore := prefix.NewStore(store, []byte(types.NewDealKeyPrefix))
  dealStoreIter := dealStore.Iterator(nil, nil)
  defer dealStoreIter.Close()
  
  var deals []types.NewDeal
  for ; dealStoreIter.Valid(); dealStoreIter.Next() {
   var val types.NewDeal
   cdc.MustUnmarshal(dealStoreIter.Value(), &val)
   deals = append(deals, val)
  }
for _, deal := range deals {
  contractStorePrefixKey := fmt.Sprintf("%s%s/",    types.NewContractKeyPrefix, deal.DealId)
  contractStore := prefix.NewStore(store,  []byte(contractStorePrefixKey))
  iterator := sdk.KVStorePrefixIterator(contractStore, []byte{})
  defer iterator.Close()
for ; iterator.Valid(); iterator.Next() {
  var val types.NewContract
  cdc.MustUnmarshal(iterator.Value(), &val)
  if val.Status == "DELIVERED" {
   contractKey := fmt.Sprintf("%s%s/", types.NewContractKeyPrefix,    val.ContractId)
   contractStore.Delete([]byte(contractKey))
 }
}
}
 return nil
}

We are iterating over the contracts for each deal, to identify and delete the expired contracts. We will invoke this script via method on migration object -

// migrations.go
package keeper
import (
v01 "github.com/Harry-027/deal/x/deal/legacy/v01"
sdk "github.com/cosmos/cosmos-sdk/types"
)
// Migrate2to3 - Migrating deal module from version 2 to 3
func (m Migrator) Migrate2to3(ctx sdk.Context) error {
 return v01.MigrateDealStore(ctx, m.keeper.storeKey, m.keeper.cdc) // v01 is package `x/deal/legacy/v01`.
}

We will now register the Migrate2to3 method under configurator via registerMigration method for consensus version 3. Note the configurator is accessible under module method- RegisterServices

// module.go
func (am AppModule) RegisterServices(cfg module.Configurator) {
types.RegisterQueryServer(cfg.QueryServer(), am.keeper)
 m := keeper.NewMigrator(am.keeper)
 if err := cfg.RegisterMigration(types.ModuleName, 3, m.Migrate2to3); err != nil {
   panic(fmt.Sprintf("failed to migrate x/deal from version 2 to 3:
%v", err))
 }
}

One can register multiple migrations on configurator.

The x/upgrade module keeps track of all module consensus versions in the VersionMap store. To trigger the migration during upgrade we also need to update the consensus version for our module. Let’s change it from 2 to 3-

// ConsensusVersion implements ConsensusVersion.
func (AppModule) ConsensusVersion() uint64 { return 3 }

Any number of migrations registered on configurator for version 3 will be triggered during chain upgrade from module consensus version 2.

The chain upgrades usually takes place via voting on proposals. Proposals are usually submitted by the stakeholders via tx corresponding to gov module. Once validators and delegators participate and vote for the submitted proposal and voting period is completed, upgrade takes place at a certain block height via upgrade keeper handler. Thus we need to set the upgrade handler to carry out the migrations.

// app.go
app.UpgradeKeeper.SetUpgradeHandler("migrateOldContracts", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {
// Register the consensus version in the version map
 fromVM["deal"] = dealmodule.AppModule{}.ConsensusVersion()
 return app.mm.RunMigrations(ctx, cfg, fromVM)
})

The upgradeHandler will invoke the RunMigrations method on module manager which will internally invoke our module migration script. RunMigrations will also return back the new consensus version which will be then persisted in VersionMap store of x/upgrade module. Thus confirming the stable upgrade and latest consensus version of module.

One can submit the proposal to governance module-

deald tx gov submit-proposal software-upgrade migrateOldContracts --title migrateOldContracts --description migrateOldContracts --upgrade-height 100 --from validator --yes

and vote for it

deald tx gov deposit 1 10000000stake --from validator --yes
deald tx gov vote 1 yes --from validator --yes

Upgrade will occur automatically at block height 100.

As part of local development, one might want to test the migration scripts locally, without going via proposal way. To achieve it, one of the hack could be invoking migration script in module EndBlock method (As EndBlock gets automatically executed after each block, we will be able to test our migration script locally however this hack is not advisable for huge amount of migration data)-

//module.go
func (am AppModule) EndBlock(ctx sdk.Context, _ abci.RequestEndBlock) []abci.ValidatorUpdate {
// for testing if migration scripts run properly
   m := keeper.NewMigrator(am.keeper)
   m.Migrate2to3(ctx)
return []abci.ValidatorUpdate{}
}

Enough learning for today :) Will cover the blockchain test simulation in next part.

Refer the source code here- source code

Series
* Part-1
* Part-2
* Part-3
* Part-4
* Part-5
* Part-6

Join Coinmonks Telegram Channel and Youtube Channel learn about crypto trading and investing

Also, Read

• Cryptocurrency Savings Accounts | Crypto Trading Bots
• BigONE Exchange Review | CEX.IO Review | Swapzone Review
• Best Bitcoin Margin Trading| Bityard Margin Trading
• Crypto Margin Trading Exchanges | Earn Bitcoin
• WazirX vs CoinDCX vs Bitbns | BlockFi vs CoinLoan vs Nexo