Anatomy of a Cosmos SDK Application
This document describes the core parts of a Cosmos SDK application, represented throughout the document as a placeholder application named app
.
Node Client
The Daemon, or Full-Node Client, is the core process of a Cosmos SDK-based blockchain. Participants in the network run this process to initialize their state-machine, connect with other full-nodes, and update their state-machine as new blocks come in.
The blockchain full-node presents itself as a binary, generally suffixed by -d
for "daemon" (e.g. appd
for app
or gaiad
for gaia
). This binary is built by running a simple main.go
function placed in ./cmd/appd/
. This operation usually happens through the Makefile.
Once the main binary is built, the node can be started by running the start
command. This command function primarily does three things:
- Create an instance of the state-machine defined in
app.go
. - Initialize the state-machine with the latest known state, extracted from the
db
stored in the~/.app/data
folder. At this point, the state-machine is at heightappBlockHeight
. - Create and start a new CometBFT instance. Among other things, the node performs a handshake with its peers. It gets the latest
blockHeight
from them and replays blocks to sync to this height if it is greater than the localappBlockHeight
. The node starts from genesis and CometBFT sends anInitChain
message via the ABCI to theapp
, which triggers theInitChainer
.
When starting a CometBFT instance, the genesis file is the 0
height and the state within the genesis file is committed at block height 1
. When querying the state of the node, querying block height 0 will return an error.
Core Application File
In general, the core of the state-machine is defined in a file called app.go
. This file mainly contains the type definition of the application and functions to create and initialize it.
Type Definition of the Application
Constructor Function
Also defined in app.go
is the constructor function, which constructs a new application of the type defined in the preceding section. The function must fulfill the AppCreator
signature in order to be used in the start
command of the application's daemon command.
loading...
Here are the main actions performed by this function:
- Instantiate a new
codec
and initialize thecodec
of each of the application's modules using the module manager. - Instantiate a new application with a reference to a
baseapp
instance, a codec, and all the appropriate store keys. - Instantiate all the
keeper
objects defined in the application'stype
using theNewKeeper
function of each of the application's modules. Note that keepers must be instantiated in the correct order, as theNewKeeper
of one module might require a reference to another module'skeeper
. - Instantiate the application's module manager with the
AppModule
object of each of the application's modules. - With the module manager, initialize the application's
Msg
services, gRPCQuery
services, legacyMsg
routes, and legacy query routes. When a transaction is relayed to the application by CometBFT via the ABCI, it is routed to the appropriate module'sMsg
service using the routes defined here. Likewise, when a gRPC query request is received by the application, it is routed to the appropriate module'sgRPC query service
using the gRPC routes defined here. The Cosmos SDK still supports legacyMsg
s and legacy CometBFT queries, which are routed using the legacyMsg
routes and the legacy query routes, respectively. - With the module manager, set the order of execution between the
InitGenesis
,PreBlocker
,BeginBlocker
, andEndBlocker
functions of each of the application's modules. Note that not all modules implement these functions. - Set the remaining application parameters:
InitChainer
: used to initialize the application when it is first started.PreBlocker
: called before BeginBlock.BeginBlocker
,EndBlocker
: called at the beginning and at the end of every block.anteHandler
: used to handle fees and signature verification.
- Mount the stores.
- Return the application.
Note that the constructor function only creates an instance of the app, while the actual state is either carried over from the ~/.app/data
folder if the node is restarted, or generated from the genesis file if the node is started for the first time.
See an example of application constructor from simapp
:
loading...
InitChainer
The InitChainer
is a function that initializes the state of the application from a genesis file (i.e. token balances of genesis accounts). It is called when the application receives the InitChain
message from the CometBFT engine, which happens when the node is started at appBlockHeight == 0
(i.e. on genesis). The application must set the InitChainer
in its constructor via the SetInitChainer
method.
In general, the InitChainer
is mostly composed of the InitGenesis
function of each of the application's modules. This is done by calling the InitGenesis
function of the module manager, which in turn calls the InitGenesis
function of each of the modules it contains. Note that the order in which the modules' InitGenesis
functions must be called has to be set in the module manager using the module manager's SetOrderInitGenesis
method. This is done in the application's constructor, and the SetOrderInitGenesis
has to be called before the SetInitChainer
.
See an example of an InitChainer
from simapp
:
loading...
PreBlocker
There are two semantics around the new lifecycle method:
- It runs before the
BeginBlocker
of all modules - It can modify consensus parameters in storage, and signal the caller through the return value.
When it returns ConsensusParamsChanged=true
, the caller must refresh the consensus parameter in the finalize context:
app.finalizeBlockState.ctx = app.finalizeBlockState.ctx.WithConsensusParams(app.GetConsensusParams())
The new ctx must be passed to all the other lifecycle methods.
BeginBlocker and EndBlocker
The Cosmos SDK offers developers the possibility to implement automatic execution of code as part of their application. This is implemented through two functions called BeginBlocker
and EndBlocker
. They are called when the application receives the FinalizeBlock
messages from the CometBFT consensus engine, which happens respectively at the beginning and at the end of each block. The application must set the BeginBlocker
and EndBlocker
in its constructor via the SetBeginBlocker
and SetEndBlocker
methods.
In general, the BeginBlocker
and EndBlocker
functions are mostly composed of the BeginBlock
and EndBlock
functions of each of the application's modules. This is done by calling the BeginBlock
and EndBlock
functions of the module manager, which in turn calls the BeginBlock
and EndBlock
functions of each of the modules it contains. Note that the order in which the modules' BeginBlock
and EndBlock
functions must be called has to be set in the module manager using the SetOrderBeginBlockers
and SetOrderEndBlockers
methods, respectively. This is done via the module manager in the application's constructor, and the SetOrderBeginBlockers
and SetOrderEndBlockers
methods have to be called before the SetBeginBlocker
and SetEndBlocker
functions.
As a sidenote, it is important to remember that application-specific blockchains are deterministic. Developers must be careful not to introduce non-determinism in BeginBlocker
or EndBlocker
, and must also be careful not to make them too computationally expensive, as gas does not constrain the cost of BeginBlocker
and EndBlocker
execution.
See an example of BeginBlocker
and EndBlocker
functions from simapp
loading...
Register Codec
The EncodingConfig
structure is the last important part of the app.go
file. This structure's purpose is to define the codecs that will be used throughout the app.
loading...
Here are descriptions of what each of the four fields means:
InterfaceRegistry
: TheInterfaceRegistry
is used by the Protobuf codec to handle interfaces that are encoded and decoded (we also say "unpacked") usinggoogle.protobuf.Any
.Any
could be thought as a struct that contains atype_url
(name of a concrete type implementing the interface) and avalue
(its encoded bytes).InterfaceRegistry
provides a mechanism for registering interfaces and implementations that can be safely unpacked fromAny
. Each application module implements theRegisterInterfaces
method that can be used to register the module's own interfaces and implementations.- You can read more about
Any
in ADR-019. - To go more into details, the Cosmos SDK uses an implementation of the Protobuf specification called
gogoprotobuf
. By default, the gogo protobuf implementation ofAny
uses global type registration to decode values packed inAny
into concrete Go types. This introduces a vulnerability where any malicious module in the dependency tree could register a type with the global protobuf registry and cause it to be loaded and unmarshaled by a transaction that referenced it in thetype_url
field. For more information, please refer to ADR-019.
- You can read more about
Codec
: The default codec used throughout the Cosmos SDK. It is composed of aBinaryCodec
used to encode and decode state, and aJSONCodec
used to output data to the users (for example, in the CLI). By default, the SDK uses Protobuf asCodec
.TxConfig
:TxConfig
defines an interface a client can utilize to generate an application-defined concrete transaction type. Currently, the SDK handles two transaction types:SIGN_MODE_DIRECT
(which uses Protobuf binary as over-the-wire encoding) andSIGN_MODE_LEGACY_AMINO_JSON
(which depends on Amino). Read more about transactions here.Amino
: Some legacy parts of the Cosmos SDK still use Amino for backwards-compatibility. Each module exposes aRegisterLegacyAmino
method to register the module's specific types within Amino. ThisAmino
codec should not be used by app developers anymore, and will be removed in future releases.
An application should create its own encoding config.
See an example of a simappparams.EncodingConfig
from simapp
:
loading...
Modules
Modules are the heart and soul of Cosmos SDK applications. They can be considered as state-machines nested within the state-machine. When a transaction is relayed from the underlying CometBFT engine via the ABCI to the application, it is routed by baseapp
to the appropriate module in order to be processed. This paradigm enables developers to easily build complex state-machines, as most of the modules they need often already exist. For developers, most of the work involved in building a Cosmos SDK application revolves around building custom modules required by their application that do not exist yet, and integrating them with modules that do already exist into one coherent application. In the application directory, the standard practice is to store modules in the x/
folder (not to be confused with the Cosmos SDK's x/
folder, which contains already-built modules).
Application Module Interface
Modules must implement interfaces defined in the Cosmos SDK and AppModule
. AppModule
handles the bulk of the module methods (including methods that require references to other modules' keeper
s). AppModule
is, by convention, defined in a file called module.go
.
AppModule
exposes a collection of useful methods on the module that facilitates the composition of modules into a coherent application. These methods are called from the module manager
, which manages the application's collection of modules.
Msg
Services
Each application module defines two Protobuf services: one Msg
service to handle messages, and one gRPC Query
service to handle queries. If we consider the module as a state-machine, then a Msg
service is a set of state transition RPC methods.
Each Protobuf Msg
service method is 1:1 related to a Protobuf request type, which must implement sdk.Msg
interface.
Note that sdk.Msg
s are bundled in transactions, and each transaction contains one or multiple messages.
When a valid block of transactions is received by the full-node, CometBFT relays each one to the application via DeliverTx
. Then, the application handles the transaction:
- Upon receiving the transaction, the application first unmarshalls it from
[]byte
. - Then, it verifies a few things about the transaction like fee payment and signatures before extracting the
Msg
(s) contained in the transaction. sdk.Msg
s are encoded using ProtobufAny
s. By analyzing eachAny
'stype_url
, baseapp'smsgServiceRouter
routes thesdk.Msg
to the corresponding module'sMsg
service.- If the message is successfully processed, the state is updated.
For more details, see transaction lifecycle.
Module developers create custom Msg
services when they build their own module. The general practice is to define the Msg
Protobuf service in a tx.proto
file. For example, the x/bank
module defines a service with two methods to transfer tokens:
loading...
Service methods use keeper
in order to update the module state.
Each module should also implement the RegisterServices
method as part of the AppModule
interface. This method should call the RegisterMsgServer
function provided by the generated Protobuf code.
gRPC Query
Services
gRPC Query
services allow users to query the state using gRPC. They are enabled by default, and can be configured under the grpc.enable
and grpc.address
fields inside app.toml
.
gRPC Query
services are defined in the module's Protobuf definition files, specifically inside query.proto
. The query.proto
definition file exposes a single Query
Protobuf service. Each gRPC query endpoint corresponds to a service method, starting with the rpc
keyword, inside the Query
service.
Protobuf generates a QueryServer
interface for each module, containing all the service methods. A module's keeper
then needs to implement this QueryServer
interface, by providing the concrete implementation of each service method. This concrete implementation is the handler of the corresponding gRPC query endpoint.
Finally, each module should also implement the RegisterServices
method as part of the AppModule
interface. This method should call the RegisterQueryServer
function provided by the generated Protobuf code.
Keeper
Keepers
are the gatekeepers of their module's store(s). To read or write in a module's store, it is mandatory to go through one of its keeper
's methods. This is ensured by the object-capabilities model of the Cosmos SDK. Only objects that hold the key to a store can access it, and only the module's keeper
should hold the key(s) to the module's store(s).
Keepers
are generally defined in a file called keeper.go
. It contains the keeper
's type definition and methods.
The keeper
type definition generally consists of the following:
- Key(s) to the module's store(s) in the multistore.
- Reference to other module's
keepers
. Only needed if thekeeper
needs to access other module's store(s) (either to read or write from them). - A reference to the application's codec. The
keeper
needs it to marshal structs before storing them, or to unmarshal them when it retrieves them, because stores only accept[]bytes
as value.
Along with the type definition, the next important component of the keeper.go
file is the keeper
's constructor function, NewKeeper
. This function instantiates a new keeper
of the type defined above with a codec
, stores keys
and potentially references other modules' keeper
s as parameters. The NewKeeper
function is called from the application's constructor. The rest of the file defines the keeper
's methods, which are primarily getters and setters.
Command-Line, gRPC Services and REST Interfaces
Each module defines command-line commands, gRPC services, and REST routes to be exposed to the end-user via the application's interfaces. This enables end-users to create messages of the types defined in the module, or to query the subset of the state managed by the module.
CLI
Generally, the commands related to a module are defined in a folder called client/cli
in the module's folder. The CLI divides commands into two categories, transactions and queries, defined in client/cli/tx.go
and client/cli/query.go
, respectively. Both commands are built on top of the Cobra Library:
- Transactions commands let users generate new transactions so that they can be included in a block and eventually update the state. One command should be created for each message type defined in the module. The command calls the constructor of the message with the parameters provided by the end-user, and wraps it into a transaction. The Cosmos SDK handles signing and the addition of other transaction metadata.
- Queries let users query the subset of the state defined by the module. Query commands forward queries to the application's query router, which routes them to the appropriate querier the
queryRoute
parameter supplied.
gRPC
gRPC is a modern open-source high performance RPC framework that has support in multiple languages. It is the recommended way for external clients (such as wallets, browsers and other backend services) to interact with a node.
Each module can expose gRPC endpoints called service methods, which are defined in the module's Protobuf query.proto
file. A service method is defined by its name, input arguments, and output response. The module then needs to perform the following actions:
- (Optional) Define a
RegisterGRPCGatewayRoutes
method onAppModule
to wire the client gRPC requests to the correct handler inside the module. - For each service method, define a corresponding handler. The handler implements the core logic necessary to serve the gRPC request, and is located in the
keeper/grpc_query.go
file.
gRPC-gateway REST Endpoints
Some external clients may not wish to use gRPC. In this case, the Cosmos SDK provides a gRPC gateway service, which exposes each gRPC service as a corresponding REST endpoint. Please refer to the grpc-gateway documentation to learn more.
The REST endpoints are defined in the Protobuf files, along with the gRPC services, using Protobuf annotations. Modules that want to expose REST queries should add google.api.http
annotations to their rpc
methods. By default, all REST endpoints defined in the SDK have a URL starting with the /cosmos/
prefix.
The Cosmos SDK also provides a development endpoint to generate Swagger definition files for these REST endpoints. This endpoint can be enabled inside the app.toml
config file, under the api.swagger
key.
Application Interface
Interfaces let end-users interact with full-node clients. This means querying data from the full-node or creating and sending new transactions to be relayed by the full-node and eventually included in a block.
The main interface is the Command-Line Interface. The CLI of a Cosmos SDK application is built by aggregating CLI commands defined in each of the modules used by the application. The CLI of an application is the same as the daemon (e.g. appd
), and is defined in a file called appd/main.go
. The file contains the following:
- A
main()
function, which is executed to build theappd
interface client. This function prepares each command and adds them to therootCmd
before building them. At the root ofappd
, the function adds generic commands likestatus
,keys
, andconfig
, query commands, tx commands, andrest-server
. - Query commands, which are added by calling the
queryCmd
function. This function returns a Cobra command that contains the query commands defined in each of the application's modules (passed as an array ofsdk.ModuleClients
from themain()
function), as well as some other lower level query commands such as block or validator queries. Query command are called by using the commandappd query [query]
of the CLI. - Transaction commands, which are added by calling the
txCmd
function. Similar toqueryCmd
, the function returns a Cobra command that contains the tx commands defined in each of the application's modules, as well as lower level tx commands like transaction signing or broadcasting. Tx commands are called by using the commandappd tx [tx]
of the CLI.
See an example of an application's main command-line file from the Cosmos Hub.
loading...
Dependencies and Makefile
This section is optional, as developers are free to choose their dependency manager and project building method. That said, the current most used framework for versioning control is go.mod
. It ensures each of the libraries used throughout the application are imported with the correct version.
The following is the go.mod
of the Cosmos Hub, provided as an example.
loading...
For building the application, a Makefile is generally used. The Makefile primarily ensures that the go.mod
is run before building the two entrypoints to the application, Node Client
and Application Interface
.
Here is an example of the Cosmos Hub Makefile.