# Node Client (Daemon)

The main endpoint of an SDK application is the daemon client, otherwise known as the full-node client. The full-node runs the state-machine, starting from a genesis file. It connects to peers running the same client in order to receive and relay transactions, block proposals and signatures. The full-node is constituted of the application, defined with the Cosmos SDK, and of a consensus engine connected to the application via the ABCI.

# Pre-requisite Readings

# main function

The full-node client of any SDK application is built by running a main function. The client is generally named by appending the -d suffix to the application name (e.g. appd for an application named app), and the main function is defined in a ./cmd/appd/main.go file. Running this function creates an executable .appd that comes with a set of commands. For an app named app, the main command is appd start, which starts the full-node.

In general, developers will implement the main.go function with the following structure:

  • First, a codec is instanciated for the application.
  • Then, the config is retrieved and config parameters are set. This mainly involves setting the bech32 prefixes for addresses and pubkeys. Copy // Config is the structure that holds the SDK configuration parameters. // This could be used to initialize certain configuration parameters for the SDK. type Config struct { fullFundraiserPath string keyringServiceName string bech32AddressPrefix map[string]string txEncoder TxEncoder addressVerifier func([]byte) error mtx sync.RWMutex coinType uint32 sealed bool }
  • Using cobra, the root command of the full-node client is created. After that, all the custom commands of the application are added using the AddCommand() method of rootCmd.
  • Add default server commands to rootCmd using the server.AddCommands(ctx, cdc, rootCmd, newApp, exportAppStateAndTMValidators) method. These commands are separated from the ones added above since they are standard and defined at SDK level. They should be shared by all SDK-based applications. They include the most important command: the start command.
  • Prepare and execute the executor.
    Copy type Executor struct { *cobra.Command Exit func(int) // this is os.Exit by default, override in tests }

See an example of main function from the gaia application:

Copy package main import ( "encoding/json" "io" "github.com/spf13/cobra" "github.com/spf13/viper" abci "github.com/tendermint/tendermint/abci/types" "github.com/tendermint/tendermint/libs/cli" "github.com/tendermint/tendermint/libs/log" tmtypes "github.com/tendermint/tendermint/types" dbm "github.com/tendermint/tm-db" "github.com/cosmos/gaia/app" "github.com/cosmos/cosmos-sdk/baseapp" "github.com/cosmos/cosmos-sdk/client" "github.com/cosmos/cosmos-sdk/server" "github.com/cosmos/cosmos-sdk/store" sdk "github.com/cosmos/cosmos-sdk/types" "github.com/cosmos/cosmos-sdk/x/auth" genutilcli "github.com/cosmos/cosmos-sdk/x/genutil/client/cli" "github.com/cosmos/cosmos-sdk/x/staking" ) const flagInvCheckPeriod = "inv-check-period" var invCheckPeriod uint func main() { cdc := app.MakeCodec() config := sdk.GetConfig() config.SetBech32PrefixForAccount(sdk.Bech32PrefixAccAddr, sdk.Bech32PrefixAccPub) config.SetBech32PrefixForValidator(sdk.Bech32PrefixValAddr, sdk.Bech32PrefixValPub) config.SetBech32PrefixForConsensusNode(sdk.Bech32PrefixConsAddr, sdk.Bech32PrefixConsPub) config.Seal() ctx := server.NewDefaultContext() cobra.EnableCommandSorting = false rootCmd := &cobra.Command{ Use: "gaiad", Short: "Gaia Daemon (server)", PersistentPreRunE: server.PersistentPreRunEFn(ctx), } rootCmd.AddCommand(genutilcli.InitCmd(ctx, cdc, app.ModuleBasics, app.DefaultNodeHome)) rootCmd.AddCommand(genutilcli.CollectGenTxsCmd(ctx, cdc, auth.GenesisAccountIterator{}, app.DefaultNodeHome)) rootCmd.AddCommand(genutilcli.MigrateGenesisCmd(ctx, cdc)) rootCmd.AddCommand( genutilcli.GenTxCmd( ctx, cdc, app.ModuleBasics, staking.AppModuleBasic{}, auth.GenesisAccountIterator{}, app.DefaultNodeHome, app.DefaultCLIHome, ), ) rootCmd.AddCommand(genutilcli.ValidateGenesisCmd(ctx, cdc, app.ModuleBasics)) rootCmd.AddCommand(AddGenesisAccountCmd(ctx, cdc, app.DefaultNodeHome, app.DefaultCLIHome)) rootCmd.AddCommand(client.NewCompletionCmd(rootCmd, true)) rootCmd.AddCommand(testnetCmd(ctx, cdc, app.ModuleBasics, auth.GenesisAccountIterator{})) rootCmd.AddCommand(replayCmd()) server.AddCommands(ctx, cdc, rootCmd, newApp, exportAppStateAndTMValidators) // prepare and add flags executor := cli.PrepareBaseCmd(rootCmd, "GA", app.DefaultNodeHome) rootCmd.PersistentFlags().UintVar(&invCheckPeriod, flagInvCheckPeriod, 0, "Assert registered invariants every N blocks") err := executor.Execute() if err != nil { panic(err) } } func newApp(logger log.Logger, db dbm.DB, traceStore io.Writer) abci.Application { var cache sdk.MultiStorePersistentCache if viper.GetBool(server.FlagInterBlockCache) { cache = store.NewCommitKVStoreCacheManager() } return app.NewGaiaApp( logger, db, traceStore, true, invCheckPeriod, baseapp.SetPruning(store.NewPruningOptionsFromString(viper.GetString("pruning"))), baseapp.SetMinGasPrices(viper.GetString(server.FlagMinGasPrices)), baseapp.SetHaltHeight(viper.GetUint64(server.FlagHaltHeight)), baseapp.SetHaltTime(viper.GetUint64(server.FlagHaltTime)), baseapp.SetInterBlockCache(cache), ) } func exportAppStateAndTMValidators( logger log.Logger, db dbm.DB, traceStore io.Writer, height int64, forZeroHeight bool, jailWhiteList []string, ) (json.RawMessage, []tmtypes.GenesisValidator, error) { if height != -1 { gapp := app.NewGaiaApp(logger, db, traceStore, false, uint(1)) err := gapp.LoadHeight(height) if err != nil { return nil, nil, err } return gapp.ExportAppStateAndValidators(forZeroHeight, jailWhiteList) } gapp := app.NewGaiaApp(logger, db, traceStore, true, uint(1)) return gapp.ExportAppStateAndValidators(forZeroHeight, jailWhiteList) }

# start command

The start command is defined in the /server folder of the Cosmos SDK. It is added to the root command of the full-node client in the main function and called by the end-user to start their node:

Copy // For an example app named "app", the following command starts the full-node appd start

As a reminder, the full-node is composed of three conceptual layers: the networking layer, the consensus layer and the application layer. The first two are generally bundled together in an entity called the consensus engine (Tendermint Core by default), while the third is the state-machine defined with the help of the Cosmos SDK. Currently, the Cosmos SDK uses Tendermint as the default consensus engine, meaning the start command is implemented to boot up a Tendermint node.

The flow of the start command is pretty straightforward. First, it retrieves the config from the context in order to open the db (a leveldb instance by default). This db contains the latest known state of the application (empty if the application is started from the first time.

With the db, the start command creates a new instance of the application using an appCreator function:

Copy package server // DONTCOVER import ( "fmt" "os" "runtime/pprof" "github.com/spf13/cobra" "github.com/spf13/viper" "github.com/tendermint/tendermint/abci/server" tcmd "github.com/tendermint/tendermint/cmd/tendermint/commands" cmn "github.com/tendermint/tendermint/libs/common" "github.com/tendermint/tendermint/node" "github.com/tendermint/tendermint/p2p" pvm "github.com/tendermint/tendermint/privval" "github.com/tendermint/tendermint/proxy" ) // Tendermint full-node start flags const ( flagWithTendermint = "with-tendermint" flagAddress = "address" flagTraceStore = "trace-store" flagPruning = "pruning" flagCPUProfile = "cpu-profile" FlagMinGasPrices = "minimum-gas-prices" FlagHaltHeight = "halt-height" FlagHaltTime = "halt-time" FlagInterBlockCache = "inter-block-cache" ) // StartCmd runs the service passed in, either stand-alone or in-process with // Tendermint. func StartCmd(ctx *Context, appCreator AppCreator) *cobra.Command { cmd := &cobra.Command{ Use: "start", Short: "Run the full node", Long: `Run the full node application with Tendermint in or out of process. By default, the application will run with Tendermint in process. Pruning options can be provided via the '--pruning' flag. The options are as follows: syncable: only those states not needed for state syncing will be deleted (keeps last 100 + every 10000th) nothing: all historic states will be saved, nothing will be deleted (i.e. archiving node) everything: all saved states will be deleted, storing only the current state Node halting configurations exist in the form of two flags: '--halt-height' and '--halt-time'. During the ABCI Commit phase, the node will check if the current block height is greater than or equal to the halt-height or if the current block time is greater than or equal to the halt-time. If so, the node will attempt to gracefully shutdown and the block will not be committed. In addition, the node will not be able to commit subsequent blocks. For profiling and benchmarking purposes, CPU profiling can be enabled via the '--cpu-profile' flag which accepts a path for the resulting pprof file. `, RunE: func(cmd *cobra.Command, args []string) error { if !viper.GetBool(flagWithTendermint) { ctx.Logger.Info("starting ABCI without Tendermint") return startStandAlone(ctx, appCreator) } ctx.Logger.Info("starting ABCI with Tendermint") _, err := startInProcess(ctx, appCreator) return err }, } // core flags for the ABCI application cmd.Flags().Bool(flagWithTendermint, true, "Run abci app embedded in-process with tendermint") cmd.Flags().String(flagAddress, "tcp://0.0.0.0:26658", "Listen address") cmd.Flags().String(flagTraceStore, "", "Enable KVStore tracing to an output file") cmd.Flags().String(flagPruning, "syncable", "Pruning strategy: syncable, nothing, everything") cmd.Flags().String( FlagMinGasPrices, "", "Minimum gas prices to accept for transactions; Any fee in a tx must meet this minimum (e.g. 0.01photino;0.0001stake)", ) cmd.Flags().Uint64(FlagHaltHeight, 0, "Block height at which to gracefully halt the chain and shutdown the node") cmd.Flags().Uint64(FlagHaltTime, 0, "Minimum block time (in Unix seconds) at which to gracefully halt the chain and shutdown the node") cmd.Flags().Bool(FlagInterBlockCache, true, "Enable inter-block caching") cmd.Flags().String(flagCPUProfile, "", "Enable CPU profiling and write to the provided file") // add support for all Tendermint-specific command line options tcmd.AddNodeFlags(cmd) return cmd } func startStandAlone(ctx *Context, appCreator AppCreator) error { addr := viper.GetString(flagAddress) home := viper.GetString("home") traceWriterFile := viper.GetString(flagTraceStore) db, err := openDB(home) if err != nil { return err } traceWriter, err := openTraceWriter(traceWriterFile) if err != nil { return err } app := appCreator(ctx.Logger, db, traceWriter) svr, err := server.NewServer(addr, "socket", app) if err != nil { return fmt.Errorf("error creating listener: %v", err) } svr.SetLogger(ctx.Logger.With("module", "abci-server")) err = svr.Start() if err != nil { cmn.Exit(err.Error()) } cmn.TrapSignal(ctx.Logger, func() { // cleanup err = svr.Stop() if err != nil { cmn.Exit(err.Error()) } }) // run forever (the node will not be returned) select {} } func startInProcess(ctx *Context, appCreator AppCreator) (*node.Node, error) { cfg := ctx.Config home := cfg.RootDir traceWriterFile := viper.GetString(flagTraceStore) db, err := openDB(home) if err != nil { return nil, err } traceWriter, err := openTraceWriter(traceWriterFile) if err != nil { return nil, err } app := appCreator(ctx.Logger, db, traceWriter) nodeKey, err := p2p.LoadOrGenNodeKey(cfg.NodeKeyFile()) if err != nil { return nil, err } UpgradeOldPrivValFile(cfg) // create & start tendermint node tmNode, err := node.NewNode( cfg, pvm.LoadOrGenFilePV(cfg.PrivValidatorKeyFile(), cfg.PrivValidatorStateFile()), nodeKey, proxy.NewLocalClientCreator(app), node.DefaultGenesisDocProviderFunc(cfg), node.DefaultDBProvider, node.DefaultMetricsProvider(cfg.Instrumentation), ctx.Logger.With("module", "node"), ) if err != nil { return nil, err } if err := tmNode.Start(); err != nil { return nil, err } var cpuProfileCleanup func() if cpuProfile := viper.GetString(flagCPUProfile); cpuProfile != "" { f, err := os.Create(cpuProfile) if err != nil { return nil, err } ctx.Logger.Info("starting CPU profiler", "profile", cpuProfile) if err := pprof.StartCPUProfile(f); err != nil { return nil, err } cpuProfileCleanup = func() { ctx.Logger.Info("stopping CPU profiler", "profile", cpuProfile) pprof.StopCPUProfile() f.Close() } } TrapSignal(func() { if tmNode.IsRunning() { _ = tmNode.Stop() } if cpuProfileCleanup != nil { cpuProfileCleanup() } ctx.Logger.Info("exiting...") }) // run forever (the node will not be returned) select {} }

Note that an appCreator is a function that fulfills the AppCreator signature. In practice, the constructor the application is passed as the appCreator.

Copy type ( // AppCreator is a function that allows us to lazily initialize an // application using various configurations. AppCreator func(log.Logger, dbm.DB, io.Writer) abci.Application // AppExporter is a function that dumps all app state to // JSON-serializable structure and returns the current validator set. AppExporter func(log.Logger, dbm.DB, io.Writer, int64, bool, []string) (json.RawMessage, []tmtypes.GenesisValidator, error) )

Then, the instance of app is used to instanciate a new Tendermint node:

Copy // create & start tendermint node tmNode, err := node.NewNode( cfg, pvm.LoadOrGenFilePV(cfg.PrivValidatorKeyFile(), cfg.PrivValidatorStateFile()), nodeKey, proxy.NewLocalClientCreator(app), node.DefaultGenesisDocProviderFunc(cfg), node.DefaultDBProvider, node.DefaultMetricsProvider(cfg.Instrumentation), ctx.Logger.With("module", "node"), )

The Tendermint node can be created with app because the latter satisfies the abci.Application interface (given that app extends baseapp). As part of the NewNode method, Tendermint makes sure that the height of the application (i.e. number of blocks since genesis) is equal to the height of the Tendermint node. The difference between these two heights should always be negative or null. If it is strictly negative, NewNode will replay blocks until the height of the application reaches the height of the Tendermint node. Finally, if the height of the application is 0, the Tendermint node will call InitChain on the application to initialize the state from the genesis file.

Once the Tendermint node is instanciated and in sync with the application, the node can be started:

Copy if err := tmNode.Start(); err != nil { return nil, err }

Upon starting, the node will bootstrap its RPC and P2P server and start dialing peers. During handshake with its peers, if the node realizes they are ahead, it will query all the blocks sequentially in order to catch up. Then, it will wait for new block proposals and block signatures from validators in order to make progress.

# Next

Learn about the store