# Node Client (Daemon)

The main endpoint of an SDK application is the daemon client, otherwise known as the full-node client. The full-node runs the state-machine, starting from a genesis file. It connects to peers running the same client in order to receive and relay transactions, block proposals and signatures. The full-node is constituted of the application, defined with the Cosmos SDK, and of a consensus engine connected to the application via the ABCI.

# Pre-requisite Readings

# main function

The full-node client of any SDK application is built by running a main function. The client is generally named by appending the -d suffix to the application name (e.g. appd for an application named app), and the main function is defined in a ./appd/cmd/main.go file. Running this function creates an executable appd that comes with a set of commands. For an app named app, the main command is appd start, which starts the full-node.

In general, developers will implement the main.go function with the following structure:

  • First, an appCodec is instanciated for the application.
  • Then, the config is retrieved and config parameters are set. This mainly involves setting the bech32 prefixes for addresses and pubkeys. Copy // Config is the structure that holds the SDK configuration parameters. // This could be used to initialize certain configuration parameters for the SDK. type Config struct { fullFundraiserPath string bech32AddressPrefix map[string]string txEncoder TxEncoder addressVerifier func([]byte) error mtx sync.RWMutex coinType uint32 sealed bool sealedch chan struct{} }
  • Using cobra (opens new window), the root command of the full-node client is created. After that, all the custom commands of the application are added using the AddCommand() method of rootCmd.
  • Add default server commands to rootCmd using the server.AddCommands() method. These commands are separated from the ones added above since they are standard and defined at SDK level. They should be shared by all SDK-based applications. They include the most important command: the start command.
  • Prepare and execute the executor.
    Copy // Executor wraps the cobra Command with a nicer Execute method type Executor struct { *cobra.Command Exit func(int) // this is os.Exit by default, override in tests }

See an example of main function from the simapp application, the SDK's application for demo purposes:

Copy package main import ( "os" "github.com/cosmos/cosmos-sdk/server" "github.com/cosmos/cosmos-sdk/simapp/simd/cmd" ) func main() { rootCmd, _ := cmd.NewRootCmd() if err := cmd.Execute(rootCmd); err != nil { switch e := err.(type) { case server.ErrorCode: os.Exit(e.Code) default: os.Exit(1) } } }

# start command

The start command is defined in the /server folder of the Cosmos SDK. It is added to the root command of the full-node client in the main function and called by the end-user to start their node:

Copy # For an example app named "app", the following command starts the full-node. appd start # Using the SDK's own simapp, the following commands start the simapp node. simd start

As a reminder, the full-node is composed of three conceptual layers: the networking layer, the consensus layer and the application layer. The first two are generally bundled together in an entity called the consensus engine (Tendermint Core by default), while the third is the state-machine defined with the help of the Cosmos SDK. Currently, the Cosmos SDK uses Tendermint as the default consensus engine, meaning the start command is implemented to boot up a Tendermint node.

The flow of the start command is pretty straightforward. First, it retrieves the config from the context in order to open the db (a leveldb (opens new window) instance by default). This db contains the latest known state of the application (empty if the application is started from the first time.

With the db, the start command creates a new instance of the application using an appCreator function:

Copy package server // DONTCOVER import ( "fmt" "os" "runtime/pprof" "time" "github.com/spf13/cobra" "github.com/tendermint/tendermint/abci/server" tcmd "github.com/tendermint/tendermint/cmd/tendermint/commands" tmos "github.com/tendermint/tendermint/libs/os" "github.com/tendermint/tendermint/node" "github.com/tendermint/tendermint/p2p" pvm "github.com/tendermint/tendermint/privval" "github.com/tendermint/tendermint/proxy" "github.com/tendermint/tendermint/rpc/client/local" "google.golang.org/grpc" "github.com/cosmos/cosmos-sdk/client" "github.com/cosmos/cosmos-sdk/client/flags" "github.com/cosmos/cosmos-sdk/server/api" "github.com/cosmos/cosmos-sdk/server/config" servergrpc "github.com/cosmos/cosmos-sdk/server/grpc" "github.com/cosmos/cosmos-sdk/server/types" storetypes "github.com/cosmos/cosmos-sdk/store/types" ) // Tendermint full-node start flags const ( flagWithTendermint = "with-tendermint" flagAddress = "address" flagTransport = "transport" flagTraceStore = "trace-store" flagCPUProfile = "cpu-profile" FlagMinGasPrices = "minimum-gas-prices" FlagHaltHeight = "halt-height" FlagHaltTime = "halt-time" FlagInterBlockCache = "inter-block-cache" FlagUnsafeSkipUpgrades = "unsafe-skip-upgrades" FlagTrace = "trace" FlagInvCheckPeriod = "inv-check-period" FlagPruning = "pruning" FlagPruningKeepRecent = "pruning-keep-recent" FlagPruningKeepEvery = "pruning-keep-every" FlagPruningInterval = "pruning-interval" FlagIndexEvents = "index-events" FlagMinRetainBlocks = "min-retain-blocks" ) // GRPC-related flags. const ( flagGRPCEnable = "grpc.enable" flagGRPCAddress = "grpc.address" ) // State sync-related flags. const ( FlagStateSyncSnapshotInterval = "state-sync.snapshot-interval" FlagStateSyncSnapshotKeepRecent = "state-sync.snapshot-keep-recent" ) // StartCmd runs the service passed in, either stand-alone or in-process with // Tendermint. func StartCmd(appCreator types.AppCreator, defaultNodeHome string) *cobra.Command { cmd := &cobra.Command{ Use: "start", Short: "Run the full node", Long: `Run the full node application with Tendermint in or out of process. By default, the application will run with Tendermint in process. Pruning options can be provided via the '--pruning' flag or alternatively with '--pruning-keep-recent', 'pruning-keep-every', and 'pruning-interval' together. For '--pruning' the options are as follows: default: the last 100 states are kept in addition to every 500th state; pruning at 10 block intervals nothing: all historic states will be saved, nothing will be deleted (i.e. archiving node) everything: all saved states will be deleted, storing only the current state; pruning at 10 block intervals custom: allow pruning options to be manually specified through 'pruning-keep-recent', 'pruning-keep-every', and 'pruning-interval' Node halting configurations exist in the form of two flags: '--halt-height' and '--halt-time'. During the ABCI Commit phase, the node will check if the current block height is greater than or equal to the halt-height or if the current block time is greater than or equal to the halt-time. If so, the node will attempt to gracefully shutdown and the block will not be committed. In addition, the node will not be able to commit subsequent blocks. For profiling and benchmarking purposes, CPU profiling can be enabled via the '--cpu-profile' flag which accepts a path for the resulting pprof file. `, PreRunE: func(cmd *cobra.Command, _ []string) error { serverCtx := GetServerContextFromCmd(cmd) // Bind flags to the Context's Viper so the app construction can set // options accordingly. serverCtx.Viper.BindPFlags(cmd.Flags()) _, err := GetPruningOptionsFromFlags(serverCtx.Viper) return err }, RunE: func(cmd *cobra.Command, _ []string) error { serverCtx := GetServerContextFromCmd(cmd) clientCtx := client.GetClientContextFromCmd(cmd) withTM, _ := cmd.Flags().GetBool(flagWithTendermint) if !withTM { serverCtx.Logger.Info("starting ABCI without Tendermint") return startStandAlone(serverCtx, appCreator) } serverCtx.Logger.Info("starting ABCI with Tendermint") // amino is needed here for backwards compatibility of REST routes err := startInProcess(serverCtx, clientCtx, appCreator) return err }, } cmd.Flags().String(flags.FlagHome, defaultNodeHome, "The application home directory") cmd.Flags().Bool(flagWithTendermint, true, "Run abci app embedded in-process with tendermint") cmd.Flags().String(flagAddress, "tcp://", "Listen address") cmd.Flags().String(flagTransport, "socket", "Transport protocol: socket, grpc") cmd.Flags().String(flagTraceStore, "", "Enable KVStore tracing to an output file") cmd.Flags().String(FlagMinGasPrices, "", "Minimum gas prices to accept for transactions; Any fee in a tx must meet this minimum (e.g. 0.01photino;0.0001stake)") cmd.Flags().IntSlice(FlagUnsafeSkipUpgrades, []int{}, "Skip a set of upgrade heights to continue the old binary") cmd.Flags().Uint64(FlagHaltHeight, 0, "Block height at which to gracefully halt the chain and shutdown the node") cmd.Flags().Uint64(FlagHaltTime, 0, "Minimum block time (in Unix seconds) at which to gracefully halt the chain and shutdown the node") cmd.Flags().Bool(FlagInterBlockCache, true, "Enable inter-block caching") cmd.Flags().String(flagCPUProfile, "", "Enable CPU profiling and write to the provided file") cmd.Flags().Bool(FlagTrace, false, "Provide full stack traces for errors in ABCI Log") cmd.Flags().String(FlagPruning, storetypes.PruningOptionDefault, "Pruning strategy (default|nothing|everything|custom)") cmd.Flags().Uint64(FlagPruningKeepRecent, 0, "Number of recent heights to keep on disk (ignored if pruning is not 'custom')") cmd.Flags().Uint64(FlagPruningKeepEvery, 0, "Offset heights to keep on disk after 'keep-every' (ignored if pruning is not 'custom')") cmd.Flags().Uint64(FlagPruningInterval, 0, "Height interval at which pruned heights are removed from disk (ignored if pruning is not 'custom')") cmd.Flags().Uint(FlagInvCheckPeriod, 0, "Assert registered invariants every N blocks") cmd.Flags().Uint64(FlagMinRetainBlocks, 0, "Minimum block height offset during ABCI commit to prune Tendermint blocks") cmd.Flags().Bool(flagGRPCEnable, true, "Define if the gRPC server should be enabled") cmd.Flags().String(flagGRPCAddress, config.DefaultGRPCAddress, "the gRPC server address to listen on") cmd.Flags().Uint64(FlagStateSyncSnapshotInterval, 0, "State sync snapshot interval") cmd.Flags().Uint32(FlagStateSyncSnapshotKeepRecent, 2, "State sync snapshot to keep") // add support for all Tendermint-specific command line options tcmd.AddNodeFlags(cmd) return cmd } func startStandAlone(ctx *Context, appCreator types.AppCreator) error { addr := ctx.Viper.GetString(flagAddress) transport := ctx.Viper.GetString(flagTransport) home := ctx.Viper.GetString(flags.FlagHome) db, err := openDB(home) if err != nil { return err } traceWriterFile := ctx.Viper.GetString(flagTraceStore) traceWriter, err := openTraceWriter(traceWriterFile) if err != nil { return err } app := appCreator(ctx.Logger, db, traceWriter, ctx.Viper) svr, err := server.NewServer(addr, transport, app) if err != nil { return fmt.Errorf("error creating listener: %v", err) } svr.SetLogger(ctx.Logger.With("module", "abci-server")) err = svr.Start() if err != nil { tmos.Exit(err.Error()) } defer func() { if err = svr.Stop(); err != nil { tmos.Exit(err.Error()) } }() // Wait for SIGINT or SIGTERM signal return WaitForQuitSignals() } // legacyAminoCdc is used for the legacy REST API func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.AppCreator) error { cfg := ctx.Config home := cfg.RootDir var cpuProfileCleanup func() if cpuProfile := ctx.Viper.GetString(flagCPUProfile); cpuProfile != "" { f, err := os.Create(cpuProfile) if err != nil { return err } ctx.Logger.Info("starting CPU profiler", "profile", cpuProfile) if err := pprof.StartCPUProfile(f); err != nil { return err } cpuProfileCleanup = func() { ctx.Logger.Info("stopping CPU profiler", "profile", cpuProfile) pprof.StopCPUProfile() f.Close() } } traceWriterFile := ctx.Viper.GetString(flagTraceStore) db, err := openDB(home) if err != nil { return err } traceWriter, err := openTraceWriter(traceWriterFile) if err != nil { return err } app := appCreator(ctx.Logger, db, traceWriter, ctx.Viper) nodeKey, err := p2p.LoadOrGenNodeKey(cfg.NodeKeyFile()) if err != nil { return err } genDocProvider := node.DefaultGenesisDocProviderFunc(cfg) tmNode, err := node.NewNode( cfg, pvm.LoadOrGenFilePV(cfg.PrivValidatorKeyFile(), cfg.PrivValidatorStateFile()), nodeKey, proxy.NewLocalClientCreator(app), genDocProvider, node.DefaultDBProvider, node.DefaultMetricsProvider(cfg.Instrumentation), ctx.Logger.With("module", "node"), ) if err != nil { return err } ctx.Logger.Debug("Initialization: tmNode created") if err := tmNode.Start(); err != nil { return err } ctx.Logger.Debug("Initialization: tmNode started") // Add the tx service to the gRPC router. app.RegisterTxService(clientCtx) var apiSrv *api.Server config := config.GetConfig(ctx.Viper) if config.API.Enable { genDoc, err := genDocProvider() if err != nil { return err } clientCtx := clientCtx. WithHomeDir(home). WithChainID(genDoc.ChainID). WithClient(local.New(tmNode)) apiSrv = api.New(clientCtx, ctx.Logger.With("module", "api-server")) app.RegisterAPIRoutes(apiSrv, config.API) errCh := make(chan error) go func() { if err := apiSrv.Start(config); err != nil { errCh <- err } }() select { case err := <-errCh: return err case <-time.After(5 * time.Second): // assume server started successfully } } var grpcSrv *grpc.Server if config.GRPC.Enable { grpcSrv, err = servergrpc.StartGRPCServer(app, config.GRPC.Address) if err != nil { return err } } defer func() { if tmNode.IsRunning() { _ = tmNode.Stop() } if cpuProfileCleanup != nil { cpuProfileCleanup() } if apiSrv != nil { _ = apiSrv.Close() } if grpcSrv != nil { grpcSrv.Stop() } ctx.Logger.Info("exiting...") }() // Wait for SIGINT or SIGTERM signal return WaitForQuitSignals() }

Note that an appCreator is a function that fulfills the AppCreator signature: +++https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/server/types/app.go#L48-L50

In practice, the constructor of the application is passed as the appCreator.

Copy func newApp(logger log.Logger, db dbm.DB, traceStore io.Writer, appOpts servertypes.AppOptions) servertypes.Application { var cache sdk.MultiStorePersistentCache if cast.ToBool(appOpts.Get(server.FlagInterBlockCache)) { cache = store.NewCommitKVStoreCacheManager() } skipUpgradeHeights := make(map[int64]bool) for _, h := range cast.ToIntSlice(appOpts.Get(server.FlagUnsafeSkipUpgrades)) { skipUpgradeHeights[int64(h)] = true } pruningOpts, err := server.GetPruningOptionsFromFlags(appOpts) if err != nil { panic(err) } snapshotDir := filepath.Join(cast.ToString(appOpts.Get(flags.FlagHome)), "data", "snapshots") snapshotDB, err := sdk.NewLevelDB("metadata", snapshotDir) if err != nil { panic(err) } snapshotStore, err := snapshots.NewStore(snapshotDB, snapshotDir) if err != nil { panic(err) } return simapp.NewSimApp( logger, db, traceStore, true, skipUpgradeHeights, cast.ToString(appOpts.Get(flags.FlagHome)), cast.ToUint(appOpts.Get(server.FlagInvCheckPeriod)), simapp.MakeTestEncodingConfig(), // Ideally, we would reuse the one created by NewRootCmd. appOpts, baseapp.SetPruning(pruningOpts), baseapp.SetMinGasPrices(cast.ToString(appOpts.Get(server.FlagMinGasPrices))), baseapp.SetHaltHeight(cast.ToUint64(appOpts.Get(server.FlagHaltHeight))), baseapp.SetHaltTime(cast.ToUint64(appOpts.Get(server.FlagHaltTime))), baseapp.SetMinRetainBlocks(cast.ToUint64(appOpts.Get(server.FlagMinRetainBlocks))), baseapp.SetInterBlockCache(cache), baseapp.SetTrace(cast.ToBool(appOpts.Get(server.FlagTrace))), baseapp.SetIndexEvents(cast.ToStringSlice(appOpts.Get(server.FlagIndexEvents))), baseapp.SetSnapshotStore(snapshotStore), baseapp.SetSnapshotInterval(cast.ToUint64(appOpts.Get(server.FlagStateSyncSnapshotInterval))), baseapp.SetSnapshotKeepRecent(cast.ToUint32(appOpts.Get(server.FlagStateSyncSnapshotKeepRecent))), ) }

Then, the instance of app is used to instanciate a new Tendermint node:

Copy tmNode, err := node.NewNode( cfg, pvm.LoadOrGenFilePV(cfg.PrivValidatorKeyFile(), cfg.PrivValidatorStateFile()), nodeKey, proxy.NewLocalClientCreator(app), genDocProvider, node.DefaultDBProvider, node.DefaultMetricsProvider(cfg.Instrumentation), ctx.Logger.With("module", "node"), )

The Tendermint node can be created with app because the latter satisfies the abci.Application interface (opens new window) (given that app extends baseapp). As part of the NewNode method, Tendermint makes sure that the height of the application (i.e. number of blocks since genesis) is equal to the height of the Tendermint node. The difference between these two heights should always be negative or null. If it is strictly negative, NewNode will replay blocks until the height of the application reaches the height of the Tendermint node. Finally, if the height of the application is 0, the Tendermint node will call InitChain on the application to initialize the state from the genesis file.

Once the Tendermint node is instanciated and in sync with the application, the node can be started:

Copy if err := tmNode.Start(); err != nil { return err }

Upon starting, the node will bootstrap its RPC and P2P server and start dialing peers. During handshake with its peers, if the node realizes they are ahead, it will query all the blocks sequentially in order to catch up. Then, it will wait for new block proposals and block signatures from validators in order to make progress.

# Other commands

To discover how to concretely run a node and interact with it, please refer to our Running a Node, API and CLI guide.

# Next

Learn about the store