cosmovisor is a small process manager for Cosmos SDK application binaries that monitors the governance module for incoming chain upgrade proposals. If it sees a proposal that gets approved,
cosmovisor can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary.
Cosmovisor is designed to be used as a wrapper for an
Cosmos SDK app:
- it will pass all arguments to the app. Running
cosmovisor arg1 arg2 ....will run
app arg1 arg2 ...;
- it will manage an app by restarting and upgrading if needed;
- it is configured using environment variables, not positional arguments.
Note: If new versions of the application are not set up to run in-place store migrations, migrations will need to be run manually before restarting
cosmovisor with the new binary. For this reason, we recommend applications adopt in-place store migrations.
Note: If validators would like to enable the auto-download option, and they are currently running an application using Cosmos SDK
v0.42, they will need to use Cosmovisor
v0.1 (opens new window). Later versions of Cosmovisor do not support Cosmos SDK
v0.42 or earlier if the auto-download option is enabled.
Cosmovisor is part of the Cosmos SDK monorepo, but it's a separate module with it's own release schedule.
Release branches have the following format
release/cosmovisor/vA.B.x, where A and B are a number (e.g.
release/cosmovisor/v0.1.x). Releases are tagged using the following format:
To install the latest version of
cosmovisor, run the following command:
Note: If you are using go
v1.15 or earlier, you will need to use
go get, and you may want to run the command outside a project directory.
# Command Line Arguments And Environment Variables
All arguments passed to
cosmovisor will be passed to the application binary (as a subprocess).
cosmovisor will return
/dev/stderr of the subprocess as its own. For this reason,
cosmovisor cannot accept any command-line arguments other than those available to the application binary, nor will it print anything to output other than what is printed by the application binary.
cosmovisor reads its configuration from environment variables:
DAEMON_HOMEis the location where the
cosmovisor/directory is kept that contains the genesis binary, the upgrade binaries, and any additional auxiliary files associated with each binary (e.g.
DAEMON_NAMEis the name of the binary itself (e.g.
DAEMON_ALLOW_DOWNLOAD_BINARIES(optional), if set to
true, will enable auto-downloading of new binaries (for security reasons, this is intended for full nodes rather than validators). By default,
cosmovisorwill not auto-download new binaries.
DAEMON_RESTART_AFTER_UPGRADE(optional, default =
true, will restart the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. Otherwise (
cosmovisorstops running after an upgrade and requires the system administrator to manually restart it. Note that
cosmovisorwill not auto-restart the subprocess if there was an error.
DAEMON_POLL_INTERVALis the interval length in milliseconds for polling the upgrade plan file. Default: 300.
false), if set to
false, will backup the data before trying the upgrade. Otherwise it will upgrade directly without doing any backup. This is useful (and recommended) in case of failures and when needed to rollback. It is advised to use backup option, i.e.,
# Folder Layout
$DAEMON_HOME/cosmovisor is expected to belong completely to
cosmovisor and the subprocesses that are controlled by it. The folder content is organized as follows:
cosmovisor/ directory incudes a subdirectory for each version of the application (i.e.
upgrades/<name>). Within each subdirectory is the application binary (i.e.
bin/$DAEMON_NAME) and any additional auxiliary files associated with each binary.
current is a symbolic link to the currently active directory (i.e.
name variable in
upgrades/<name> is the URI-encoded name of the upgrade as specified in the upgrade module plan.
Please note that
$DAEMON_HOME/cosmovisor only stores the application binaries. The
cosmovisor binary itself can be stored in any typical location (e.g.
/usr/local/bin). The application will continue to store its data in the default data directory (e.g.
$HOME/.gaiad) or the data directory specified with the
$DAEMON_HOME is independent of the data directory and can be set to any location. If you set
$DAEMON_HOME to the same directory as the data directory, you will end up with a configuation like the following:
The system administrator is responsible for:
- installing the
- configuring the host's init system (e.g.
- appropriately setting the environmental variables
- manually installing the
- manually installing the
cosmovisor will set the
current link to point to
genesis at first start (i.e. when no
current link exists) and then handle switching binaries at the correct points in time so that the system administrator can prepare days in advance and relax at upgrade time.
In order to support downloadable binaries, a tarball for each upgrade binary will need to be packaged up and made available through a canonical URL. Additionally, a tarball that includes the genesis binary and all available upgrade binaries can be packaged up and made available so that all the necessary binaries required to sync a fullnode from start can be easily downloaded.
DAEMON specific code and operations (e.g. tendermint config, the application db, syncing blocks, etc.) all work as expected. The application binaries' directives such as command-line flags and environment variables also work as expected.
# Detecting Upgrades
cosmovisor is polling the
$DAEMON_HOME/data/upgrade-info.json file for new upgrade instructions. The file is created by the x/upgrade module in
BeginBlocker when an upgrade is detected and the blockchain reaches the upgrade height.
The following heuristic is applied to detect the upgrade:
- When starting,
cosmovisordoesn't know much about currently running upgrade, except the binary which is
current/bin/. It tries to read the
current/update-info.jsonfile to get information about the current upgrade name.
- If neither
cosmovisorwill wait for
data/upgrade-info.jsonfile to trigger an upgrade.
cosmovisor/current/upgrade-info.jsondoesn't exist but
cosmovisorassumes that whatever is in
data/upgrade-info.jsonis a valid upgrade request. In this case
cosmovisortries immediately to make an upgrade according to the
cosmovisorwaits for changes in
upgrade-info.json. As soon as a new upgrade name is recorded in the file,
cosmovisorwill trigger an upgrade mechanism.
When the upgrade mechanism is triggered,
cosmovisor will start by auto-downloading a new binary (if
DAEMON_ALLOW_DOWNLOAD_BINARIES is enabled) into
<name> is the
cosmovisor will then update the
current symbolic link to point to the new directory and save
cosmovisor requires that the system administrator place all relevant binaries on disk before the upgrade happens. However, for people who don't need such control and want an easier setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option.
DAEMON_ALLOW_DOWNLOAD_BINARIES is set to
true, and no local binary can be found when an upgrade is triggered,
cosmovisor will attempt to download and install the binary itself. The plan stored in the upgrade module has an info field for arbitrary JSON. This info is expected to be outputed on the halt log message. There are two valid formats to specify a download in such a message:
- Store an os/architecture -> binary URI map in the upgrade plan info field as JSON under the
"binaries"key. For example:
- Store a link to a file that contains all information in the above format (e.g. if you want to specify lots of binaries, changelog info, etc. without filling up the blockchain). For example:
cosmovisor is triggered to download the new binary,
cosmovisor will parse the
"binaries" field, download the new binary with go-getter (opens new window), and unpack the new binary in the
upgrades/<name> folder so that it can be run as if it was installed manually.
Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS.
go-getter will always ensure the downloaded file matches the checksum if it is provided.
go-getter will also handle unpacking archives into directories (in this case the download link should point to a
zip file of all data in the
To properly create a sha256 checksum on linux, you can use the
sha256sum utility. For example:
The result will look something like the following:
You can also use
sha512sum if you would prefer to use longer hashes, or
md5sum if you would prefer to use broken hashes. Whichever you choose, make sure to set the hash algorithm properly in the checksum argument to the URL.
# Example: SimApp Upgrade
The following instructions provide a demonstration of
cosmovisor using the simulation application (
simapp) shipped with the Cosmos SDK's source code. The following commands are to be run from within the
First, check out the latest
~/.simapp (never do this in a production environment):
simd binary for testing:
Initialize the node and overwrite any previous genesis file (never do this in a production environment):
Set the minimum gas price to
Create a new key for the validator, then add a genesis account and transaction:
Set the required environment variables:
Set the optional environment variable to trigger an automatic restart:
Create the folder for the genesis binary and copy the
For the sake of this demonstration, amend
genesis.json to a reduced time of 20 seconds (
Next, we will hardcode a modification in
simapp to simulate a code change. In
simapp/app.go, find the line containing the
UpgradeKeeper initialization. It should look like the following:
After that line, add the following:
Now recompile the
simd binary with the added upgrade handler:
Create the folder for the upgrade binary and copy the
Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other):
The upgrade will occur automatically at height 20.