Testing
The Cosmos SDK contains different types of tests. These tests have different goals and are used at different stages of the development cycle. We advice, as a general rule, to use tests at all stages of the development cycle. It is adviced, as a chain developer, to test your application and modules in a similar way than the SDK.
The rationale behind testing can be found in ADR-59.
Unit Tests
Unit tests are the lowest test category of the test pyramid. All packages and modules should have unit test coverage. Modules should have their dependencies mocked: this means mocking keepers.
The SDK uses mockgen
to generate mocks for keepers:
loading...
You can read more about mockgen here.
Example
As an example, we will walkthrough the keeper tests of the x/gov
module.
The x/gov
module has a Keeper
type requires a few external dependencies (ie. imports outside x/gov
to work properly).
loading...
In order to only test x/gov
, we mock the expected keepers and instantiate the Keeper
with the mocked dependencies. Note that we may need to configure the mocked dependencies to return the expected values:
loading...
This allows us to test the x/gov
module without having to import other modules.
loading...
We can test then create unit tests using the newly created Keeper
instance.
loading...
Integration Tests
Integration tests are at the second level of the test pyramid.
In the SDK, we locate our integration tests under /tests/integrations
.
The goal of these integration tests is to test a component with a minimal application (i.e. not simapp
). The minimal application is defined with the help of depinject
– the SDK dependency injection framework, and includes all necessary modules to test the component. With the helps of the SDK testing package, we can easily create a minimal application and start the application with a set of genesis transactions: https://github.com/cosmos/cosmos-sdk/blob/main/testutil/sims/app_helpers.go.
Example
Here, we will walkthrough the integration tests of the x/distribution
module. The x/distribution
module has, in addition to keeper unit tests, integration tests that test the x/distribution
module with a minimal application. This is expected as you may want to test the x/distribution
module with actual application logic, instead of only mocked dependencies.
For creating a minimal application, we use simtestutil.Setup
and an AppConfig
of the x/distribution
minimal dependencies.
For instance, the AppConfig
of x/distribution
is defined as:
This is a stripped down version of the simapp
AppConfig
:
You can as well use the AppConfig
configurator
for creating an AppConfig
inline. There no difference between those two ways, use whichever you prefer.
loading...
Now the types are injected and we can use them for our tests:
loading...
Deterministic and Regression tests
Tests are written for queries in the Cosmos SDK which have module_query_safe
Protobuf annotation.
Each query is tested using 2 methods:
- Use property-based testing with the
rapid
library. The property that is tested is that the query response and gas consumption are the same upon 1000 query calls. - Regression tests are written with hardcoded responses and gas, and verify they don't change upon 1000 calls and between SDK patch versions.
Here's an example of regression tests:
loading...
Simulations
Simulations uses as well a minimal application, built with depinject
:
Following is an example for x/gov/
simulations:
loading...
loading...
End-to-end Tests
End-to-end tests are at the top of the test pyramid.
They must test the whole application flow, from the user perspective (for instance, CLI tests). They are located under /tests/e2e
.
For that, the SDK is using simapp
but you should use your own application (appd
).
Here are some examples:
- SDK E2E tests: https://github.com/cosmos/cosmos-sdk/tree/main/tests/e2e.
- Cosmos Hub E2E tests: https://github.com/cosmos/gaia/tree/main/tests/e2e.
- Osmosis E2E tests: https://github.com/osmosis-labs/osmosis/tree/main/tests/e2e.
The SDK is in the process of creating its E2E tests, as defined in ADR-59. This page will eventually be updated with better examples.
Summary
Scope | App Fixture | Mocks? |
---|---|---|
Unit | None | Yes |
Integration | depinject | Some |
Simulation | depinject | No |
E2E | appd | No |