Retention Policy

Convoy supports exporting events to any S3 compatible cold storage.

Note:

Retention policies are available only on a paid license.

Partition conversions and per-project retention windows are part of that license. There is no separate feature flag to turn partitioning on.

Configuring Retention

JSON Configuration

You can configure read replicas by adding the storage_policy object to your database configuration JSON.

Key	Description
`type`	Storage type, either “s3” or “on_prem”
`s3.prefix`	Prefix for S3 storage path
`s3.bucket`	Name of the S3 bucket
`s3.access_key`	AWS access key for S3 authentication
`s3.secret_key`	AWS secret key for S3 authentication
`s3.region`	AWS region where the S3 bucket is located
`s3.session_token`	Temporary session token for AWS authentication (optional)
`s3.endpoint`	Custom endpoint URL for S3-compatible storage
`on_prem.path`	File system path for on-premises storage location

using storage policy via JSON configuration

{
	"storage_policy": {
		"type": "s3 | on_prem",
		"s3": {
			"prefix": "<insert-s3-prefix>",
			"bucket": "<insert-s3-bucket>",
			"access_key": "<insert-s3-access-key>",
			"secret_key": "<insert-s3-secret-key>",
			"region": "<insert-s3-region>",
			"session_token": "<insert-s3-session-token>",
			"endpoint": "<insert-s3-endpoint>"
		},
		"on_prem": {
			"path": "<insert-on-prem-path>"
		}
	}
}

Environment Variables Configuration

Alternatively, you can supply environment variables to configure the storage policy.

Environment Variable	Description
`CONVOY_STORAGE_POLICY_TYPE`	Storage type, either “s3” or “on_prem”
`CONVOY_STORAGE_AWS_PREFIX`	Prefix for S3 storage path
`CONVOY_STORAGE_AWS_BUCKET`	Name of the S3 bucket
`CONVOY_STORAGE_AWS_ACCESS_KEY`	AWS access key for S3 authentication
`CONVOY_STORAGE_AWS_SECRET_KEY`	AWS secret key for S3 authentication
`CONVOY_STORAGE_AWS_REGION`	AWS region where the S3 bucket is located
`CONVOY_STORAGE_AWS_SESSION_TOKEN`	Temporary session token for AWS authentication (optional)
`CONVOY_STORAGE_AWS_ENDPOINT`	Custom endpoint URL for S3-compatible storage
`CONVOY_STORAGE_PREM_PATH`	File system path for on-premises storage location

using storage policy via environment variable configuration

CONVOY_STORAGE_POLICY_TYPE=''
CONVOY_STORAGE_AWS_PREFIX=''
CONVOY_STORAGE_AWS_BUCKET=''
CONVOY_STORAGE_AWS_ACCESS_KEY=''
CONVOY_STORAGE_AWS_SECRET_KEY=''
CONVOY_STORAGE_AWS_REGION=''
CONVOY_STORAGE_AWS_SESSION_TOKEN=''
CONVOY_STORAGE_AWS_ENDPOINT=''
CONVOY_STORAGE_PREM_PATH=''

Retention Using Postgres Partitions

Convoy retains webhook history with Postgres-native daily partitions (go_partman). Licensed instances can convert tables from the dashboard or the CLI. Partitioning is a paid feature; it is not a feature flag. Before Convoy v25.1.1 the retention job both uploaded to S3 and ran DELETE on the same tables. Timeouts and network errors left duplicates or skipped deletes, and large deletes triggered autovacuum pressure. Partitions drop a day at a time instead. You can set a different retention duration per project from project settings.

Tables

These tables can be partitioned:

delivery_attempts
event_deliveries
events
events_search

Back up the database, or try the conversion on a copy, before you convert a large production table.

Convert from the admin dashboard

On a licensed instance, open Admin → Table partitions. Pick a table and an operation. Convert one table at a time. Attach (the usual conversion) keeps ingestion running. The existing table becomes one history partition that holds everything written up to the cutoff; new rows go into daily partitions after that. Retention expires that history partition in full once every row in it is older than your window, then expires later days one partition at a time. Detach / copy unpartition convert back to a plain table. Copy unpartition should run in a maintenance window: rows written during the copy are not carried over, so pause ingestion first. Progress for the in-flight run is shown on the same page.

Convert from the CLI

partition all tables

convoy utils partition

Or one table at a time:

partition tables one after the other

convoy utils partition delivery_attempts
convoy utils partition event_deliveries
convoy utils partition events
convoy utils partition events_search

If you are running Convoy in Docker:
partitioning using docker exec
docker exec -it convoy-staging /cmd util partition

Un-partition:

unpartition

convoy utils unpartition

unpartition one table

convoy utils unpartition delivery_attempts
convoy utils unpartition event_deliveries
convoy utils unpartition events
convoy utils unpartition events_search

Breaking change in v26.7.2

Partitions created by convoy utils partition used to be ignored by retention because their names were folded to lower case. They are managed now. History that survived only because those partitions were skipped is dropped at CONVOY_RETENTION_POLICY on the next maintenance run. Confirm that duration before you upgrade.

Indexes after conversion

An interrupted index build, or an upgrade that drops a large index so migrate can finish, leaves work on Admin → Table indexes. convoy utils indexes reports the same list. From v26.7.3 server and agent work through that list themselves, in the background after boot, one index at a time and unique indexes first. No rebuild holds a lock against traffic, but a large index takes hours. Use Rebuild on that page to retry one that failed.

Breaking change in v26.7.3

Boot also drops any index Postgres left INVALID (a killed CREATE INDEX CONCURRENTLY) and queues it the same way, instead of leaving it until the next migrate with the planner ignoring it silently. An invalid unique index still enforces its key. Dropping one gives that up until the rebuild finishes, so upgrade in a window where losing the constraint for a few hours is acceptable, and watch the page until every owed index is valid again. A rebuild that meets duplicate rows is recorded as blocked, with the offending key, and skipped from then on. Resolve those duplicates by hand before you retry it.

Home

Guides

Deployment

Product Manual

Command Line

SDK

Cloud

Legal

Business & Enterprise

Glossary

Resources

FAQ

Retention Policy

Configuring Retention

JSON Configuration

Environment Variables Configuration

Retention Using Postgres Partitions

Tables

Convert from the admin dashboard

Convert from the CLI

Breaking change in v26.7.2

Indexes after conversion

Breaking change in v26.7.3

​Configuring Retention

​JSON Configuration

​Environment Variables Configuration

​Retention Using Postgres Partitions

​Tables

​Convert from the admin dashboard

​Convert from the CLI

​Breaking change in v26.7.2

​Indexes after conversion

​Breaking change in v26.7.3

Configuring Retention

JSON Configuration

Environment Variables Configuration

Retention Using Postgres Partitions

Tables

Convert from the admin dashboard

Convert from the CLI

Breaking change in v26.7.2

Indexes after conversion

Breaking change in v26.7.3