Configuration

Replicator uses a configuration file in YAML format. The file must be called appsettings.yaml and located in the config subdirectory, relative to the tool working directory.

The settings file has the replicator root level, all settings are children to that root. It allows using the same format for the values override file when using Helm.

Reference

In the reference table below, all options are listed without replicator. prefix. Remember to add it when using the configuration file. For example, reader.connectionString should be specified as replicator.reader.connectionString.

Available configuration options are:

Option	Description
reader.connectionString	Connection string for the source cluster or instance
reader.protocol	Reader protocol (tcp or grpc)
reader.pageSize	Reader page size (only applicable for TCP protocol
sink.connectionString	Connection string for the target cluster or instance
sink.protocol	Writer protocol (tcp or grpc)
sink.partitionCount	Number of partitioned concurrent writers
sink.partitioner	Custom JavaScript partitioner
sink.bufferSize	Size of the sink buffer, 1000 events by default
scavenge	Enable real-time scavenge
runContinuously	Set to false if you want Replicator to stop when it reaches the end of $all stream. Default is true, so the replication continues until you stop it explicitly.
filters	Add one or more of provided filters
transform	Configure the event transformation
transform.bufferSize	Size of the prepare buffer (filtering and transformations), 1000 events by default
checkpoint.type	Type of checkpoint store (file or mongo), file by default
checkpoint.path	The file path or connection string, ./checkpoint by default
checkpoint.checkpointAfter	The number of events that must be replicated before a checkpoint is stored, 1000 events by default
checkpoint.database	The name of the Mongo database, replicator by default
checkpoint.instanceId	The name of the replicator instance to isolate checkpoints with in the Mongo database, default by default
checkpoint.seeder.type	Type of checkpoint seeder to use (none or chaser), none by default
checkpoint.seeder.path	The file path of the chaser.chk, empty by default
restartDelayInSeconds	The number of seconds between replication restarts, 5 by default
reportMetricsFrequencyInSeconds	The frequency at which to report certain metrics expressed in seconds, 5 by default

Enable verbose logging

You can enable debug-level logging by setting the REPLICATOR_DEBUG environment variable to any value.

Example configuration

The following example configuration will instruct Replicator to read all the events from a local cluster with three nodes (es1.acme.org, es2.acme.org and es3.acme.org) using TCP protocol, and copy them over to the Event Store Cloud cluster with cluster ID c2etr1lo9aeu6ojco781 using gRPC protocol. Replicator will also call an HTTP transformation function at https://my.acme.org/transform.

The global order of events will be the same, as partitionCount is set to one.

Scavenge filter is disabled, so Replicator will also copy deleted events, which haven’t been scavenged by the server yet.

replicator:
  reader:
    protocol: tcp
    connectionString: "GossipSeeds=es1.acme.org:2113,es2.acme.org:2113,es3.acme.org:2113; HeartBeatTimeout=500; DefaultUserCredentials=admin:changeit; UseSslConnection=false;"
    pageSize: 2048
  sink:
    protocol: grpc
    connectionString: "esdb://admin:changeit@c2etr1lo9aeu6ojco781.mesdb.eventstore.cloud:2113"
    partitionCount: 1
  transform:
    type: http
    config: https://my.acme.org/transform
  scavenge: false
  filters: []
  checkpoint:
    path: "./checkpoint"