Sync service

The Electric sync service is an Elixir application that manages active-active replication between your Postgres database and your local apps.

The main way to run the sync service is using the official Docker image. See Usage -> Installation -> Sync service for more instructions.

Configuration options are passed as environment variables, e.g.:

docker run \
    -e "DATABASE_URL=postgresql://..." \
    -e "ELECTRIC_WRITE_TO_PG_MODE=direct_writes" \
    -e "PG_PROXY_PASSWORD=..." \
    -e "AUTH_JWT_ALG=HS512" \
    -e "AUTH_JWT_KEY=..." \
    -p 5133:5133 \
    -p 5433:5433 \
    electricsql/electric

This page documents all the configuration options, including the:

• core config of DATABASE_URL, HTTP_PORT, etc.
• write-to-PG mode with associated networking and database user permissions
• migrations proxy, authentication and telemetry

info

For a longer form description of how to successfully deploy the sync service and what needs to connect where, see Deployment -> Concepts.

A note on ports

Your configuration options affect the number and type of ports that need to be exposed. You must always expose the HTTP_PORT. Your write-to-PG mode and migrations proxy config then determines whether you need to expose up-to two TCP ports: the LOGICAL_PUBLISHER_PORT and PG_PROXY_PORT respectively.

In development using Docker you usually want to map all the necessary ports to your host network (-p 5133:5133 and -p 5433:5433 in the example above).

In production you must make sure your hosting infrastructure exposes the necessary ports and protocols. If not, you can use the workarounds provided in the form of direct writes mode and the proxy tunnel.

Again, see Deployment -> Concepts for more information.

Core config

Configure how Electric connects to Postgres and exposes its HTTP/WebSocket API.

DATABASE_URL

Variable	DATABASE_URL required
Description	Postgres connection string. Used to connect to the Postgres database. The connection string must be in the libpg Connection URI format of postgresql://[userspec@][hostspec][/dbname][?sslmode=<sslmode>]. The userspec section of the connection string specifies the database user that Electric connects to Postgres as. The permissions required for this user depend on your choice of write-to-PG-mode. See the database user permissions section below for more information. The optional sslmode query parameter may be set to one of the following values: disable allow prefer require Including sslmode=require in the database connection string is equivalent to setting DATABASE_REQUIRE_SSL=true (which also happens to be the default). Any other value for sslmode is equivalent to setting DATABASE_REQUIRE_SSL=false. Do note that if you explicitly set DATABASE_REQUIRE_SSL, the sslmode query parameter in DATABASE_URL will be ignored.
Example	DATABASE_URL=postgresql://user:password@example.com:54321/electric

DATABASE_REQUIRE_SSL

Variable	DATABASE_REQUIRE_SSL
Default	true
Description	Set to false to enable Electric to fallback to using unencrypted connections in case the database is not configured to work with SSL. Be mindful of changing this default, more often than not it's a bad idea to use unencrypted database connections because all data flowing between your database and Electric may get intercepted by an unauthorized party. Whether Electric will use SSL encryption for its database connections can also be configured with the sslmode query parameter in DATABASE_URL but only if DATABASE_REQUIRE_SSL is not explicitly set.
Example	DATABASE_REQUIRE_SSL=false

DATABASE_USE_IPV6

Variable	DATABASE_USE_IPV6
Default	false
Description	Set to true to prioritise connecting to the database over IPv6. Electric will fall back to an IPv4 DNS lookup if the IPv6 lookup fails.
Example	DATABASE_USE_IPV6=true

ELECTRIC_USE_IPv6

Variable	ELECTRIC_USE_IPV6
Default	true
Description	Set to false to force Electric to only listen on IPv4 interfaces. By default, Electric will accept inbound connections over both IPv6 and IPv4 when running on Linux. On Windows and some BSD systems inbound connections over IPv4 will not be accepted unless this option is disabled.
Example	ELECTRIC_USE_IPV6=false

HTTP_PORT

Variable	HTTP_PORT
Default	5133
Description	Port for HTTP connections. This exposes an HTTP API used by the CLI and Generator on /api and a WebSocket API used by the Satellite replication protocol on /ws. When using the Proxy tunnel, connections to the Migrations proxy are also tunneled over this port.
Example	HTTP_PORT=8080

ELECTRIC_RESUMABLE_WAL_WINDOW

Variable	ELECTRIC_RESUMABLE_WAL_WINDOW
Default	2G
Description	Maximum size that the sync service is allowed to use for keeping around old WAL records in Postgres. Measured in bytes unless one of the following suffixes is used: k for KB; K for KiB; m for MB; M for MiB; g for GB; G for GiB. Normally, Postgres discards WAL records as soon as they are acknowledged by the replica. However, for the sync service to be able to restore its caches after a restart, it needs to hold on to old WAL records since those may contain transactions that affect electrified tables. Setting this to a low value may lead to clients having to discard their local copy of the server state and restart their replication stream from scratch. In the future Electric may store change diffs or employ techniques for compacting the retained WAL records with the goal of extending the time frame during which a client may catch up to the latest server state without discarding its local state.
Example	ELECTRIC_RESUMABLE_WAL_WINDOW=800M

ELECTRIC_TXN_CACHE_SIZE

Variable	ELECTRIC_TXN_CACHE_SIZE
Default	256M
Description	Size of the in-memory cache for transactions the sync service consumes from Postgres over the logical replication stream. Measured in bytes unless one of the following suffixes is used: k for KB; K for KiB; m for MB; M for MiB; g for GB; G for GiB. This cache is used to quickly catch up newly connected clients to the current state in Postgres. When the sync service restarts, it will repopulate this in-memory cache by streaming already committed transactions from retained Postgres' WAL records, going as far back in time as the size of the WAL window configured with ELECTRIC_RESUMABLE_WAL_WINDOW allows.
Example	ELECTRIC_TXN_CACHE_SIZE=2g

LOG_LEVEL

Variable	LOG_LEVEL
Default	info
Description	Verbosity of Electric's log output. Available levels, in the order of increasing verbosity: error warning info debug
Example	LOG_LEVEL=debug

Write-to-PG mode

Electric writes data to Postgres using one of two modes:

1. logical replication
2. direct writes

caution

The mode you choose affects your networking config and database user permissions.

ELECTRIC_WRITE_TO_PG_MODE

Variable	ELECTRIC_WRITE_TO_PG_MODE
Default	logical_replication
Description	The mode to use when writing data from Electric to Postgres. The allowed values are: logical_replication and direct_writes. In logical_replication mode, Electric provides a logical replication publisher service over TCP that speaks the Logical Streaming Replication Protocol. Postgres connects to Electric and establishes a subscription to this. Writes are then streamed in and applied using logical replication. In direct_writes mode, Electric writes data to Postgres using a standard interactive client connection. This avoids the need for Postgres to be able to connect to Electric and reduces the permissions required for the database user that Electric connects to Postgres as.
Example	ELECTRIC_WRITE_TO_PG_MODE=direct_writes

Logical replication mode

In logical_replication mode, Electric exposes a logical replication publisher service over TCP that speaks the Logical Streaming Replication Protocol.

Postgres connects to Electric on LOGICAL_PUBLISHER_HOST:LOGICAL_PUBLISHER_PORT and establishes a logical replication subscription to this publisher service. Writes are then streamed in and applied using the logical replication subscription.

         | <----------- DATABASE_URL ----------- |
Postgres |                                       | Electric
         | ---- LOGICAL_PUBLISHER_HOST:PORT ---> |

Caution

In logical replication mode:

1. the database user that Electric connects to Postgres as must have the SUPERUSER role attribute
2. Postgres must be able to connect to Electric (i.e.: must be able to establish an outbound TCP connection) on the host and port that you configure

As a result, you must make sure (in terms of networking / firewalls) not only that Postgres is reachable from Electric but also that Electric is reachable from Postgres. And Electric must know its own address, in order to provide it to Postgres when setting up the logical replication publication that allows writes to be replicated into Postgres.

LOGICAL_PUBLISHER_HOST

Variable	LOGICAL_PUBLISHER_HOST required
Description	Hostname of the electric instance that Postgres connects when establishing the logical replication subscription. Electric must be accessible from the Postgres instance on this hostname. Required when-and-only-when running in logical replication mode.
Example	LOGICAL_PUBLISHER_HOST=example.com

LOGICAL_PUBLISHER_PORT

Variable	LOGICAL_PUBLISHER_PORT
Default	5433
Description	Port number of the TCP service exposed by Electric that Postgres connects to when establishing the logical replication subscription. The port must be accessible from the Postgres instance.
Example	LOGICAL_PUBLISHER_PORT=65433

Direct writes mode

In direct_writes mode, Electric writes data to Postgres using a standard interactive client connection. This avoids the need for Postgres to be able to connect to Electric and reduces the permissions required for the database user that Electric connects to Postgres as.

Postgres | <----------- DATABASE_URL ----------- | Electric

No additional configuration is required for direct writes mode. The LOGICAL_PUBLISHER_HOST and LOGICAL_PUBLISHER_PORT variables are not required and are ignored.

Why are there two modes?

Originally (prior to v0.8) all writes to Postgres were made using logical replication.

In version 0.8, Electric added direct writes mode to reduce the database user permissions required and increase compatibility with Postgres hosting providers.

In future, we may deprecate logical replication and consolidate on direct writes. However, logical replication may have performance advantages and, for now, we've kept both modes available while direct writes mode gains maturity and we learn more about the operational characteristics of both modes.

Database user permissions

The Electric sync service connects to Postgres using the DATABASE_URL connection string, in the format postgresql://[userspec@][hostspec][/dbname].

The userspec section of this connection string specifies the database user that Electric connects to Postgres as. This user must have the following permissions.

Permissions for logical replication mode

In logical replication mode, the database user must have the LOGIN and SUPERUSER role attributes. You can create a user with these permissions using, e.g.:

CREATE ROLE electric
  WITH LOGIN
       PASSWORD '...'
       SUPERUSER;

Permissions for direct writes mode

In direct writes mode, the database user must have LOGIN, REPLICATION and then either ALL on the database and public schema or at a minimum:

• CONNECT, CREATE and TEMPORARY on the database
• CREATE, EXECUTE on ALL and USAGE on the public schema

Plus ALTER DEFAULT PRIVILEGES to grant the same permissions on any new tables in the public schema. For example, to create a user with the necessary permissions:

CREATE ROLE electric
  WITH LOGIN
    PASSWORD '...'
    REPLICATION;

GRANT ALL
  ON DATABASE '...'
  TO electric;

GRANT ALL
  ON ALL TABLES
  IN SCHEMA public
  TO electric;

ALTER DEFAULT PRIVILEGES
  IN SCHEMA public
  GRANT ALL
    ON TABLES
    TO electric;

Migrations proxy

Electric exposes a Migrations proxy as a TCP service. This must be secured using PG_PROXY_PASSWORD and is exposed on PG_PROXY_PORT.

The PG_PROXY_PORT supports a special http value that allows you to connect to the migrations proxy over a TCP-over-HTTP tunnel. This enables the use of the Proxy Tunnel. This is a CLI command that tunnels the Migrations proxy connection over the HTTP_PORT.

PG_PROXY_PASSWORD

Variable	PG_PROXY_PASSWORD required
Description	Password to use when connecting to the Postgres proxy via psql or any other Postgres client.
Example	PG_PROXY_PASSWORD=b3aed739144e859a

PG_PROXY_PORT

Variable	PG_PROXY_PORT
Default	65432
Description	Port number for connections to the Migrations proxy. Electric provides a migrations proxy service over TCP that speaks the Postgres protocol. This configures the port that this service is exposed on. If you have Electric deployed behind a restrictive firewall that only allows HTTP/HTTPS connections, you can set the value to http or add http: as a prefix to the port number, e.g. http:1234. This will enable tunelling mode in which the migrations proxy will accept WebSocket connections from the Proxy tunnel while the TCP service will be listening for connections on port 1234. Setting the value to http makes the TCP service listen on the default port.
Example	PG_PROXY_PORT=http:65432

Authentication

Electric provides two authentication modes:

1. secure (the default)
2. insecure

In secure more, AUTH_JWT_ALG and AUTH_JWT_KEY are required. In insecure mode, all other authentication variables can be omitted.

AUTH_MODE

Variable	AUTH_MODE
Default	secure
Description	Authentication mode to use to authenticate clients. The allowed values are: secure and insecure. Secure mode is the default and is strongly recommended for production use. Insecure mode should only be used in development.
Example	AUTH_MODE=insecure

AUTH_JWT_NAMESPACE

Variable	AUTH_JWT_NAMESPACE optional
Description	This is an optional setting that specifies the location inside the token of custom claims that are specific to Electric. Currently, only the sub custom claim (formerly user_id) is required.
Example	AUTH_JWT_NAMESPACE=example

Secure mode

In secure mode, Electric authenticates its replication connections by obtaining a JWT from each client and verifying its validity before allowing data streaming in either direction.

See Usage -> Authentication -> Secure mode

AUTH_JWT_ALG

Variable	AUTH_JWT_ALG required
Description	The algorithm to use for JWT verification. The following algorithms are supported: HS256, HS384, HS512: HMAC-based cryptographic signature that relies on the SHA-2 family of hash functions RS256, RS384, RS512: RSA-based algorithms for digital signature ES256, ES384, ES512: ECC-based algorithms for digital signature
Example	AUTH_JWT_ALG=HS512

AUTH_JWT_KEY

Variable	AUTH_JWT_KEY required
Description	The key to use for JWT verification. Must be appropriate for the chosen signature algorithm. For HS* algorithms, the symmetric key can be base64-encoded, provided that you also configure AUTH_JWT_KEY_IS_BASE64_ENCODED. For RS* and ES* algorithms, the public key must be in the PEM format: -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQY... ... -----END PUBLIC KEY-----
Example	AUTH_JWT_KEY=x\a0CA7Q4_gc...

AUTH_JWT_KEY_IS_BASE64_ENCODED

Variable	AUTH_JWT_KEY_IS_BASE64_ENCODED optional
Description	A boolean value that indicates whether the key configured in AUTH_JWT_KEY needs to be base64-decoded before it is used for token signature verification.
Example	AUTH_JWT_KEY_IS_BASE64_ENCODED=true

AUTH_JWT_ISS

Variable	AUTH_JWT_ISS optional
Description	Specify the "issuer" that will be matched against the iss claim extracted from JWT auth tokens. This can be used to ensure that only tokens created by the expected party are used to authenticate your client.
Example	AUTH_JWT_ISS=example.com

AUTH_JWT_AUD

Variable	AUTH_JWT_AUD optional
Description	Specify the "audience" that will be matched against the aud claim extracted from JWT auth tokens. This can be used to ensure that only tokens for a specific application are used to authenticate your client.
Example	AUTH_JWT_AUD=example.com

Insecure mode

Insecure mode is designed for development or testing. It supports unsigned JWTs that can be generated anywhere, including on the client, as well as signed JWTs which are accepted with no signature verification.

All other authentication variables (aside from AUTH_MODE) can be omitted.

See Usage -> Authentication -> Insecure mode for more information.

Telemetry

By default, ElectricSQL collects aggregated, anonymous usage data and sends them to our telemetry service. See Reference -> Telemetry for more information.

ELECTRIC_TELEMETRY

Variable	ELECTRIC_TELEMETRY
Default	enabled
Description	Telemetry mode. The allowed values are: enabled and disabled. Telemetry is enabled by default. Set to disabled to disable collection. It's extremely helpful to leave telemetry enabled if you can.
Example	ELECTRIC_TELEMETRY=disabled