Connections

A connection is a named, reusable credential bundle. Define it once in connections:, then reference it by name in every pipeline that needs it. The connection is the single authority for connector type and credentials — change it in one place, all pipelines pick it up on the next apply.

Defining connections

Connections live in the same YAML file as pipelines and are applied with nanosync apply:

connections:
  - name: prod-postgres
    type: postgres
    dsn: "postgres://nanosync:${env:PG_PASSWORD}@db.prod.internal:5432/mydb?sslmode=require"

  - name: prod-bigquery
    type: bigquery
    properties:
      project_id: my-project
      dataset_id: replication
      credentials_file: /path/to/key.json

DSN vs properties

Source connectors (databases) use a dsn: string. Destination connectors (cloud services) use properties: key-value pairs. Both support ${env:VAR_NAME} for secret injection.

connections:
  - name: prod-postgres
    type: postgres
    dsn: "postgres://nanosync:${env:PG_PASSWORD}@db.prod.internal:5432/mydb?sslmode=require"

  - name: prod-sqlserver
    type: sqlserver
    dsn: "sqlserver://nanosync:${env:MSSQL_PASSWORD}@db.prod.internal:1433?database=mydb"

connections:
  - name: prod-bigquery
    type: bigquery
    properties:
      project_id: my-project
      dataset_id: replication
      credentials_file: /etc/nanosync/bq-sa.json

  - name: prod-kafka
    type: kafka
    properties:
      brokers: "kafka1.prod.internal:9092,kafka2.prod.internal:9092"
      tls_enabled: "true"
      sasl_mechanism: SCRAM-SHA-256
      sasl_username: nanosync
      sasl_password: "${env:KAFKA_PASSWORD}"

Secret management

Never put passwords or API keys directly in the config file. Use ${env:VAR_NAME} — values are injected from the process environment at startup.

connections:
  - name: prod-postgres
    type: postgres
    dsn: "postgres://nanosync:${env:PG_PASSWORD}@db.prod.internal:5432/mydb?sslmode=require"

Set the variable before running nanosync:

export PG_PASSWORD=my-secret
nanosync apply --file pipelines.yaml

For secrets stored in GCP Secret Manager, AWS Secrets Manager, or HashiCorp Vault, configure the secrets_backend in your server config and use ${secret:path/to/secret} instead of ${env:...}. See Configuration reference for details.

Testing a connection

Verify a connection is reachable before using it in a pipeline:

nanosync test connection prod-postgres
# ✓  prod-postgres connected in 12 ms

nanosync test connection prod-bigquery
# ✓  prod-bigquery connected in 34 ms

If the connection fails, the error is specific:

✗  prod-postgres: connection refused — dial tcp 10.0.1.4:5432: connect: connection refused
✗  prod-bigquery: permission denied — bigquery.tables.create required on dataset replication

Fix the issue before applying pipelines — don’t debug a running pipeline when you can catch it here.

Reusing connections across pipelines

One connection can be referenced by any number of pipelines. This is the main reason to use named connections rather than inlining credentials.

connections:
  - name: prod-postgres
    type: postgres
    dsn: "postgres://nanosync:${env:PG_PASSWORD}@db.prod.internal:5432/mydb?sslmode=require"

  - name: prod-bigquery
    type: bigquery
    properties:
      project_id: my-project
      dataset_id: replication
      credentials_file: /etc/nanosync/bq-sa.json

pipelines:
  - name: orders-pipeline
    source:
      connection: prod-postgres
      tables: [public.orders]
    sink:
      connection: prod-bigquery

  - name: users-pipeline
    source:
      connection: prod-postgres
      tables: [public.users]
    sink:
      connection: prod-bigquery

Both pipelines share the same credentials definition. If the database password rotates, update the environment variable and send SIGHUP to the server — no pipeline changes needed.

Available connection types

For connector-specific property keys, see the source and sink reference pages under Sources and Sinks.