SQL Server Source

Nanosync supports two change-capture modes for SQL Server:

ModeMechanismRequirement
cdc (default)SQL Server CDC change tables + LSN watermarkingCDC enabled on database and tables; FULL or BULK_LOGGED recovery model
tlogsys.fn_dblog transaction log direct readVIEW DATABASE STATE only — no CDC setup required

Mode: cdc (default)

Prerequisites

Setup

  1. Enable CDC on the database

    USE mydb;
    EXEC sys.sp_cdc_enable_db;
  2. Enable CDC per table

    EXEC sys.sp_cdc_enable_table
      @source_schema    = N'dbo',
      @source_name      = N'orders',
      @role_name        = NULL,
      @capture_instance = N'dbo_orders',
      @supports_net_changes = 1;

    Repeat for each table you want to replicate.

  3. Enable snapshot isolation

    Nanosync uses snapshot isolation during the initial backfill to read a consistent point-in-time view of each table. Without it, concurrent writes during the snapshot can produce duplicates or missing rows — especially on tables without primary keys.

    ALTER DATABASE mydb SET ALLOW_SNAPSHOT_ISOLATION ON;
  4. Start the SQL Server Agent and keep it running

    CDC capture and cleanup jobs run inside the SQL Server Agent. If the Agent is down for an extended period, transaction logs accumulate and may get truncated before nanosync reads them — causing permanent data loss.

    Verify the Agent is running:

    SELECT name, enabled FROM msdb.dbo.sysjobs WHERE name LIKE 'cdc.%';

    Set the Agent service to restart automatically to prevent extended outages.

  5. Configure CDC job retention

    Two separate jobs control CDC behaviour. Set both:

    Capture job — how often SQL Server scans the transaction log for new changes (@pollinginterval is in seconds):

    USE mydb;
    EXEC sys.sp_cdc_change_job @job_type = 'capture', @pollinginterval = 86399;
    EXEC sp_cdc_stop_job 'capture';
    EXEC sp_cdc_start_job 'capture';

    Cleanup job — how long rows stay in CDC change tables before being deleted (@retention is in minutes; default is 4320 = 3 days). Increase this if pipelines might be down for extended periods:

    EXEC sys.sp_cdc_change_job @job_type = 'cleanup', @retention = 14400;  -- 10 days

    If nanosync is offline longer than the retention window, rows are deleted before they can be read — the pipeline must re-snapshot affected tables.

  6. Create the login and grant permissions

    CREATE LOGIN nanosync WITH PASSWORD = 'secret';
    
    USE mydb;
    CREATE USER nanosync FOR LOGIN nanosync;
    EXEC sp_addrolemember 'db_datareader', 'nanosync';
    GRANT SELECT ON SCHEMA::cdc TO nanosync;
    GRANT EXECUTE ON sys.sp_cdc_help_change_data_capture TO nanosync;
    GRANT VIEW DATABASE STATE TO nanosync;
    
    -- Also add the user to master so server-level grants work
    USE master;
    CREATE USER nanosync FOR LOGIN nanosync;
  7. Verify CDC is working

    SELECT * FROM cdc.change_tables;

Configuration

connections:
  - name: prod-sqlserver
    type: sqlserver
    dsn: "sqlserver://nanosync:${env:SQL_PASSWORD}@host:1433?database=mydb"

pipelines:
  - name: orders-pipeline
    source:
      connection: prod-sqlserver
      tables:
        - dbo.orders
        - dbo.order_items
      properties:
        cdc_mode:      cdc
        poll_interval: "5s"

Mode: tlog

Reads changes directly from SQL Server transaction log files — no CDC feature required on the source database.

Prerequisites

Setup

  1. Create the login

    CREATE LOGIN nanosync WITH PASSWORD = 'secret';
    
    USE mydb;
    CREATE USER nanosync FOR LOGIN nanosync;
    
    -- Also add the user to master so server-level grants work
    USE master;
    CREATE USER nanosync FOR LOGIN nanosync;
  2. Grant permissions on the source database

    USE mydb;
    EXEC sp_addrolemember 'db_datareader', 'nanosync';
    -- db_owner is required to read tlog; db_denydatawriter prevents accidental writes
    EXEC sp_addrolemember 'db_owner', 'nanosync';
    EXEC sp_addrolemember 'db_denydatawriter', 'nanosync';
    GRANT VIEW DATABASE STATE TO nanosync;
  3. Grant server-level and master permissions

    USE master;
    GRANT VIEW SERVER STATE TO nanosync;
    GRANT SELECT ON sys.fn_dblog TO nanosync;
  4. Grant msdb permissions

    Nanosync checks SQL Server Agent job state to detect log truncation risk.

    USE msdb;
    CREATE USER nanosync FOR LOGIN nanosync;
    GRANT SELECT ON dbo.sysjobs TO nanosync;

Configuration

pipelines:
  - name: orders-tlog
    source:
      connection: prod-sqlserver
      tables:
        - dbo.orders
      properties:
        cdc_mode:       tlog
        poll_interval:  "200ms"
        log_batch_size: "10000"

Connection configuration

sqlserver://[user]:[password]@[host]:[port]?database=[db]&[options]
DSN optionExampleDescription
databasemydbDatabase name
encrypttrueEnable TLS encryption
TrustServerCertificatefalseSkip certificate verification (dev/test only)
MultiSubnetFailovertrueEnable for Always On Availability Groups
connection timeout30Connection timeout in seconds

Always On Availability Group:

dsn: "sqlserver://nanosync:${env:SQL_PASSWORD}@ag-listener:1433?database=mydb&MultiSubnetFailover=true&encrypt=true"

Source properties

Connection

PropertyDefaultDescription
cdc_modecdcChange-capture mode: cdc or tlog
poll_interval5s (cdc) / 200ms (tlog)How often to poll for new changes
max_reconnect_attempts5Maximum consecutive reconnect retries before the pipeline errors

Snapshot

PropertyDefaultDescription
chunk_size10000Rows fetched per page during the initial backfill

CDC mode properties

PropertyDefaultDescription
schema_refresh_interval5mTTL for the internal schema cache

tlog mode properties

PropertyDefaultDescription
log_batch_size10000Maximum fn_dblog rows fetched per query
max_xact_memory268435456 (256 MiB)In-memory byte limit across all open transactions
max_xact_age4hDuration after which a long-running transaction triggers a WARN log
lob_fetch_timeout5sPer-read timeout for off-row LOB columns

Output

PropertyDefaultDescription
track_historyfalseInclude temporal table history-table inserts in the change stream
split_updatesfalseEmit UPDATE as DELETE(before) + INSERT(after) pairs — improves BigQuery and Iceberg compatibility

Supported operations

Operationcdc modetlog mode
INSERTYesYes
UPDATEYes — before + afterYes — before + after
DELETEYesYes
TRUNCATEDetected via full scanNot captured
DDL changesSchema drift detection onlySchema drift detection only

Limitations


Monitoring

nanosync metrics pipeline orders-pipeline
MetricDescription
ns_pipeline_replication_lag_secondsEnd-to-end source-to-sink latency
ns_cdc_events_totalCDC events read from change tables or tlog
ns_snapshot_rows_totalRows written during the initial snapshot
ns_tlog_read_lag_secondsAge of oldest unprocessed tlog record (tlog mode only)

Troubleshooting

sp_cdc_enable_db fails with “Database is not eligible for CDC”

The database must use FULL or BULK_LOGGED recovery model:

ALTER DATABASE mydb SET RECOVERY FULL;

No changes appear after CDC is enabled

Check that the CDC capture job is active — run the verification query from step 4 of the setup. If the Agent service was restarted, jobs may need to be re-enabled.

tlog mode: “transaction exceeded max_xact_memory”

Increase max_xact_memory or switch to cdc mode.

Always On AG: changes stop after failover

Point the DSN at the AG listener and set MultiSubnetFailover=true. The connector reconnects automatically and resumes from the last committed LSN.