SQL Server Source
Nanosync supports two change-capture modes for SQL Server:
| Mode | Mechanism | Requirement |
|---|---|---|
cdc (default) | SQL Server CDC change tables + LSN watermarking | CDC enabled on database and tables; FULL or BULK_LOGGED recovery model |
tlog | sys.fn_dblog transaction log direct read | VIEW DATABASE STATE only — no CDC setup required |
Mode: cdc (default)
Prerequisites
- SQL Server 2016 or later
- Database recovery model:
FULLorBULK_LOGGED - CDC feature enabled on the database and each replicated table
Setup
-
Enable CDC on the database
USE mydb; EXEC sys.sp_cdc_enable_db; -
Enable CDC per table
EXEC sys.sp_cdc_enable_table @source_schema = N'dbo', @source_name = N'orders', @role_name = NULL, @capture_instance = N'dbo_orders', @supports_net_changes = 1;Repeat for each table you want to replicate.
-
Enable snapshot isolation
Nanosync uses snapshot isolation during the initial backfill to read a consistent point-in-time view of each table. Without it, concurrent writes during the snapshot can produce duplicates or missing rows — especially on tables without primary keys.
ALTER DATABASE mydb SET ALLOW_SNAPSHOT_ISOLATION ON; -
Start the SQL Server Agent and keep it running
CDC capture and cleanup jobs run inside the SQL Server Agent. If the Agent is down for an extended period, transaction logs accumulate and may get truncated before nanosync reads them — causing permanent data loss.
Verify the Agent is running:
SELECT name, enabled FROM msdb.dbo.sysjobs WHERE name LIKE 'cdc.%';Set the Agent service to restart automatically to prevent extended outages.
-
Configure CDC job retention
Two separate jobs control CDC behaviour. Set both:
Capture job — how often SQL Server scans the transaction log for new changes (
@pollingintervalis in seconds):USE mydb; EXEC sys.sp_cdc_change_job @job_type = 'capture', @pollinginterval = 86399; EXEC sp_cdc_stop_job 'capture'; EXEC sp_cdc_start_job 'capture';Cleanup job — how long rows stay in CDC change tables before being deleted (
@retentionis in minutes; default is 4320 = 3 days). Increase this if pipelines might be down for extended periods:EXEC sys.sp_cdc_change_job @job_type = 'cleanup', @retention = 14400; -- 10 daysIf nanosync is offline longer than the retention window, rows are deleted before they can be read — the pipeline must re-snapshot affected tables.
-
Create the login and grant permissions
CREATE LOGIN nanosync WITH PASSWORD = 'secret'; USE mydb; CREATE USER nanosync FOR LOGIN nanosync; EXEC sp_addrolemember 'db_datareader', 'nanosync'; GRANT SELECT ON SCHEMA::cdc TO nanosync; GRANT EXECUTE ON sys.sp_cdc_help_change_data_capture TO nanosync; GRANT VIEW DATABASE STATE TO nanosync; -- Also add the user to master so server-level grants work USE master; CREATE USER nanosync FOR LOGIN nanosync; -
Verify CDC is working
SELECT * FROM cdc.change_tables;
Configuration
connections:
- name: prod-sqlserver
type: sqlserver
dsn: "sqlserver://nanosync:${env:SQL_PASSWORD}@host:1433?database=mydb"
pipelines:
- name: orders-pipeline
source:
connection: prod-sqlserver
tables:
- dbo.orders
- dbo.order_items
properties:
cdc_mode: cdc
poll_interval: "5s"
Mode: tlog
Reads changes directly from SQL Server transaction log files — no CDC feature required on the source database.
Prerequisites
- SQL Server 2016 or later
- Database recovery model:
FULLorBULK_LOGGED(notSIMPLE) VIEW DATABASE STATEpermission
Setup
-
Create the login
CREATE LOGIN nanosync WITH PASSWORD = 'secret'; USE mydb; CREATE USER nanosync FOR LOGIN nanosync; -- Also add the user to master so server-level grants work USE master; CREATE USER nanosync FOR LOGIN nanosync; -
Grant permissions on the source database
USE mydb; EXEC sp_addrolemember 'db_datareader', 'nanosync'; -- db_owner is required to read tlog; db_denydatawriter prevents accidental writes EXEC sp_addrolemember 'db_owner', 'nanosync'; EXEC sp_addrolemember 'db_denydatawriter', 'nanosync'; GRANT VIEW DATABASE STATE TO nanosync; -
Grant server-level and master permissions
USE master; GRANT VIEW SERVER STATE TO nanosync; GRANT SELECT ON sys.fn_dblog TO nanosync; -
Grant msdb permissions
Nanosync checks SQL Server Agent job state to detect log truncation risk.
USE msdb; CREATE USER nanosync FOR LOGIN nanosync; GRANT SELECT ON dbo.sysjobs TO nanosync;
Configuration
pipelines:
- name: orders-tlog
source:
connection: prod-sqlserver
tables:
- dbo.orders
properties:
cdc_mode: tlog
poll_interval: "200ms"
log_batch_size: "10000"
tlog mode holds all events for an open transaction in memory until commit. Large transactions exceeding max_xact_memory (default 256 MiB) will cause the pipeline to error. For large-transaction workloads, prefer cdc mode.
Connection configuration
sqlserver://[user]:[password]@[host]:[port]?database=[db]&[options]
| DSN option | Example | Description |
|---|---|---|
database | mydb | Database name |
encrypt | true | Enable TLS encryption |
TrustServerCertificate | false | Skip certificate verification (dev/test only) |
MultiSubnetFailover | true | Enable for Always On Availability Groups |
connection timeout | 30 | Connection timeout in seconds |
Always On Availability Group:
dsn: "sqlserver://nanosync:${env:SQL_PASSWORD}@ag-listener:1433?database=mydb&MultiSubnetFailover=true&encrypt=true"
Source properties
Connection
| Property | Default | Description |
|---|---|---|
cdc_mode | cdc | Change-capture mode: cdc or tlog |
poll_interval | 5s (cdc) / 200ms (tlog) | How often to poll for new changes |
max_reconnect_attempts | 5 | Maximum consecutive reconnect retries before the pipeline errors |
Snapshot
| Property | Default | Description |
|---|---|---|
chunk_size | 10000 | Rows fetched per page during the initial backfill |
CDC mode properties
| Property | Default | Description |
|---|---|---|
schema_refresh_interval | 5m | TTL for the internal schema cache |
tlog mode properties
| Property | Default | Description |
|---|---|---|
log_batch_size | 10000 | Maximum fn_dblog rows fetched per query |
max_xact_memory | 268435456 (256 MiB) | In-memory byte limit across all open transactions |
max_xact_age | 4h | Duration after which a long-running transaction triggers a WARN log |
lob_fetch_timeout | 5s | Per-read timeout for off-row LOB columns |
Output
| Property | Default | Description |
|---|---|---|
track_history | false | Include temporal table history-table inserts in the change stream |
split_updates | false | Emit UPDATE as DELETE(before) + INSERT(after) pairs — improves BigQuery and Iceberg compatibility |
Supported operations
| Operation | cdc mode | tlog mode |
|---|---|---|
INSERT | Yes | Yes |
UPDATE | Yes — before + after | Yes — before + after |
DELETE | Yes | Yes |
TRUNCATE | Detected via full scan | Not captured |
| DDL changes | Schema drift detection only | Schema drift detection only |
Limitations
TRUNCATEis not captured intlogmode. Incdcmode it is detected via a full table scan, not a streaming event.tlogmode holds all events for an open transaction in memory — large transactions exceedingmax_xact_memory(default 256 MiB) cause the pipeline to error.- SQL Server Express edition does not support per-table CDC. Use
tlogmode on Express. - Azure SQL Database:
tlogmode is not available. Usecdcmode only. - DDL changes are detected but not replayed automatically. The pipeline pauses on breaking changes.
Monitoring
nanosync metrics pipeline orders-pipeline
| Metric | Description |
|---|---|
ns_pipeline_replication_lag_seconds | End-to-end source-to-sink latency |
ns_cdc_events_total | CDC events read from change tables or tlog |
ns_snapshot_rows_total | Rows written during the initial snapshot |
ns_tlog_read_lag_seconds | Age of oldest unprocessed tlog record (tlog mode only) |
Troubleshooting
sp_cdc_enable_db fails with “Database is not eligible for CDC”
The database must use FULL or BULK_LOGGED recovery model:
ALTER DATABASE mydb SET RECOVERY FULL;
No changes appear after CDC is enabled
Check that the CDC capture job is active — run the verification query from step 4 of the setup. If the Agent service was restarted, jobs may need to be re-enabled.
tlog mode: “transaction exceeded max_xact_memory”
Increase max_xact_memory or switch to cdc mode.
Always On AG: changes stop after failover
Point the DSN at the AG listener and set MultiSubnetFailover=true. The connector reconnects automatically and resumes from the last committed LSN.