What's New in v0.0.9

What’s New in v0.0.9

Bug Fixes

• Elasticsearch CDC failed on text watermark fields (Fielddata is disabled): When cdc_watermark_field pointed at a text field (e.g. id), every poll returned 400 illegal_argument_exception: Fielddata is disabled because text fields have no doc_values and fielddata is off by default since ES 8.x. The error was only logged as a warning and retried forever, so the affected index silently never advanced. The CDC poll now resolves a sortable representation of the watermark field — preferring the conventional <field>.keyword multi-field — and sorts/range-filters on that, while still reading the watermark value from _source at the base field. Indices with a genuinely unsortable watermark are caught at preflight with an actionable message instead of spinning at runtime.
• Non-date watermarks dropped documents at the safety-lag boundary: The poll’s upper bound (now − cdc_safety_lag) is an RFC3339 timestamp and was compared lexicographically even for keyword/numeric/id watermarks, filtering rows arbitrarily. The upper bound is now applied only for date-family watermarks; non-date watermarks paginate on the gt lower bound plus the ascending sort alone. A non-date watermark must still be monotonically increasing (e.g. a ULID or zero-padded sequence).
• Elasticsearch missing from the connector wizard: The manifest declared an off-list search category, which the connector grid silently dropped, so Elasticsearch never appeared in Add Connection. It is now categorised under Databases (alongside MongoDB/DynamoDB), and both grid views defensively render any unrecognised category instead of dropping it.

Improvements

• Connection edit form is now manifest-driven for every connector: The Edit Connection page previously used hardcoded per-connector forms, so connectors without a bespoke branch (Elasticsearch, DynamoDB, Iceberg, Pub/Sub, …) showed “No configuration form available”. It now renders from the connector manifest — the same source the Add Connection wizard uses — which also corrects latent field-key mismatches in the old forms (e.g. MySQL username vs the config’s user, BigQuery dataset/credentials_json vs dataset_id/credentials_file).
• Connection-based pipelines are self-describing about their connector type: A pipeline that references a connection by name (e.g. created via the CLI) stored no inline source/sink type, so the pipeline canvas couldn’t match a manifest and fell back to a generic Host/Endpoint field. The connector type is now denormalised from the referenced connection wherever a definition is persisted (Create, SaveDraft, Update, Apply), with a one-time idempotent backfill on startup so pre-existing definitions become self-describing without a re-save. The connection remains the authority for DSN and properties.

Installation

curl -fsSL https://nanosync.dev/install.sh | sh