# A Delta migration can create tags and change timestamp history

Iceberg 1.12.0 migrated two versions of an empty synthetic Delta table into two snapshots and created four tags from the source version numbers and stored commit-info times. The destination published one timestamp-history entry, for the final snapshot. In a second migration, two equal stored commit times produced a duplicate tag error. The destination stayed absent from the local catalog, and two staged Avro files remained beneath its warehouse path. The [fixture](ClockReplay.java) and [observations](observations.json) use empty local tables with Delta Standalone 3.3.3 and HadoopCatalog. [Migration action](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java#L177-L189)

## Two snapshots and one published history entry

The test sets the Delta log-file times two seconds apart. Delta's legacy timestamp selector uses those file times. The source's independently stored commit-info times are one millisecond apart in the successful case. Iceberg uses those stored values to name the `delta-ts` tags and also creates a `delta-version` tag for each source version. [Delta history clocks](https://github.com/delta-io/delta/blob/v3.3.3/connectors/standalone/src/main/scala/io/delta/standalone/internal/DeltaHistoryManager.scala#L196-L236) · [Iceberg tag assignment](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java#L429-L440)

| Lookup | Observed result |
| --- | --- |
| Delta timestamp lookup at the second source log-file time | Source version 1 |
| Iceberg `delta-version-0` tag | The earlier migrated snapshot |
| Iceberg timestamp lookup at the second Delta log-file time | No matching published history entry |
| Iceberg timestamp lookup at the first target snapshot's creation time | No matching published history entry |
| Iceberg timestamp lookup at the destination's published history time | The final migrated snapshot |

The migration stages both replays in one create transaction. Iceberg's metadata builder filters intermediate main-snapshot entries from the published history. The earlier snapshot and its tags remain available. The timestamp resolver searches the published history. Our first fixture expected two entries and stopped on that assertion. The corrected fixture checked both tagged snapshots and the final-only history separately. [Intermediate log entry filter](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1840-L1885) · [History resolver](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java#L314-L337) · [Earlier history change](https://github.com/apache/iceberg/pull/5568)

## Repeated stored times

In the paired case, the source still has two ordered log-file times, so both original version cutoffs remain readable. Its two stored commit-info timestamps are equal. The second replay tries to create the same `delta-ts-1577836800000` tag and hits Iceberg's duplicate-reference check before the create transaction commits. The catalog reports that the table is absent. A separate file inspection found two staged Avro files totaling 8,641 bytes. The fixture exercises the legacy stored timestamp used by the action. Newer Delta's `inCommitTimestamp` is a separate protocol field. A proposed Delta Kernel migration implementation already includes logic to skip repeated time tags. [Duplicate-reference check](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java#L61-L66) · [Calculation of the Delta tag name](https://github.com/apache/iceberg/blob/212aa894e919ff9fdf89160da4aadcbe6858ac01/delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java#L429-L440) · [Delta protocol](https://github.com/delta-io/delta/blob/v3.3.3/PROTOCOL.md#in-commit-timestamps) · [Proposed Kernel handling](https://github.com/apache/iceberg/blob/0275a908e9314cced92dbaafb65aa3f794b39663/delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeKernelTableAction.java#L615-L629)

For a migration that serves historical consumers, save the source version, source selector clock and target snapshot reference together. Rehearse named-reference and timestamp reads separately. A failed create also calls for separate catalog and destination-file checks. Our runs measured these paths on empty synthetic tables and a local HadoopCatalog. Spark SQL, record equality, retry behavior and other catalogs remain unmeasured.

## Reproduce

Download [ClockReplay.java](ClockReplay.java), [pom.xml](pom.xml) and the [observations](observations.json) into one folder. From that folder, use JDK 17 and Maven. The verified run used Liberica JDK Lite 17.0.21.0.3+5 and Maven 3.9.16. Maven resolves the fixed dependency versions into the temporary directory. The Java source launcher runs the migration against two newly written pairs of log files. [Maven classpath goal](https://maven.apache.org/plugins-archives/maven-dependency-plugin-3.8.1/build-classpath-mojo.html) · [Java source launcher](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html)

```sh
set -eu
sapien_delta_run=$(mktemp -d)
trap 'rm -rf -- "$sapien_delta_run"' EXIT
mkdir "$sapien_delta_run/fixture"
mvn -C -B -q -f pom.xml -Dmaven.repo.local="$sapien_delta_run/repository" \
  org.apache.maven.plugins:maven-dependency-plugin:3.8.1:build-classpath \
  -DincludeScope=runtime -Dmdep.outputFile="$sapien_delta_run/classpath.txt"
java --class-path "$(cat "$sapien_delta_run/classpath.txt")" --source 17 ClockReplay.java \
  "$sapien_delta_run/fixture"
sapien_delta_warehouse="$sapien_delta_run/fixture/warehouse/fixture"
cat "$sapien_delta_warehouse/distinct_times/metadata/v1.metadata.json"
find "$sapien_delta_warehouse/repeated_time" -type f -exec wc -c {} +
```

The checker prints these two outcome lines.

```text
DISTINCT sourceCutoffVersion=1 migratedSnapshots=2 migrationTags=4 publishedHistoryEntries=1 targetCutoffSnapshot=null
REPEATED duplicateTimestampTagRejected=true catalogTableExists=false
```

The metadata readback exposes `snapshots`, `refs` and `snapshot-log`. The final command inventories the failed destination before temporary cleanup. Snapshot IDs, creation times and staged filenames can vary between runs. Compare the retained tag mapping and the catalog and timestamp outcomes.
