Core: populate manifest created/replaced/kept count when commit a snapshot

[dramaticlly]

Currently iceberg commit does not expose the manifest level information, so it's difficult to see how metadata will evolve with given snapshot.

Now snapshot summary will also include following information about the table commit

• manifests-created: new manifest was created by the same snapshot
• manifests-replaced: manifest was filtered (in-place rewrite)/merged by the merging snapshot producer
• manifest-kept: existing manifest was carried over where it's created by the older snapshot prior to the commit

// append create 1 manifest for 2 files
table.newAppend().appendFile(FILE_A).appendFile(FILE_B).commit()
assertThat(appendFile.summary())
    .containsEntry(SnapshotSummary.CREATED_MANIFESTS_COUNT, "1")
    .containsEntry(SnapshotSummary.REPLACED_MANIFESTS_COUNT, "0")
    .containsEntry(SnapshotSummary.KEPT_MANIFESTS_COUNT, "0");

// delete will copy-on-write to replace one of the manifest-entry for the existing manifest 
table.newDelete().deleteFile(FILE_A).commit()
assertThat(deleteFile.summary())
    .containsEntry(SnapshotSummary.CREATED_MANIFESTS_COUNT, "1")
    .containsEntry(SnapshotSummary.REPLACED_MANIFESTS_COUNT, "1")
    .containsEntry(SnapshotSummary.KEPT_MANIFESTS_COUNT, "0");

[dramaticlly]

@nastra @amogh-jahagirdar if you can help take a look

[dramaticlly]

also @huaxingao and @stevenzwu, if you are interested to take a look.

[RussellSpitzer]

Just a note because of the normal append path in merging snapshot producer this can also be original manifest != appended manifest

[RussellSpitzer]

ah nvm, didn't see we were in manifest filter manager

[RussellSpitzer]

I'm generally +1 on this idea but don't have time to do a full review right now, I do think we should consider using "existing" instead of "kept"? Or maybe skip it all together since I think we aren't tracking manifests which are not scanned in the first place right?

[dramaticlly]

I'm generally +1 on this idea but don't have time to do a full review right now, I do think we should consider using "existing" instead of "kept"? Or maybe skip it all together since I think we aren't tracking manifests which are not scanned in the first place right?

Thanks @RussellSpitzer , discussed offline as we reuse the SnapshotSummary already defined in https://iceberg.apache.org/spec/#optional-snapshot-summary-fields. Previously we only populate such for rewrite-manifest operation, this change I want to introduce them for all commits result in a new snapshot, like append, row-delta and delete etc

[RussellSpitzer]

You are totally right! Better to keep with the original definitions then