[smengcl]

Generated-by: Claude Code (Opus 5.5)

What changes were proposed in this pull request?

HDDS-17151 (OM fails to start after a downgrade because of db.snapdiff) was not caught by the upgrade acceptance test. The common non-rolling upgrade callback does not touch snapshots, and after each restart the test only waits for one OM leader. ozone admin om roles reports peers from config, so it does not notice a follower that failed to start either.

This PR:

• Adds smoketest/upgrade/snapshot.robot, which creates a snapshot of a bucket, reads the key through the .snapshot path of snapshots from earlier phases, and diffs against the previous snapshot until the job is DONE.
• Runs it in every phase of both the non-rolling (old1 bucket) and rolling (overwrite bucket) upgrade suites, with independent callbacks. The old version also runs a diff before the upgrade: the new version replays that WAL into db.snapdiff SST files, which the old version must read after the downgrade. Skipped when upgrading from versions before 1.4.0.
• Waits for the RPC port of every OM after each restart (wait_for_all_oms). The rolling suite already waited for each restarted OM, so there it is added only after the initial start and the non-rolling downgrade/re-upgrade restarts.

Suggested by @errose28 in #11633 (comment).

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-17192

How was this patch tested?

Ran compose/upgrade/test.sh locally: ha non-rolling 2.2.1 to current passed. test.sh does not run the rolling suite yet, so it was enabled locally (current to current) and passed.

With the HDDS-17151 fix reverted in the OM jar, the run fails in the downgraded phase: one OM does not start (Corrupt or unsupported format_version: 6 in file /data/metadata/db.snapdiff/MANIFEST-000028), and wait_for_all_oms times out on it.

🤖 Generated with Claude Code

[Copilot]

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

[errose28]

Thanks for adding this. For ZDU we forked the callbacks from the non-rolling upgrade suite, so we should add these to the rolling upgrade suite as well. The rolling suite will be used once the first ZDU enabled release goes out.

I actually don't recall whether the plan was to delete the non-rolling suite at this point, or whether both will remain and we should look at having them use the same set of callbacks. Let me check on this with the other devs who worked on this. In the mean time, probably best to apply the test to both suites independently.

[smengcl]

For ZDU we forked the callbacks from the non-rolling upgrade suite, so we should add these to the rolling upgrade suite as well.

Thanks @errose28 . Added the same checks to the rolling upgrade suite as well.