[release-4.21] MCO-2017: Backport OSImageStreams to 4.21

[pablintino]

- What I did
This change is a manual cherry-pick of multiple PRs that backports the OSImageStreams to 4.21.

- How to verify it

TBD

- Description for the changelog
This change is a manual cherry-pick of multiple PRs that backports the OSImageStreams to 4.21.

[openshift-ci-robot]

@pablintino: This pull request references MCO-2017 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

Details

In response to this:

- What I did
This change is a manual cherry-pick of multiple PRs that backports the OSImageStreams to 4.21.

- How to verify it

TBD

- Description for the changelog
This change is a manual cherry-pick of multiple PRs that backports the OSImageStreams to 4.21.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pablintino

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [pablintino]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@pablintino: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-images	420c2a7	link	true	/test okd-scos-images
ci/prow/bootstrap-unit	420c2a7	link	false	/test bootstrap-unit

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[ptalgulk01]

Pre-merge verified:

Environment Setup
OpenShift Version: 4.21.0-0-2026-01-08-104425-test-ci-ln-zjmf0hk-latest
Platform: AWS, GCP, Azure, Vsphere, FIPS on TechPreview Cluster

Verification Steps

Checking Switch

• Enable the TechPreview and check OSStream enabled

status:
  featureGates:
           - name: OSStreams

• Default is rhel-9

$  oc get mcp worker -ojsonpath='{.status.osImageStream.name}'
 empty
$ oc debug node/ip-10-0-71-60.us-east-2.compute.internal -- chroot /host rpm-ostree status
Starting pod/ip-10-0-71-60us-east-2computeinternal-debug-qqxzm ...
To use host binaries, run `chroot /host`
State: idle
Deployments:
* ostree-unverified-registry:registry.build05.ci.openshift.org/ci-ln-qz3x1kb/stable@sha256:707f4ec6193865bb7eda34e8e9bb5f1310b00eb4342fd666ae72a55c218811aa
                   Digest: sha256:707f4ec6193865bb7eda34e8e9bb5f1310b00eb4342fd666ae72a55c218811aa
                  Version: 9.6.20251219-0 (2025-12-25T12:49:21Z)

• Switch to rhel-10 (Tested on both Master and Worker pool)

 $ oc patch mcp worker --type merge -p '{"spec":{"osImageStream":{"name":"rhel-10"}}}'
machineconfigpool.machineconfiguration.openshift.io/worker patched

$ oc debug node/ip-10-0-8-235.us-east-2.compute.internal -- chroot /host rpm-ostree status
Starting pod/ip-10-0-8-235us-east-2computeinternal-debug-tglv9 ...
To use host binaries, run `chroot /host`
State: idle
Deployments:
* ostree-unverified-registry:registry.build05.ci.openshift.org/ci-ln-qz3x1kb/stable@sha256:8eab89184784583b739b785772d1697a73b8bd09f8dfb7b1f61dbb34d3a56bd0
                   Digest: sha256:8eab89184784583b739b785772d1697a73b8bd09f8dfb7b1f61dbb34d3a56bd0
                  Version: 10.1.20251222-0 (2025-12-26T23:56:41Z)

PIS Verification (No-issue)
Applied PIS on rhel-9 updated to rhel-10
Applied on rhel10 updated to rhel-9

Kernel Argument Verification (No-issue)
Applied the Kernel argument MC

$ oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker  
  name: 05-worker-kernelarg-selinuxpermissive  
spec:
  kernelArguments:
    - enforcing=0
EOF

Updated to rhel10.
Applied MC on rhel-10 -> switch

Custom Pool (No-issue)

• Created custom Pool and check the switch on the node

Scaling Node (No-issue)

• Able to Scale node with rhel-9 and rhel-10

Scaling node on Custom Machineset (No-issue)

• Created the Custom MS and scaled the node for it

$ oc get machinesets.machine.openshift.io -n openshift-machine-api ppt-0701a-6m9l2-worker-us-east-1d -o yaml > ms10.yaml

• Able to create custom MS and scale node for it for rhel-9 and rhel-10

Pause (No-issue)

• Pause the MCP with rhel-9 -> Switch to rhel10 -> Unpause -> Switched to rhel10
• Pause the MCP with rhel-10 -> Switch to rhel9-> Unpause -> Switched to rhel9

Max Unavailable (No-issue)

Kernel Argument (issue)

• Applied MC on rhel-9 cluster -> Switch to rhel10 -> Error
• Applied MC on rhel-10 -> Switch to rhel-9 -> MCP updated with error and modified to rhel10

oc create -f - << EOF                                                       
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker 
  name: worker-kernel-rt
spec:
  kernelType: realtime

EOF
machineconfig.machineconfiguration.openshift.io/worker-kernel-rt created

Error seen

  - lastTransitionTime: "2026-01-08T12:42:26Z"
    message: 'Node ip-10-0-62-222.us-east-2.compute.internal is reporting: "Node ip-10-0-62-222.us-east-2.compute.internal
      upgrade failure. error running rpm-ostree override remove kernel kernel-core
      kernel-modules kernel-modules-core kernel-modules-extra --install kernel-rt-core
      --install kernel-rt-modules --install kernel-rt-modules-extra --install kernel-rt-kvm:
      error: Packages not found: kernel-rt-kvm\n: exit status 1", Node ip-10-0-62-222.us-east-2.compute.internal
      is reporting: "error running rpm-ostree override remove kernel kernel-core kernel-modules
      kernel-modules-core kernel-modules-extra --install kernel-rt-core --install
      kernel-rt-modules --install kernel-rt-modules-extra --install kernel-rt-kvm:
      error: Packages not found: kernel-rt-kvm\n: exit status 1"'
    reason: 1 nodes are reporting degraded status on sync
    status: "True"
    type: NodeDegraded

Extension (No-issue)

• Applied MC on rhel-9 cluster -> Switch to rhel10 -> MCP got updated to rhel10
• Applied MC on rhel-10 -> Switch to rhel-9 -> MCP updated with error and modified to rhel10

oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: master-ext
spec:
  config:
    ignition:
      version: 3.1.0
  extensions:
    -  usbguard
EOF

SNO (no-issue)

FIPS based cluster (No-issue)

Verified the Day-1 installation (No-issue)

• Both for rhel10 and rhel-9