L4 — Platform services (Day-2, GitOps)
The core Day-2 stack, delivered by Argo (hub/ + spoke/base + overlays). Grouped by concern.
Cluster management & trust
| Component | Where | Notes |
|---|---|---|
| ACM / MCE | hub (hub/acm MultiClusterHub) |
manages wl, wl-dr, hub-dr; ManagedClusterSet spokes |
| cert-manager | both (*/cert-manager) |
ClusterIssuer vault-issuer (Vault PKI, audience vault://vault-issuer) |
| External Secrets (ESO) | both (*/eso) |
ClusterSecretStore vault, reads apps/ |
| cluster-trust | both (*/cluster-trust) |
user-ca-bundle = Airgap CA + Quay CA (both required) |
| mirror-config | both (*/mirror-config) |
IDMS/ITMS → Quay; regenerated each mirror run |
⚠️ user-ca-bundle needs BOTH CAs — missing Airgap CA = pods can’t reach MinIO/SSO (x509); missing Quay CA = nodes can’t pull new images. It also carries each cluster’s ingress CA for Ramen S3 trust (see DR).
Storage
| Cluster | Storage |
|---|---|
| hub / hub-dr | LVMS (lvms-vg1) |
| wl / wl-dr | ODF on LVM — deviceset count:1 replica:3, cluster-wide + per-OSD Vault-KMS encryption |
StorageClasses (wl/wl-dr): lvms-vg1 (default), ocs-storagecluster-ceph-rbd, -cephfs, -ceph-rgw, noobaa. ODF nodes labeled cluster.ocs.openshift.io/openshift-storage.
Security / PCI-DSS
| Control | Detail |
|---|---|
| FIPS | all clusters + infra VMs (GitLab excepted) |
| etcd/API encryption | aesgcm both clusters (*/etcd-encryption) |
| Compliance | Compliance Operator ocp4-pci-dss (*/compliance), daily scan 01:00 |
| ACS / StackRox | Central on hub; SecuredCluster sensors on spokes; Privileged-Container policy → FAIL_DEPLOYMENT_CREATE |
| File integrity | FileIntegrity node FIM (*/file-integrity) |
| Container security | Quay CVE scanning of running images (*/container-security) |
| Gatekeeper (OPA) | */gatekeeper — OperatorGroup must be AllNamespaces |
| NetworkPolicies | default-deny + scoped-allow on non-mesh app ns (mesh ns use ztunnel mTLS instead) |
| Supply chain | cosign-signed demo image; ACS enforced “require trusted cosign signature” |
Observability
| Signal | Stack | S3 (MinIO) |
|---|---|---|
| Metrics | user-workload-monitoring (both) + ACM MCO (hub aggregates) | acm-observability |
| Logs | OpenShift Logging (Vector) + LokiStack | loki-hub / loki-spoke |
| Traces (wl) | OTel Collector otel → TempoStack tracing (tenant dev) |
tempo |
| Dashboards | COO Monitoring UIPlugin + Perses (auto-provisioned) |
— |
| Alerting | Alertmanager → webhook → Wazuh SIEM (.52:9099) |
— |
⚠️ TempoStack is delivered by the tempo-operator; if it wedges (status frozen, no workloads), delete+recreate the TempoStack CR to force a fresh reconcile. Argo needs the custom health check (scripts/argo-spoke-tempostack-healthcheck.sh).
GitOps vs. out-of-band
Almost entirely GitOps. Out-of-band: ocs-kms-token (fresh Vault token before the ODF wave), IDMS re-apply on mirror changes (commit in the same motion).