Commit 25fa595
Rebase lug 2026
* Upgrade Go to 1.26.3
* Update dependencies
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* gomod: tidy and verify all modules
Replace the hardcoded module lists in the `gomod` make target and
verify-gomod.sh with dynamic discovery, so no module is left untidied or
unverified. Hidden directories are excluded.
* e2e: set the v3 runtime class path in apiserver templates
* build: use gcr.io/distroless/static as base image
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* feat: add gVisor RuntimeClass support for containerd
Add support for running workloads under gVisor (runsc) as a containerd
runtime handler, following the existing NVIDIA GPU runtime pattern.
gVisor is gated to Debian-family distributions only. The runsc apt
package provides both runsc and containerd-shim-runsc-v1.
Assisted by Opus 4.6
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
* Use protobuf
* hetzner: enable Cluster Autoscaler
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* chore(channels): promote alpha to stable
Promotes kubernetes#18367 with latest k8s versions
Signed-off-by: Moshe Vayner <moshe@vayner.me>
* chore(networking): bump aws cni to version 1.21.2
Signed-off-by: Moshe Vayner <moshe@vayner.me>
* test: hack/update-expected
Signed-off-by: Moshe Vayner <moshe@vayner.me>
* chore: don't try to use proto for bare-metal tooling
* Release 1.36.0-alpha.1 (kubernetes#18413)
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* etcd: add ListenClientHTTPURLs field to EtcdManagerSpec
* generated changes
* Release notes for 1.36 (alpha)
* chore: Add hashes for additional May releases
* Add logs at v2 level to ensure we have slow pods data in logs
* chore: upgrade containerd to v2.3.1
* ./hack/update-expected.sh
* Upgrade kube-router to v2.10.0
* Disable strict external IP validation
* azure: Use IMDS attested metadata document for node identity
Replace the resourceID+vmID token with a PKCS7-signed attested metadata document from the Azure IMDS. The authenticator queries the attested document endpoint, and the verifier validates the PKCS7 signature chain, checks the nonce and expiration, then cross-verifies the signed vmId against the Azure API response.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* azure: Address review feedback on IMDS attestation
- Thread context.Context through the IMDS query helpers and verifier
client construction (http.NewRequestWithContext). CreateToken and the
nodeidentity client use context.TODO() where no context is available.
- Require the signer certificate to carry at least one issuer identifier
(RawIssuer or AuthorityKeyId) before matching a fetched intermediate,
so the per-field checks cannot degrade to accepting any CA.
- Include the source URL when an intermediate certificate fails to parse.
- Document the PKCS7 verification trust progression and clarify that
validateFetchedIntermediateForSigner is a structural check only; trust
is established by verifySignerCertChain.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* azure: Reflow comments to 100 columns
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* azure: Resolve intermediate CA chain across multiple AIA hops
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* aws: pin LBC selectors to fix in-place upgrades
The helm+kustomize migration switched the controller Deployment and
webhook Service selectors from {component, name} to the chart's
{instance, name}. A Deployment's spec.selector is immutable, so upgrading
from kops <1.36 has the apply rejected: the pod template is never updated,
the webhook Service loses its endpoints, and the mutating webhooks
(failurePolicy: Fail) time out, breaking target registration.
Pin both selectors back to the historical {component, name} labels and add
the component label to the pod template so it still satisfies the selector,
keeping upgrades working without touching the immutable field.
* cert-manager: set AWS_REGION on controller for Route53 dns-01
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* cert-manager: upgrade to v1.19.5
* ./hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Default omitted authorization to RBAC instead of AlwaysAllow
A cluster spec applied with the top-level `authorization` field omitted
(kops create -f / replace -f) defaulted to AlwaysAllow, while
`kops create cluster` has always defaulted to RBAC. Align the v1alpha2
and v1alpha3 SetDefaults_ClusterSpec with the CLI: an omitted or empty
authorization now defaults to RBAC. AlwaysAllow is still honored when set
explicitly.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* cilium: allow disabling masquerade in ENI IPAM mode
kOps forbids setting masquerade (disableMasquerade) when Cilium ENI IPAM is
used, erroring with "Masquerade must be enabled when ENI IPAM is used". This
blocks users who want Cilium's upstream no-masquerade behavior for ENI (e.g.
private-topology clusters with NAT-gateway egress, or clusters using VPC
endpoints), forcing them to patch the cilium-config ConfigMap after the fact,
which races new nodes during rolling updates.
Remove the validation so masquerade can be set in either direction for ENI.
The default is intentionally left unchanged (masquerade stays on): flipping it
off by default breaks pod egress to external endpoints (e.g. IRSA -> STS) on
public-topology clusters whose pod ENI IPs have no public address, as shown by
a red pull-kops-e2e-cni-cilium-eni run. Users who can route pod egress without
masquerading opt in via masquerade=false (or via extraConfig).
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* channels: surface addon apply failures via a readiness probe
Apply errors were only logged, so a manifest the apiserver rejects (e.g. an
immutable Deployment selector) left the cluster silently broken. /readyz now
reflects the last apply outcome; the pod is system-node-critical, so a NotReady
fails `kops validate cluster` and halts the rolling update before workers roll.
* etcd-manager: bump etcd to latest patches (3.5.31, 3.6.12)
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* etcd-manager: upgrade to v3.0.20260531
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* ./hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Build only linux/amd64 in kubernetes scalability presubmits
* chore: downgrade containerd to v2.2.4
* ./hack/update-expected.sh
* Skip the live kubelet version skew probe for Terraform golden tests
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Ignore the default kubeconfig for hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Allow feature gates to be set in scalability tests
* dump: add --node-dump-timeout flag for per-node dump timeout
* docs: add gVisor RuntimeClass support to 1.36 release notes
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
* Remove gVisor package config
* Restrict gVisor runtime to worker instance groups
gVisor (runsc) was previously installable on any instance group role.
Restrict it to nodes with role Node: reject cluster/IG configs that
enable gVisor on control plane, apiserver, or bastion roles, strip the
gVisor config from non-worker nodeup configs, and only apply the
gVisor node label and RuntimeClass addon when a worker has it enabled.
Also harden nil handling for cluster.Spec.Containerd in nodeup config
and bootstrapchannelbuilder. Update release notes and add tests across
validation, nodeup config, gvisor builder, and instancegroup spec.
* Enable misspell in golangci-lint
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
* build(deps): bump actions/checkout from 6.0.2 to 6.0.3
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@de0fac2...df4cb1c)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: 6.0.3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* calico: add NFTablesMode setting
Surface Calico's Felix NFTablesMode (Disabled, Enabled, Auto) as a
field on CalicoNetworkingSpec and propagate it to the calico-node
DaemonSet via FELIX_NFTABLESMODE. When left unset, the upstream
Calico chart default applies, preserving existing behavior.
On distributions where iptables is only present as a shim over
nftables (e.g. RHEL10+, Rocky10+), routing Felix's data plane
through iptables-nft / nft_compat has produced BGP session flapping
and broken pod networking on GCE. This field lets clusters opt
their Calico install into native nftables on those nodes.
* flagbuilder: only shell-quote values for the joined string form
* etcd-manager: switch to go-runner-based distroless image
* ./hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* kube-proxy: assert buildPod command in unit test
* Remove unused util/pkg/exec package
* gVisor: add HasGVisor() helper function
* Default node dump timeout to 5m in scalability run-test.sh
* Support etcd 3.7.0-rc.0 and allow overriding etcd version in scalability scenario
Signed-off-by: Jefftree <jeffrey.ying86@live.com>
* Regenerate integration golden outputs for etcd 3.7.0-rc.0
Signed-off-by: Jefftree <jeffrey.ying86@live.com>
* Fix GCE backend service ownership filtering during delete
containsOnlyListedIGMs returned true for backend services with no
backends (vacuous truth), causing listBackendServices to claim
unrelated regional backend services. This cascaded into
listHealthchecks selecting their health checks too. The delete loop
then retried those foreign resources indefinitely.
Return false when Backends is empty so only backend services with
backends pointing at cluster IGMs are selected for deletion.
Signed-off-by: Jathavedhan M <jathavedhan.m@ibm.com>
* azure: Simplify IMDS attestation code
* Filter randomized AWS zones by instance type availability
* Release 1.36.0-beta.1 (kubernetes#18464)
* Improve Google Storage Bucket support.
Ensuring the image stored in gs:// can be retrieved.
Switching from 'gsutil' to 'gcloud storage' where appropriate.
Switching from 'gsutil' to 'gcloud buckets' where appropriate.
go get cloud.google.com/go/storage
./hack/update-goimports.sh
./hack/verify-gomod.sh
./hack/update-expected.sh
* gce: Reconcile HTTP health check changes on existing health checks
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* aws: Reconcile target group health check changes on existing target groups
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Enable gocritic and gosec linters
Changes are fixes from running 'make verify-golangci-lint'
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
* Added support for APIServer using LB
Added support for APIServer without local Etcd
Etcd is hosted on ControlPlane nodes.
For GCP setup a LB and plumb it's address into /opt/kops/conf/kube_env.yaml
Fix up /etc/hosts to point the etcd host to the LB address.
Also added support for gs: storage buckets.
Needed to use the hack/dev-build-gce.sh build.
make gomod
hack/update-expected.sh
Fixed template for google storage copy & check
Wired context up to make call stack more obvious
Also provided better reuse of context object
make gofmt
make goimports
Fixed "APIServer role forbidden on GCE with dns=None" test to reflect
that use case should no longer generate an error
Fixed unit test.
* feat(linode): implement SSH key management and associated tasks
Signed-off-by: Moshe Vayner <moshe@vayner.me>
* Add serathius as scalability test owner
* Report experiment variant into scalability run metadata
* Asserted fixes from justinsb's feedback.
One significant change was renaming EtcdLBAddresses as EtcdIPs.
Cleanup k8s tf test file.
* chore(channels): bump k8s versions in alpha channel
Signed-off-by: Moshe Vayner <moshe@vayner.me>
* Upgrade Karpenter to v1.13.0
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Replace deprecated karpenter.sh/provisioner-name with karpenter.sh/nodepool
The karpenter.sh/provisioner-name label is a Karpenter v1alpha5 relic. Since
v1beta1/v1, Karpenter applies karpenter.sh/nodepool to the nodes it provisions.
Update the controller's anti-self-scheduling affinity to match Karpenter's own
v1 label (also the upstream chart default), and stop setting the obsolete
provisioner-name label on Karpenter-managed nodes.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Remove obsolete node-role.kubernetes.io/master affinity term
kOps supports Kubernetes 1.31+, where node-role.kubernetes.io/master no longer
exists (removed in 1.24/1.25) and kOps only labels control-plane nodes with
node-role.kubernetes.io/control-plane. The second nodeSelectorTerm keyed on the
master label can never match, so the control-plane term alone suffices.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Generate karpenter addon manifest with kustomize
Adopt the same kustomize-based generation as the Azure CCM/CSI addons: the
upstream chart is pulled via a kustomization.yaml helmChart, kops customizations
(image, dnsPolicy, feature gates) are declarative patches, and the manifest is
regenerated with regenerate.sh. This replaces the manual 'helm template' command
plus hand-applied customizations, which required a 3-way merge to preserve on
every version bump. Controller replicas are now fixed at 1, matching the other
kustomize-generated addons.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* channels: Add Kubernetes 1.36.0 support
* Skip ImageVolume tests on COS 121
* gce: emit kops.k8s.io/instancegroup node label
The GCE node identifier reads the instance-group name from the MIG
instance template metadata but never adds it to the returned labels,
so kops-controller never patches kops.k8s.io/instancegroup onto GCE
nodes. The label has been missing since the cluster-api refactor in
a7f1b4f.
* Register Karpenter nodes with karpenter.sh/unregistered taint
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* hack/update-expected.sh
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Add missing EC2 read permissions to Karpenter IAM policy
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* scaletest: report experiment variant after kubetest2 runs
* nodeup: load ip_set module and disable firewalld on RHEL10+/Rocky10+
Two related fixes for Calico on the ForceNftables() distros (RHEL10+,
Rocky10+, etc.).
Load the ip_set kernel module alongside nf_tables and nf_conntrack.
Calico's Felix unconditionally starts an ipsetsManager that shells out
to "ipset list -name" during dataplane resync, even when NFTablesMode
is Enabled. On RHEL10-family kernels ip_set is not auto-loaded, so the
ipset call returns EINVAL and Felix panics in a tight loop, crashing
calico-node and blocking cluster Up on every arm64 grid cell.
Disable and mask firewalld via a new disableFirewalld step on
FirewallBuilder, gated on Distribution.ForceNftables(). firewalld's
default-reject filter_INPUT/filter_FORWARD policies and periodic-reload
behavior conflict with the iptables/nftables rules CNIs install for
pod and service traffic; Calico's own requirements doc and RKE2 both
document that firewalld must be disabled on hosts running these CNIs.
The disable/mask sequence is idempotent and a no-op where firewalld is
not installed, so this is net-neutral on the cloud images that already
strip firewalld (AWS RHEL/Rocky AMIs, Rocky GenericCloud) and net-
positive on the GCE-optimized Rocky 10 image where firewalld ships
active and breaks Calico BGP keepalives in BPF mode.
* Adding flag to enable machine type for APIServer only.
Added flag --api-server-size to be consistent with other machine type
flags.
Added doc on the flag reflecting my testing.
Adding GCE test for APIServer only option.
Fixed comment from previous PR.
apiserver only DNS check for AWS comment is now correct.
Removed k8s version flag from doc.
make gen-cli-docs
* Adding flag to enable machine type for APIServer only.
* Adding GCE test for APIServer only option.
* Fixing LB behavior when you have both APIServer and Control Plane.
Initially the LB sent traffic to both.
The DNS None is a new case.
Now we only send traffic to the APIServer in this case.
This protects the Control Plane nodes to do core controller work.
Remove separate tests.
Regenerated docs.
* Remove namespace from DO ClusterRole
* chore(channels): promote alpha to stable
* chore(channels): bump node images
* chore(channels): recommend kOps 1.35.1 for k8s 1.30-1.35, 1.34.3 for 1.29
* chore(channels): add arm64 GCE and Azure noble node images
* ./hack/update-expected.sh
* scale-test: bind etcd metrics to all interfaces
* build(deps): bump actions/checkout from 6.0.3 to 7.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.3 to 7.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@df4cb1c...9c091bb)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: 7.0.0
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* e2e: test the PR's own channels, not master's
In e2e, `kops create cluster --channel=alpha` reads the channel from the
kops master branch, so a PR's edits to channels/alpha or channels/stable
are never exercised by its own e2e jobs.
When kops is built from the PR checkout, the deployer now rewrites
--channel to a file:// path into that checkout's channels/ directory
(defaulting to alpha when --channel is unset), so the build uses the PR's
channels. Downloaded release/marker binaries don't match the checkout and
keep using master's channels.
* tests/e2e: refine externalTrafficPolicy=Local and hostNetwork skips
The externalTrafficPolicy=Local source-IP-preservation tests only fail on
Cilium (the client IP is SNATed to a pod IP instead of being preserved),
tracked upstream in cilium/cilium#37613. Move the "implement NodePort and
HealthCheckNodePort correctly when ExternalTrafficPolicy changes" skip into
the Cilium block next to its sibling so other CNIs run the test.
The hostNetwork "function for service endpoints" test was fixed in k8s 1.37
by kubernetes/kubernetes#139819 (it now reads spec.nodeName via the Downward
API instead of os.Hostname()), so drop its skip gate from < 1.38 to < 1.37.
Also clean up stale/incorrect issue references in the surrounding comments
(wrong Azure issue, superseded hostname WIP PR, and the unrelated #129221).
* Update CCM pods to tolerate all taints
* ./hack/update-expected.sh
* Allow setting missing slice elements from the command line
Grow slices for explicit indexes while processing --set paths, so paths like cluster.spec.addons[0].manifest can create the first element.
* tests/ai-conformance: install Gateway API CRDs through kOps addons
Configure the scenario to install Gateway API CRDs via cluster.spec.addons, using the Gateway API version documented by Istio 1.29.
* channels: apply direct manifests from spec.addons
* Add managed Karpenter EC2NodeClass and NodePool
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* Switching from comparison on Role to helper.
Preperation for something like kubernetes#18495.
Moving away from direct comparison (== or !=) on IG role.
Using helper methods such as HasNode() or HasControlPlane().
Also added a hack test so we don't backtrack.
Should help prepare for supporting more control plane roles.
* tests/e2e: skip implement-NodePort ETP=Local test on more CNIs
The "Services should implement NodePort and HealthCheckNodePort correctly
when ExternalTrafficPolicy changes" test was previously gated to Cilium
only, but the e2e-kops-aws-cni-* periodic jobs show it also fails on
flannel, kopeio and kube-router: the client source IP is SNATed to a pod IP
instead of being preserved (kube-router instead times out reaching the local
endpoint). It is the sole failure in those three jobs' latest runs.
Move it out of the Cilium block into a condition covering cilium, flannel,
kopeio and kube-router. amazon-vpc, calico and kindnet preserve the source
IP and continue running the test.
The sibling "externalTrafficPolicy=Local for type=NodePort" test passes on
every non-Cilium CNI, so it stays gated to Cilium only.
* chore(channels): pin Azure noble image to a deployable version
Azure retired the pinned Ubuntu 24.04 daily images from the uksouth
marketplace, so VMSS creation fails with PlatformImageNotFound and every
Azure e2e job dies in the Up phase:
The platform image
'Canonical:ubuntu-24_04-lts:server:24.04.202606120' is not available.
`az vm image show` (the query path a deployment uses) confirms
24.04.202606120 (amd64) and 24.04.202606110 (arm64) are no longer
available, while `az vm image list` still lists them from a stale
catalog. 24.04.202606060 is the newest version that `az vm image show`
confirms deployable for both server and server-arm64, so pin both arches
to it.
* tests/e2e: also skip implement-NodePort ETP=Local test on calico+GCE
The "Services should implement NodePort and HealthCheckNodePort correctly
when ExternalTrafficPolicy changes" test fails on calico on GCE but passes
on calico on AWS.
On GCE the VPC drops packets with arbitrary calico pod-CIDR source/dest
addresses, so calico must IPIP-encapsulate inter-node pod traffic (routes go
via tunl0). The IPIP/masquerade path rewrites the ETP=Local NodePort
traffic's source to the node's tunnel address (a pod-CIDR IP) instead of
preserving the client IP. On AWS kops disables the EC2 source/dest check, so
calico routes pod traffic natively over the VPC (dev ens5, no encapsulation)
and the source IP is preserved.
Extend the skip to calico when the cloud provider is GCE. amazon-vpc and
kindnet continue running the test on both clouds.
* build(deps): bump actions/setup-go from 6.4.0 to 6.5.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6.4.0 to 6.5.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](actions/setup-go@4a36011...924ae3a)
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-version: 6.5.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Add an experimental roles feature flag.
No new functionality yet.
Added 4 new role placholders, etcd, scheduler, ccm and kcm.
Sets up the CLI API as well as the accessor functions.
* azure: Scope nodes-to-API NSG rules to the NAT gateway public IP
The AllowNodesToKubernetesAPI and AllowNodesToKopsController rules
allowed any source, which bypassed the spec.api.access allowlist on
port 443 and exposed kops-controller to the internet on clusters with
a public API load balancer. Node traffic to the public frontend
egresses through the NAT gateway, so its public IP is the only source
these rules need.
* fix: use JoinHostPost to tolerate IPv6 addresses
* azure: Bump azuredisk-csi-driver to v1.34.4
The kops Azure e2e jobs (e.g. e2e-kops-azure-gossip-ha) flake on slow
Azure disk attach/detach. The v1.34.x patch stream fixes that path:
- kubernetes-sigs/azuredisk-csi-driver#3481
(v1.34.1): fix: handling dangling detaches in a better way
- kubernetes-sigs/azuredisk-csi-driver#3547
(v1.34.3): fix: incorrect node attached on waitingForDetached call
* ./hack/update-expected.sh
* azure: Move cloud-controller-manager tolerations into the kustomize patch
"Update CCM pods to tolerate all taints" hand-edited the generated
k8s-1.31.yaml.template to set the cloud-controller-manager and
cloud-node-manager tolerations to a single "operator: Exists". That
override lived only in the template, not in any kustomize input, so
regenerate.sh reverted it to the chart's default tolerations and would
silently drop the override on the next regeneration.
Move the override into the kustomization.yaml patches block so
regenerate.sh reproduces it. The rendered template is unchanged.
* azure: Bump azure-cloud-controller-manager to v1.36.2
cloud-provider-azure stopped publishing images to
mcr.microsoft.com/oss/kubernetes after v1.34.3 and now publishes to
mcr.microsoft.com/oss/v2/kubernetes, which the upstream Helm chart
selects for Kubernetes 1.32 and newer. Switch the cloud-controller-manager
and cloud-node-manager image defaults to the oss/v2 path so kOps can
track current releases, and bump both from v1.34.3 to v1.36.2.
Also bump the pinned Helm chart from 1.34.5 to 1.36.0 to keep the chart
minor aligned with the image minor. The rendered manifest is unchanged:
the chart's Deployment and DaemonSet templates are identical between the
two versions, and kOps patches the image fields out with the
{{ .ExternalCloudControllerManager.* }} placeholders.
Notable upstream changes since v1.34.3:
- New service.beta.kubernetes.io/azure-disable-load-balancer-nsg-rule
annotation to skip CCM-managed LoadBalancer NSG rules.
- External LoadBalancer Services with an invalid or non-public pinned IP
now fail fast instead of listing Public IPs.
- Multiple standard load balancer fixes: backend pool IP deduplication,
serialized backend pool updates, and IP sharing across services.
- Network-isolated clusters always use the managed identity credential.
- ACR credential provider adds KSA-based authentication.
Release notes:
https://github.com/kubernetes-sigs/cloud-provider-azure/releases
* ./hack/update-expected.sh
* chore: update syntax of CLAUDE.md
The @ syntax is more readily followed by claude.
* azure: Grant control-plane VMSS Contributor instead of Owner
The built-in Owner role includes Microsoft.Authorization/roleAssignments/write,
a privilege-escalation path reachable by any pod via the instance metadata
endpoint. No cluster component uses Microsoft.Authorization; Contributor
covers everything the control plane needs.
Existing clusters keep the old Owner role assignment until it is removed
manually or the cluster is deleted; the terraform target prunes it on apply.
* coredns: Honor node taints in hostname topologySpreadConstraint
The hostname topologySpreadConstraint uses whenUnsatisfiable: DoNotSchedule, but CoreDNS autoscales via the cluster-proportional-autoscaler. On clusters with a single schedulable worker (e.g. one control-plane + one worker), the tainted control-plane node was counted as an empty topology domain, so the second CoreDNS replica could never satisfy maxSkew and stayed Pending.
Setting nodeTaintsPolicy: Honor excludes nodes with untolerated taints (control-plane, cordoned/draining) from the skew calculation, keeping the hard spread across workers while allowing the replicas to co-locate when there is only one eligible node.
* ./hack/update-expected.sh
* Expose GCP project as PROJECT env for ClusterLoader2 on GCE
ClusterLoader2's managed Prometheus client
(clusterloader2/pkg/prometheus/clients/gcp_managed.go) reads the GCP
project from the PROJECT env var to build the Cloud Monitoring query
URL. The kubetest2-kops deployer only exported GCP_PROJECT, so for
scalability jobs that enable the "Quotas total usage" measurement
(via preset-e2e-scalability-common's CL2_ENABLE_QUOTAS_USAGE_MEASUREMENT)
the query URL had an empty project and Cloud Monitoring rejected it
with HTTP 400 INVALID_ARGUMENT, failing the load test.
Also export PROJECT alongside GCP_PROJECT so the measurement can query
the correct project. This is set from d.GCPProject, which is populated
even when the project is acquired from boskos.
* hetzner: fix Cluster Autoscaler node group membership
hetzner: improve Cluster Autoscaler integration
The Cluster Autoscaler Hetzner provider keys node group membership,
initial target size and Nodes() entirely off the hcloud/node-group
server label, which kOps never set. As a result kOps-created servers
were invisible to the autoscaler.
- Label Node-role servers with hcloud/node-group at creation. Servers
from earlier kOps versions get the label when they are replaced.
- Treat minSize as a floor: servers beyond Count are no longer marked
needs-update, so autoscaler scale-ups survive kops update cluster.
Shrink with 'kops delete instance' or autoscaler scale-down.
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* tests/e2e: skip implement-NodePort ETP=Local test on calico+Azure
The "Services should implement NodePort and HealthCheckNodePort correctly
when ExternalTrafficPolicy changes" test fails on calico on Azure for the
same reason it does on GCE: the underlay cannot route the pod CIDR, so calico
must encapsulate inter-node pod traffic (VXLAN vxlan.calico on Azure, IPIP
tunl0 on GCE). On the node-local ExternalTrafficPolicy=Local NodePort
short-circuit path the masquerade then rewrites the client source IP to the
node's tunnel address (a pod-CIDR IP) instead of preserving it, so the test
fails on every run once the cluster comes up.
Calico preserves the source IP only on AWS, where kOps disables the
source/dest check and routes pod traffic natively. Extend the calico skip
from gce to also cover azure.
* Update cluster-autoscaler to v1.36.0
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
* tests: enable cluster-autoscaler in the ha_gce integration test
The nodes instance group in this scenario spans two zones, so kOps creates
two zonal InstanceGroupManagers, but only the first zone's MIG is registered
with cluster-autoscaler. The expected output captures this current behavior.
* gce: register all zonal MIGs of a multi-zone instance group with cluster-autoscaler
For a GCE instance group spanning multiple zones, kOps creates one zonal
InstanceGroupManager per zone, but only registered the first zone's MIG
with cluster-autoscaler, carrying the full min/max of the instance group.
The MIGs in the remaining zones were invisible to the autoscaler and were
never scaled, so scale-ups could get stuck in a single zone (for example
when that zone has no spot capacity).
Register each zonal MIG as its own cluster-autoscaler node group, splitting
the instance group's min/max sizes across zones with the same algorithm
used for the MIG target sizes. Single-zone instance groups render exactly
as before.
* tests: update expected output for ha_gce
* Update Go to 1.26.5 and bump golang.org/x modules
* clusterapi: fix build by making gen.go a library package
* pkg/assets: keep container registry clients out of runtime binaries
* rebase with upstream lug 2026
* update: vendor
* feat: logic up to the region choose func
* feat: added various integrations
* feat: daemons api endpoints and other improvements
* various errors resolution
* feat: working prototype up to store phase
* working create config generation
* rebase kops lug 2026
* feat: servergroup initial implementation
* fix: compatibility issues
* fix: copute instance creation
* feat: volume support
* fix: compatibility with elemento lib
* fix: bugs
* fix: remove verify token between nodes for development purpose
* fix: minor
* fix: nodeup auto script
* chore: changed module repo from tesi-paolobeci to ecloud-go
* fix: version elemento
* fix: minor
* feat: custom ssh key support
* feat: limited task execution retries on error
* fix: remote ecloud package on go.mod
* fix: release secret
* fix: deactivate useless gh actions
* fix: ecloud version
* fix: minor
---------
Signed-off-by: Ciprian Hacman <ciprian@hakman.dev>
Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
Signed-off-by: Moshe Vayner <moshe@vayner.me>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jefftree <jeffrey.ying86@live.com>
Signed-off-by: Jathavedhan M <jathavedhan.m@ibm.com>
Co-authored-by: Ciprian Hacman <ciprian@hakman.dev>
Co-authored-by: Kubernetes Prow Robot <20407524+k8s-ci-robot@users.noreply.github.com>
Co-authored-by: Arnaud Meukam <ameukam@gmail.com>
Co-authored-by: Marek Siarkowicz <serathius@users.noreply.github.com>
Co-authored-by: Moshe Vayner <moshe@vayner.me>
Co-authored-by: justinsb <justinsb@google.com>
Co-authored-by: Jefftree <jeffrey.ying86@live.com>
Co-authored-by: Harish K <hakuna@amazon.com>
Co-authored-by: Peter Rifel <pgrifel@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jathavedhan M <jathavedhan.m@ibm.com>
Co-authored-by: Walter Fender <wfender@google.com>
Co-authored-by: kubernetes-prow[bot] <181008794+kubernetes-prow[bot]@users.noreply.github.com>1 parent 4660a60 commit 25fa595
4,089 files changed
Lines changed: 301377 additions & 255968 deletions
File tree
- .github/workflows
- addons
- ambassador
- cluster-autoscaler
- ingress-citrix
- ingress-nginx
- kube-ingress-aws-controller
- kube-state-metrics
- kubernetes-dashboard
- logging-elasticsearch
- metrics-server
- prometheus-operator
- channels
- cmd/channels
- pkg
- channels
- cmd
- nodelabeler
- cloudmock
- aws
- mockec2
- mockelbv2
- gce
- mockcompute
- openstack
- mockcompute
- mockdns
- mocknetworking
- scaleway/mockdns
- clusterapi
- bootstrap/controllers
- config/crds
- controlplane/kops/api/v1beta1
- examples
- manifests
- build
- cluster-api-provider-gcp
- cluster-api
- patches
- pkg/builders
- cmd
- kops-controller
- controllers
- pkg
- config
- server
- kops
- test
- util
- kube-apiserver-healthcheck
- nodeup
- dev/tasks
- discovery
- apis/discovery.kops.k8s.io/v1alpha1
- cmd/discovery-server
- dev/tasks
- docs
- k8s
- pkg/discovery
- scripts
- dns-controller
- cmd/dns-controller
- pkg/watchers
- dnsprovider/pkg/dnsprovider
- providers
- aws/route53/stubs
- google/clouddns
- internal
- openstack/designate
- docs
- advanced
- cli
- contributing
- examples
- addons
- getting_started
- networking
- operations
- releases
- tutorial
- hack
- boilerplate
- images/mkdocs
- k8s
- crds
- nodeup/pkg
- bootstrap
- model
- networking
- resources
- tests
- channels
- containerdbuilder
- complex
- flatcar
- simple
- v3
- dockerbuilder
- docker_19.03.11
- healthcheck
- logflags
- skipinstall
- golden
- audit
- awsiam
- dedicated-apiserver
- envvars
- hooks-containerd-exec
- minimal
- oidc
- side-loading
- without-etcd-events
- kubelet
- featuregates
- warmpool
- protokube
- updateservicebuilder
- automatic
- external
- pkg
- apis
- kops
- install
- model
- util
- v1alpha2
- v1alpha3
- validation
- nodeup
- applylib
- applyset
- testdata/managedfields
- csa-migration
- merge-owners
- mocks/mockkubeapiserver
- assets
- assetcopy
- assetdata
- bootstrap
- pkibootstrap
- client
- clientset_generated
- clientset
- fake
- typed/kops
- internalversion
- fake
- v1alpha2
- fake
- v1alpha3
- fake
- internalclientset
- fake
- typed/kops
- internalversion
- fake
- v1alpha2
- fake
- v1alpha3
- fake
- simple
- api
- vfsclientset
- clouds
- commands
- commandutils
- toolbox
- clusterapi
- controllers/clusterapi
- dump
- featureflag
- flagbuilder
- instancegroups
- jsonutils
- k8sversion
- kopscontrollerclient
- kubeconfig
- kubemanifest
- model
- awsmodel
- azuremodel
- components
- addonmanifests
- awscloudcontrollermanager
- awsebscsidriver
- awsloadbalancercontroller
- certmanager
- clusterautoscaler
- dnscontroller
- karpenter
- kuberouter
- nodeterminationhandler
- channels
- tests
- container_registry
- minimal
- etcdmanager
- tests
- interval
- minimal
- overwrite_settings
- proxy
- kubeapiserver
- tests/minimal
- kubescheduler
- tests
- kubeschedulerconfig
- minimal
- mixing
- domodel
- gcemodel
- hetznermodel
- iam
- tests
- linodemodel
- openstackmodel
- tests/servergroup
- resources
- tests/data
- nodeidentity
- aws
- azure
- clusterapi
- do
- gce
- hetzner
- linode
- openstack
- scaleway
- nodelabels
- nodemodel
- wellknownassets
- pki
- resources
- aws
- azure
- digitalocean
- gce
- hetzner
- linode
- openstack
- ops
- scaleway
- spotinst
- templates
- testutils
- golden
- urls
- util/templater
- validation
- values
- wellknownports
- wellknownservices
- wellknownusers
- protokube
- cmd/protokube
- pkg
- etcd
- gossip
- aws
- azure
- dns/hosts
- do
- hetzner
- memberlist
- mesh
- openstack
- scaleway
- protokube
- tests
- codecs
- e2e
- csi-manifests
- azure-disk
- dobs
- gcp-pd
- kubetest2-kops
- aws
- azure
- builder
- deployer
- gce
- pkg
- kops
- tester
- scenarios
- addon-resource-tracking
- ai-conformance
- testartifacts
- tools
- check-aws-availability
- write-conformance-report
- validators
- accelerators
- dra_support
- testdata
- driver_runtime_management
- testdata
- conformance
- kubeobjects
- networking/ai_inference
- testdata
- observability/accelerator_metrics
- testdata
- operator/robust_controller
- testdata
- scheduling-orchestration
- clusterautoscaling
- testdata
- gangscheduling
- testdata
- podautoscaling
- testdata
- security/secure_accelerator_access
- testdata
- aws-ebs-csi
- aws-lb-controller
- bare-metal
- cilium-connectivity-test
- clusterapi
- discovery-service
- karpenter
- keypair-rotation
- kops-upgrade
- lib
- metrics-server
- podidentitywebhook
- scalability
- smoketest
- splitkcp
- upgrade-ab-gossip
- upgrade-ab
- templates
- integration
- channel
- simple
- conversion
- aws
- azure
- canal
- cilium
- do
- gce
- minimal
- openstack
- create_cluster
- apiserver-gce
- calico_bpf
- cilium-eni
- complex-private
- complex
- different-amis
- gce_byo_sa
- gossip-aws
- gossip-azure
- gossip-gce
- gossip-hetzner
- ha_encrypt
- ha_gce
- ha_hetzner
- ha_openstack_nodns
- ha_openstack_octavia
- ha_openstack
- ha
- ingwspecified
- ipv6
- karpenter
- minimal-1.27
- minimal-1.28
- minimal-1.29
- minimal-1.30
- minimal-1.31
- minimal-1.32
- minimal-1.35
- minimal-1.36
- minimal-arm64
- minimal-gce-dns-none
- minimal-gce
- minimal-irsa
- minimal_feature-gates
- minimal_hetzner
- ngwspecified
- overrides
- private_gce
- private
- volume_type
- update_cluster
- additionalobjects
- data
- apiservernodes
- data
- aws-lb-controller
- data
- bastionadditional_user-data
- data
- cluster-autoscaler-priority-expander-custom
- data
- cluster-autoscaler-priority-expander
- data
- complex
- data
- compress
- data
- containerd-custom
- data
- containerd
- data
- digit
- data
- existing_iam
- data
- existing_sg
- data
- external_dns_irsa
- data
- external_dns
- data
- externallb
- data
- externalpolicies
- data
- gossip-aws
- data
- gossip-azure
- data
- gossip-gce
- data
- gossip-hetzner
- data
- ha_gce
- data
- ha
- data
- irsa
- data
- karpenter
- data
- lifecycle_phases
- many-addons-ccm-irsa
- data
- many-addons-ccm
- data
- many-addons-gce
- data
- many-addons
- data
- minimal-1.27
- data
- minimal-1.28
- data
- minimal-1.29
- data
- minimal-1.30
- data
- minimal-1.31
- data
- minimal-1.32
- data
- minimal-1.33
- data
- minimal-1.34
- data
- minimal-1.35
- data
- minimal-1.36
- data
- minimal-aws
- data
- minimal-dns-none
- data
- minimal-etcd
- data
- minimal-gp3
- data
- minimal-ipv6-calico
- data
- minimal-ipv6-cilium
- data
- minimal-ipv6-kindnet
- data
- minimal-ipv6-no-subnet-prefix
- data
- minimal-ipv6
- data
- minimal-longclustername
- data
- minimal-warmpool
- data
- minimal_azure
- data
- minimal_gce_dns-none
- data
- minimal_gce_ilb_cilium_etcd
- data
- minimal_gce_ilb_longclustername
- data
- minimal_gce_ilb
- data
- minimal_gce_longclustername
- data
- minimal_gce_plb_apiserver
- data
- minimal_gce_plb
- data
- minimal_gce_private
- data
- minimal_gce
- data
- minimal_gossip_irsa
- data
- minimal_gossip
- data
- minimal_hetzner
- data
- minimal_scaleway
- data
- mixed_instances_spot
- data
- mixed_instances
- data
- nth-imds-processor-irsa
- data
- nth-imds-processor
- data
- nvidia/data
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | | - | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
29 | 35 | | |
30 | 36 | | |
31 | 37 | | |
32 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
33 | 45 | | |
34 | 46 | | |
| 47 | + | |
35 | 48 | | |
36 | 49 | | |
37 | | - | |
| 50 | + | |
38 | 51 | | |
39 | 52 | | |
40 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
| 68 | + | |
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
| |||
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
90 | | - | |
| 90 | + | |
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
| 95 | + | |
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
96 | 101 | | |
97 | 102 | | |
98 | | - | |
99 | | - | |
| 103 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
1 | 2 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 3 | + | |
16 | 4 | | |
17 | | - | |
| 5 | + | |
18 | 6 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
This file was deleted.
0 commit comments