Shipping a Kubernetes App to the Azure Marketplace
Kubernetes application offers are one of the smallest categories in the Azure Marketplace, and the certification bar is the reason. This is a technical account of the approach that clears it: engineering container images to Microsoft Defender's standard, packaging with CNAB, designing CI for non-root build workers, and validating the purchase flow the way a customer will exercise it.
Talk to an EngineerWhy So Few Containers Are in the Marketplace
The Azure Marketplace lists tens of thousands of offers, but filter it down to Kubernetes application offers - software that installs into the customer's own AKS cluster from the portal - and the catalog thins to a small fraction of that. The scarcity is not lack of demand. It reflects what this offer type requires of the publisher.
A Kubernetes application offer is not a listing with a download link. It is a CNAB bundle (Cloud Native Application Bundle): a self-contained package holding the application's container images, its Helm chart, an ARM template, the portal UI definition, and the installer logic that Azure executes inside the buyer's subscription when they click Create. Microsoft certifies the entire unit: every image is scanned against the Defender for Cloud vulnerability database, the bundle is deployed into a real cluster to validate installation, and submissions can be routed to manual review by a human at Microsoft.
The result is a distribution channel where the platform vendor has verified your supply chain, your installer, and your purchase flow before a customer ever sees the listing. That is why enterprise procurement favors it - and why the catalog is thin. We publish and operate a Kubernetes application offer (APIM Backup, automated backup and disaster recovery for Azure API Management), and this page documents the approach in enough detail to reproduce it.
The unit of publication and the gauntlet it clears: one CNAB bundle, four gates, and a listing Microsoft has verified end to end. The sections below cover each stage in the order the bundle meets it.
The Certification Pipeline as an Engineering Target
Certification is deterministic enough to engineer against once its stages and their properties are mapped:
| Stage | What is checked | Observed timing |
|---|---|---|
| Automated validation | Bundle structure, manifest integrity, registry reachability | Minutes |
| Container scan (policy 400.3.2) | Every image layer against the Defender for Cloud vulnerability database | Included in certification, verdict in about 1.5 hours |
| Deployment validation | The bundle is installed into a real AKS cluster, including repeat-install scenarios | Same window |
| Manual validation | Human review of the offer and purchase flow; triggered by repeated submissions or flags | Around 2 business days, business hours only |
Two properties of the pipeline dominate release planning. First, the scan of record is Microsoft's, not yours: certification is scored against Defender's findings on the exact image digests submitted, so internal scan results are a prediction, not a guarantee. Second, a new submission cancels one already under review and restarts its clock. The correct response to an in-flight review is therefore to stage the next release as a ready fallback - built, published, verified - and submit it only after a verdict. Resubmitting early converts a two-day wait into a four-day one.
The pipeline, as Partner Center reports it. Automated offer validation completes in minutes; certification transfers the technical assets and, on this submission, routes to manual validation with a two-business-day window. Preview creation, publisher signoff, and the final publish stages follow only after a passing verdict - the timing properties that shape the release engineering described below.
Image Supply Chain: Engineering to Defender Zero
Policy 400.3.2 - container image vulnerabilities - is where most of the certification effort concentrates, and it resolves into two distinct problems.
Problem one is bulk, and the method is subtraction. A general-purpose base image carries software the application never executes: cloud SDKs, package managers, build toolchains, development headers. Each is surface area for findings. Our runtime image entered this process with 364 Critical and High findings by package-database scan; removing the unused SDK, stripping npm and yarn from the final stage, purging development packages, and patching the OS brought the package-database count to zero. None of this is exotic - it is the standard minimal-image discipline, applied until the scanner agrees.
Problem two is scanner parity, and it is the one that fails otherwise-clean submissions. Package-database scanners enumerate what the package manager installed. Defender additionally matches files that no package database owns. In our data, a package-clean image still produced seven Defender findings in two categories:
| Category | Mechanism | Remediation |
|---|---|---|
| Side-effect installations | Libraries pulled in as dependencies of an OS package (here: Python cryptography libraries installed with apt certbot), invisible to the distro database once flagged upstream | Replace the OS-packaged tool with a language-package-managed equivalent (pip-managed certbot), then remove the distro packages entirely |
| Vendored copies | Duplicates of a library bundled inside another tool's directory tree, never registered with any package manager | Locate every copy by file scan and overlay the patched version in place |
The operational rule that follows: treat Defender's named findings as the specification. Remediate each named package individually, then verify the fix against the exact image digest to be submitted - by file inspection where no package database applies. An internal zero is a necessary condition, never a sufficient one.
Packaging: Anatomy of the Bundle
The deployable unit is assembled by Porter, the CNAB reference toolchain, from a manifest that embeds every artifact the Azure portal needs:
| Component | Role |
|---|---|
| Porter manifest (porter.yaml) | Declares the bundle: name, version, images, install and uninstall actions |
| Helm chart (base64-embedded) | The application's Kubernetes resources, packaged and inlined into the manifest |
| ARM template | Provisions the Azure-side resources, including cluster creation when the buyer requests one |
| UI definition (createUIDefinition.json) | Drives the portal's purchase wizard: cluster selector, sizing, parameters |
| Extension registration parameters | Registers the application as a cluster extension in the buyer's subscription |
Three practices keep this layer stable:
- Version lockstep. The bundle version, the application image tag, and the release tag are the same string, derived from one source of truth, so a certification report always maps to exactly one build.
- Deterministic assembly. The bundle is rendered from templates at release time - chart version, image references, and registry paths injected by the pipeline - so no hand-edited manifest can drift from what CI built.
- Toolchain pinning with checksums. Porter, Helm, and the registry client are installed at pinned versions with verified checksums, because the packaging container is itself an image that must pass the same vulnerability bar as the product.
CI Design for Non-Root Build Workers
Certification hardening pushes build containers to run as an unprivileged user, and this is where pipelines that predate the requirement fail - because a root-running build container acquires permissions implicitly, and dropping root converts each implicit permission into an explicit denial. The failures surface sequentially, each masked by the one before it. Empirically, exactly three permission surfaces account for the class:
| Surface | Failure mode | Design rule |
|---|---|---|
| Mounted workspace | The CI runner owns the checkout directory; the container user cannot write build artifacts into the mount | The staging step that prepares the directory grants group and world write (a+rwX) as part of staging, not as a fix afterward |
| Image-internal tooling | Tools installed by root during image build, after the home directory is chowned, are unreadable by the runtime user | Dockerfile ordering: install everything first, chown the tool tree immediately before the USER directive - and test the image as its runtime UID, not as root |
| Host Docker socket | The socket is owned by root and the host's docker group; the container user belongs to neither | Grant socket access on the ephemeral runner, scoped to the job; container-side group mapping is not portable across hosts |
Two verification rules close the loop. Reproduce and prove locally as the unprivileged UID: every fix above is testable in seconds with a local container run as the runtime user, executing the exact operation that failed - before any release is cut. Fix at the source: workspace permissions belong in the pipeline, tool ownership belongs in the worker image (released upstream, then consumed as a version bump), socket access belongs in the job definition. A chmod in the wrong layer works once and regresses on the next image update.
In the worker image, anything installed while the build runs as root has to be handed to the runtime user explicitly, and the order matters: install first, chown the tool tree, and only then drop privileges. If the chown happens before the install (a common arrangement, since base images chown the home directory early), the tools land root-owned and the runtime user cannot execute them.
RUN porter mixin install exec
RUN chown -R "${USER}:${USER}" "${HOME}/.porter"
USER ${USER}In the CI job, the container needs two things the host owns: write access to the directory it mounts as a workspace (owned by the runner's user, not the container's), and access to the Docker socket (owned by root and the host's docker group). The write grant belongs to the script that stages the directory, so it can never be forgotten separately; the socket grant sits in the job definition, scoped to the ephemeral runner VM.
- name: Stage CNAB runtime
run: ./ci/scripts/stage-cnab-runtime.sh
# the staging script grants the container write access:
# chmod -R a+rwX "$RUNTIME_DIR"
- name: Allow worker access to Docker socket
run: sudo chmod 666 /var/run/docker.sockEach fix lives in the layer that owns the problem: tool ownership in the image, workspace and socket access in the job. A chmod in the wrong layer works once and regresses on the next image update.
Deployment Validation and Purchase Rehearsal
Certification deploys the bundle into a real cluster, and it does so under conditions first-party testing tends to skip: repeat installations, and subscriptions where resources of the same name already exist. Our canonical example is the auto-generated node resource group: deploying the offer twice into one subscription collided on its name (NodeResourceGroupAlreadyExists) - a defect invisible to every fresh-account demo and guaranteed to affect exactly the customers who buy again. The general requirement is idempotency of the install path: generated resource names must be unique per deployment, and every install action must either be repeatable or fail with an actionable message.
The purchase flow itself - portal wizard, cluster selector, parameter validation, deployment into a resource group - is part of the certified surface and needs rehearsal at release cadence, not launch cadence. To make that economical we operate a dedicated test identity whose multi-factor authentication is satisfiable programmatically (a TOTP seed held in an automation vault, codes computed at login), so an automated agent can execute the full portal purchase end to end without a human in the loop. The rehearsal validates the same path a customer exercises, against the preview version of the offer, before publisher signoff releases it to the public listing.
Release Operations Under Certification
The certification pipeline's timing properties (verdicts in hours to days, resubmission resets the clock) impose a specific release-engineering shape:
Human gates, automated intervals. Two decisions stay with named people: the production release gate and the final publisher signoff. Everything between - merge, build, publish, failure diagnosis, fix verification, rerun - runs without waiting on a person. In this program, a three-surface CI failure chain was root-caused, fixed across two repositories including a released upstream image, and verified, entirely between two human gate approvals.
Operational state is written down, not remembered. Submission identifiers, in-flight review status, staged fallback versions, and standing rules (never resubmit during manual validation) live in persistent operator notes that survive session and shift boundaries. Any operator - human or automated - resumes from the record, not from recollection.
Artifacts are verified in the registry, not inferred from CI. A release is complete when the bundle tag is present and pullable in the production registry and the published image passes the same checks locally that certification will apply remotely. Green pipelines are evidence; the artifact is proof.
The summary metrics for the current certification round: package-database findings reduced 364 to 0; 7 Defender-only findings remediated individually and verified per-digest; 3 CI permission surfaces closed at their sources; releases staged as fallback rather than resubmitted against an in-flight review.
Array
The parts of our practice this story draws on.
The GitOps automation platform that runs these release pipelines.
Preview Environments for Dynamic Sites
The companion discipline: rehearse the risky moment before customers see it.
CI/CD pipelines with security hardening designed in, not bolted on.
The broader architecture practice marketplace products are built on.
Our guide to getting applications into containers in the first place.
Direct access to the engineers who run your stack.
"
Talk to an Expert
Connect with our diverse group of UDX experts that can help you implement successful DevOps practices within your organization.