Context
When starting with containers and designing a container runtime/orchestration platform one should also design a container registry deployment model. Which model you choose will have consequences on cost, security, and maintenance. Especially when the number of containerized applications starts to run through the ceiling.
This blog tries to explain the different deployment models of Azure Container Registry (ACR) with their advantages and disadvantages. Although we use Azure ACR, this article can also be applied to other container registries.
Container Registry Deployment Models
One container registry for build and one dedicated registry per environment
In our first model container images are created during build phase and pushed to a general container registry. From there images are first pushed to the test container registry before they are deployed to the test Kubernetes environment. Once approval is given for the acceptance environment, the deployment pipeline takes the images from the test container registry and pushes the images to the acceptance container registry before the image is deployed to that acceptance Kubernetes environment. The same for production, after the approval for production is given, the deployment pipeline takes the image from the acceptance registry and pushes the images to the production registry, after which the image is deployed in production.
This model has the following characteristics:
- Images for each environment are stored in the registry of the corresponding environment. This increases visibility on which image version is installed in which environment.
- Each environment specific container registry has its own governance policies, retention policies, security, and authorization rules, fine-tuned according to the environment.
- Each Kubernetes environment uses its own dedicated container registry.
- Older images can be safely removed from the image registry. The last image version in a registry is the image which is deployed in the corresponding environment. Typically, we use retention policies to only keep the last 5 image versions.
- Not the cheapest deployment model due to increased costs of the registries and related maintenance tasks.
- By using different registries in different environments, drift can occur where the image deployed in production differs from the image deployed in the test environment.
One container registry for build and non-production environments and one registry for production
In this deployment model we use two container registries. One for all non-production environments and one for the production environment.
This model has the following characteristics:
- We can enforce more strict security and authorization rules on the production registry
- We limit the number of container registries to two decreasing costs and maintenance.
- Production images are managed separately and thus reducing the risk of losing production images.
One general purpose container registry
In this setup we use one container registry for all the different environments. After the image is built in the CD pipeline the image is pushed to ACR and from there the same image is deployed into all the different environments.
- Ensures the ‘one image for all environments’ principle
- Simplified management
- Requires extensive image tagging and locking strategy to prevent that container images are deleted by retention scripts!
- Minimizes costs since only one registry is used.
- Setting authorization rules can be challenging.
Conclusion
Which container registry deployment model suits best for your enterprise depends on, most importantly, your requirements together with other factors, like size of the organization, organizational structure, security constraints, availability and off course your application code deployment model.
In my experience, if possible, try to foresee at least two container registries in which you have a separate container registry for production and the non-production environment.
Some other factors worth considering which were not discussed are organizational structure, proximity and network latency! Try to place your container registry geographically close to your container runtime environments. If your containers are scattered all over the world, consider adding more container registries in those regions. Some container registries, like Azure Container Registry for example, supports the concept of geo-replication which simplifies registry management while minimizing latency.
