Hierarchical Namespace Controller in Kubernetes
Kubernetes namespaces are a flat list, but real organizations are trees of companies, departments, teams and projects. The Hierarchical Namespace Controller teaches the cluster about parents and children, lets rules flow down the tree, and lets teams create their own namespaces safely. This guide explains how it works inside, and how to use it well.
Think of a large company’s filing room. At first there is a single long shelf with one folder per team: Payments, Search, Mobile. As the company grows, the shelf becomes crowded. Each department wants its own folders for development, testing and production. The company wants one rule, such as “all folders must be locked at night”, to apply to everything under a department without repeating it a hundred times. A flat shelf cannot express this. A filing cabinet with drawers inside drawers can. The Hierarchical Namespace Controller adds those inner drawers to Kubernetes.
1What the Hierarchical Namespace Controller Is
A short definition first, so that every later chapter has something solid to stand on.
The Hierarchical Namespace Controller, usually shortened to HNC, is an add-on for Kubernetes that lets namespaces form a tree. One namespace can be the parent of others, and the others are its children. A child can have children of its own. Once the relationships exist, HNC does two main things for you. It copies chosen objects, such as permissions and policies, from a parent down into all its descendants. And it lets people who control a parent namespace create new child namespaces without asking a cluster administrator.
HNC does not change Kubernetes itself. A namespace is still just a namespace as far as the core system is concerned, and the cluster still keeps them in a flat list. HNC adds a layer on top. It stores the parent and child links in its own objects, watches for changes, and keeps the right copies in the right places.
Imagine a school where every classroom has its own set of rules on the wall. A headteacher can walk into each room and write the same rule again and again, or the school can have a system where a rule written in the “Year 5” office automatically appears on the wall of every Year 5 classroom. HNC is that system for namespaces.
Namespace
A named box inside a cluster that holds objects such as Pods and Services, and that can have its own permissions and limits.
Parent and child
A parent namespace sits one level above a child. Each namespace can have at most one parent, and any number of children.
Ancestors and descendants
Ancestors are all the namespaces above one, up to the top. Descendants are all the namespaces below it, at any depth.
Propagation
The automatic copying of selected objects from a namespace down to its descendants.
Subnamespace
A child namespace that is created by asking inside the parent, with no cluster-level rights needed.
HNC is a separate project from the core of Kubernetes. It lives in the Kubernetes community under the multi-tenancy effort. Before adopting it, check the current state of the project, the supported Kubernetes versions and how actively it is maintained.
2A Quick Refresher on Namespaces
To see what is missing, we first need to be clear about what namespaces give us today.
A namespace is a way to divide one cluster into several virtual sections. Most objects, such as Pods, Services, Deployments and ConfigMaps, belong to exactly one namespace. Names only need to be unique inside a namespace, so two teams can each have a Deployment called web without a clash. A few objects, such as Nodes and PersistentVolumes, are cluster-wide and do not live in any namespace.
Namespaces are also the main unit for control. Permissions are usually granted per namespace, using Roles and RoleBindings. Resource limits are applied per namespace, using ResourceQuotas. Network rules are written per namespace, using NetworkPolicies. A policy engine or an admission rule often looks at the namespace to decide what is allowed.
| What namespaces give you | Example |
|---|---|
| Name scoping | Two teams can both own a Service named api |
| Access control boundary | A team’s engineers can edit their namespace and nothing else |
| Resource limits | A quota caps how much CPU a team may use |
| Network boundary | Policies can allow or block traffic between namespaces |
| Policy target | Admission rules can be applied to selected namespaces |
What namespaces do not give you
Namespaces have no relationship to each other. There is no such thing as a namespace inside a namespace. Each is an equal in a flat list. Rules written in one namespace do not affect any other. To apply the same rule to twenty namespaces, someone has to create twenty copies and keep them all up to date by hand or with automation.
A flat list of namespaces is like a street of identical houses. Each has its own front door and its own rules, but there is no concept of a neighborhood, a district or a city. If the city wants every house to have a smoke alarm, someone must visit each house one by one.
Many people assume that putting dashes in names, like team-a-dev and team-a-test, creates a hierarchy. It does not. The dashes are only part of the name. Nothing in Kubernetes treats these two as related.
3The Pain That Hierarchy Removes
Hierarchy is not a luxury. It solves several very concrete problems that grow with the size of a cluster.
Repeated setup for every namespace
Every new namespace needs the same baseline: access rules for the team, a default network policy, a pull secret for the image registry, and often a quota. Doing this by hand is slow and error-prone, and automation scripts drift out of date.
A bottleneck at the platform team
In most clusters, only administrators may create namespaces, because a namespace is a cluster-level object. Every team that wants a new environment must file a ticket and wait. Platform teams become a queue.
Giving too much power to delegate creation
The alternative, giving team leads the right to create any namespace, is risky. They could create namespaces with names that collide with others, or take over names meant for other teams, and nobody controls where those new namespaces fit.
No way to say “everything under this team”
Policies often need to apply to a whole group of namespaces: all of Team A’s environments, or all production namespaces of a department. A flat list has no way to describe such groups except by relying on labels that people must remember to add.
Inconsistent permissions
When roles are copied by hand into many namespaces, copies fall out of step. One namespace still grants access to someone who left the team, because nobody remembered to update it.
Imagine a company where only the head of facilities may ever open a new office room. Every team that needs a meeting room must queue to ask. If the head of facilities gives out master keys to everyone to solve the queue, rooms are opened without any plan. A better arrangement lets each department head open rooms inside the department’s own floor, and nowhere else. That is what subnamespaces provide.
4The Core Idea: Trees of Namespaces
Everything HNC does grows from a very small idea about how namespaces are connected.
HNC arranges namespaces in a forest, which is simply a collection of trees. Each tree has one namespace at the top, called the root. Every other namespace in that tree has exactly one parent. A namespace with no parent is a root. A namespace with no children is a leaf. Cycles are not allowed, so a namespace can never be its own ancestor.
flowchart TD
ORG["org: root"] --> TA["team-a"]
ORG --> TB["team-b"]
TA --> TAD["team-a-dev"]
TA --> TAS["team-a-staging"]
TA --> TAP["team-a-prod"]
TB --> TBD["team-b-dev"]
TB --> TBP["team-b-prod"]
Two simple rules about direction
Things flow in one direction only. Objects and rules move down the tree, from a parent to its children, grandchildren and so on. Nothing flows upward. A rule in a child never affects its parent, and a rule in one child never affects its sibling. This one-way flow keeps behavior easy to predict: to know everything that applies to a namespace, you look at it and at its ancestors, and nowhere else.
| Word | Meaning | Example from the tree |
|---|---|---|
| Root | Namespace with no parent | org |
| Parent | The namespace directly above | team-a is the parent of team-a-dev |
| Child | A namespace directly below | team-a-prod is a child of team-a |
| Ancestor | Any namespace above, at any distance | org and team-a are ancestors of team-a-dev |
| Descendant | Any namespace below, at any distance | All of the namespaces under org |
| Sibling | Another child of the same parent | team-a-dev and team-a-prod |
| Leaf | A namespace with no children | team-b-dev |
A family tree has the same shape. Grandparents pass things down to parents and children, but a child cannot pass anything up, and cousins do not share with each other directly. Each person has exactly one set of parents in the tree.
5Architecture: Controller, Webhooks and the Forest
Now we open the box and look at the parts that make the tree real.
HNC is installed as a program that runs inside the cluster, in its own namespace, usually called hnc-system. It is built like most Kubernetes add-ons. It has custom resource definitions that add new object types. It has a set of controllers that watch objects and make changes. And it has admission webhooks that check changes before they are saved.
Custom resource definitions
Add new types to the API: the hierarchy configuration, the subnamespace anchor and the HNC configuration.
Controllers
Watch namespaces and objects, and create, update or delete copies so that the cluster matches the tree.
Admission webhooks
Reject changes that would break the rules, such as a cycle in the tree or an edit to an inherited object.
The in-memory forest
A data structure inside the controller that holds the whole tree of namespaces, so that decisions are made quickly without asking the API server each time.
flowchart LR
U["User or automation"] --> API["API Server"]
API -->|"Admission check"| W["HNC webhooks"]
W --> API
API -->|"Watch events"| C["HNC controllers"]
C F["In-memory forest"]
C -->|"Create and update copies"| API
The forest in memory
When HNC starts, it reads all namespaces and hierarchy objects from the API server and builds the forest in memory. After that, it keeps the forest up to date by listening for changes. Whenever a controller needs to know “who are the descendants of this namespace”, it asks the forest and gets an instant answer. This makes propagation quick. It also means that after a restart, HNC rebuilds its picture from the objects stored in the cluster, so the real source of truth stays in the API server.
A receptionist keeps a copy of the company’s organization chart on the desk, so that when someone asks “who works under Maria?” the answer is immediate. If the desk copy is lost, the receptionist rebuilds it from the official records in the archive. The forest is the desk copy, and the API server is the archive.
The webhooks are part of the path of requests. If HNC is unavailable, some operations on namespaces and on inherited objects may be delayed or blocked, depending on how the webhooks are configured. Plan for HNC’s availability like any other control plane component.
6Installing HNC and Using the kubectl Plugin
Getting started is straightforward, and a small command line plugin makes daily use friendly.
HNC is installed by applying the manifests published with each release, which create the controller, the definitions and the webhooks in the HNC namespace. Choose a release that supports your Kubernetes version, and read its notes. Because HNC adds webhooks that can affect namespaces and objects across the cluster, it is wise to try it first in a test cluster.
HNC also comes with a plugin for the kubectl command line tool, usually installed through the krew plugin manager. The plugin adds a command called kubectl hns with several subcommands. These do not do anything special that could not be done by editing objects directly, but they make common tasks much easier and their output much clearer.
| Plugin command | What it does | When you use it |
|---|---|---|
| create | Creates a subnamespace under a given parent | A team lead wants a new environment |
| tree | Prints the tree of namespaces below a given one, with any problems | You want to see the shape of the hierarchy |
| set | Changes settings such as the parent of a namespace | You attach an existing namespace to a tree |
| describe | Shows details of one namespace: its parent, children, conditions and inherited objects | You are debugging why something did or did not propagate |
| config | Views and changes which object types propagate | You want a new kind of object to flow down the tree |
What changes after installation
Right after installation, nothing dramatic happens. Existing namespaces simply become roots of their own trees, because none of them has a parent yet. By default, RBAC objects, meaning Roles and RoleBindings, are already set up to propagate. From that point, you can start attaching namespaces together or creating subnamespaces.
Install HNC in a test cluster, create a small tree, and watch what happens when you add roles, delete namespaces and break rules on purpose. A hands-on feel for the behavior is worth hours of reading.
Installing HNC is like installing an intercom system in a building. On day one, every room still works exactly as before. The new capabilities appear only when you start connecting rooms into groups and sending announcements.
7Subnamespaces and Anchors
Subnamespaces are the part of HNC that makes self-service possible, and the trick behind them is clever.
Creating a namespace normally needs cluster-level permission, because a namespace is not inside any other namespace. HNC solves this with a small helper object called a subnamespace anchor. An anchor is an ordinary object that lives inside the parent namespace. Because it is namespaced, a person with rights in the parent can create it, with no cluster-level rights at all.
When the anchor appears, HNC reacts. Its controller, which does have cluster-level rights, creates a real namespace with the name given in the anchor, and records that the new namespace is a child of the parent. The person never needed the power to create namespaces. They only needed the ordinary power to create an anchor object in their own namespace, and HNC did the rest on their behalf.
sequenceDiagram
participant T as Team lead
participant A as API Server
participant H as HNC controller
T->>A: Create anchor named dev inside team-a
A-->>H: Watch event: new anchor
H->>A: Create namespace dev
H->>A: Mark team-a as parent of dev
H->>A: Update anchor status to Ok
A-->>T: Anchor shows the new namespace is ready
Rights are granted in the parent only
The platform team gives the team lead permission to create anchors in the team’s namespace. This is an ordinary namespaced permission.
The lead creates an anchor
The name of the anchor becomes the name of the new namespace.
HNC creates the namespace
It also marks the new namespace as a subnamespace, and records the parent link and an annotation that points back to the parent.
Inherited objects arrive
Roles, role bindings and any other configured objects flow down from the parent into the new namespace at once.
A manager who may not hire staff directly can fill in a request form kept in the department’s own folder. The human resources system watches that folder, and whenever a form appears, it hires the person and places them in the right team. The form is the anchor, and HR is the controller.
Namespace names are still unique across the whole cluster, because the core system keeps them in a flat list. A subnamespace called dev cannot be created if any other namespace is already called dev. A common habit is to include the parent’s name in the child’s name, such as team-a-dev, to avoid clashes.
8Full Namespaces: Attaching Existing Namespaces to a Tree
Subnamespaces are not the only way to build a tree. You can also connect namespaces that already exist.
HNC distinguishes two kinds of child namespaces. A subnamespace is created through an anchor, and is tied to its parent by that anchor. A full namespace is a normal namespace that was created in the usual way by an administrator, and then attached to a parent afterwards by setting its parent. Both kinds behave the same way for propagation. The difference is mostly in how they are created and how they are deleted.
| Aspect | Subnamespace | Full namespace |
|---|---|---|
| Created by | An anchor object inside the parent | Normal namespace creation by an administrator |
| Who needs cluster rights | Nobody. HNC acts on the user’s behalf | Whoever creates the namespace |
| Parent is set by | HNC automatically, when it creates the namespace | Explicitly, by changing the hierarchy setting of the child |
| Can be moved to another parent | Limited. It is tied to its anchor | Yes, by changing the parent, if permissions allow |
| Deleted by | Deleting the anchor in the parent | Normal namespace deletion |
| Typical use | Self-service environments for teams | Bringing older namespaces into a tree |
Who may set a parent
Setting the parent of a namespace is a powerful action, because it changes which rules flow into it. HNC therefore checks permissions on both sides. The person must be allowed to change the hierarchy setting of the child, and must also have the right to add children to the new parent. This prevents someone from attaching their own namespace to a powerful tree in order to inherit its permissions.
A new employee cannot simply decide to report to the chief executive. Both the employee’s current manager and the new boss must agree. HNC requires the same agreement from both sides before it accepts a new parent.
Use subnamespaces for everything that teams create on their own, and use full namespaces mainly for migration of existing namespaces. This keeps the creation path clean and keeps cluster-level rights away from teams.
Attaching an existing namespace to a tree without checking what will flow into it. The moment it gets a parent, it receives all the inherited roles and objects, which may grant people access they should not have there.
9The HierarchyConfiguration Object
Each namespace in the tree has one small object that records its place in it.
The place of a namespace in the tree is stored in an object called the hierarchy configuration. There is at most one of these in each namespace, and it has a fixed name, so that it can always be found. Its spec, the part that people edit, holds the name of the parent and a setting about whether deleting this namespace’s anchor or parent should also remove everything below. Its status, the part that HNC fills in, lists the children of the namespace and a set of conditions that describe any problems.
Parent
The name of the parent namespace. Empty means this namespace is a root.
Allow cascading deletion
A switch that says whether removing this namespace should also remove all its descendants.
Children
The list of direct children, maintained by HNC, which is used to draw the tree.
Conditions
Notes about problems, such as a missing parent or a conflict, with a reason and a message for people to read.
Conditions are the health report
Conditions tell you what is wrong, and they are the first place to look when something seems off. A condition can warn that the named parent does not exist, that a cycle was detected, that an ancestor has a problem, or that an object could not be copied because of a conflict. Some conditions are only warnings about a single object. Others are serious, and cause HNC to stop activities in the affected part of the tree until the problem is fixed, to avoid making things worse.
Every employee file has a line saying who the manager is, a line listing direct reports, and a section of notes from human resources about anything unusual. The hierarchy configuration is that file for a namespace, and conditions are the notes from human resources.
Show conditions on a dashboard and alert on serious ones. A tree with a serious condition has frozen parts, and nobody should discover that by accident when a new subnamespace fails to appear.
Editing the hierarchy configuration by hand without understanding the permission checks. Prefer the plugin commands, which show clear error messages when a change is refused.
10Propagation: What Flows Down the Tree
Propagation is the second big feature of HNC, and it is where most of the daily value comes from.
When you create an object in a namespace that is marked as a source, HNC makes a copy of it in every descendant of that namespace. If you change the original, the copies change too. If you delete the original, the copies are deleted. The copies are real objects in each descendant, so Kubernetes treats them like any other, and a Role copied into a child namespace grants real permissions there.
flowchart TD
S["Source object created in team-a"] --> C1["Copy in team-a-dev"]
S --> C2["Copy in team-a-staging"]
S --> C3["Copy in team-a-prod"]
C3 --> C4["Copy in any further child below team-a-prod"]
What kinds of objects can propagate
By default, two kinds of objects propagate: Roles and RoleBindings. These are the objects that control who may do what, and they are by far the most common reason for using a hierarchy. Other kinds of objects can be added to the list by configuration, and we will cover how in the next chapters. Typical additions are NetworkPolicies, ResourceQuotas, ConfigMaps, Secrets and limit ranges. Almost any namespaced object type can be added, including custom resources.
| Object type | Propagates by default | Typical reason to propagate |
|---|---|---|
| Role and RoleBinding | Yes | Give a team’s people the same access in all its environments |
| NetworkPolicy | No, can be enabled | Apply a baseline of network rules to every namespace in a tree |
| ResourceQuota and LimitRange | No, can be enabled | Give every child the same standard limits |
| ConfigMap | No, can be enabled | Share standard settings such as certificate authority bundles |
| Secret | No, can be enabled with care | Share an image registry pull secret with all environments |
| Custom resources | No, can be enabled | Share policy or settings objects defined by operators |
When a head office updates the safety poster in the company handbook, every branch office automatically receives the new poster, and nobody needs to print and mail them. If head office removes the poster, it disappears from every branch too.
Propagation always goes downward, and copies are never sent up or sideways. A Role created directly in a child stays in that child. This is what keeps the behavior predictable.
11Sharing Permissions Down the Tree with RBAC
The most loved feature of the controller is also the simplest: give someone access once, and every child namespace inherits it.
Kubernetes controls who may do what with a system called RBAC, short for role-based access control. A Role is a list of allowed actions, such as reading Pods or editing Deployments. A RoleBinding connects a Role to a person, a group or a service account inside one namespace. Without hierarchy, a RoleBinding in one namespace says nothing about any other namespace. If a team owns twelve namespaces, somebody must create twelve bindings and keep all twelve in sync.
The controller changes this by treating Roles and RoleBindings as things that flow downward by default. When you create a RoleBinding in a parent namespace, the controller creates an identical copy in every child, grandchild and deeper descendant. When you change the original, every copy changes. When you delete the original, every copy disappears. The people named in the binding therefore gain the same access across the whole subtree with one action.
Why this is a safe default
Access flows only downward. A person who is granted rights in a child gains nothing in the parent. This matches how most organizations think: a department head can see everything inside the department, while a single project member sees only that project. The direction of flow makes the permission model easy to explain and easy to audit.
Think of a master key at a school. The principal’s key opens the main gate, every hallway and every classroom. A teacher’s key opens only one classroom. When the school hands the principal a new key, nobody needs to visit each classroom to change a lock. Permissions that flow downward work the same way.
Roles and bindings travel together
A binding points to a Role by name. If only the binding were copied and the Role were left behind, the copy would point at nothing. For this reason both objects propagate together. A team can define a Role called “namespace editor” in the parent and bind it to the team’s group. Every child then has the Role and the binding, and they work at once.
| Situation | Without hierarchy | With the controller |
|---|---|---|
| New team member joins | Edit bindings in every team namespace | Edit one binding in the parent |
| New environment namespace created | Copy all bindings by hand or by script | Bindings appear automatically |
| Person leaves the team | Hunt for every binding that names them | Remove the one original binding |
| Audit question: who can edit here? | Read each namespace separately | Read the namespace plus its ancestors |
Bind groups, not individuals. When bindings name a group from your identity provider, joining or leaving the team is handled by group membership, and the hierarchy only carries the rule that the group has access.
Forgetting that a permission placed high in the tree reaches every descendant, including namespaces created later by other people. Place broad rights only at levels where you are happy for them to reach everything below.
12Choosing More Types with HNCConfiguration
Roles and bindings are only the start. A single cluster-wide object tells the controller which other kinds of objects should travel.
The controller has one cluster-wide settings object called HNCConfiguration. There is exactly one of it, and it lists resource types together with a mode for each type. A resource type is a kind of object, such as ConfigMap, Secret, NetworkPolicy or ResourceQuota. The mode decides what the controller does with objects of that type.
The common modes
The names and details can change between versions, so check the documentation for your release. In general there are three ideas. The first is propagate: objects of this type that you place in a parent are copied down to descendants. The second is remove: the controller deletes inherited copies, which is useful when you want to stop a type from flowing without deleting the originals. The third is ignore: the controller does nothing at all for the type, which is the state of every type not on the list.
Propagate
Originals in a parent are copied into every descendant and kept in sync.
Remove
Inherited copies are cleaned up. The originals stay where they are.
Ignore
The controller leaves this type alone. Nothing flows and nothing is removed.
Imagine a school office with a photocopier and a list on the wall. The list says which papers get copied to every classroom: the fire plan, the timetable. Other papers are never copied. HNCConfiguration is that list on the wall. A new type of paper goes on the list only when someone decides it should travel.
Which types make good candidates
Good candidates are objects that every namespace in a subtree should share and that rarely differ. Examples include policies that apply to every workload, a shared configuration file with a company-wide setting, or a pull credential for a private image registry. Poor candidates are objects that hold unique data for each application, or objects that vary by environment.
| Object type | Good to propagate? | Reason |
|---|---|---|
| Role and RoleBinding | Yes, on by default | Shared access rules are the main purpose of hierarchy |
| NetworkPolicy | Often yes | Baseline traffic rules, such as default deny, should apply everywhere |
| ConfigMap with shared settings | Sometimes | Useful for company-wide values; avoid for per-app data |
| Secret for a registry | Carefully | Convenient, but copies of secrets spread access to the secret |
| ResourceQuota | Rarely | Each copy is a separate limit, so the totals add up in surprising ways |
Switching on a type without checking existing objects. When a type starts to propagate, every existing object of that type in every parent immediately flows down, which can surprise teams that already had a different object with the same name in a child.
13Selecting What Propagates and What Does Not
Sometimes one specific object should stay home, or should go only to some children. Annotations give you fine control.
Switching a whole type to propagate is a blunt tool. Real life has exceptions: a Role that is meant only for the parent, or a Secret that should reach just one branch of the tree. The controller offers small notes called annotations that you attach to the individual object. An annotation is a label-like key and value stored in the object’s metadata that tools can read. The controller reads a few of them to decide how that one object should travel.
The ideas behind the selectors
There are four main ideas. One note can say that the object should not propagate at all. Another can say it should propagate everywhere, even if the type is normally excluded in some situations. A third can pick children by their names or labels, in the style of a label selector. A fourth can pick whole subtrees through the tree labels you will meet soon. The exact annotation keys have changed over time, so read the documentation for the key names in your version rather than copying them from a blog post.
Type is listed to propagate
HNCConfiguration says Secrets travel, so the controller considers every Secret in a parent.
The object is checked for notes
The controller reads the object’s annotations to see whether it has been told to stay, to go everywhere or to go to chosen children only.
Each descendant is tested
For every descendant, the controller asks whether this object is allowed to reach it.
Copies are created or removed
Allowed descendants receive a copy. Disallowed ones have any old copy removed.
A teacher hands out a worksheet to the whole class by default. One sheet has a sticky note saying “only for group B”. The teacher reads the note and gives that sheet to group B only. Annotations are the sticky notes on individual objects.
Why selectors are powerful but risky
Selectors let you share a credential with exactly the right children. They also create invisible rules that live inside individual objects. A reader looking at a child namespace sees a copy but cannot tell why the other children did not get one. Keep a short written record of every selected object, and prefer the tree structure to express most of your intent.
Use the tree itself for the common case and annotations for rare exceptions. If many objects need annotations, the tree probably does not match how the organization works and deserves a redesign.
14Inherited Copies, Labels and Read-Only Rules
A copy in a child is not an ordinary object. It is marked, protected and tied to its source.
When the controller creates a copy of a parent object in a child, it labels that copy with a marker whose key is hnc.x-k8s.io/inherited-from and whose value is the name of the namespace where the original lives. This label is the passport of the copy. It tells people and tools that the object is inherited, and it tells the controller that this object is under its management.
Why copies are read-only
The controller’s webhooks refuse changes to inherited copies made by ordinary users. If anyone could edit a copy, the next sync would overwrite their change, or worse, the child would drift from the parent without anyone noticing. To change a copy, you change the original in the ancestor namespace. This is a strong guarantee. It means that what a child namespace receives from above is exactly what the owner above intended.
A photocopy of a school notice pinned in a classroom has a stamp that says “copy of the office notice”. Students may not write on it to change the rules. If the notice must change, the office issues a new one and all copies are replaced.
How a copy is kept up to date
The controller watches both the original and the copies. If the original changes, copies are updated. If a copy is somehow deleted or altered by an administrator who bypasses the webhook, the controller restores it. If the original is deleted, all the copies are removed. This reconciling loop is the same pattern that all Kubernetes controllers use, and it is why the system recovers from small accidents without human help.
Consequences you should know
Easy to find origin
Listing objects with the label quickly shows which ones are inherited and from where.
Edit at the source
Every change to a shared object must be made in the namespace that owns the original.
Names must be free
A child that already has an object with the same name and type blocks the copy, and a condition reports the clash.
Trying to customize an inherited Role in a child by editing it. The edit is rejected. If a child needs a slightly different Role, create a new Role with another name in the child, and leave the inherited one alone.
15Tree Labels and Policies That Understand the Hierarchy
Besides copying objects, the controller stamps every namespace with labels that describe where it sits in the tree.
Many Kubernetes features select namespaces by label. A NetworkPolicy, for example, can say “allow traffic from namespaces that carry this label”. Core Kubernetes knows nothing about trees, so a plain policy cannot say “allow everything from the finance department and all its sub-teams”. The controller fills this gap by adding labels to each namespace that name its ancestors.
What the tree labels look like
For every ancestor of a namespace, including the namespace itself, the controller adds a label whose key includes the ancestor’s name followed by the suffix .tree.hnc.x-k8s.io/depth. The value is a number. Zero means the namespace itself, one means a child of that ancestor, two means a grandchild, and so on. A policy can then select every namespace that carries the label for the finance department, regardless of how deep the namespace sits.
flowchart TD
F["finance (depth 0 label on itself)"] --> P["payroll (finance depth 1)"]
F --> A["audit (finance depth 1)"]
P --> PD["payroll-dev (finance depth 2, payroll depth 1)"]
P --> PP["payroll-prod (finance depth 2, payroll depth 1)"]
Figure 5: Every namespace carries a label for each of its ancestors, with a number telling how far away that ancestor is.
In the figure, a policy that selects the finance label matches finance itself, payroll, audit and both payroll environments. A policy that selects the payroll label matches payroll and its two environments only. The same idea works for any feature that selects namespaces by label.
A family tree where each person wears a badge for every ancestor: “descendant of Grandma”, “descendant of Aunt Rosa”. If you invite “everyone wearing the Grandma badge” to a party, you get every child and grandchild without listing names.
Why this is valuable for network rules
A common requirement is that workloads inside one department may talk to each other, but not to other departments. Without tree labels, you list each namespace by name and update the list whenever a team adds a namespace. With tree labels, you write one rule per department. New namespaces join the department simply by being created under it.
| Goal | Selection by names | Selection by tree label |
|---|---|---|
| Allow traffic inside a department | List every namespace and edit the list on each change | One rule using the department label |
| Allow a shared service to be reached by all teams | Add each new team by hand | One rule using the organization label |
| Block one department from another | Maintain two lists | Rules that match each department label |
Remember that the policy engine, not the controller, enforces the rule. Test that your network plugin honors namespace selectors, and test with a real connection, not only by reading the objects.
16Passing Labels and Annotations to Child Namespaces
Some settings live on the namespace object itself. The controller can be told to hand selected ones down to new subnamespaces.
Namespaces carry labels and annotations of their own. Many cluster tools depend on them. A security feature might read a label that sets how strict the Pod rules are. A cost tool might read a label that names the team that pays. A service mesh might read a label that turns on automatic sidecars. If subnamespaces do not carry the same labels as their parent, those tools treat them differently, often by accident.
What the controller can do
Depending on version and startup settings, the controller can copy chosen labels and annotations from a parent namespace to its children, so that a team’s namespaces share the same cost label or security level. Which keys are allowed to propagate is a setting for the administrator, because blindly copying everything would be unsafe. Features in this area have changed across releases, so confirm what your version supports before depending on it.
A new student in a school class receives the same house badge as the rest of the class. The badge decides which colour team they play for and which fees are billed. If the badge is forgotten, the student is counted in no team at all.
Why it matters in practice
Team ownership
A cost label on the parent lets every child be charged to the same team.
Pod safety level
A strict level on the parent stops children from starting with a weaker one.
Sidecar injection
Children join the same mesh setting without manual labeling.
Allowing too many keys to propagate. If a privileged label is allowed to flow, a person who can create a subnamespace may receive privileges they were never granted. Keep the list short and review it as carefully as any permission.
If your version does not support label propagation, you can reach a similar result with a policy engine that adds labels at creation time, or with automation that runs when a subnamespace appears. The key point is to decide on purpose how children get their labels, rather than leaving it to chance.
17Inside the Reconciliation Loop
Here we follow what the controller does, step by step, when a single object changes in a parent.
A reconciler is a piece of code that compares what should exist with what does exist and fixes the difference. The controller has several reconcilers. One looks after hierarchy objects and keeps the forest up to date. Another looks after anchors. Others look after each propagated type. They run whenever the API server tells them that something has changed.
sequenceDiagram
participant U as User
participant API as API server
participant W as Webhook
participant R as Reconciler
participant F as Forest
U->>API: Change a RoleBinding in the parent
API->>W: Validate the change
W->>F: Check hierarchy rules
F-->>W: Allowed
W-->>API: Accept
API->>R: Notify of the change
R->>F: Find all descendants
F-->>R: List of children
R->>API: Create or update copies in each child
Figure 6: A change in a parent reaches every descendant through a webhook check and then a reconciling step.
The steps in words
The event arrives
The API server notifies the controller that an object of a watched type changed.
The forest answers a question
The reconciler asks the in-memory forest which namespaces sit below the changed object’s namespace.
Each child is compared
For every descendant, the reconciler compares the existing copy with what the original now says.
Differences are written
Missing copies are created, stale ones updated, unwanted ones deleted.
Status is reported
If something went wrong, a condition is recorded so people can see it.
A school janitor walks the corridors with a checklist. Each time the head teacher posts a new notice, the janitor checks every classroom: is the notice there, is it the latest version, should an old one be taken down? Reconciliation is that walk, repeated whenever something changes.
Why the loop is forgiving
Because the reconciler compares the whole desired state to the whole actual state, it does not matter if it missed an event or crashed in the middle. After a restart it simply walks the corridors again. This idea, called level-based reconciliation, is what makes Kubernetes controllers resilient. It also means the controller can be slow for large trees, which we will discuss later.
18Conflicts, Cycles and Conditions
When something cannot be done safely, the controller refuses and tells you why, using conditions on the namespace.
A hierarchy can go wrong in several ways. Two people can try to give a namespace different parents at the same time. Someone can set a parent that does not exist yet. Someone can create a loop where A is a child of B and B is a child of A. Someone can try to copy an object into a child that already has a different object of the same name. The controller meets each of these with a clear rule.
| Problem | What the controller does | How you see it |
|---|---|---|
| Cycle in the tree | Webhook rejects the change, or all activity in the loop is halted | A condition on the affected namespaces |
| Missing parent | Treats the namespace as if it had no ancestors until the parent appears | A condition noting the parent is missing |
| Name conflict on copy | Does not overwrite the existing object in the child | A condition describing the conflicting object |
| Ancestor in trouble | Stops propagating through the troubled namespace | A condition on descendants that mentions the ancestor |
| Forbidden parent change | Webhook denies the request | An error message to the user |
A railway signal turns red when two trains could meet on the same track. The signal does not fix the trains. It stops traffic and shows everyone that something needs attention. Conditions are the red signal of the hierarchy.
Halting is a safe choice
When a namespace is in trouble, the controller stops copying objects into and out of it. This is called halting. It may feel annoying, because teams stop receiving updates. It is much safer than guessing, because a wrong guess might give access to the wrong people. Once the problem is fixed, the condition clears and normal flow resumes.
Build an alert on namespaces that carry a problem condition for more than a few minutes. A condition that nobody sees is the same as a silent failure.
Fixing the symptom, for example by deleting the conflicting object in the child, without asking why it existed. Sometimes the child owner created it for a reason, and the right answer is to rename the object in the parent.
19Deleting Namespaces and Cascading Deletion
Deleting a branch of the tree can remove a lot of work. The controller therefore adds guard rails.
In core Kubernetes, deleting a namespace deletes everything inside it. In a hierarchy, a parent may have many children, each full of applications. A careless deletion of the parent could, if cascading were automatic, remove the whole branch in seconds. The controller treats this very carefully.
Subnamespaces and their anchors
A subnamespace is owned by its anchor in the parent. If you delete the anchor, the controller deletes the subnamespace, but only when the subnamespace has no subnamespaces of its own, or when cascading deletion is allowed. This is a deliberate speed bump. It makes you stop and think when the deletion would affect several levels.
The cascading permission
The HierarchyConfiguration of a namespace has a setting, allowCascadingDeletion, which defaults to off. When it is off for a parent, the controller refuses to delete the parent’s subnamespace children by removing their anchors in a chain. When it is on, deletion may sweep downward through the whole subtree. The setting may be placed on a namespace or on its ancestors, and it lets the owner decide how much freedom to give.
A building has a master switch that turns off the power to every floor. The switch has a plastic cover so nobody flips it by accident. Cascading deletion is the cover. You can lift it on purpose, but it should not be opened casually.
| Action | Cascading off | Cascading on |
|---|---|---|
| Delete a leaf subnamespace anchor | Allowed | Allowed |
| Delete an anchor whose subnamespace has children | Refused | Allowed, whole branch removed |
| Delete a full namespace that has children | Children are not deleted; they lose their parent link | Depends on the rules for that kind of child |
Turning on cascading deletion everywhere to make clean-up scripts work. Doing so removes the safety cover for every person on the team. Turn it on only on the namespaces and in the moments where clean-up truly needs it.
Remember too that deleting copied objects does nothing useful. The controller recreates them. To remove an inherited object you must delete the original in its owning namespace or change the settings that make it travel.
20Delegation: Who May Do What
The point of subnamespaces is to let teams act on their own. This chapter shows the exact line between the platform team and the tenants.
Delegation means handing part of a job to someone else together with the right to do it. In this system, the platform team prepares a team’s top namespace and gives the team permission to create anchors inside it. After that, the team can create, name and use subnamespaces without asking anyone, and the platform team does not need cluster-wide privileges for the work.
Three layers of permission
Cluster administrators
Install the controller, set HNCConfiguration and create top-level namespaces.
Team leads
Hold rights in their team namespace, including creating anchors, so they can create subnamespaces.
Team members
Work inside the subnamespaces with the rights that the hierarchy gives them.
A school principal gives each head of department a budget and the right to hire helpers inside that budget. The principal does not approve each pencil. The heads work freely inside their space, but cannot touch another department’s cupboard.
Where the guard rails are
The webhook checks that a person who sets a parent has the right to do so on both sides. Moving a namespace under a new parent needs authority over both the namespace and the new parent. This stops a person from attaching their namespace beneath someone else’s tree to inherit that tree’s permissions. Without this check, hierarchy would become a way to steal access.
Write down the three layers for your organization and review them every quarter. When the line between layers is clear, people stop asking for cluster-wide rights they do not need.
21Hierarchy, Quotas and Limits
People often expect a parent to share a budget of CPU and memory with its children. The truth is more careful.
A ResourceQuota in Kubernetes caps how much a single namespace may use: total CPU, memory, storage or number of objects. In a hierarchy the natural wish is to place one quota on a department and let the children share it. The project explored this idea in an early experimental version, but it is not part of the current stable features. Check the documentation of your release before planning around it.
What you can do today
You can propagate a ResourceQuota object downward, but each copy is its own separate limit. If a department has ten children and the quota says ten CPUs, the department can actually use a hundred CPUs. That is rarely what anyone wants. A better approach is to place a quota on each child by a different mechanism, such as an automation tool or policy engine that sets the right numbers at creation time, or a more advanced multi-tenant system that manages shared budgets.
A family gives each child a sweet jar with ten sweets. If the rule is copied to five children, the family hands out fifty sweets. If the family really wanted fifty sweets in total, they should have counted once, in the pantry, and handed out shares.
| Approach | Effect | Trade-off |
|---|---|---|
| Propagate a quota object | Each child gets an equal separate limit | Total can be much more than intended |
| Create quotas by automation | Each child gets a planned share | Extra tooling to run |
| Use a platform with shared quota | One budget for the whole subtree | Another component with its own lifecycle |
| Use cost labels and reports | No hard limit, only visibility | Overuse is found after the fact |
Assuming that a quota on a parent namespace limits its children. A quota counts only what runs inside the namespace where it sits.
22What the Controller Does Not Do
Knowing the limits is as useful as knowing the features. These are the common misunderstandings.
The controller is a small, focused tool. It adds a tree to namespaces and copies chosen objects. It is not a complete multi-tenancy platform. Many wrong plans come from expecting it to do things that belong to other tools.
Not isolation
It does not separate workloads. Network rules, node pools and Pod security remain your job.
Not a separate control plane
All namespaces share one API server. A cluster-wide problem affects every tenant.
Not upward sharing
Nothing flows from child to parent. The flow is downward only.
Not a cluster-wide object copier
Only types you list in the configuration travel, and only through the tree.
Not a name system
Namespace names still must be unique across the cluster, whatever their position in the tree.
Not a billing tool
It can help with labels, but it does not measure or report cost.
A hallway map on the school wall shows how rooms relate to one another. It does not lock the doors, count the pupils or pay the electricity bill. The map is very useful, and other systems still do the rest.
This is why serious multi-tenant clusters combine the controller with network policies, Pod security settings, admission policies, quotas and sometimes dedicated nodes. The controller removes the repetition from these tools. It does not replace them.
23Worked Example: A Platform Team Serving Many Product Teams
Let us follow a made-up company called Northwind, which has one shared cluster and four product teams.
Northwind’s platform team runs the cluster. Four product teams, called checkout, search, payments and profile, each need several namespaces: a development one, a testing one and a production one. Before the controller, the platform team created twelve namespaces by hand, then copied the same access rules into each, then answered a steady stream of requests such as “please add a namespace for my experiment”.
The new design
The platform team creates one full namespace for each team, such as team-checkout. They bind the team’s group to a Role in that namespace that allows creating anchors and managing ordinary workloads. Then they step back. The checkout team creates subnamespaces called checkout-dev, checkout-test and checkout-prod by themselves. The access rules flow down at once. Network rules flow down too, because the platform team listed NetworkPolicy in HNCConfiguration.
flowchart TD
NW["northwind-root (platform team)"] --> TC["team-checkout"]
NW --> TS["team-search"]
NW --> TP["team-payments"]
NW --> TR["team-profile"]
TC --> CD["checkout-dev"]
TC --> CT["checkout-test"]
TC --> CP["checkout-prod"]
TP --> PD["payments-dev"]
TP --> PP["payments-prod"]
Figure 7: Northwind’s tree. The platform team owns the top. Each team owns its own branch.
What changed for people
Requests dropped
Teams no longer wait for a namespace. They create it in seconds with their own rights.
Consistency improved
Every namespace receives the same baseline rules because they are inherited, not retyped.
Audits became shorter
Reviewers read one namespace per team plus the root, rather than twelve or more.
The platform team focused on the platform
Time once spent on tickets went to better monitoring and safer upgrades.
A landlord who once painted every tenant’s room now hands out a rulebook for the building and lets each tenant arrange their own flat. The landlord’s visits drop, and the house rules stay the same for everyone.
Start with one pilot team. Learn from their first month before you switch the other teams over.
24Worked Example: Sharing a Registry Credential and a Default-Deny Network Rule
Two chores that every new namespace needs, solved once at the top of the tree.
Every new namespace at Northwind needs two things before an application can run properly. First, a credential for the private image registry, because Pods cannot pull company images without it. Second, a default-deny network rule, which blocks all traffic by default so teams must add explicit allowances. Without hierarchy, both items were forgotten about as often as they were remembered.
How the platform team handles the credential
The platform team places one credential object in a platform namespace. They mark the type as one that propagates in HNCConfiguration, and they add a note so that the credential reaches only the product team branches, not other special namespaces. Each subnamespace receives a copy automatically. When the registry password rotates, the platform team replaces the one original and every copy updates within moments.
How the default-deny rule is handled
The default-deny NetworkPolicy is placed in each team’s top namespace. It propagates to every child. Teams then add their own allow-rules in their own namespaces to open the connections they need. Because the inherited rule cannot be edited by tenants, no team can remove the safety net by mistake.
| Chore | Where the original lives | What teams do | What the platform team does on a change |
|---|---|---|---|
| Registry credential | Platform namespace | Nothing | Replace one object |
| Default-deny rule | Team top namespace or root | Add allow-rules for their needs | Edit one object |
| Standard Role for developers | Team top namespace | Add people to a group | Edit the Role once |
A hotel gives every new guest a key card and a copy of the house rules at check-in, without the guest asking. When the hotel changes the Wi-Fi password, the front desk updates the card printer once.
Treating a propagated secret as private to the platform team. Every copy is readable by whoever has permission to read secrets in that child. Check who holds that permission, and consider a registry credential that has pull access only.
25Working with GitOps and Policy Engines
Many teams keep cluster settings in a version-controlled repository. Inherited copies need a little care in that world.
GitOps means that a repository is the source of truth, and a tool in the cluster keeps the cluster equal to the repository. Such tools often have a pruning feature that deletes anything in the cluster that is not described in the repository. Inherited copies are created by the controller, not by the repository, so a pruning tool may see them as strangers and try to delete them. The controller then recreates them, and the two tools fight in a loop.
How to avoid the fight
Most GitOps tools allow you to exclude objects by label. Because every inherited copy carries the inherited-from label, you can tell the tool to ignore objects that have it. Another safe habit is to keep the originals in the repository and let the controller own all copies. Teams should never place the same object by hand into a child namespace when the parent already sends it.
flowchart LR
G["Git repository"] --> S["GitOps tool"]
S --> P["Original objects in parent namespaces"]
P --> H["Hierarchical namespace controller"]
H --> C["Inherited copies in children"]
S -. "ignore objects labeled inherited-from" .-> C
Figure 8: The repository owns the originals. The controller owns the copies. The GitOps tool is told to leave the copies alone.
Policy engines
A policy engine, also called an admission controller, checks new objects against rules, for example “every Pod must have a team label”. The controller creates copies by writing to the API server like any other client, so the engine’s rules apply to those copies too. If your rules reject something the controller creates, you will see a condition about a failed copy. Make sure your policies allow what the controller needs, or exempt the controller’s service account where that is safe.
A school has a caretaker who tidies anything left in the corridor. A teacher who pins a notice on the corridor wall would see it thrown away unless the notice carries an approved stamp. The label on inherited copies is that stamp for the caretaker tool.
Test the whole chain in a small cluster first: repository, GitOps tool, policy engine and the controller together. Surprises show up in minutes there and in hours in production.
26Scale and Performance
The controller does work in proportion to the size of your trees and the number of objects that propagate.
Every change to a propagated object means copies in every descendant. A Role copied into ten children is ten writes. A Secret that changes every hour and travels to five hundred namespaces is five hundred writes every hour. Those writes go through the API server, are stored in etcd, and are watched by many other components. At small scale this is invisible. At large scale it can matter.
The factors that grow cost
Number of namespaces
The forest lives in memory, and every namespace adds a few entries to it.
Children per parent
One change in a wide parent triggers many writes at the same moment.
Rate of changes
Objects that change often make more copies and more events.
Number of propagated types
Each listed type adds its own watching and reconciling.
A teacher who photocopies one page for thirty pupils finishes in a minute. A teacher who must photocopy a new page every ten minutes for three hundred pupils spends the whole day at the copier. Both are the same job at different sizes.
Keeping it healthy
Keep propagated objects small and slow-changing. Avoid very deep trees; two to four levels are common and easier to reason about. Watch the controller’s CPU, memory and queue length, and the API server’s request rates. If you grow to thousands of namespaces, test in a staging cluster with similar numbers before going live, and read the project’s notes on tested limits for your release.
Propagating an object that changes constantly, such as a token that is renewed every few minutes, to a very wide tree. Share the source of the secret in some other way instead.
27Observability: Metrics, Logs, Events and Conditions
A good operator can tell in a minute whether the hierarchy is healthy. Four sources of signals make that possible.
Observability means being able to understand what a system is doing from the outside. For this controller there are four natural places to look. Each answers a different question.
| Source | Question it answers | How to use it |
|---|---|---|
| Conditions on namespaces | Is any branch of the tree in trouble? | List namespaces and alert on problem conditions |
| Controller metrics | Is the controller busy, slow or failing? | Graph reconcile counts, durations and errors |
| Controller logs | What happened to this specific object? | Search by object and namespace name |
| Kubernetes events | What was the most recent action? | Read events in the affected namespace |
| The tree view of the plugin | What does the hierarchy look like right now? | Print the tree for a root and compare with the plan |
A car has a dashboard light for general trouble, a trip computer for numbers, a service history for detail and a mirror for recent events. You need all four to drive with confidence.
What to watch for first
Is the controller running?
Check that its Pod is ready and that its webhook answers requests quickly.
Are there problem conditions?
Any condition lasting more than a few minutes needs a person.
Are reconcile errors rising?
A rising error count often shows permission trouble or a rejected copy.
Is latency growing?
Slow propagation hints at too many copies or an overloaded API server.
Because the webhook sits in the path of namespace and hierarchy changes, track its latency and its error rate as carefully as you track the API server itself.
28Upgrades, Backups and Disaster Recovery
What do you need to save, and what happens if the controller disappears for an afternoon?
The controller holds no data of its own. The forest in its memory is rebuilt from the API server each time it starts. The real information lives in Kubernetes objects: the hierarchy settings, the anchors, the configuration and the originals. If you back up those objects as part of your normal cluster backup, you can rebuild the whole hierarchy.
What happens during an outage
If the controller is down, existing copies stay in place and keep working, because they are ordinary objects. Applications are not affected. New changes in parents do not reach children, and new anchors are not turned into namespaces until it returns. If the webhook is configured to fail closed, hierarchy changes may be refused during the outage. Choose that setting knowing the trade-off between safety and availability.
If the school photocopier breaks, the pages already pinned in classrooms stay up. Only the new pages cannot be delivered until the machine is repaired. Learning goes on.
| Item | Where it lives | Back it up? |
|---|---|---|
| HierarchyConfiguration objects | In each namespace | Yes |
| SubnamespaceAnchor objects | In parent namespaces | Yes |
| HNCConfiguration | Cluster-wide | Yes |
| Original propagated objects | In parent namespaces | Yes |
| Inherited copies | In children | Not needed; they are rebuilt |
| The in-memory forest | Controller memory | No; it is rebuilt at start |
Upgrade habits
Read the release notes before each upgrade, especially for changes to the custom resource definitions and to configuration names. Upgrade in a test cluster first. Take a backup. After the upgrade, check that conditions stay clear and that a test change in a parent reaches a test child.
Restoring only some objects after a disaster. If anchors come back before their parents, or originals come back late, you may see temporary conditions and name conflicts. Restore namespaces first, then hierarchy, then originals, and let the controller settle.
29Security Considerations
A tool that copies permissions across namespaces deserves a careful security review.
The controller itself needs broad rights. To copy Roles and Secrets it must be able to read and write them across the cluster. Anyone who takes control of the controller could, in theory, create access in any namespace. Treat its service account, its Pod and its images as high-value assets.
Main risks and answers
Privilege creep
High-level bindings reach many children. Keep broad rights near the top and few.
Stolen controller identity
Restrict who can edit its Deployment, limit network access to it and pin its image.
Secret spread
Every copy of a Secret widens the circle of readers. Share only what is needed.
Re-parenting attacks
Moving a namespace under a powerful parent could inherit its rights. The webhook demands rights on both sides.
Webhook bypass
If the webhook is down and set to ignore failures, rules may not be checked. Choose the failure policy with care.
The person who cuts the master keys for a building must be trusted more than any tenant. The key-cutting shop, its tools and its records must be guarded better than any single flat.
Record who may create anchors and who may set parents. Review the list regularly, and use the audit log of the API server to watch for changes to HierarchyConfiguration objects outside normal work hours.
Remember too that hierarchy is a convenience for permissions, not a wall. Strong isolation needs network policies, Pod security rules and, for the strictest needs, separate clusters.
30Troubleshooting Guide
When something looks wrong, a calm order of questions finds the cause faster than guessing.
Start with the tree, then conditions, then the controller, then the object. This order moves from the widest view to the narrowest and avoids chasing small details before checking the big picture.
| Symptom | Likely cause | First check |
|---|---|---|
| A child did not receive an object | Type not configured, selector excludes it, or a name conflict | Look at HNCConfiguration and the object’s annotations, then conditions on the child |
| Anchor stays in a waiting state | Controller not running, name already taken, or parent in trouble | Read the anchor’s status and the controller logs |
| Copy keeps reappearing after delete | The original still exists in an ancestor | Find the origin using the inherited-from label |
| Cannot change a Role in a child | It is an inherited, read-only copy | Edit the original in the owning namespace |
| Cannot set a parent | Missing rights on one side, or it would form a cycle | Read the webhook’s error message |
| Changes are slow to arrive | Large fan-out or overloaded API server | Check controller metrics and API server load |
| GitOps tool keeps deleting copies | Pruning sees copies as strangers | Exclude objects with the inherited-from label |
A doctor first looks at the whole patient, then the chart, then the organ. A good engineer treats a cluster the same way: the map, the signals, the machine, the single object.
Keep a short runbook with these questions and the exact commands your team uses. Under stress, a checklist beats memory.
Disabling the webhook to get past a refusal. The refusal usually protects something. Read the message, fix the cause, and keep the guard in place.
31Design Patterns and Anti-Patterns
These are patterns that work, and mistakes that appear again and again.
Patterns that work
One root per organization
A single top namespace holds baseline rules for everything below it.
Team branch with environments
A namespace per team, with dev, test and prod beneath it.
Platform services branch
Shared services live under their own branch with a clear owner.
Anti-patterns
Very deep trees. Six levels of nesting make permissions hard to reason about and slow to audit. Keep depth modest and flat where possible.
Cluster-admin bindings in the root. Such a binding reaches every namespace beneath it. Use narrow Roles and bind them lower down.
Propagating everything. Turning on many types makes the tree noisy, increases cost and creates unexpected name clashes. Add types one at a time, each with a reason.
Mirroring the org chart exactly. Companies reorganize often, and namespaces are awkward to move. Model stable things such as products and environments instead of people and managers.
Hand-editing inherited objects. The controller rejects or reverts such edits. Always change the original.
Treating the tree as isolation. Hierarchy shares permissions. It does not stop one workload from calling another. Add network and Pod rules separately.
A good filing cabinet has few drawers, clear labels and the same layout every time. A bad one has twelve nested folders and names that change with every new manager.
32Advantages, Disadvantages and Alternatives
No tool is right for every cluster. Here is an honest comparison.
Advantages
- Self-service namespaces without cluster-wide rights for teams.
- Permissions and baseline rules defined once and inherited.
- Tree labels make network and policy rules short and stable.
- Small and simple: it adds a tree and copies selected objects.
- Works with ordinary Kubernetes objects and tools.
Disadvantages
- Namespaces still share one control plane and one cluster.
- No built-in shared quota for a subtree in current features.
- Extra moving parts: controller, webhooks and custom resources.
- Large fan-out can strain the API server.
- Check the project’s current maintenance status before adopting.
Alternatives to consider
| Option | Strength | Weakness |
|---|---|---|
| Hierarchical namespace controller | Light and focused on trees and propagation | Limited to namespaces in one cluster |
| Multi-tenancy operators that group namespaces into tenants | Tenant concept with quotas and policies built in | A different model, not a tree |
| Virtual clusters per team | Stronger separation, own API server per tenant | More resources and more to operate |
| Separate clusters per team | Strongest isolation | Highest cost and operations load |
| Own automation and policy engines | Fits your exact rules | You build and maintain it |
Rooms in one house with a shared rulebook, flats in one building with separate locks, or separate houses on separate streets. Each costs more and separates more than the last. Pick the level that matches the risk.
Choose by the trust level of your tenants. Teams inside one company that trust each other usually fit trees of namespaces. Unknown outside customers usually need stronger separation.
33A Step-by-Step Adoption Plan
Introduce the controller slowly. Each step is small, safe and easy to undo.
Check fit and status
Confirm the project’s current maintenance state, supported Kubernetes versions and the features you need. Decide whether the alternatives suit you better.
Draw the tree on paper
Keep it shallow. Name each level by what it is for. Agree on owners.
Install in a test cluster
Add the controller and the plugin. Practice creating subnamespaces, setting parents and watching copies appear.
Choose the first propagated types
Begin with Roles and Bindings. Add network rules once the team is comfortable.
Add monitoring and alerts
Alert on problem conditions, webhook failures and reconcile errors before real teams depend on it.
Pilot with one team
Move one willing team into the tree. Ask what is confusing and write it down.
Adopt existing namespaces gradually
Set parents on old namespaces one at a time, checking for name conflicts each time.
Document and review
Write a short guide for teams and review the tree every quarter.
Moving a school to a new filing system is done one cabinet at a time. You do not carry every paper out of every room on the same Monday morning.
Moving existing production namespaces under a parent before checking name conflicts and existing bindings. The first inherited copy may collide with something that production relies on.
34Where Teams Use It in Real Life
The pattern fits several kinds of organizations, each for a slightly different reason.
Internal developer platforms
Platform teams offer each product team a root namespace and let them create their own environments. The team gets speed and the platform team gets consistency.
Large enterprises with many departments
Each department receives a branch. Audit teams read access by branch, and compliance rules sit at the department level.
Training and workshop clusters
Each learner receives a subnamespace under a class namespace with the same baseline rules. At the end, the class anchor is removed and everything is cleaned up together.
CI preview environments
Each pull request creates a short-lived subnamespace under a CI namespace, which inherits the shared credentials and rules, and is deleted when the request closes.
Managed services and internal software vendors
A group that runs databases for others gives each customer team a subnamespace with the right access and network rules already in place.
A university gives each department a floor and lets the department arrange its own rooms. The same fire rules hang on every wall, and the building manager does not repaint a door for every student.
35Naming and Designing Your Tree
A tree is easy to build and hard to rearrange. A little thought about names and levels at the start saves months of cleaning later.
Namespace names must be unique across the whole cluster. Position in the tree does not give a name its own private space. Two teams cannot both have a namespace called dev, even if they sit in different branches. The simplest way out is to put the team or product name inside the namespace name, so that checkout-dev and search-dev can live side by side.
Rules of thumb for names
Start with the owner
Begin with the team or product, then add the purpose, such as checkout-prod. Sorted lists then group related namespaces together.
Keep it stable
Choose words that will outlive reorganizations. Product names change less often than manager names.
Keep it short
Tree labels use the ancestor name as part of a label key, and long names can hit length limits.
A school gives every locker a number with the class letter in front of it, such as 5B-17. A pupil never has to guess which hallway holds the locker, and two classes never receive the same number by mistake.
Choosing the levels
Each level in the tree should answer one question. A common three-level design asks: which organization, which team, which environment. A level that does not answer a clear question is probably unnecessary. Before you add a level, ask what rule or permission will be attached to it. If you cannot name one, leave the level out.
| Level | Question it answers | What usually lives here |
|---|---|---|
| Root | What rules apply to everyone? | Baseline network rules, auditor access, shared credentials |
| Team | Who owns this branch? | Team group bindings, permission to create anchors |
| Environment | What is this space used for? | The running workloads, stricter rules in production |
Putting environment above team, so that all production namespaces sit together. That looks tidy but gives the production owner power over every team’s production workloads, which often does not match who should be in charge.
Draw the tree and read it aloud to someone outside the platform team. If they cannot guess what each level is for, the design needs another pass.
36A Day in the Life of a Subnamespace
Let us follow one subnamespace from its birth to its removal and watch what the controller does at each step.
Morning: a developer asks for space
A developer in the payments team needs a place to test a new feature. She creates an anchor in the team namespace with the name she wants.
Seconds later: the namespace appears
The controller sees the anchor, creates the namespace, sets its parent and records it as a child of the team namespace.
Right after: rules arrive
The team’s bindings, the default-deny rule and the registry credential are copied in. The tree labels are added.
Midday: work happens
She starts test workloads and adds an allow-rule so they can talk to a test database. Her own objects are hers to change.
Afternoon: the platform team changes a rule
The credential is rotated at the top. Within moments her namespace holds the new copy, and she does nothing.
Evening: the test is done
She deletes the anchor. The controller removes the namespace and everything inside it, because it has no children of its own.
stateDiagram-v2
[*] --> Requested: anchor created
Requested --> Created: namespace made by controller
Created --> Configured: inherited objects and labels added
Configured --> InUse: team deploys workloads
InUse --> InUse: parent changes flow in
InUse --> Removing: anchor deleted
Removing --> [*]: namespace removed
Figure 9: The life of a subnamespace, from the first request to its removal.
A hotel guest checks in, receives a key and the house rules, uses the room, hears about a new Wi-Fi password through a card slid under the door, and then checks out. The room is cleaned and ready for the next guest. Nobody had to ring the manager at any step.
What is worth noticing is how few people were involved. The developer acted on her own. The platform team acted once, at the top, for the credential change. Nobody with cluster-wide rights was needed for any step in her day.
37Testing and Validating Your Setup
A setup that works on the first day can silently break on the hundredth. Regular checks keep it honest.
Testing a tree means checking that the right things travel to the right places, and that the wrong things do not. It is useful to test the positive case, where an object should arrive, and the negative case, where it should not. Negative tests find most of the dangerous mistakes, since they show leaks of access.
A short list of checks
| Check | What you expect | Why it matters |
|---|---|---|
| Create a test object in a parent | A labeled copy appears in each descendant | Proves propagation works end to end |
| Change the test object | All copies update | Proves that updates flow |
| Delete the test object | All copies disappear | Proves clean-up works |
| Create the same object in a sibling branch | It does not reach the other branch | Proves isolation between branches |
| Try to edit an inherited copy as a normal user | The change is refused | Proves read-only protection |
| Try to create a cycle | The change is refused | Proves the safety guard |
| Test a real network connection between two namespaces | Allowed or blocked as the rules say | Proves that tree labels match policies |
A school holds a fire drill every term. The drill is dull, but it shows whether the alarms work and whether the doors open. Testing the tree is a fire drill for permissions.
Make the checks routine
Run the same list after every upgrade of the controller, after every change to HNCConfiguration, and on a regular schedule such as once a week from an automated job. Keep the results somewhere the whole team can read. A test that fails loudly the day after a change is far cheaper than a leak found during an audit.
Use a dedicated test branch of the tree with throwaway namespaces. That way tests never touch real team workloads, and anyone can run them without fear.
Only testing that things arrive. A setup can pass every positive test and still leak a Secret to a branch that should never see it. Always include at least one test that proves something does not arrive.
38A Plain-Words Glossary
Keep this page nearby while you read other material about the controller. Every term is explained in everyday language.
| Term | Plain meaning | Everyday picture |
|---|---|---|
| Namespace | A named area inside a cluster where objects live | A room in a house |
| Hierarchy | Parents and children arranged as a tree | A family tree |
| Forest | The set of all trees the controller knows about | Many family trees in one town |
| Parent and child | A namespace above another, and the one below it | Grandmother and grandchild |
| Ancestor and descendant | Any namespace above or below, at any distance | All relatives up or down the line |
| Subnamespace | A child created by an anchor in its parent | A room a team adds inside its own floor |
| Anchor | An object in the parent that asks for a subnamespace | A request slip handed to the office |
| Propagation | Copying chosen objects from parent to descendants | A notice photocopied to every classroom |
| Inherited copy | An object in a child that came from an ancestor | A stamped photocopy |
| Condition | A status message that reports a problem or state | A warning light |
| Webhook | A check that runs before a change is accepted | A guard at the door |
| Reconciler | Code that compares wanted and actual state and fixes gaps | A janitor with a checklist |
| Cascading deletion | Removing a whole branch with one action | Cutting a branch with all its leaves |
A glossary works like the legend on a map. The symbols mean nothing until someone tells you what they stand for, and then the whole map becomes easy to read.
If any of these words still feels unclear, return to the chapter that introduces it. Each chapter explains one idea with a story, so that the meaning stays in your memory.
39Frequently Asked Questions
Short answers to questions that newcomers ask most often.
It is an ordinary Kubernetes namespace that has been placed in a tree, so that it can inherit selected objects and permissions from the namespaces above it.
No. Core Kubernetes namespaces are flat. The tree is added by an extra controller that you install in the cluster.
Down only. Child namespaces inherit from ancestors. Nothing in a child affects its parents.
A subnamespace is created through an anchor in the parent by someone with rights only in the parent. A full namespace is created the normal way by someone with cluster rights, and then attached to a parent. The tree treats both the same once attached.
No. Inherited copies are read-only to ordinary users. Change the original in the ancestor.
Roles and RoleBindings. Other types need to be added to the cluster-wide configuration.
Not in the current stable features. Propagating a quota object gives each child its own separate limit, so totals can add up to more than intended. Plan shared budgets with other tools.
Existing copies stay in place and applications keep running. New changes do not propagate until it returns.
No. It shares permissions and rules but does not stop network traffic or workloads from affecting each other. Use network policies, Pod security rules and, where needed, separate clusters.
No. Namespace names are unique across the entire cluster, regardless of position in the tree.
As shallow as works. Two to four levels are common. Deeper trees are harder to understand and audit.
Project activity changes over time. Check the repository and release notes for the current state, supported Kubernetes versions and any announced changes before you plan on it.
40Summary and Key Takeaways
A tree of namespaces turns repeated manual work into a rule you write once.
We began with a flat list of namespaces and the headaches it creates: copied permissions, forgotten rules, endless requests. We then saw how the controller adds parents and children, lets teams create subnamespaces with anchors, and copies chosen objects downward while keeping them read-only. We looked at the controller’s inner workings, its conditions and its safety limits, then at real examples, risks and a careful path to adoption.
- Core Kubernetes namespaces are flat. The hierarchical namespace controller adds parent and child relationships on top.
- Roles and bindings flow down the tree by default. Other object types flow only when you list them in the cluster-wide configuration.
- Inherited copies are labeled and read-only. Always change the original in the owning ancestor.
- Subnamespaces let teams create their own namespaces through anchors without cluster-wide rights, and deletion has safety guards.
- Tree labels let network and policy rules select a whole subtree by one label.
- The tree shares rules and permissions, but it does not isolate workloads. Combine it with network policies, Pod security rules and quotas.
- Verify the project’s current maintenance status and supported features before adopting, and roll out slowly with monitoring on conditions and the webhook.