Grey Matter supports the use of SPIRE for automatic certificate management and rotation, and for workload authentication using mTLS to operate securely within the SPIFFE framework.


The SPIFFE specification defines a set of standards to securely identify software systems so that they can mutually authenticate in heterogeneous environments.

The SPIFFE Standard

Three major components make up the SPIFFE standard - the SPIFFE ID, the SPIFFE Verifiable Identity Document, and the Workload API.

A SPIFFE Identity is a valid URI of the form spiffe://<trust-domain>/<path> with trust-domain corresponding to the trust root of a system, and path allowing for a unique identification for a specific workload.

The SPIFFE Verifiable Identity Document (SVID) is comprised of a SPIFFE ID, a valid signature from a signing authority within the corresponding trust domain, and an optional public key. The signing authority must carry an SVID of its own with no path component, which forms the basis of trust for the trust domain.

Lastly, the Workload API defines the method by which workloads and processes obtain their SVIDs. Grey Matter uses SPIRE to implement the SPIFFE Workload API.

General Architecture

SPIRE provides an implementation of the SPIFFE Workload APIs that securely issues and verifies SVID's to workloads and processes. A SPIRE deployment consists of the Server and Agent(s).

Key Concepts:

  • workload: a workload is any process operating in the software system.

  • node: a node is a subset of the overall network that houses workloads

  • attestation: attestation is a process of verifying an entity's identity by gathering information about it from another trusted source


The SPIRE server acts as an identity registry for a set of workloads in it's trust domain. By default, the server acts as its own signing authority, but can be configured to use a certificate authority from a different PKI system. See configuring an upstream CA for information on how to set this up.

All workloads need to be registered with the SPIRE server in order to retrieve identities and authenticate within the system. Workloads are registered with a SPIFFE ID and a set of properties that the workload must possess in order to be issued the identity. These properties, known as selectors, are used during workload attestation to verify the workload's identity.

The Grey Matter helm charts configure the registrar service to automatically register workloads and their selectors with the SPIRE server.


A SPIRE agent is run on every node in the system where a workload runs. Once an agent is authenticated via node attestation, it keeps a cached set of SVIDs requested from the server, and presents identified workloads with their SVIDs when they request them. The agent identifies these workloads by exposing the SPIFFE Workload API on the node and attesting workload identities as they call it.

Node Attestation

The first form of attestation that the agent preforms is Node Attestation. This authenticates and verifies the agent and the node that it is running on to the server by providing information that only the node itself could be in possession of.

The node attestation plugin used in the Grey Matter helm charts is k8s_psat, which provides credentials provisioned by Kubernetes for a Projected Service Account token specific to the node that the agent is running on. Other options for node attestation can be found in the server and agent build in plugins.

When configuring a SPIRE deployment, the desired node attestation plugins are specified and configured for the server in the server's configuration file, which is specified in an argument to the server statefulset server container as args: ["-config", "/run/spire/config/server.conf"]. In our helm charts this is server.conf, and can be configured here. The desired node attestation plugins are specified and configured for the agent in the agent's config file, specified in an argument to the agent daemonset agent container as args: ["-config", "/run/spire/config/server.conf"]. In our helm charts this is agent.conf, and can be configured here.

Workload Attestation

Once an agent is verified through Node Attestation, it will request the SVID's that are registered with the server, and keep them cached to present to identified workloads when they call the Workload API to ask for them. The process of identifying the workloads is Workload Attestation.

When a workload requests it's SVID from the Workload API, the agent running on that node compares the set of properties (or selectors) that were registered in the SPIRE server with that SVID with information it retrieves from locally available authorities.

The SPIRE agents configured in the Grey Matter helm charts use workload attestation plugin k8s. The k8s workload attestor configures the agent to get the workload's pod id from its cgroup and look to the kubelet for more information about the pod, specifically namespace. The workload attestation plugins can be configured in the agent config file in the same way as the node attestation plugins here.

Workload Registrar Service

The Grey Matter helm charts use the SPIRE Kubernetes Workload Registrar to register workloads with the SPIRE server. The registrar service watches as pods are created and deleted, and automatically generates or deletes entries in the server.

The registrar can be configured to register an entry for each pod's service account, for each pod with a specific label, or for each pod with a specific annotation. This specification can be configured in the registrar services's config file, which is specified in an argument to the server statefulset registrar container as args: ["-config", "/run/spire/config/registrar.conf"]. In the helm charts, this is registrar.conf and can be configured here. To use service account based registration, remove the line pod_label entirely, to use pod annotation based registration, simple change it to pod_annotation = <desired annotation>.

The Grey Matter helm charts configures the registrar service to register entries with the pod label "greymatter.io/control", which is also used in Grey Matter Control to indicate which services to discover instances for. Each entry will then have spiffe id: spiffe://quickstart.greymatter.io/<service-name> where service-name is the value of the pod's label "greymatter.io/control.

The registration entries from the registrar service in the Grey Matter helm charts will look like the following example which for Grey Matter Control API running in the namespace fabric:

Entry ID : 973818fc-26d4-4565-b2a8-720bfe1c0135
SPIFFE ID : spiffe://quickstart.greymatter.io/control-api
Parent ID : spiffe://quickstart.greymatter.io/k8s-workload-registrar/quickstart/node
TTL : default
Selector : k8s:ns:fabric
Selector : k8s:pod-name:control-api-0

If the registrar is configured with pod_label, it a SPIFFE ID and corresponding selectors based on the pod's service account by default. In this case, the spiffe ID will look like spiffe://<trust-domain>/ns/<namespace>/sa/<default>, with path identifying the namespace and name of the service account.