Light Dark Auto

Setup Spire

Introduction

Zero-trust networking is key to the security of modern cloud environments. One way greymatter enacts a zero-trust architecture is through SPIFFE/SPIRE. SPIFFE/SPIRE (SPIRE) secures microservice-to-microservice connections with mTLS using scoped x509 certificates and automatic rotation. Greymatter integrates seamlessly with SPIRE, easing the logistical and security burden of managing, minting, and rotating hundreds of certificates.

This guide will show you how to configure tenant services to utilize SPIRE for their East-West traffic connections. We will add SPIRE mTLS security policies between to the edge node and the service, from the service to the greymatter health checking subsystem, and finally from the service to another service within the mesh.

Prerequisites

  • An installed greymatter 1.8.x instance
  • An installed SPIRE instance
  • A fully initialized and deployed tenant project with a service
  • A core mesh with SPIRE mTLS enabled
  • A tenant edge with some form of TLS enabled at its ingress

1. Update the Edge Manifests

The Operator manages the SPIRE setup for services, but not for the edge itself. From a default generated edge manifest, we need to make a few additions. Open the tenant edge manifest found in the tenant project at k8s/manifest.yaml. To the spec.selector.matchlabels object add the field:

greymatter.io/workload: gm-operator.<mesh name>.<namespace>-edge

Add the exact same field to the spec.template.metadata.labels object.

In spec.template.spec add the field hostPID:

hostPID: true

Inside the sidecar element of the containers list, locate the env block. Add another environment variable named SPIRE_PATH:

- name: SPIRE_PATH
  value: /run/spire/socket/agent.sock

Still inside the sidecar block, add a volume mount:

volumeMounts:
- name: spire-socket
  mountPath: /run/spire/socket

Simply append the spire-socket block if the edge already has a volumeMount block.

Finally, add a new volume to the pod:

volumes:
- name: spire-socket
  hostPath:
    path: /run/spire/socket
    type: DirectoryOrCreate

Make sure to save the file. You will need to bring down the existing edge pod and replace it with the newly updated one:

kubectl delete -f k8s/manifest.yaml
kubectl apply -f k8s/manifest.yaml

2. Enable SPIRE between the Edge and the Service

To handle requests going to the service from outside the tenant project, we need to secure the link between the edge and the service. The #Service definition contains a struct called edge with an upstream object that controls the configuration for this network link.

Open the GSL file for the service. Inside the edge struct, locate the #Upstream block:

edge: {
    ...
    routes: {
        "<route path>": {
            ...
            upstreams: {
                "<name of the service>": {
                    ...
                    gsl.#Upstream

                }
            }
        }
    }
}

#Upstreams have a special mixin object called #SpireUpstream. Embed it inside the #Upstream object like so:

"<name of the service>": {
  gsl.#Upstream
  ...

  gsl.#SpireUpstream & {}
}

Merge the #SpireUpstream mixin with these two attributes:

gsl.#SpireUpstream & {
    #context: {
        service_name: "<edge name>"
        globals.globals
    }
    #subjects: ["<namespace-prefixed-service-name"]
}

The #context attribute injects special values that GSL uses to generate the SPIRE configurations. This format above is used only for edge blocks inside the #Service definition. We’ll see the second format in a later section.

The #subjects attribute lists the services the outbound request can reach. Since the outbound request originates from the edge, the subjects list should only include the service. The subject names must adhere to the pattern <service namespace>-<service name>.

Once you have filled in the details, save and dry-run:

greymatter sync --dry-run

Then, push your changes.

You can test the configuration change by making a request to the service. It will fail because now the ingress security policy for the service mismatches the egress security policy of the edge.

3. Enable SPIRE on the Service’s Ingress Listener

Although the edge utilizes SPIRE-populated secrets to handle proxy requests to the service, the service itself needs to accept those secrets. We need to convert the non-SPIRE ingress listener to a SPIRE one. Inside the ingress listener, embed the #SpireListener mixin:

// GSL service file
gsl.#Service & {
    ingress: (name): {
        gsl.#HTTPListener
        gsl.#SpireListener & {

        }

        ...
    }
}

A #SpireListener accepts the same attributes used for the #SpireUpstream, except the values of the subject names refer to the services allowed to connect, not the service to connect to.

gsl.#SpireListener & {
    #context: context.SpireContext
    #subjects: ["<namespace>-<edge name>"]
}

Save the file and verify correctness by running:

greymatter sync --dry-run

Push your changes. After Sync applies them, attempt to access the service through the edge. You should receive a successful response.

4. Enable Spire on Health Checks

The greymatter health check subsystem can also secure its connection with SPIRE. All configurations options for the subsystem are set in the health_options struct. Locate your health_options struct, or add one if it does not exist, and insert a #SpireUpstream:

gsl.#Service & {
    ...
    health_options: {
        spire: gsl.#SpireUpstream & {
            #context: context.SpireContext
            #subjects: ["greymatter-datastore"]
        }
    }
}

Make sure to save and dry run:

greymatter sync --dry-run

If the CLI detected no errors, then push your configuration.

5. Enable SPIRE between Services

In the previous sections, we fully configured the service to accept ingress traffic and report health statuses using SPIRE-based mTLS. If this is all your service needs, then feel free to skip to the conclusion of this document. However, if your service also requires an egress link to another service, then continue on with this section.

Enabling SPIRE between services follows the same pattern we used in the previous sections. The request originator needs to set a SPIRE security policy on the upstream between itself and its destination, and the destination needs to set a SPIRE security policy on its primary ingress listener.

Inside the source service’s egress block, embed a #SpireUpstream mixin:

egress: {
    "<egress listener name>": {
        ...
        routes: {
            "<route path>": {
                upstreams: {
                    "<upstream name>": {
                        gsl.#Upstream
                        ...
                        gsl.#SpireUpstream & {
                            #context: context.SpireContext
                            #subjects: ["<namespace>-<upstream name>"]
                        }
                    }
                }
            }
        }
    }
}

Open the GSL file of the upstream service and embed a #SpireListener in its primary ingress listener:

ingress: "<name of service>": {
    ...
    gsl.#SpireListener & {
        #context: context.SpireContext
        #subjects: ["<namespace>-<source service name>"]
    }
}

Dry run the evaluation:

greymatter sync --dry-run

If the CLI successfully validates the changes, then commit and push them. After Sync accepts them, verify your service can complete a successful egress connection.

Conclusion

Congratulations, you have made a strong first step into establishing a zero trust mesh. By using simple mixin definitions, we fully secured a service’s health checking, ingress, and egress connections with SPIRE-based mTLS. If you are interested in tweaking the defaults, see the reference document on the SPIRE mixins.