Install on Kubernetes

This guide covers the necessary steps to install and configure the Grey Matter service mesh on an AWS EKS cluster. While these instructions should support all version of AWS EKS, they have been tested and confirmed against versions 1.15, 1.16, 1.17, 1.18.

Warning: The Grey Matter mesh installed at the end of this guide is not intended for production use. Contact Grey Matter Customer Support for more information on a Production deployment.


  1. helm v3

  2. eksctl

  3. kubectl

  4. Grey Matter credentials requested via Grey Matter Support


1. Install Kubernetes

NOTE: If you already have a Kubernetes cluster up and running, move to Step 2. Just verify you can connect to the cluster with a command like kubectl get nodes.

For this deployment, we'll use Amazon EKS to automatically provision a Kubernetes cluster for us. eksctl will use your preconfigured default AWS credentials to create master and worker nodes to our specifications, and configure kubectl so we can manipulate the cluster.

The regions, node type/size and other settings can be tuned to your use case. The values given are basic settings. Run, the following command:

eksctl create cluster \
--name production \
--version 1.17 \
--nodegroup-name workers \
--node-type m5.2xlarge \
--nodes=2 \
--node-ami auto \
--region us-east-1 \
--zones us-east-1a,us-east-1b \
--profile default

After 10 - 15 minutes, your cluster should be ready. You can test that your eksctl configuration is correct by running:

eksctl get cluster --region us-east-1 --profile default
eksctl get nodegroup --region us-east-1 --profile default --cluster production

2. Set up Credentials

The credentials identified in the prerequisite steps are used to create a Kubernetes Image Pull Secret that grants access to the Grey Matter Docker images in the Grey Matter Nexus Repository.

If you do not have credentials yet, please contact Grey Matter support.

3. Get the Grey Matter Helm Charts

The Grey Matter Helm Charts are available from our GitHub helm-charts repository. Run the following command to add our helm charts repository.

helm repo add greymatter
helm repo update

4. Set up Secrets

Using your credentials, create the following file and save it as credentials.yaml.

- registry:
email: <username>
username: <username>
password: <password>

5. Generate Configurations

Run the following to download the base configuration file:


This file is where you can specify any custom configurations for the installation. The file downloaded as is will install Grey Matter with all default values. If you wish to modify the defaults, make changes to any existing values in global.yaml.

Some configuration changes will change the installation process.

If you set global.spire.enabled to false, skip the server and agent release installations in step 6.

If you set to true, you will need to add the data installation to part 3 of step 6, helm install data greymatter/data -f global.yaml --set=global.environment=eks --set=global.waiter.service_account.create=false.

To set more complex configurations like image versions, service environment variables, etc., check out the values files in each of the Grey Matter helm charts. Any additional configurations you wish to set can be added to the global.yaml file with the same directory structure as found in the <chart>/values.yaml file.

For example, to set the version of the Grey Matter proxy for the edge proxy, just simply replace the version as indicated below:

version: '<desired-version>'

6. Install

Grey Matter is made up of a handful of components, each handling different pieces of the overall platform. Once you have set up your credentials.yaml and global.yaml files, run the following steps in order:

  1. Install the necessary secrets

    Grey Matter requires several Kubernetes secrets. The necessary secrets have been extracted into a single helm-chart for ease of installation.

    helm install secrets greymatter/secrets --version {{ book.helm_versions.secrets }} -f credentials.yaml -f global.yaml
  2. Install SPIRE server and agent

    This guide will use SPIRE to issue certificates to enable mTLS through the mesh. These commands will install the SPIRE server and agents into the spire namespace.

    helm install spire-server greymatter/server --version {{ book.helm_versions.spire_server }} -f global.yaml

    Before installing the agent, watch the server pod come up:

    kubectl get pods -n spire -w

    Wait until the server pod is 2/2:

    server-0 2/2 Running 1 3h54m

    Then, install the spire agent:

    helm install spire-agent greymatter/agent --version {{ book.helm_versions.spire_agent }} -f global.yaml

    Verify SPIRE installation

    kubectl get pods -n spire -w

    The SPIRE agent runs as a DaemonSet, so the number of Agent pods is directly related to how many nodes are in your cluster.

    agent-5d7q4 1/1 Running 0 3h54m
    agent-8c9lq 1/1 Running 0 3h54m
    agent-s2svz 1/1 Running 0 3h54m
    server-0 2/2 Running 1 3h54m
  3. Install Grey Matter Charts

    At this point, we're ready to install Grey Matter. Grey Matter is installed through a series of Helm Charts that install the different components of Grey Matter. You'll notice we're also setting a few values on the command line. These could be updated in the global.yaml file, but we wanted to call them to your attention here.

    • global.environment: This is set to eks to drive EKS specific configurations

    • edge.ingress.type: This is set to LoadBalancer to update the Edge service to modify it to a type of LoadBalancer.

    • global.waiter.service_account.create: This is set to false to prevent the Sense helm-chart from attempting to create the waiter service account.

    helm install fabric greymatter/fabric --version {{ book.helm_versions.fabric }} -f global.yaml --set=global.environment=eks
    helm install edge greymatter/edge --version {{ book.helm_versions.edge }} -f global.yaml --set=global.environment=eks --set=edge.ingress.type=LoadBalancer
    helm install sense greymatter/sense --version {{ book.helm_versions.sense }} -f global.yaml --set=global.environment=eks --set=global.waiter.service_account.create=false

    Notice in the edge installation we are setting --set=edge.ingress.type=LoadBalancer, this value sets the service type for edge. The default is ClusterIP. In this example we want an AWS ELB to be created automatically for edge ingress (see below), thus we are setting it to LoadBalancer. See the Kubernetes publishing services docs for guidance on what this value should be in your specific installation.

    If you receive Error: could not find tiller after running the above helm commands, then you're running an older version of Helm and must install Helm v3. If you need to manage multiple versions of Helm, we highly recommend using helmenv to easily switch between versions.

    While these are being installed, you can use the kubectl command to check if everything is running. When all pods are Running or Completed, the install is finished and Grey Matter is ready to go.

    kubectl get pods

    The running output will look like the following:

    catalog-75d4c66477-h9nnr 2/2 Running 0 49s
    catalog-init-v778j 0/1 Completed 0 49s
    control-78f8ccf5f6-krlbd 1/1 Running 0 111s
    control-api-0 2/2 Running 0 111s
    control-api-init-pblsb 0/1 Completed 0 111s
    dashboard-dd95bddbc-hrzm5 2/2 Running 0 49s
    edge-d56fd6795-qdctl 1/1 Running 0 63s
    jwt-redis-6b84846ffc-fkpjg 1/1 Running 0 111s
    jwt-security-7bc8bfb9f-jbz6l 2/2 Running 0 111s
    mesh-redis-0 1/1 Running 0 111s
    postgres-slo-0 1/1 Running 0 49s
    prometheus-0 2/2 Running 0 49s
    slo-d98698db9-p7ksz 2/2 Running 0 49s

7. Accessing the application

NOTE: For easy set up, access to this deployment was provisioned with the quickstart SSL certificates. For access to the application via the public access point, download our quickstart certificate from our helm charts and import it into your browser of choice - the password is password.

An Amazon ELB will be created automatically when we specified the flags --set=global.environment=eks and --set=edge.ingress.type=LoadBalancer during installation. The ELB is accessible through the randomly created URL attached to the edge service:

kubectl get svc edge

The output will look like the following:

edge LoadBalancer 10808:31098/TCP,8081:31216/TCP 4m

Visit the url listed under EXTERNAL-IP (e.g. in the browser to access the Grey Matter application:

Grey Matter application

Configure the Grey Matter CLI

In order to add or modify service configurations, make sure you have the greymatter CLI installed.

To configure it for this installation, replace {edge-amazon-elb-dns} with the edge service external IP from step 5, and replace <path/to/greymatter/helm-charts> with the path on your machine to the Grey Matter Helm Charts directory in the block below:

export EDITOR=vim
export GREYMATTER_API_HOST={edge-amazon-elb-dns}:10808
export GREYMATTER_API_PREFIX=/services/control-api/latest
export GREYMATTER_API_SSLCERT=<path/to/greymatter/helm-charts>/certs/quickstart.crt
export GREYMATTER_API_SSLKEY=<path/to/greymatter/helm-charts>/certs/quickstart.key
export GREYMATTER_CATALOG_HOST={edge-amazon-elb-dns}:10808
export GREYMATTER_CATALOG_PREFIX=/services/catalog/latest
export GREYMATTER_CATALOG_SSLCERT=<path/to/greymatter/helm-charts>/certs/quickstart.crt
export GREYMATTER_CATALOG_SSLKEY=<path/to/greymatter/helm-charts>/certs/quickstart.key
export GREYMATTER_CATALOG_ZONENAME=zone-default-zone

Now if you can run greymatter list cluster and greymatter list catalog_cluster and there are no errors, the CLI is properly configured.


Delete the Grey Matter Installation

helm uninstall sense edge fabric spire-agent spire-server secrets

Delete The EKS Cluster

NOTE: this deletion actually takes longer than the output would indicate to terminate all resources. Attempting to create a new cluster with the same name will fail for some time until all resources are purged from AWS.

eksctl delete cluster --name production
[] using region us-east-1
[] deleting EKS cluster "production"
[] kubeconfig has been updated
[] cleaning up LoadBalancer services
[] 2 sequential tasks: { delete nodegroup "workers", delete cluster control plane "prod" [async] }
[] will delete stack "eksctl-production-nodegroup-workers"
[] waiting for stack "eksctl-production-nodegroup-workers" to get deleted
[] will delete stack "eksctl-production-cluster"
[] all cluster resources were deleted