Light Dark Auto

TLS

Configure TLS

Transportation Security Layer (TLS) are a set of protocols designed to ensure secure communication across a network. Envoy's and Grey Matter's filters allow for diverse types of TLS connections throughout the mesh. This document explains the different types of supported TLS and technical steps on how to configure them within Grey Matter.

One-Way TLS

In One-Way TLS, only one certificate trust file is evaluated: either the client's or the server's. This can be configured with different values in the cluster vs. domain objects. Crucially, the domain configures TLS parameters (e.g. certificates, ssl types) for incoming requests whereas the cluster object controls TLS setting used for outward requests leaving the proxy.

  • To validate client certificates, add the "trust_file" parameter to the domain object.
  • To validate server certificates, add the "trust_file" parameter to the cluster object.

mTLS / Re-Encrypt

Mutual TLS is a type of TLS communication where the server and client both verify each other's certificates during the TLS handshake, hence it is more secure than One-Way TLS. In its current implementation in Envoy, this is a form of re-encryption where the incoming request context is terminated and the outgoing request is formed from a new certificate pair.

To add mTLS to your mesh, the following configuration needs to applied to each domain (incoming requests to the proxy)

{
  ...,
  "ssl_config": {
      "protocols": [
        "TLSv1.2"
      ],
    "require_client_certs": true,
    "trust_file": "/etc/proxy/tls/sidecar/ca.crt",
    "cert_key_pairs": [
      {
        "certificate_path": "/etc/proxy/tls/sidecar/server.crt",
        "key_path": "/etc/proxy/tls/sidecar/server.key"
      }
    ]
    "crl": {
      "filename": "/etc/proxy/tls/sidecar/ca.crl"
    },
  },
  "force_https": true,
}

and cluster (outgoing requests from the proxy):

{
  ...,
  "ssl_config": {
      "protocols": [
        "TLSv1.2"
      ],
    "require_client_certs": true,
    "trust_file": "/etc/proxy/tls/sidecar/ca.crt",
    "cert_key_pairs": [
      {
        "certificate_path": "/etc/proxy/tls/sidecar/server.crt",
        "key_path": "/etc/proxy/tls/sidecar/server.key"
      }
    ],
    "crl": {
      "filename": "/etc/proxy/tls/sidecar/ca.crl"
    },
  },
  "require_tls": true,
}

This forces each gm-proxy instance to serve HTTPS and validate incoming requests from a given certificate pair. For more information on TLS configuration parameters, see API documentation for cluster and domain TLS configs.

Passthrough

SSL Passthrough differs from other methods for implementing TLS. Instead of handling encryption / decrypting at the proxy, the TLS configuration is forwarded "as is." No layer 7 actions are taken such as access control, redirects, blocking, or cookie / session management. The responsibility for this logic is deferred downstream to the workload.

At this writing, envoy does not support generic TLS passthrough. It does support TCP and gRPC bridges to connect directly to TCP or gRPC servers directly. For more on this see our Envoy Network Filters documentation.

Changing Protocols

A common use case of sidecars is to change the incoming protocol to something that can be used by the backed service. This gives assurance that upstream requests will be of a compatible type certain with each service's supported protocols.

Origination

Origination occurs when a http context is "upgraded" to https. To enable origination, we first need to allow incoming http requests checks to pass validation. This can be done by updating the TLS options in the domain to look like the following:

{
  ...,
  "ssl_config": {},
  "force_https": false
}

This removes TLS checks on incoming requests. Then, we need to configure outgoing requests to use a TLS context. This can be done by updating the cluster to look like the following:

{
  ...,
  "ssl_config": {
      "protocols": [
        "TLSv1.2"
      ],
    "require_client_certs": true,
    "trust_file": "/etc/proxy/tls/sidecar/ca.crt",
    "cert_key_pairs": [
      {
        "certificate_path": "/etc/proxy/tls/sidecar/server.crt",
        "key_path": "/etc/proxy/tls/sidecar/server.key"
      }
    ]
  },
  "require_tls": true,
}

This ensures that outgoing requests from the proxy will use a TLS context from the provided certificates.

Termination

Termination occurs when a https context is "downgraded" to http, the opposite of origination. To enable termination, we need to first ensure that incoming requests are validated with a TLS security context. This is done by updating the domain to use the following TLS configs:

{
  ...,
  "ssl_config": {
      "protocols": [
        "TLSv1.2"
      ],
    "require_client_certs": true,
    "trust_file": "/etc/proxy/tls/sidecar/ca.crt",
    "cert_key_pairs": [
      {
        "certificate_path": "/etc/proxy/tls/sidecar/server.crt",
        "key_path": "/etc/proxy/tls/sidecar/server.key"
      }
    ]
  },
  "force_https": true,
}

Now, we need to tell each gm-proxy instance to use plain http for outgoing requests. This is done by removing TLS configs from the cluster:

{
  ...,
  "ssl_config": {},
  "require_tls": false,
}

Incoming requests must now be TLS and outgoing requests from the proxy will be in plain http.

X-Forwarded Proto

When making requests to services which change protocols, the header x-forwarded-proto should be set explicitly. Otherwise, if set implicitly, it will default to the original client protocol. When the upstream service receives the request, it will examine the x-forwarded-proto value and think the request is not the right protocol. The request will be rejected with a 301 redirect. Here is an example of this workflow:

  1. Instantiate a listener on localhost:80 using TLS origination
  2. Perform a request curl http://localhost:80. By default, the header is set as x-forwarded-proto: http
  3. Sidecar receives the request, converts it to an SSL request, finds the correct cluster, and proxies the request.
  4. The upstream service receives the request, performs the SSL handshake, reads the request, sees that x-forward-proto: http. The service is confused because it doesn't handle http and issues a 301 redirect back.

There are two ways of ensuring that the right x-forwarded-proto header is set for outgoing requests:

  • Specify x-forwarded-proto in each incoming request. This is not recommended for deployments or as a long term solution.
  • Add x-forwarded-proto as a custom header in the domain. Here is an example of setting this for origination:
{
  ...,
  "custom_headers": [
    {
      "key": "x-forwarded-proto",
      "value": "https"
    }
  ]
}

Now, every outgoing request using this domain will include the header "x-forwarded-proto: https".

Certificate Revocation

Client certificate revocation can also be enforced through sidecar configuration by specifying any number of PEM-encoded certificate revocation lists. These may be appended directly to the provided trust_file or added separately to the crl parameter of a SSL configuration object.

It is important to note that if a CRL is provided for any certificate authority in a trust chain, a CRL must be provided for all certificate authorities in that chain in order for verification to be enforced for revoked and unrevoked certificates.

For more information on CRL configuration, see documentation for cluster and domain SSL configs.

Multiple Listeners

Each type of TLS listed here can be combined with multiple listeners on one sidecar. For example, a sidecar handling both plaintext and encrypted traffic has two listeners. Each listener uses a different domain to control how to handle incoming requests:

  • listener:localhost:80 domain: configured to accept http requests
  • listener:localhost:443 domain: configured to accept https requests

After an incoming request is validated by the corresponding domain, it is routed to the corresponding cluster, using the cluster's specified TLS configuration.

Together, adding different types listeners and specifying the protocol for each cluster allows for powerful and dynamic configuration of TLS for each service (i.e. mTLS, One-Way TLS, origination, etc).