Rate Limiting
Envoy allows configuration of how many requests / second a service can field. This is useful in preventing DDOS attacks and otherwise making sure your server's resources aren't easily overrun.
Unlike circuit breakers which are configured on each cluster, rate limiting is configured across multiple listeners or clusters. Overall, circuit breakers are good for avoiding cascading failures of a bad downstream host. However, when:
[A] large number of hosts are forwarding to a small number of hosts and the average request latency is low (e.g., connections/requests to a database server). If the target hosts become backed up, the downstream hosts will overwhelm the upstream cluster. In this scenario it is extremely difficult to configure a tight enough circuit breaking limit on each downstream host such that the system will operate normally during typical request patterns but still prevent cascading failure when the system starts to fail. Global rate limiting is a good solution for this case.
This is because rate limiting sets a global number of requests / second across a service, which is independent of the number of instances configured for a cluster. This guide shows how to configure rate limiting on the edge node in a Grey Matter deployment to an
edge namespace.Prerequisites
- 1.
- 2.
- 3.
kubectlorocsetup with access to the cluster
Steps
1. Deploy Ratelimit Service
Rate limiting relies on an external service to regulate and calculate the current number of requests / second. For this example we use envoy proxy's open source rate limit service which is based on Lyft's original rate limiting service. This blog post is a good example of this architecture.
Begin by creating a deployment for the rate limit service by applying these configs to kubernetes. Note for all these examples I use
edge as the namespace – this may differ for your deployment. Additionally, be sure to update the Redis password with your own:1
---
2
apiVersion: apps/v1
3
kind: Deployment
4
metadata:
5
labels:
6
app: ratelimit
7
name: ratelimit
8
namespace: default
9
spec:
10
replicas: 1
11
selector:
12
matchLabels:
13
app: ratelimit
14
template:
15
metadata:
16
labels:
17
app: ratelimit
18
spec:
19
serviceAccountName: default
20
containers:
21
- name: ratelimit
22
image: "envoyproxy/ratelimit:v1.4.0"
23
imagePullPolicy: IfNotPresent
24
env:
25
- name: USE_STATSD
26
value: "false"
27
- name: LOG_LEVEL
28
value: "debug"
29
- name: REDIS_SOCKET_TYPE
30
value: "tcp"
31
- name: REDIS_URL
32
value: "redis.default.svc:6379"
33
- name: RUNTIME_ROOT
34
value: "/"
35
- name: RUNTIME_SUBDIRECTORY
36
value: "ratelimit"
37
- name: REDIS_AUTH
38
valueFrom:
39
secretKeyRef:
40
name: redis-password
41
key: password
42
command: ["/bin/sh","-c"]
43
args: ["mkdir -p /ratelimit/config && cp /data/ratelimit/config/config.yaml /ratelimit/config/config.yaml && cat /ratelimit/config/config.yaml && /bin/ratelimit"]
44
ports:
45
- name: server
46
containerPort: 8081
47
- name: debug
48
containerPort: 6070
49
volumeMounts:
50
- name: ratelimit-config
51
mountPath: /data/ratelimit/config
52
readOnly: true
53
volumes:
54
- name: ratelimit-config
55
configMap:
56
name: ratelimit
57
---
58
kind: Service
59
apiVersion: v1
60
metadata:
61
name: ratelimit
62
labels:
63
app: ratelimit
64
spec:
65
ports:
66
- name: server
67
port: 8081
68
protocol: TCP
69
targetPort: 8081
70
- name: debug
71
port: 6070
72
protocol: TCP
73
targetPort: 6070
74
selector:
75
app: ratelimit
76
sessionAffinity: None
77
type: ClusterIP
78
---
79
apiVersion: v1
80
kind: ConfigMap
81
metadata:
82
name: ratelimit
83
namespace: default
84
data:
85
config.yaml: |-
86
---
87
domain: edge
88
descriptors:
89
- key: path
90
value: "/"
91
rate_limit:
92
unit: second
93
requests_per_unit: 1
Copied!
When applied with
kubectl, this service will serve as a central hub to limit the number of requests coming from the domain "edge" to 100 requests / second.2. Configure Grey Matter Sidecar To Use Ratelimit Filter
In order to configure rate limiting on a sidecar, we need to configure the sidecar with an additional cluster that points to the rate limit service. As a convenience, Grey Matter Sidecar allows for defining a cluster using environment variables. The following sample environment variables define a cluster
ratelimit that points to our deployed ratelimit service:1
...
2
tcp_cluster:
3
type: 'value'
4
value: 'ratelimit'
5
tcp_host:
6
type: 'value'
7
value: 'ratelimit.default.svc.cluster.local'
8
tcp_port:
9
type: 'value'
10
value: '8081'
11
...
Copied!
Make sure that you can see this cluster on whichever sidecar you have configured. It's available on
localhost:8001/clusters under the ratelimit cluster.Now let's update the listener config for the sidecar we've configured. Edit the listener in the Grey Matter CLI and add the following attributes:
1
...
2
"active_network_filters": [
3
"envoy.rate_limit"
4
],
5
"network_filters": {
6
"envoy_rate_limit": {
7
"stat_prefix": "edge",
8
"domain": "edge",
9
"failure_mode_deny": true,
10
"descriptors": [
11
{
12
"entries": [
13
{
14
"key": "path",
15
"value": "/"
16
}
17
]
18
}
19
],
20
"rate_limit_service": {
21
"grpc_service": {
22
"envoy_grpc": {
23
"cluster_name": "ratelimit"
24
}
25
}
26
}
27
}
28
},
29
...
Copied!
You should see that requests to the
ratelimit cluster succeed on the localhost:8001/clusters endpoint on the sidecar. You should also see logs in the ratelimit pod on every request since the logs are set to debug. If there are ever more than 1 request / second, all requests are blocked until the 1 request / second threshold is released.3. Trust but Verify
The ratelimit we set can be tested by changing the number of requests / second to 1 and spamming the sidecar. Make a series of curl requests to the edge. The response code should be 429, although if TLS is enabled this often is just an SSL error. You should see also something like the following logs in the ratelimit pod:
1
time="2020-09-03T21:54:40Z" level=debug msg="returning normal response"
2
time="2020-09-03T21:54:40Z" level=debug msg="cache key: edge_path_/_1599170080 current: 1"
3
time="2020-09-03T21:54:40Z" level=debug msg="returning normal response"
4
time="2020-09-03T21:54:40Z" level=debug msg="starting get limit lookup"
5
time="2020-09-03T21:54:40Z" level=debug msg="looking up key: path_/"
6
time="2020-09-03T21:54:40Z" level=debug msg="found rate limit: path_/"
7
time="2020-09-03T21:54:40Z" level=debug msg="starting cache lookup"
8
time="2020-09-03T21:54:40Z" level=debug msg="looking up cache key: edge_path_/_1599170080"
9
time="2020-09-03T21:54:40Z" level=debug msg="cache key: edge_path_/_1599170080 current: 3"
Copied!
This shows that the ratelimit service is registering calls from edge and is checking the requests / second against the limit of a maximum of 1 request / second. If you wish to raise the limit in a production environment, 100 requests / second is a good starting point.
References
Last modified 1yr ago