In greymatter, when requests must flow to non-identical servers, we can rely on traffic shaping to manage request flow into two or more upstreams. These rules are user-defined and can be fall into two categories: split or shadow/mirror.
GSL Configuration
traffic_options may only be defined in the context of an upstream.
All traffic_options declarations must be unified with one of two potential GSL traffic manipulation objects:
// Shadowing/mirroring traffic between multiple upstreams:
traffic_options: gsl.#ShadowTraffic & {
weight: integer
}
// Splitting traffic between multiple upstreams:
traffic_options: gsl.#SplitTraffic & {
weight: integer
}
Weights
Traffic shaping controls the percentage of traffic that goes to each upstream through the weight attribute.
When weights are configured, traffic is calculated for each upstream by taking weight / (sum all weights). For example, assigning weights to 1,2,3 and clusters a,b,c respectively, would route traffic to cluster a 17% of the time (1/6), cluster b 33% of the time (2/6) and cluster c 50% of the time (3/6). Assigning the weights 2,4,6 yields the same result.
Ex: With the weights updated to 9 and 1, fakeservice will receive 90% of traffic where version fakeservice2 will receive 10%. Over time, if version 2 performs well, we can increment the amount of traffic it receives until it fields 100% of requests and we can remove fakeservice. Conversely, if version 2 performs poorly, traffic can be safely diverted from the failing cluster until the issue is resolved.
Canary or Blue/Green Deployments
A common use case is to ramp up or ramp down a new release of a service. Consider a stable release of a fakeservice. If a newly optimized version was released: fakeservice2, it would be useful to slowly ramp up the use of version 2 without a disruption of the current user base.
In the below example we have a data-plane proxy with a route defined for incoming local traffic. We also specify a secondary upstream called "fakeservice2". In this deployment we can see that all incoming traffic will be split equally between both the upstream "local" and "fakeservice2". (The weight calculation for each service is 1 / (1 + 1) = 50%.) Note that a traffic_options setting is required for both upstreams.
routes: {
"/": {
upstreams: {
"local": {
instances: [
{
host: "127.0.0.1"
port: 9090
},
],
traffic_options: gsl.#SplitTraffic & {
weight: 1
}
}
"fakeservice2": {
namespace: "team-b"
traffic_options: gsl.#SplitTraffic & {
weight: 1
}
}
}
}
}
Alternatively, if mirroring traffic was the desired outcome, swapping the gsl.#SplitTraffic declaration for a gsl.#ShadowTraffic and adjusting the weights would yield request mirroring.
"fakeservice2": {
namespace: "team-b"
traffic_options: gsl.#ShadowTraffic & {
weight: 1
}
}