Observables

Search…
Observables
Observables are live event taps that can be configured at any Sidecar in the Fabric mesh. By taking metadata from the request and response at each point throughout the mesh, Grey Matter can perform high-level monitoring of user experience, create audit trails for service access, or any other required analysis.
Observable Content
Field
Description
​
id
A GUID, unique for each event
​
eventChain
An optional chain of GUIDs
​
schemaVersion
The version of the Observable schema in use structure
​
originatorTokens
Identifiers (e.g. USER_DN) for originators of this event
​
topic
Human readable string to identify/indicate what this event is about.  E.g. service-a-prod, dev-west, greymatter-core
​
timestamp
Unix timestamp of when the observable was produced according to the host system
​
xForwardedForIp
IP address of the original requester
​
systemIp
IP address of the system originating the event
​
action
Action being performed on the service
create, read, update, delete, undelete
payload
Request/Response information (see below)
​
payload.isSuccessful
true if service responded with a 2xx code
​
payload.request
Information about the initial request
​
payload.requestEndpoint
The URI path that was invoked.  E.g. /admin, /services/catalog/1.0/, /apps
​
payload.request.headers
all headers sent along with the request
​
payload.request.body
(optional) The full body sent with the request
​
payload.response
Information about the returned response
​
payload.response.code
returned HTTP code
​
payload.response.headers
all headers sent back to the user
​
payload.response.body
(optional) The full body sent back to the user
​
Example Dashboard
The creation and emission of Observables is core to Grey Matter, but the aggregation and analysis is left to third party tools like Kibana, Splunk, or custom aggregation/analysis tools. An example dashboard showing the geo-location of requests, response bodies, etc is shown below.
Observables Kibana Dashboard
Base Payload
Shown below is the base payload that is emitted for every Observable event across the Sidecars. In addition to the metadata below, the request/response bodies can also be emitted for each transaction. However, due to the high data volume this can incur on the supporting infrastructure, this should be turned on only for systems that can support it or for Sidecars/routes in which the data volumes are acceptable.
1
{
2
  "eventId": "96536082-7077-11ea-89c3-8e1665400fd4",
3
  "eventChain": [
4
    "96536082-7077-11ea-89c3-8e1665400fd4"
5
  ],
6
  "schemaVersion": "1.0",
7
  "originatorToken": [
8
    "CN=Acert,OU=Engineering,O=Decipher Technology Studios,L=Alexandria,ST=Virginia,C=US"
9
  ],
10
  "eventType": "edge",
11
  "timestamp": 1585346951,
12
  "xForwardedForIp": "192.168.48.196",
13
  "systemIp": "192.168.11.69",
14
  "action": "GET",
15
  "payload": {
16
    "isSuccessful": false,
17
    "request": {
18
      "endpoint": "/services/catalog/1.0/summary",
19
      "headers": {
20
        ":authority": "aeb8ee2746fcd11ea84dd123e1026b56-1506059117.us-east-1.elb.amazonaws.com:80",
21
        ":method": "GET",
22
        ":path": "/static/js/runtime~main.4e755ff0.js",
23
        "referer": "https://aeb8ee2746fcd11ea84dd123e1026b56-1506059117.us-east-1.elb.amazonaws.com:80/",
24
        "ssl_client_s_dn": "CN=Acert,OU=Engineering,O=Decipher Technology Studios,L=Alexandria,ST=Virginia,C=US",
25
        "user_dn": "CN=Acert,OU=Engineering,O=Decipher Technology Studios,L=Alexandria,ST=Virginia,C=US",
26
        "x-forwarded-for": "192.168.48.196",
27
        "x-forwarded-proto": "https",
28
        "x-real-ip": "192.168.48.196",
29
        "x-request-id": "0ac0108f-cbfb-9b37-a9f7-801baf338710"
30
      }
31
    },
32
    "response": {
33
      "code": 502,
34
      "headers": {
35
        ":status": "502"
36
      }
37
    }
38
  }
39
}
Copied!
Example Enhanced Payload
Though the base payload includes only metadata that can be universally gained from all requests flowing throughout the mesh, the aggregation process can add additional information and data transformations as desired. The below payload demonstrates the ability to add geo-location information during this aggregation step.
1
{
2
  "_index": "observables-2020.03.04",
3
  "_type": "_doc",
4
  "_id": "nBlCpnABwokfVRiKGisP",
5
  "_version": 1,
6
  "_score": null,
7
  "_source": {
8
    "eventType": "fibonacci",
9
    "@version": "1",
10
    "action": "GET",
11
    "originatorToken": [
12
      "cn=jeanice.theroux, dc=berkshirehathaway, dc=com",
13
      ""
14
    ],
15
    "@timestamp": "2020-03-04T15:52:36.000Z",
16
    "timestamp": 1583337156,
17
    "eventChain": [
18
      "2aff9c91-5e30-11ea-b062-0a580a8102c8"
19
    ],
20
    "payload": {
21
      "request": {
22
        "endpoint": "/9",
23
        "headers": {
24
          "cache-control": "no-cache",
25
          "cookie": "OauthExpires=1583397155; OauthSignature=RBYj1AO52AaAOMC1ZSmbOmp7WEk; OauthUserDN=cn%3Djeanice.theroux%2C+dc%3Dberkshirehathaway%2C+dc%3Dcom",
26
          "x-gm-domain": "*:9080",
27
          "user-agent": "PostmanRuntime/7.16.2",
28
          "x-gm-shared-rules": "fibonacci",
29
          "content-length": "0",
30
          "user_dn": "cn=jeanice.theroux, dc=berkshirehathaway, dc=com",
31
          "x-forwarded-port": "443",
32
          "external_sys_dn": "",
33
          "x-gm-rule": "DEFAULT",
34
          ":method": "GET",
35
          "x-forwarded-host": "oauth.demonstration.deciphernow.com",
36
          ":authority": "oauth.demonstration.deciphernow.com",
37
          "forwarded": "for=3.10.150.42;host=oauth.demonstration.deciphernow.com;proto=https;proto-version=",
38
          "x-gm-route": "edge-oauth.fibonacci.route",
39
          "accept": "*/*",
40
          "x-forwarded-for": "3.10.150.42",
41
          ":path": "/9",
42
          "postman-token": "acce6a87-1e2f-4e64-8362-cd388b49b890",
43
          "x-request-id": "d47d7f62-6eeb-414f-963b-3c43abc26f19",
44
          "accept-encoding": "gzip, deflate",
45
          "x-envoy-original-path": "/services/fibonacci/latest/9",
46
          "x-forwarded-proto": "https"
47
        }
48
      },
49
      "response": {
50
        "body": "34\n",
51
        "headers": {
52
          "content-type": "text/plain; charset=utf-8",
53
          ":status": "200",
54
          "date": "Wed, 04 Mar 2020 15:52:36 GMT",
55
          "content-length": "3",
56
          "x-envoy-upstream-service-time": "0"
57
        },
58
        "code": 200
59
      },
60
      "isSuccessful": true
61
    },
62
    "xForwardedForIp": "3.10.150.42",
63
    "geoip": {
64
      "country_name": "United Kingdom",
65
      "region_name": "England",
66
      "ip": "3.10.150.42",
67
      "country_code2": "GB",
68
      "location": {
69
        "lon": -0.093,
70
        "lat": 51.5164
71
      },
72
      "latitude": 51.5164,
73
      "city_name": "London",
74
      "country_code3": "GB",
75
      "timezone": "Europe/London",
76
      "region_code": "ENG",
77
      "longitude": -0.093,
78
      "continent_code": "EU",
79
      "postal_code": "EC2V"
80
    },
81
    "schemaVersion": "1.0",
82
    "systemIp": "10.129.2.200",
83
    "eventId": "2aff9c91-5e30-11ea-b062-0a580a8102c8"
84
  },
85
  "fields": {
86
    "@timestamp": [
87
      "2020-03-04T15:52:36.000Z"
88
    ]
89
  },
90
  "sort": [
91
    1583337156000
92
  ]
93
}
Copied!
Usage - Previous
Telemetry
Historical Metrics
Last modified 1yr ago
Export as PDF
Copy link
Contents
Observable Content
Example Dashboard
Base Payload
Example Enhanced Payload