openobserve/ua_regex
Ankur Srivastava 5b01947491
feat: rum/maxmind integration (#1520)
This PR implements the first draft for the Real User Monitoring.

The endpoints at the moment are:

- `/rum/v1/{org_id}/rum`
- `/rum/v1/{org_id}/replay`
- `/rum/v1/{org_id}/logs`

For now the authentication is done using `oo-api-key` which is
being passed in the query-param using `browser-sdk`.

As of now, the data is being collected in the following three streams:

- `_rumdata` for rum data
- `_rumlog` for log data
- `_sessionreplay` for session-replay data

The data is being ingested using the multi-json implementation, with a
small change where some extra data is also ingested in the json from
the headers, query-params etc. using a middleware extractor and stores
the data in `HashMap<String, String>` and gets ingested in this
multi-json.

A file containing regexes for common user-agents is also committed which
is read during the initialization time which is used to parse incoming
user-agents and browser, os etc are inserted into the incoming row of
data.

Added the endpoints for `get`, `put`, `post` for rum-tokens.

- `/api/{org_id}/organizations/rumtoken`

#### maxmind integration 
Integrated the maxmind-db data in the source. Currently the data
is injected via the middleware and the file gets uploaded every
24 hours.
The file pointer gets mutated and updated when there is a change
in the mmdb file. To efficiently do this, we also query the sha256
of our file and compare it using the .sha256 file available on the
public bucket.
2023-10-18 15:09:24 +02:00
..
regexes.yaml feat: rum/maxmind integration (#1520) 2023-10-18 15:09:24 +02:00