RD-12747 - document the secret scrubbing behavior

README.md

@@ -117,14 +117,73 @@ class MyFunction implements RequestHandler<String, String> {
     }
 ```
 
-### Support Java 11 and Above 
+### Support Java 11 and Above
 
 Add the environment variable `JAVA_TOOL_OPTIONS` to your Lambda functions and set it to
 `-Djdk.attach.allowAttachSelf=true` in addition to the manual code mentioned above.
 
 ### Supported Instrumentation Libraries
 
-- Aws SDK V1 
+- Aws SDK V1
 - Aws SDK V2
 - Apache HTTP Client
-- Apache Kafka
+- Apache Kafka
+
+### Secret scrubbing
+
+The tracer will automatically scrub values for keys in payload objects such as HTTP request / response body,  Lambda events, return value etc. that match (case-sensitively) the following regex patterns at any depth:
+- `.*pass.*`
+- `.*key.*`
+- `.*secret.*`
+- `.*credential.*`
+- `.*passphrase.*`
+- `SessionToken`
+- `x-amz-security-token`
+- `Signature`
+- `Authorization`
+This behavior can be overridden by setting the `LUMIGO_SECRET_MASKING_REGEX` environment variable to a JSON array of regex patterns to match, e.g.: `[".+top.secret.+", ".+pazzword.+"]`.
+
+#### Notes
+1. providing a bad regex pattern (e.g., invalid JSON string) will result in an error and fallback to the default patterns.
+2. Only values that are strings are redacted - objects, numbers etc. will stay intact even though their keys match the patterns.
+
+#### Escaping special characters
+When the patterns contain special characters such as double quotes (`"`) or backslashes (`\`), those should be escaped with a backslash (`\`).
+
+For example, the pattern for keys with whitespaces and quotes like `"key\s+spaced"` becomes `\"key\\\\s+spaced\"`. That's because each double quotes turns into `\"`, and the `\s+` expression requires the backslash character to be escaped both in the string context (`\s+` => `\\s+`) and again in a JSON string context (`\\s+` => `\\\\s+`). When placed into the env-var as an array-item, this becomes:
+```
+["\\"key\\\\s+spaced\\""]
+```
+
+#### Examples
+
+`LUMIGO_SECRET_MASKING_REGEX` set to `[".*top\\\\s+secret.*", ".*password.*"]` for a payload object like:
+```json
+{
+    "top    secret": {
+        "password": "123456"
+    },
+    "top secret object": {
+        "this will not be scrubbed since the parent is an object": "123456"
+    },
+    "password": "123456",
+    "top secret:": "123456",
+    "not so secret": "value",
+    "ToP sEcReT": "is case sensitive"
+}
+```
+will result in the following payload shown in the Lumigo platform:
+```json
+{
+    "top    secret": {
+        "password": "****"
+    },
+    "top secret object": {
+        "this will not be scrubbed since the parent is an object": "123456"
+    },
+    "password": "****",
+    "top secret:": "****",
+    "not so secret": "value",
+    "ToP sEcReT": "is case sensitive"
+}
+```