Organizational - Three way Cross-Account- Kubernetes only (no TF) - Threat Detection and Compliance

Use-Case explanation

Organizational setup
- Pre-Existing organizational-cloudtrail
  - Cloudtrail-SNS activated (in management account)
  - Cloudtrail-S3 is not in the management account, but in a log-archive member account
- Pre-Existing Kubernetes Cluster, where Sysdig compute will be deployed
- Sysdig features: Threat-detection and Compliance for selected org accounts. No image scanning
- Dynamic environments with accounts created and destroyed on-demand
Due to dynamic nature of customer's environment, a heavy programmatically ops tooling are used (not including Terraform) .
A summary of the required resources/permissions will be done, so they're provisioned for the Secure for Cloud features to work.

Infrastructure Solution

As a quick summary we'll need

Organizational Management Account
- Pre-Existing organizational-cloudtrail with SNS activated
Member Account - Logs Account
- Pre-Existing Cloudtrail-S3 bucket
- Create a specific role for CrossAccount S3 access
Member Account - Sysdig Compute Account
- Pre-Existing K8s cluster for cloud-connector (SFC compute)
- Create a topic for Cloudtrail-SNS-SQS wiring, from organizational cloudtrail into cloud-connector compute module
Member Account(s)
- Sysdig Compliance Role (aws:SecurityAudit policy)

Note:

All event ingestion resource (cloudtrail-sns, and cloudtrail-s3 bucket) live in same AWS_REGION AWS region. Otherwise, contact us, so we can alleviate this limitation.

We suggest to

start with Cloud-Connector module required infrastructure wiring and deployment; this will cover threat-detection side
then move on to Compliance role setup

Cloud-Connector wiring: Organizational Cloudtrail S3 + SNS - SQS

We will leverage Secure for Cloud - cloudtrail ingestion, in order to consume organizational events. For that, we will need to prepare an SQS to ingest the events, and the Cloudtrail-S3 bucket, in order to allow cross-account read.

Verify that your Organizational Cloudtrail has the Cloudtrail-SNS notification activated within same account and region.
In your organization, choose a member account as SYSDIG_ACCOUNT_ID.
Ideally, this account will gather the EKS cluster where Sysdig compute workload will be deployed, and the SQS topic to ingest Cloudtrail events.
In SYSDIG_ACCOUNT_ID, create an SQS queue (in the same region as the SNS/EKS).
- Default queue parametrization is enough.
- Subscribe the Cloudtrail-SNS topic to it.
- Due to cross-account limitations, you may need to enable SNS:Subscribe permissions on the queue first
```
{
  "Sid": "AllowCrossAccountSNSSubscription",
  "Effect": "Allow",
  "Principal": {
    "AWS": "<SUBSCRIPTION_ACTION_USER_ARN>"
  },
  "Action": "sns:Subscribe",
  "Resource": "<CLOUDTRAIL_SNS_ARN>"
}
```
- Save SYSDIG_CLOUDTRAIL_SNS_SQS_URL and SYSDIG_CLOUDTRAIL_SNS_SQS_ARN for later
Configure Cross-Account S3 access-credentials
- In the organizational account where Cloudtrail-S3 bucket is placed, create a new role to handle following permissions and save SYSDIG_S3_ACCESS_ROLE_ARN
```
{
    "Sid": "AllowSysdigReadS3",
    "Effect": "Allow",
    "Action": [
      "s3:GetObject"
    ],
    "Resource": "<CLOUDTRAIL_S3_ARN>/*"
}
```
- Now we will need to perform same permissions setup on the S3 bucket. Add following Statement to the Bucket policy
```
{
    "Sid": "AllowSysdigReadS3",
    "Effect": "Allow",
    "Principal": {
      "AWS": "<SYSDIG_S3_ACCESS_ROLE_ARN>"
    },
    "Action": "s3:GetObject",
    "Resource": "<CLOUDTRAIL_S3_ARN>/*"
 }
```
- Last step, is to allow cross-account assumeRole, but first we need to create the user/role on next steps. We will come to this IAM role afterwards.

Secure for Cloud Compute Deployment

In the SYSDIG_ACCOUNT_ID account.

Kubernetes Credentials creation
- This step is not really required if Kubernetes role binding is properly configured for the deployment, with an IAM role with required permissions listed in following points.
- Otherwise, we will create an AWS user SYSDIG_K8S_USER_ARN, with SYSDIG_K8S_ACCESS_KEY_ID and SYSDIG_K8S_SECRET_ACCESS_KEY, in order to give Kubernetes compute permissions to be able to handle S3 and SQS operations
- Secure for Cloud does not manage IAM key-rotation, but find some suggestions to rotate access-key

Permissions Setup
Check previous point K8s credentials, IAM user or Role, has the following permissions

Be able to assumeRole on SYSDIG_S3_ACCESS_ROLE_ARN.
Firstly, this must be setup on the SYSDIG_S3_ACCESS_ROLE_ARN

{
  "Effect": "Allow",
  "Principal": {
    "AWS": "<SYSDIG_K8S_USER/ROLE_ARN>"
  },
  "Action": "sts:AssumeRole"
}

As well as in the current SYSDIG_K8S_USER_ARN user

{
    "Sid": "AllowSysdigAssumeRoleCrossS3",
    "Effect": "Allow",
    "Action": [
        "sts:AssumeRole"
    ],
    "Resource": "<SYSDIG_S3_ACCESS_ROLE_ARN>"
}

For the SYSDIG_K8S_USER_ARN user, also give SQS access to process events

 {
  "Sid": "AllowSysdigProcessSQS",
  "Effect": "Allow",
  "Action": [
    "SQS:ReceiveMessage",
    "SQS:DeleteMessage"
  ],
  "Resource": "<SYSDIG_CLOUDTRAIL_SNS_SQS_ARN>"
    }

Sysdig Helm cloud-connector chart will be used with following parametrization

Locate your <SYSDIG_SECURE_ENDPOINT> and <SYSDIG_SECURE_API_TOKEN>.
Howto fetch ApiToken
Create a values.yaml file with following manifest

rules: []
scanners: []
sysdig:
  url: <SYSDIG_SECURE_ENDPOINT>
  secureAPIToken: <SYSDIG_SECURE_API_TOKEN>

aws:
  region: <AWS_REGION>
  accessKeyId: <SYSDIG_K8S_ACCESS_KEY_ID>
  secretAccessKey: <SYSDIG_K8S_SECRET_ACCESS_KEY>

# not required but would help developing product :)
telemetryDeploymentMethod: "helm_aws_k8s_org"

ingestors:
  - aws-cloudtrail-sns-sqs:
      queueURL: <SYSDIG_CLOUDTRAIL_SNS_SQS_URL>
      assumeRole: <SYSDIG_S3_ACCESS_ROLE_ARN>

We will install it

$ helm upgrade --install --create-namespace -n sysdig-cloud-connector sysdig-cloud-connector sysdig/cloud-connector -f values.yaml

Test it

$ kubectl logs -f -n sysdig-cloud-connector deployment/sysdig-cloud-connector

And if desired uninstall it

$ helm uninstall -n sysdig-cloud-connector sysdig-cloud-connector

Check that deployment logs throw no errors and can go to confirm services are working for threat detection functionality checkup.

Compliance

On each member-account where compliance wants to be checked (AWS_ACCOUNT_ID), we need to provide a role for Sysdig to be able to impersonate and perform SecurityAudit tasks.

In addition, we must make Sysdig aware of this accounts and role. We will guide you to provide, on the Sysdig Secure SaaS backend, the following resources:

a cloud-account for each member account of your organization where compliance is wanted to be checked
a task that will run "aws_foundations_bench-1.3.0" schema on previously defined accounts

Sysdig Side

Register Organization accounts on Sysdig

For each account you want to provision for the Compliance feature, we need to register it on Sysdig Secure
For Sysdig Secure backend API communication How to use development tools. Also, we have this AWS provisioning script as reference, but we will explain it here too.

$ curl "https://<SYSDIG_SECURE_ENDPOINT>/api/cloud/v2/accounts?upsert=true" \
--header "Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>" \
-X POST \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
   "accountId": "<AWS_ACCOUNT_ID>",
   "alias": "<AWS_ACCOUNT_ALIAS>",
   "provider": "aws",
   "roleAvailable": true,
   "roleName": "SysdigComplianceRole"
}'

Register Benchmark Task

Create a single task to scope the organization account ids to be assessed.
This script does not cover it, but specific regions can be scoped too. Check Benchmarks-V2 REST-API for more detail

 $ curl -s "https://<SYSDIG_SECURE_ENDPOINT>/api/benchmarks/v2/tasks" \
 --header "Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>" \
 -X POST \
 -H 'Accept: application/json' \
 -H 'Content-Type: application/json' \
 -d '{
   "name": "Sysdig Secure for Cloud (AWS) - Organization",
   "schedule": "0 3 * * *",
   "schema": "aws_foundations_bench-1.2.0",
   "scope": "aws.accountId in ('<AWS_ACCOUNT_ID_1>',...,'<AWS_ACCOUNT_ID_N>')'",
   "enabled": true
 }'

Get Sysdig Federation Trusted Identity

For later usage, fetch the Trusted Identity SYSDIG_AWS_TRUSTED_IDENTITY_ARN

$ curl -s 'https://<SYSDIG_SECURE_ENDPOINT>/api/cloud/v2/aws/trustedIdentity' \
--header 'Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>'

Response pattern:

arn:aws:iam::SYSDIG_AWS_ACCOUNT_ID:role/SYSDIG_AWS_ROLE_NAME

Get Sysdig ExternalId
- For later usage, fetch SYSDIG_AWS_EXTERNAL_ID from one of the previously registered GCP accounts. All accounts will have same id (you only need to run it once).
```
$ curl -s "https://<SYSDIG_SECURE_ENDPOINT>/api/cloud/v2/accounts/<AWS_ACCOUNT_ID>?includeExternalId=true" \
--header "Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>"
```
From the resulting payload get the externalId attribute value.

Customer's Side

Now create SysdigCompliance role on each member account using the values gathered in previous step.

Add arn:aws:iam::aws:policy/SecurityAudit AWS managed policy
Allow following Trusted-Identity

{
  "Effect": "Allow",
  "Action": "sts:AssumeRole",
  "Principal": {
    "AWS": [ "<SYSDIG_AWS_TRUSTED_IDENTITY_ARN>" ]
  },
  "Condition": {
    "StringEquals": {"sts:ExternalId": "<SYSDIG_AWS_EXTERNAL_ID>"}
  }
}

End-To-End Validation

Validate if Sysdig <-> Customer infra connection is properly made using /cloud/accounts/{accountId} /validateRole

$ https://<SYSDIG_SECURE_ENDPOINT>/api/cloud/v2/accounts/<AWS_ACCOUNT_ID>/validateRole \
--header 'Authorization: Bearer <SYSDIG_SECURE_API_TOKEN>'

You should get success or the reason of failure.

Confirm services are working

Official Docs Check Guide
Forcing events

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nontf-org-three-cross-account-k8s-theat-compliance.md

nontf-org-three-cross-account-k8s-theat-compliance.md

Organizational - Three way Cross-Account- Kubernetes only (no TF) - Threat Detection and Compliance

Use-Case explanation

Infrastructure Solution

Cloud-Connector wiring: Organizational Cloudtrail S3 + SNS - SQS

Secure for Cloud Compute Deployment

Compliance

Sysdig Side

Customer's Side

End-To-End Validation

Confirm services are working

Files

nontf-org-three-cross-account-k8s-theat-compliance.md

Latest commit

History

nontf-org-three-cross-account-k8s-theat-compliance.md

File metadata and controls

Organizational - Three way Cross-Account- Kubernetes only (no TF) - Threat Detection and Compliance

Use-Case explanation

Infrastructure Solution

Cloud-Connector wiring: Organizational Cloudtrail S3 + SNS - SQS

Secure for Cloud Compute Deployment

Compliance

Sysdig Side

Customer's Side

End-To-End Validation

Confirm services are working