New Relic Monitoring Setup #4423
Conversation
📝 WalkthroughWalkthroughThis pull request updates the New Relic integration for Kubernetes. It enhances the README with installation instructions, adds a new instrumentation configuration for APM auto-instrumentation, and introduces environment-specific values files for production and staging. Additionally, it revises the Node resources for both airqo and airqo-stage clusters by adding new YAML definitions with updated labels and removing legacy JSON configurations. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Helm
participant KubeCluster as Kubernetes Cluster
participant NewRelic as New Relic Bundle
User->>Helm: Run helm install command with parameters
Helm->>KubeCluster: Deploy New Relic bundle & enable features
KubeCluster->>NewRelic: Initialize monitoring agents
User->>KubeCluster: Apply instrumentation YAML for APM auto-instrumentation
Suggested reviewers
Poem
Warning Review ran into problems🔥 ProblemsErrors were encountered while retrieving linked issues. Errors (1)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## staging #4423 +/- ##
===========================================
- Coverage 11.24% 10.07% -1.17%
===========================================
Files 156 156
Lines 17930 21144 +3214
Branches 388 499 +111
===========================================
+ Hits 2016 2130 +114
- Misses 15912 19012 +3100
Partials 2 2 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🔭 Outside diff range comments (1)
k8s/nodes/airqo-stage-k8s-worker-2.yaml (1)
1-7:⚠️ Potential issueFix naming inconsistency between file and node name.
The filename is
airqo-stage-k8s-worker-2.yamlbut the node name in the configuration is set toairqo-stage-k8s-worker-1. This inconsistency could lead to confusion in node management.Apply this diff to align the node name with the filename:
apiVersion: v1 kind: Node metadata: - name: airqo-stage-k8s-worker-1 + name: airqo-stage-k8s-worker-2 labels: role: high-mem
🧹 Nitpick comments (4)
k8s/nodes/airqo-stage-k8s-worker-2.yaml (1)
5-6: Consider adding more descriptive labels for better node management.While the
role: high-memlabel is good for indicating the node's resource characteristics, consider adding additional labels to improve node selection and management. This is particularly important for New Relic monitoring.Consider adding these common Kubernetes labels:
labels: role: high-mem + kubernetes.io/os: linux + node.kubernetes.io/instance-type: high-memory + topology.kubernetes.io/zone: your-zone + env: stagingk8s/new-relic/instrumentation.yaml (1)
4-6: Consider adding resource limits and security context.The instrumentation configuration should include resource limits and security context to ensure proper resource allocation and security.
metadata: name: newrelic-instrumentation namespace: newrelic-monitoring + labels: + app.kubernetes.io/name: newrelic-instrumentation + app.kubernetes.io/part-of: monitoring spec: + resources: + limits: + memory: "128Mi" + cpu: "100m" + requests: + memory: "64Mi" + cpu: "50m" + securityContext: + runAsNonRoot: true + runAsUser: 1000k8s/new-relic/README.md (1)
1-4: Enhance installation documentation with security considerations.Add a section about security best practices, including:
- License key management using Kubernetes secrets
- RBAC configuration
- Network policies
🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
3-3: Bare URL used
null(MD034, no-bare-urls)
k8s/new-relic/values.stage.yaml (1)
136-140: Review privileged mode requirement.Privileged mode is enabled which poses security risks. Verify if this is absolutely necessary.
Consider implementing a more restrictive security context if possible:
- privileged: true + privileged: false + containerSecurityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (22)
k8s/new-relic/README.md(1 hunks)k8s/new-relic/instrumentation.yaml(1 hunks)k8s/new-relic/values.prod.yaml(1 hunks)k8s/new-relic/values.stage.yaml(1 hunks)k8s/nodes/airqo-k8s-controller.yaml(1 hunks)k8s/nodes/airqo-k8s-worker-0.json(0 hunks)k8s/nodes/airqo-k8s-worker-1.json(0 hunks)k8s/nodes/airqo-k8s-worker-1.yaml(1 hunks)k8s/nodes/airqo-k8s-worker-2.json(0 hunks)k8s/nodes/airqo-k8s-worker-2.yaml(1 hunks)k8s/nodes/airqo-k8s-worker-3.json(0 hunks)k8s/nodes/airqo-k8s-worker-3.yaml(1 hunks)k8s/nodes/airqo-k8s-worker-4.json(0 hunks)k8s/nodes/airqo-stage-k8s-controller.yaml(1 hunks)k8s/nodes/airqo-stage-k8s-worker-0.json(0 hunks)k8s/nodes/airqo-stage-k8s-worker-1.json(0 hunks)k8s/nodes/airqo-stage-k8s-worker-1.yaml(1 hunks)k8s/nodes/airqo-stage-k8s-worker-2.json(0 hunks)k8s/nodes/airqo-stage-k8s-worker-2.yaml(1 hunks)k8s/nodes/airqo-stage-k8s-worker-3.json(0 hunks)k8s/nodes/airqo-stage-k8s-worker-3.yaml(1 hunks)k8s/nodes/airqo-stage-k8s-worker-4.json(0 hunks)
💤 Files with no reviewable changes (10)
- k8s/nodes/airqo-k8s-worker-3.json
- k8s/nodes/airqo-stage-k8s-worker-0.json
- k8s/nodes/airqo-k8s-worker-1.json
- k8s/nodes/airqo-k8s-worker-2.json
- k8s/nodes/airqo-stage-k8s-worker-3.json
- k8s/nodes/airqo-k8s-worker-4.json
- k8s/nodes/airqo-stage-k8s-worker-1.json
- k8s/nodes/airqo-stage-k8s-worker-4.json
- k8s/nodes/airqo-stage-k8s-worker-2.json
- k8s/nodes/airqo-k8s-worker-0.json
✅ Files skipped from review due to trivial changes (7)
- k8s/nodes/airqo-k8s-worker-3.yaml
- k8s/nodes/airqo-stage-k8s-controller.yaml
- k8s/nodes/airqo-k8s-worker-2.yaml
- k8s/nodes/airqo-stage-k8s-worker-1.yaml
- k8s/nodes/airqo-k8s-controller.yaml
- k8s/nodes/airqo-stage-k8s-worker-3.yaml
- k8s/nodes/airqo-k8s-worker-1.yaml
🧰 Additional context used
🪛 LanguageTool
k8s/new-relic/README.md
[grammar] ~8-~8: There seems to be a noun/verb agreement error. Did you mean “adds” or “added”?
Context: ...lease using the values file. helm repo add newrelic https://helm-charts.newrelic.c...
(SINGULAR_NOUN_VERB_AGREEMENT)
[duplication] ~8-~8: Possible typo: you repeated a word.
Context: ...lic/nri-bundle -n newrelic-monitoring --values values.yaml 4. Run this command to enable APM...
(ENGLISH_WORD_REPEAT_RULE)
🪛 markdownlint-cli2 (0.17.2)
k8s/new-relic/README.md
3-3: Bare URL used
null
(MD034, no-bare-urls)
8-8: Bare URL used
null
(MD034, no-bare-urls)
🪛 YAMLlint (1.35.1)
k8s/new-relic/values.stage.yaml
[error] 35-35: trailing spaces
(trailing-spaces)
[warning] 113-113: wrong indentation: expected 4 but found 6
(indentation)
[warning] 117-117: wrong indentation: expected 16 but found 14
(indentation)
[error] 143-143: trailing spaces
(trailing-spaces)
[error] 151-151: trailing spaces
(trailing-spaces)
k8s/new-relic/values.prod.yaml
[error] 35-35: trailing spaces
(trailing-spaces)
[warning] 117-117: wrong indentation: expected 4 but found 6
(indentation)
[warning] 121-121: wrong indentation: expected 16 but found 14
(indentation)
[error] 147-147: trailing spaces
(trailing-spaces)
[error] 155-155: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Analyze (javascript)
🔇 Additional comments (2)
k8s/new-relic/instrumentation.yaml (1)
22-26: Verify namespace selector configuration.The namespace selector is hardcoded to "staging". Ensure this aligns with your deployment strategy.
❌ Verification inconclusive
🏁 Script executed:
#!/bin/bash # Verify if the staging namespace exists and has the correct label kubectl get ns staging --show-labelsLength of output: 136
Manual Verification Required: Confirm Namespace Selector Configuration
The script couldn’t retrieve any output because the environment lacks the
kubectlcommand. Please manually verify on your cluster that the "staging" namespace exists with the appropriate label, ensuring that this hardcoded selector meets your deployment strategy.k8s/new-relic/README.md (1)
3-3: Verify helm chart version compatibility.The command uses latest versions. Let's verify compatibility with our cluster version.
❌ Verification inconclusive
🌐 Web query:
What is the latest stable version of New Relic's nri-bundle helm chart and its Kubernetes compatibility matrix?💡 Result:
The latest stable version of New Relic's
nri-bundleHelm chart is version 4, which includes the Kubernetes integrationappVersionv3[1]. This update focuses on configurability and security improvements while maintaining backward compatibility with reported data formats[1].For compatibility:
- The
nri-bundlesupports Kubernetes clusters running versions withinn-3of the Kubernetes client API used by Helm[9]. For example:
Helm Version Supported Kubernetes Versions 3.17.x 1.32.x - 1.29.x ... ... Key configuration changes in
nri-bundlev4 include:
- Mandatory
--portand--tlsflags for discovery commands[1]- Simplified integrations configuration format[1]
- Modular architecture with optional components like kube-state-metrics (KSM) and Prometheus integrations[2][4]
To verify your environment:
helm repo update helm search repo newrelic/nri-bundle --versionsFor upgrades requiring KSM adjustments[4]:
kube-state-metrics: enabled: false # For custom KSM instances newrelic-infrastructure: ksm: config: selector: "app.kubernetes.io/name=kube-state-metrics"Citations:
- 1: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/advanced-configuration/k8s-version2/changes-since-v3/
- 2: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/get-started/kubernetes-components/
- 3: https://docs.rafay.co/recipes/monitoring/newrelic/
- 4: https://github.com/newrelic/helm-charts/blob/master/charts/nri-bundle/README.md
- 5: https://github.com/newrelic/nri-kubernetes/blob/main/charts/newrelic-infrastructure/README.md
- 6: https://freshbrewed.science/2021/11/17/newrelic.html
- 7: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/installation/upgrading-kubernetes/
- 8: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/get-started/kubernetes-integration-compatibility-requirements/
- 9: https://helm.sh/docs/topics/version_skew/
- 10: https://artifacthub.io/packages/helm/newrelic/nri-bundle/2.5.0
Helm Chart Compatibility Needs Manual Verification
The web query confirms that the latest stable version of New Relic’s nri-bundle chart is version 4—with notable configuration changes (e.g., mandatory
--portand--tlsflags and simplified integrations). It also outlines that the chart supports Kubernetes clusters running versions within “n-3” of the Helm client’s API (for example, Helm 3.17.x supports Kubernetes 1.29.x–1.32.x).In the current README snippet, the chart version isn’t explicitly pinned, and you’re still using a legacy KSM image tag (
v2.13.0). While there’s no immediate issue with the command itself, you should verify that your cluster’s Kubernetes version falls within the supported range for the version of Helm you’re using—and consider any required updates or configuration adjustments when moving to nri-bundle v4.🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
3-3: Bare URL used
null(MD034, no-bare-urls)
| cluster: staging | ||
|
|
||
| # -- The license key for your New Relic Account. This will be preferred configuration option if both `licenseKey` and `customSecret` are specified. | ||
| licenseKey: <<LICENSE-KEY>> |
There was a problem hiding this comment.
Secure the license key using Kubernetes secrets.
Instead of using a placeholder, consider using Kubernetes secrets for the license key.
- licenseKey: <<LICENSE-KEY>>
+ customSecretName: "newrelic-license"
+ customSecretLicenseKey: "license-key"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| licenseKey: <<LICENSE-KEY>> | |
| customSecretName: "newrelic-license" | |
| customSecretLicenseKey: "license-key" |
| # -- (bool) Reduces number of metrics sent in order to reduce costs | ||
| # @default -- false | ||
| lowDataMode: true | ||
|
|
||
| # -- (bool) In each integration it has different behavior. See [Further information](#values-managed-globally-3) but all aims to send less metrics to the backend to try to save costs | | ||
| # @default -- false | ||
| privileged: true | ||
|
|
There was a problem hiding this comment.
Review production security settings.
Similar to staging, production has privileged mode enabled. This should be carefully reviewed.
Consider implementing Pod Security Policies or OPA Gatekeeper policies to enforce security standards across both environments.
| k8s-agents-operator: | ||
| # k8s-agents-operator.enabled -- Install the [`k8s-agents-operator` chart](https://github.com/newrelic/k8s-agents-operator/tree/main/charts/k8s-agents-operator) | ||
| enabled: false | ||
|
|
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Verify agent configuration differences between environments.
The k8s-agents-operator is disabled in production but enabled in staging. This inconsistency should be reviewed.
🏁 Script executed:
#!/bin/bash
# Compare agent configurations between environments
diff k8s/new-relic/values.stage.yaml k8s/new-relic/values.prod.yaml | grep "enabled:"Length of output: 141
Agent Configuration Discrepancy Between Environments
I verified that in the staging configuration the k8s-agents-operator.enabled flag is set to true, whereas in production it is set to false. This inconsistency should be reviewed to ensure it is intentional—especially considering any security or observability implications. Please confirm if production is meant to disable the operator or if this was an oversight.
Description
New Relic Monitoring Setup
Summary by CodeRabbit