Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log Annotation Modules #16

Merged
merged 39 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
da4cc09
Naming
bentonam Apr 9, 2024
fd078be
Logs WIP
bentonam Apr 9, 2024
d6f7616
Formatting
bentonam Apr 9, 2024
69bc05f
Export Value
bentonam Apr 9, 2024
35d3336
Updates
bentonam Apr 10, 2024
e120d6c
Drop Log Levels
bentonam Apr 10, 2024
088311d
Merge branch 'main' of github.com:grafana/alloy-modules into log-modules
bentonam Apr 11, 2024
60d8223
Log Utils
bentonam Apr 11, 2024
906c6ff
Kubernetes Log Utils
bentonam Apr 11, 2024
28fed12
Allow logs folder
bentonam Apr 11, 2024
1fae838
Restructure
bentonam Apr 11, 2024
78730b0
Renamed
bentonam Apr 11, 2024
87ac783
Pods
bentonam Apr 11, 2024
89618a6
Updates
bentonam Apr 11, 2024
71633de
Fixed Regex
bentonam Apr 11, 2024
b50c66c
Cleanup
bentonam Apr 11, 2024
6a79690
Core Logs
bentonam Apr 11, 2024
6dc1177
New Annotations
bentonam Apr 11, 2024
5762b4f
Luhn and Sampling
bentonam Apr 11, 2024
ec175e4
Documentation
bentonam Apr 11, 2024
38178ba
Formatting
bentonam Apr 11, 2024
157f123
Log Metrics
bentonam Apr 11, 2024
4184369
Removed forward to
bentonam Apr 11, 2024
6f808c4
Moved normalize_filename
bentonam Apr 11, 2024
27091fe
Log Annotation Modules
bentonam Apr 11, 2024
b21d8a9
Updates
bentonam Apr 12, 2024
a4e0a5f
chore(dev-deps): bump markdownlint-cli2 from 0.12.1 to 0.13.0
dependabot[bot] Apr 8, 2024
7cc0b52
docs: update usage example to reflect new repo layout
hbjydev Apr 11, 2024
e47a19d
docs: add warning about pyroscope being in public preview
hbjydev Apr 11, 2024
b1e7d32
docs: module, not import
hbjydev Apr 12, 2024
39b76b6
docs: resolve change requests in MR
hbjydev Apr 12, 2024
01ca0d9
README: fix logo for dark mode, rename references to flow/agent
rfratto Apr 11, 2024
58a3afd
metrics.alloy fixes
Imshelledin21 Apr 17, 2024
8e360e6
Load Module Updates
bentonam May 1, 2024
021e920
Alloy references
bentonam May 1, 2024
c292453
Merge branch 'main' into log-modules
bentonam May 1, 2024
a64c84e
Removed unsupported annotations
bentonam May 1, 2024
551dbd3
cri-o support
bentonam May 2, 2024
a6734b2
Updates
bentonam May 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@ Backup of *.doc*
# NodeJS
# ---------------------------------------------------
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
Expand Down
4 changes: 2 additions & 2 deletions modules/broker/rabbitmq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all rabbitmq instances in cluster.

```river
```alloy
import.git "rabbitmq" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -136,7 +136,7 @@ prometheus.remote_write "local" {

The following example will scrape rabbitmq for metrics on the local machine.

```river
```alloy
import.git "rabbitmq" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
2 changes: 1 addition & 1 deletion modules/cloud/grafana/cloud/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The token must have permissions to read stack information. The setup of these pe

### `stack`

```river
```alloy
import.git "grafana_cloud" {
repository = "https://github.com/grafana/alloy-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/collector/agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all agents in cluster.

```river
```alloy
import.git "agent" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape the agent for metrics on the local machine.

```river
```alloy
import.git "agent" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
2 changes: 1 addition & 1 deletion modules/collector/push-gateway/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all push-gateway in cluster.

```river
```alloy
import.git "push_gateway" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/kv/etcd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all etcd instances in cluster.

```river
```alloy
import.git "etcd" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -134,7 +134,7 @@ prometheus.remote_write "local" {

The following example will scrape etcd for metrics on the local machine.

```river
```alloy
import.git "etcd" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/kv/memcached/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all memcached instances in cluster.

```river
```alloy
import.git "memcached" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape memcached for metrics on the local machine.

```river
```alloy
import.git "memcached" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/kv/redis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all redis instances in cluster.

```river
```alloy
import.git "redis" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape redis for metrics on the local machine.

```river
```alloy
import.git "redis" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/sql/mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all mysql instances in cluster.

```river
```alloy
import.git "mysql" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape mysql for metrics on the local machine.

```river
```alloy
import.git "mysql" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/sql/postgres/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all postgres instances in cluster.

```river
```alloy
import.git "postgres" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape postgres for metrics on the local machine.

```river
```alloy
import.git "postgres" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/timeseries/loki/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all Loki instances in cluster.

```river
```alloy
import.git "loki" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -134,7 +134,7 @@ prometheus.remote_write "local" {

The following example will scrape Loki for metrics on the local machine.

```river
```alloy
import.git "loki" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/timeseries/mimir/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all Mimir instances in cluster.

```river
```alloy
import.git "mimir" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape Mimir for metrics on the local machine.

```river
```alloy
import.git "mimir" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/timeseries/pyroscope/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all pyroscope instances in cluster.

```river
```alloy
import.git "pyroscope" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape pyroscope for metrics on the local machine.

```river
```alloy
import.git "pyroscope" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
4 changes: 2 additions & 2 deletions modules/databases/timeseries/tempo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The following labels are automatically added to exported targets.

The following example will scrape all Tempo instances in cluster.

```river
```alloy
import.git "tempo" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down Expand Up @@ -135,7 +135,7 @@ prometheus.remote_write "local" {

The following example will scrape Tempo for metrics on the local machine.

```river
```alloy
import.git "tempo" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
Expand Down
26 changes: 13 additions & 13 deletions modules/kubernetes/annotations/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Kubernetes Annotation Modules
# Kubernetes Metric Annotation Module

**Modules:**

- [`metrics.river`](#metricsriver)
- [`probes.river`](#probesriver)
- [`metrics.alloy`](#metricsalloy)
- [`probes.alloy`](#probesalloy)

## `metrics.river`
## `metrics.alloy`

This module is meant to be used to automatically scrape targets based on a certain role and set of annotations. This module can be consumed
multiple times with different roles. The supported roles are:
Expand Down Expand Up @@ -143,8 +143,8 @@ The following labels are automatically added to exported targets.
| :---------------- | :------- | :---------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------- |
| `targets` | _yes_ | `list(map(string))` | List of targets to scrape |
| `forward_to` | _yes_ | `list(MetricsReceiver)` | Must be a where scraped should be forwarded to |
| `keep_metrics` | _no_ | [see code](module.river#L228) | A regular expression of metrics to keep |
| `drop_metrics` | _no_ | [see code](module.river#L235) | A regular expression of metrics to drop |
| `keep_metrics` | _no_ | [see code](module.alloy#L228) | A regular expression of metrics to keep |
| `drop_metrics` | _no_ | [see code](module.alloy#L235) | A regular expression of metrics to drop |
| `scrape_interval` | _no_ | `60s` | How often to scrape metrics from the targets |
| `scrape_timeout` | _no_ | `10s` | How long before a scrape times out |
| `max_cache_size` | _no_ | `100000` | The maximum number of elements to hold in the relabeling cache. This should be at least 2x-5x your largest scrape target or samples appended rate. |
Expand All @@ -156,7 +156,7 @@ N/A

---

## `probes.river`
## `probes.alloy`

This module is meant to be used to automatically scrape targets based on a certain role and set of annotations. This module can be consumed
multiple times with different roles. The supported roles are:
Expand Down Expand Up @@ -267,8 +267,8 @@ The following labels are automatically added to exported targets.
| :---------------- | :------- | :---------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------- |
| `targets` | _yes_ | `list(map(string))` | List of targets to scrape |
| `forward_to` | _yes_ | `list(MetricsReceiver)` | Must be a where scraped should be forwarded to |
| `keep_metrics` | _no_ | [see code](module.river#L228) | A regular expression of metrics to keep |
| `drop_metrics` | _no_ | [see code](module.river#L235) | A regular expression of metrics to drop |
| `keep_metrics` | _no_ | [see code](module.alloy#L228) | A regular expression of metrics to keep |
| `drop_metrics` | _no_ | [see code](module.alloy#L235) | A regular expression of metrics to drop |
| `scrape_interval` | _no_ | `60s` | How often to scrape metrics from the targets |
| `scrape_timeout` | _no_ | `10s` | How long before a scrape times out |
| `max_cache_size` | _no_ | `100000` | The maximum number of elements to hold in the relabeling cache. This should be at least 2x-5x your largest scrape target or samples appended rate. |
Expand All @@ -286,11 +286,11 @@ N/A

The following example will scrape all metric annotation instances in cluster.

```river
```alloy
import.git "metric_annotations" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
path = "modules/kubernetes/annotations/metrics.river"
path = "modules/kubernetes/annotations/metrics.alloy"
pull_frequency = "15m"
}

Expand Down Expand Up @@ -324,11 +324,11 @@ prometheus.remote_write "local" {

The following example will scrape all probe annotation instances in cluster.

```river
```alloy
import.git "probe_annotations" {
repository = "https://github.com/grafana/flow-modules.git"
revision = "main"
path = "modules/kubernetes/annotations/probes.river"
path = "modules/kubernetes/annotations/probes.alloy"
pull_frequency = "15m"
}

Expand Down
Loading