Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable prometheus jmx agent when running on ECS #121

Merged
merged 23 commits into from
Feb 1, 2024
Merged

Enable prometheus jmx agent when running on ECS #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
3414dcc
Added logs to trace datadog
Jan 11, 2024
27c5ca1
Added deploy snapshot.yml
Jan 11, 2024
419de78
Added deploy snapshot.yml
Jan 11, 2024
8b47069
Added the deploy_snapshot.yml
Jan 11, 2024
dd211c8
Added for the inside condition
Jan 16, 2024
0d11f5e
echo ECS_CONTAINER_METADATA_URI
Jan 23, 2024
12532d7
kubernetes exporter here: EXPORTER_OPTS and HIVE_METASTORE_HADOOP_OPTS
Jan 24, 2024
3c91ff7
kubernetes exporter here: EXPORTER_OPTS and HIVE_METASTORE_HADOOP_OPTS
Jan 24, 2024
7d6a8fd
logs
Jan 24, 2024
e196f32
Added EXPORTER_OPTS
Jan 30, 2024
457e09e
removed th eexports
Jan 30, 2024
da5eb5b
trying HADOOP_OPTS
Jan 30, 2024
99f4f2e
trying EXPORTER_OPTS
Jan 30, 2024
e676eed
trying curl
Jan 30, 2024
b341577
Working before the last commit but only 10 metrics flowing
Jan 30, 2024
698ad48
Updated hive_site.xml to use com.expediagroup.apiary.extensions.metas…
Jan 30, 2024
510193b
Removed echo
Jan 31, 2024
e32defb
Removed echo
Jan 31, 2024
b328a27
Removed echo
Jan 31, 2024
5d35279
Removed echo
Jan 31, 2024
e754cf3
Removed echo
Feb 1, 2024
53b7b8a
Removed echo
Feb 1, 2024
bab0c1d
Removed echo
Feb 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [3.0.17] - 2024-01-31
### Added
- Enable prometheus jmx agent when running on ECS by exporting `EXPORTER_OPTS`

## [3.0.16] - 2024-01-11
### Added
- Added snapshot.yaml for pushing docker image from feature branch.
Expand Down
4 changes: 4 additions & 0 deletions files/startup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,14 @@ if [[ -n $RANGER_AUDIT_DB_URL ]]; then
fi
fi


if [ -n "$ENABLE_METRICS" ]; then
update_property.py hive.metastore.metrics.enabled true /etc/hive/conf/hive-site.xml
#configure to send metrics to cloudwatch when running on ECS
if [ -n "$ECS_CONTAINER_METADATA_URI" ]; then

# enable prometheus jmx agent when running on ECS
export EXPORTER_OPTS="-javaagent:/usr/lib/apiary/jmx_prometheus_javaagent-${EXPORTER_VERSION}.jar=8080:/etc/hive/conf/jmx-exporter.yaml"
export CLOUDWATCH_NAMESPACE="${INSTANCE_NAME}-metastore"
export ECS_TASK_ID=$(wget -q -O - ${ECS_CONTAINER_METADATA_URI}/task|jq -r .TaskARN|awk -F/ '{ print $NF }')
update_property.py hive.service.metrics.class com.expediagroup.apiary.extensions.metastore.metrics.CodahaleMetrics /etc/hive/conf/hive-site.xml
Expand Down
Loading