Skip to content

Add feature id tracking for credentials #3216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 61 commits into from
Apr 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
29e6a37
Add comments for feature id locations
richardwang1124 Mar 18, 2025
ffea46f
Merge branch 'version-3' into feature/feature-ids
richardwang1124 Mar 18, 2025
16c86cc
Add new feature ids to METRICS
richardwang1124 Mar 18, 2025
2938a7c
Add comments and change priority of user agent
richardwang1124 Mar 20, 2025
696e1a6
Add skeleton check
richardwang1124 Mar 20, 2025
ed8a0fc
Add skeleton for all except assume role
richardwang1124 Mar 21, 2025
14db8f9
Delete metrics from opts once read
richardwang1124 Mar 21, 2025
e9a5fe0
Add assume role skeleton
richardwang1124 Mar 21, 2025
04fe61d
Fix some test failures
richardwang1124 Mar 21, 2025
374e46b
Add attribute readers
richardwang1124 Mar 21, 2025
260ef00
Revert built changes
richardwang1124 Mar 21, 2025
179901c
Revert built changes
richardwang1124 Mar 21, 2025
fc0528e
Revert built changes
richardwang1124 Mar 21, 2025
31411f8
Add with_metric for service calls
richardwang1124 Mar 21, 2025
2eb610e
Remove some puts
richardwang1124 Mar 21, 2025
8fc119c
Wrap intermediate calls with metric
richardwang1124 Mar 24, 2025
8cde58b
Change priority and revert chaining
richardwang1124 Mar 24, 2025
0806c60
Remove comments
richardwang1124 Mar 24, 2025
5434f86
Fix process credentials
richardwang1124 Mar 24, 2025
6df4fd6
Add small fixes
richardwang1124 Mar 24, 2025
af62faf
Add CHANGELOG entry
richardwang1124 Mar 24, 2025
5b7cf4b
Fix assume role logic
richardwang1124 Mar 24, 2025
015ece2
Fix name
richardwang1124 Mar 24, 2025
804826e
Start to add test
richardwang1124 Mar 25, 2025
e17ba4d
Try source approach
richardwang1124 Mar 25, 2025
01eca88
Partially fix extra metrics issue
richardwang1124 Mar 25, 2025
0299ccd
Fix extra 'i' metric error
richardwang1124 Mar 26, 2025
1dda7e2
Fix some tests
richardwang1124 Mar 26, 2025
3c42148
Fix code to align with correct credentials_code
richardwang1124 Mar 26, 2025
7cd276f
Cleanup
richardwang1124 Mar 26, 2025
1612b21
Fix SSO credential source
richardwang1124 Mar 26, 2025
b1eb340
Start to add unit tests to sign spec
richardwang1124 Mar 27, 2025
b5828b0
Add source testing
richardwang1124 Mar 27, 2025
bdac7df
Add metrics tests
richardwang1124 Mar 27, 2025
e2be224
Add missing test cases
richardwang1124 Mar 27, 2025
c68e112
Add with metrics to refresh
richardwang1124 Mar 28, 2025
cb816ab
Add partial refresh tests
richardwang1124 Mar 28, 2025
9693b9e
Remove metrics from refresh
richardwang1124 Mar 28, 2025
0398f24
Clean up code
richardwang1124 Mar 28, 2025
a509fc8
Fix merge conflicts
richardwang1124 Mar 28, 2025
ad3e569
Make changes from PR feedback
richardwang1124 Apr 1, 2025
bf62f37
Fix tests
richardwang1124 Apr 1, 2025
57e0889
Add some documentation
richardwang1124 Apr 1, 2025
1408d57
Add testing for service call metrics
richardwang1124 Apr 2, 2025
d975eab
Initial fix to simplify return
richardwang1124 Apr 2, 2025
acf8375
Fix duplicate metrics issue with AssumeRoleCredentials
richardwang1124 Apr 2, 2025
1630188
Add keyword argument for metrics_source
richardwang1124 Apr 3, 2025
4cf1af3
Clean up code
richardwang1124 Apr 4, 2025
484288b
'Disable' client side user-agent metric?
richardwang1124 Apr 4, 2025
8d84436
Fix syntax error
richardwang1124 Apr 4, 2025
c58aaec
Revert to setter
richardwang1124 Apr 7, 2025
abf7e56
Add sso stub
richardwang1124 Apr 8, 2025
734b436
Update based on PR comments
richardwang1124 Apr 9, 2025
d2d05e4
Minor edits
richardwang1124 Apr 9, 2025
18ae624
More minor edits
richardwang1124 Apr 9, 2025
ac4305f
Add double arguments
richardwang1124 Apr 9, 2025
9f1f25c
Remove dup
richardwang1124 Apr 9, 2025
281fc0d
Add changes from PR comments
richardwang1124 Apr 10, 2025
147f5e7
Add some rubocop fixes
richardwang1124 Apr 10, 2025
5a5fa02
Fix resolve source
richardwang1124 Apr 11, 2025
ab2034f
Omit unused metrics
richardwang1124 Apr 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions gems/aws-sdk-core/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
Unreleased Changes
------------------

* Issue - Additional metrics collection for credentials in the User-Agent plugin.

3.222.1 (2025-03-28)
------------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ def initialize(options = {})
end
@client = client_opts[:client] || STS::Client.new(client_opts)
@async_refresh = true
@metrics = ['CREDENTIALS_STS_ASSUME_ROLE']
super
end

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ def initialize(options = {})
@assume_role_web_identity_params[:role_session_name] = _session_name
end
@client = client_opts[:client] || STS::Client.new(client_opts.merge(credentials: nil))
@metrics = ['CREDENTIALS_STS_ASSUME_ROLE_WEB_ID']
super
end

Expand Down
4 changes: 4 additions & 0 deletions gems/aws-sdk-core/lib/aws-sdk-core/credential_provider.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ module CredentialProvider
# @return [Time]
attr_reader :expiration

# @api private
# Returns UserAgent metrics for credentials.
attr_accessor :metrics

# @return [Boolean]
def set?
!!@credentials && @credentials.set?
Expand Down
34 changes: 27 additions & 7 deletions gems/aws-sdk-core/lib/aws-sdk-core/credential_provider_chain.rb
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,14 @@ def providers

def static_credentials(options)
if options[:config]
Credentials.new(
creds = Credentials.new(
options[:config].access_key_id,
options[:config].secret_access_key,
options[:config].session_token,
account_id: options[:config].account_id
)
creds.metrics = ['CREDENTIALS_PROFILE']
creds
end
end

Expand Down Expand Up @@ -76,7 +78,9 @@ def static_profile_assume_role_credentials(options)

def static_profile_credentials(options)
if options[:config] && options[:config].profile
SharedCredentials.new(profile_name: options[:config].profile)
creds = SharedCredentials.new(profile_name: options[:config].profile)
creds.metrics = ['CREDENTIALS_PROFILE']
creds
end
rescue Errors::NoSuchProfileError
nil
Expand All @@ -85,7 +89,11 @@ def static_profile_credentials(options)
def static_profile_process_credentials(options)
if Aws.shared_config.config_enabled? && options[:config] && options[:config].profile
process_provider = Aws.shared_config.credential_process(profile: options[:config].profile)
ProcessCredentials.new([process_provider]) if process_provider
if process_provider
creds = ProcessCredentials.new([process_provider])
creds.metrics << 'CREDENTIALS_PROFILE_PROCESS'
creds
end
end
rescue Errors::NoSuchProfileError
nil
Expand All @@ -96,12 +104,14 @@ def env_credentials(_options)
secret = %w[AWS_SECRET_ACCESS_KEY AMAZON_SECRET_ACCESS_KEY AWS_SECRET_KEY]
token = %w[AWS_SESSION_TOKEN AMAZON_SESSION_TOKEN]
account_id = %w[AWS_ACCOUNT_ID]
Credentials.new(
creds = Credentials.new(
envar(key),
envar(secret),
envar(token),
account_id: envar(account_id)
)
creds.metrics = ['CREDENTIALS_ENV_VARS']
creds
end

def envar(keys)
Expand All @@ -117,7 +127,9 @@ def determine_profile_name(options)

def shared_credentials(options)
profile_name = determine_profile_name(options)
SharedCredentials.new(profile_name: profile_name)
creds = SharedCredentials.new(profile_name: profile_name)
creds.metrics = ['CREDENTIALS_PROFILE']
creds
rescue Errors::NoSuchProfileError
nil
end
Expand All @@ -126,7 +138,11 @@ def process_credentials(options)
profile_name = determine_profile_name(options)
if Aws.shared_config.config_enabled?
process_provider = Aws.shared_config.credential_process(profile: profile_name)
ProcessCredentials.new([process_provider]) if process_provider
if process_provider
creds = ProcessCredentials.new([process_provider])
creds.metrics << 'CREDENTIALS_PROFILE_PROCESS'
creds
end
end
rescue Errors::NoSuchProfileError
nil
Expand Down Expand Up @@ -156,7 +172,11 @@ def assume_role_web_identity_credentials(options)
role_session_name: ENV['AWS_ROLE_SESSION_NAME']
}
cfg[:region] = region if region
AssumeRoleWebIdentityCredentials.new(cfg)
Aws::Plugins::UserAgent.metric('CREDENTIALS_ENV_VARS_STS_WEB_ID_TOKEN') do
creds = AssumeRoleWebIdentityCredentials.new(cfg)
creds.metrics << 'CREDENTIALS_ENV_VARS_STS_WEB_ID_TOKEN'
creds
end
elsif Aws.shared_config.config_enabled?
profile = options[:config].profile if options[:config]
Aws.shared_config.assume_role_web_identity_credentials_from_config(
Expand Down
6 changes: 6 additions & 0 deletions gems/aws-sdk-core/lib/aws-sdk-core/credentials.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ def initialize(access_key_id, secret_access_key, session_token = nil,
@secret_access_key = secret_access_key
@session_token = session_token
@account_id = kwargs[:account_id]
@metrics = ['CREDENTIALS_CODE']
end

# @return [String]
Expand All @@ -28,6 +29,11 @@ def initialize(access_key_id, secret_access_key, session_token = nil,
# @return [String, nil]
attr_reader :account_id

# @api private
# Returns the credentials source. Used for tracking credentials
# related UserAgent metrics.
attr_accessor :metrics

# @return [Credentials]
def credentials
self
Expand Down
1 change: 1 addition & 0 deletions gems/aws-sdk-core/lib/aws-sdk-core/ecs_credentials.rb
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ def initialize(options = {})
@http_debug_output = options[:http_debug_output]
@backoff = backoff(options[:backoff])
@async_refresh = false
@metrics = ['CREDENTIALS_HTTP']
super
end

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ def initialize(options = {})
@token = nil
@no_refresh_until = nil
@async_refresh = false
@metrics = ['CREDENTIALS_IMDS']
super
end

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,6 @@ def call(context)
complete_opts = {
latency: end_time - start_time,
attempt_count: context.retries + 1,
user_agent: context.http_request.headers["user-agent"],
final_error_retryable: final_error_retryable,
final_http_status_code: context.http_response.status_code,
final_aws_exception: final_aws_exception,
Expand Down
16 changes: 15 additions & 1 deletion gems/aws-sdk-core/lib/aws-sdk-core/plugins/sign.rb
Original file line number Diff line number Diff line change
Expand Up @@ -41,20 +41,28 @@ def self.signer_for(auth_scheme, config, sigv4_region_override = nil, sigv4_cred
class Handler < Seahorse::Client::Handler
def call(context)
# Skip signing if using sigv2 signing from s3_signer in S3
credentials = nil
unless v2_signing?(context.config)
signer = Sign.signer_for(
context[:auth_scheme],
context.config,
context[:sigv4_region],
context[:sigv4_credentials]
)
credentials = signer.credentials if signer.is_a?(SignatureV4)
signer.sign(context)
end
@handler.call(context)
with_metrics(credentials) { @handler.call(context) }
end

private

def with_metrics(credentials, &block)
return block.call unless credentials&.respond_to?(:metrics)

Aws::Plugins::UserAgent.metric(*credentials.metrics, &block)
end

def v2_signing?(config)
# 's3' is legacy signing, 'v4' is default
config.respond_to?(:signature_version) &&
Expand Down Expand Up @@ -92,6 +100,8 @@ def sign_event(*args)

# @api private
class SignatureV4
attr_reader :signer

def initialize(auth_scheme, config, sigv4_overrides = {})
scheme_name = auth_scheme['name']

Expand Down Expand Up @@ -155,6 +165,10 @@ def sign_event(*args)
@signer.sign_event(*args)
end

def credentials
@signer.credentials_provider
end

private

def apply_authtype(context, req)
Expand Down
25 changes: 23 additions & 2 deletions gems/aws-sdk-core/lib/aws-sdk-core/plugins/user_agent.rb
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,27 @@ class UserAgent < Seahorse::Client::Plugin
"FLEXIBLE_CHECKSUMS_REQ_WHEN_SUPPORTED" : "Z",
"FLEXIBLE_CHECKSUMS_REQ_WHEN_REQUIRED" : "a",
"FLEXIBLE_CHECKSUMS_RES_WHEN_SUPPORTED" : "b",
"FLEXIBLE_CHECKSUMS_RES_WHEN_REQUIRED" : "c"
"FLEXIBLE_CHECKSUMS_RES_WHEN_REQUIRED" : "c",
"DDB_MAPPER": "d",
"CREDENTIALS_CODE" : "e",
"CREDENTIALS_ENV_VARS" : "g",
"CREDENTIALS_ENV_VARS_STS_WEB_ID_TOKEN" : "h",
"CREDENTIALS_STS_ASSUME_ROLE" : "i",
"CREDENTIALS_STS_ASSUME_ROLE_WEB_ID" : "k",
"CREDENTIALS_PROFILE" : "n",
"CREDENTIALS_PROFILE_SOURCE_PROFILE" : "o",
"CREDENTIALS_PROFILE_NAMED_PROVIDER" : "p",
"CREDENTIALS_PROFILE_STS_WEB_ID_TOKEN" : "q",
"CREDENTIALS_PROFILE_SSO" : "r",
"CREDENTIALS_SSO" : "s",
"CREDENTIALS_PROFILE_SSO_LEGACY" : "t",
"CREDENTIALS_SSO_LEGACY" : "u",
"CREDENTIALS_PROFILE_PROCESS" : "v",
"CREDENTIALS_PROCESS" : "w",
"CREDENTIALS_HTTP" : "z",
"CREDENTIALS_IMDS" : "0",
"SSO_LOGIN_DEVICE" : "1",
"SSO_LOGIN_AUTH" : "2"
}
METRICS

Expand Down Expand Up @@ -196,7 +216,8 @@ def metric_metadata
end
end

handler(Handler, step: :sign, priority: 97)
# Priority set to 5 in order to add user agent as late as possible after signing
handler(Handler, step: :sign, priority: 5)
end
end
end
2 changes: 1 addition & 1 deletion gems/aws-sdk-core/lib/aws-sdk-core/process_credentials.rb
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def initialize(process)
@process = process
@credentials = credentials_from_process
@async_refresh = false

@metrics = ['CREDENTIALS_PROCESS']
super
end

Expand Down
Loading
Loading