Skip to content

Commit 9aae1f5

Browse files
ddragosdselfxp
authored andcommitted
Add STSCredentials provider besides IAMCredentials provider (#6)
* WIP to add STSCredentials provider besides IAMCredentials provider * refactored AwsService.lua to specify `aws_credentials` as init option which allows users to specify any implementation for retrieving the AWS credentials * finalized the initial implementation for AWSSTSCredentials * Documentation on how to use the SecurityTokenService * Documentation on how to use the SecurityTokenService * Documentation on how to use the SecurityTokenService * [Doc] - Documented the AWS Credentials Providers. * Updated documentation after review * [Doc] - added suggestions in the review
1 parent 22e48b5 commit 9aae1f5

8 files changed

+670
-73
lines changed

Diff for: README.md

+124-16
Original file line numberDiff line numberDiff line change
@@ -29,24 +29,77 @@ the [ngx_lua module](http://wiki.nginx.org/HttpLuaModule), [LuaJIT 2.0](http://l
2929
[api-gateway-hmac](https://github.com/adobe-apiplatform/api-gateway-hmac) module, and
3030
[lua-resty-http](https://github.com/pintsized/lua-resty-http) module.
3131

32-
### AWS V4 Signature
33-
This library supports the latest AWS V4 signature which means you can use any of the latest AWS APIs without any problem.
32+
### AWS Credentials Provider
33+
Requests to AWS Services must supply valid credentials and this library provides a few credentials providers for signing AWS Requests.
34+
`aws_credentials` config option specifies which provider to use.
35+
36+
If no `aws_credentials` is provided then the library will try to find one using the following order:
37+
38+
1. if `aws_access_key` and `aws_secret_key` are provided then Basic Credentials Provider is used.
39+
2. Otherwise the IAM Credentials Provider is used.
40+
41+
>INFO: This library supports the latest AWS V4 signature which means you can use any of the latest AWS APIs without any problem.
42+
43+
44+
45+
#### Basic Credentials
46+
Basic credentials work with `secret_key` and `access_key`.
47+
```lua
48+
aws_credentials = {
49+
provider = "api-gateway.aws.AWSBasicCredentials",
50+
access_key = "replace-me",
51+
secret_key = "replace-me"
52+
}
53+
```
54+
>INFO: For better security inside the AWS environment use IAM or STS credentials.
55+
56+
#### IAM Credentials
57+
This is probably the most popular credentials provider to be used inside the AWS environment.
58+
To learn more about IAM Credentials see [IAM Roles for Amazon EC2](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html).
59+
60+
This credentials provider discovers automatically the `IAM Role` associated to the EC2 instance retrieving its credentials and caching them.
61+
This is a more secure method for signing AWS requests as credentials are short lived and the NGINX configuration doesn't need to maintain any `access_key` or `secret_key` nor worry about rotating the keys.
62+
63+
```lua
64+
aws_credentials = {
65+
provider = "api-gateway.aws.AWSIAMCredentials",
66+
shared_cache_dict = "my_dict" -- the name of a shared dictionary used for caching IAM Credentials
67+
}
68+
```
69+
70+
#### STS Credentials
71+
AWS Security Token Service(STS) provides a great way to get limited-privilege credentials for accessing AWS Services.
72+
To learn more about STS Credentials see [Getting Temporary Credentials with STS](http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/prog-services-sts.html) and the [STS](#sts) section bellow.
73+
```lua
74+
aws_credentials = {
75+
provider = "api-gateway.aws.AWSSTSCredentials",
76+
role_ARN = "arn:aws:iam::111111:role/assumed_role_kinesis", -- ARN of the role to assume
77+
role_session_name = "kinesis-session", -- a name for this session
78+
shared_cache_dict = "shared_cache" -- shared dict for caching the credentials
79+
}
80+
```
81+
Unlike IAM Credentials that exposes a single IAM Role for each EC2 instance, STS Credentials allows an EC2 instance to assume multiple roles each with its own access policy.
82+
83+
This credentials provider uses [SecurityTokenService](src/lua/api-gateway/aws/sts/SecurityTokenService.lua) for making requests to STS and SecurityTokenService uses the IAM Credentials provider for making the call.
84+
It is strongly recommended to provide the `shared_cache_dict` in order to improve performance. The temporary credentials obtained from STS are stored in the `shared_cache_dict` for up to 60 minutes.
3485

3586
### AwsService wrapper
36-
`AwsService` is a generic Lua class to interact with any AWS API. All the actual implementations extend form this class.
37-
It's very straight forward to configure it:
38-
39-
```lua
40-
local service = AwsService:new({
41-
aws_service = "sns",
42-
aws_region = "us-east-1",
43-
aws_secret_key = "--replace--me",
44-
aws_access_key = "--replace--me",
45-
aws_debug = true, -- print warn level messages on the nginx logs. useful for debugging
46-
aws_conn_keepalive = 60000, -- how long to keep the sockets used for AWS open
47-
aws_conn_pool = 100 -- the connection pool size for sockets used to connect to AWS
48-
})
49-
```
87+
[AwsService](src/lua/api-gateway/aws/AwsService.lua) is a generic Lua class to interact with any AWS API. The actual implementations extend this class.
88+
Its configuration is straight forward:
89+
90+
```lua
91+
local service = AwsService:new({
92+
aws_service = "sns",
93+
aws_region = "us-east-1",
94+
aws_credentials = {
95+
provider = "api-gateway.aws.AWSIAMCredentials",
96+
shared_cache_dict = "my_dict" -- the name of a shared dictionary used for caching IAM Credentials
97+
}
98+
aws_debug = true, -- print warn level messages on the nginx logs. useful for debugging
99+
aws_conn_keepalive = 60000, -- how long to keep the sockets used for AWS open
100+
aws_conn_pool = 100 -- the connection pool size for sockets used to connect to AWS
101+
})
102+
```
50103

51104

52105
Synopsis
@@ -215,6 +268,61 @@ aws lambda get-policy --function-name hello-world-lambda-fn --region=us-east-1
215268

216269
```
217270

271+
### STS
272+
273+
The AWS Security Token Service (STS) provides access to temporary, limited-privilege credentials for AWS Identity and IAM users.
274+
This can be useful for communicating with a a third party AWS account, without having access to some long-term credentials. (ex. IAM user's access key).
275+
276+
The [SecurityTokenService](src/lua/api-gateway/aws/sts/SecurityTokenService.lua) is a AWS STS API wrapper and it provides support for the `AssumeRole` requests. It can be used as follows:
277+
278+
```lua
279+
280+
local SecuriyTokenService = require "api-gateway.aws.sts.SecuriyTokenService"
281+
282+
local service = SecuriyTokenService:new({
283+
aws_region = ngx.var.aws_region,
284+
aws_secret_key = ngx.var.aws_secret_key,
285+
aws_access_key = ngx.var.aws_access_key
286+
})
287+
288+
local response, code, headers, status, body = sts:assumeRole(role_ARN,
289+
role_session_name,
290+
policy,
291+
security_credentials_timeout,
292+
external_id)
293+
```
294+
295+
These are the steps that need to be followed in order to be able to generate temporary credentials:
296+
297+
Let's say that the AWS account `A` needs to send records to a Kinesis stream in account `B`.
298+
299+
* Create a role in account `B` that grants permission to write to Kinesis and update the `Trust Relationship` as follows:
300+
```json
301+
{
302+
"Version": "2012-10-17",
303+
"Statement": [
304+
{
305+
"Effect": "Allow",
306+
"Principal": {
307+
"AWS": "arn:aws:iam::{A-account-number}:root"
308+
},
309+
"Action": "sts:AssumeRole"
310+
}
311+
]
312+
}
313+
```
314+
* The role from account `A` should be allowed to perform `sts:AssumeRole` actions.
315+
* Call AWS STS AssumeRole API to obtain temporary credentials.
316+
```lua
317+
local response, code, headers, status, body = sts:assumeRole(role_ARN,
318+
role_session_name,
319+
policy,
320+
security_credentials_timeout,
321+
external_id)
322+
```
323+
324+
>INFO: For more information on how to configure the accounts see [How to Use an External ID When Granting Access to Your AWS Resources to a Third Party](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html).
325+
218326
[Back to TOC](#table-of-contents)
219327

220328
Developer guide

Diff for: src/lua/api-gateway/aws/AWSBasicCredentials.lua

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
--[[
2+
Copyright 2016 Adobe Systems Incorporated. All rights reserved.
3+
4+
This file is licensed to you under the Apache License, Version 2.0 (the
5+
"License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
6+
7+
http://www.apache.org/licenses/LICENSE-2.0
8+
9+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR RESPRESENTATIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
10+
]]
11+
12+
--
13+
-- Basic Credentials provider using access_key and secret.
14+
-- User: ddascal
15+
-- Date: 4/10/16
16+
-- Time: 21:26
17+
--
18+
19+
local _M = {}
20+
21+
---
22+
-- @param o Init object
23+
-- o.access_key -- required. AWS Access Key Id
24+
-- o.secret_key -- required. AWS Secret Access Key
25+
--
26+
function _M:new(o)
27+
o = o or {}
28+
setmetatable(o, self)
29+
self.__index = self
30+
if (o ~= nil) then
31+
self.aws_secret_key = o.secret_key
32+
self.aws_access_key = o.access_key
33+
end
34+
return o
35+
end
36+
37+
function _M:getSecurityCredentials()
38+
return self.aws_access_key, self.aws_secret_key
39+
end
40+
41+
return _M
42+

Diff for: src/lua/api-gateway/aws/AWSIAMCredentials.lua

+1-2
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ local DEFAULT_SECURITY_CREDENTIALS_URL = "/latest/meta-data/iam/security-credent
1818
-- use GET /latest/meta-data/iam/security-credentials/ to auto-discover the IAM Role
1919
local DEFAULT_TOKEN_EXPIRATION = 60*60*24 -- in seconds
2020

21-
-- configur cache Manager for IAM crendentials
21+
-- configure cache Manager for IAM crendentials
2222
local iamCache = cacheCls:new()
2323

2424
-- per nginx process cache to store IAM credentials
@@ -45,7 +45,6 @@ local function initIamCache(shared_cache_dict)
4545
dict = shared_cache_dict,
4646
ttl = function (value)
4747
local value_o = cjson.decode(value)
48-
ngx.log(ngx.DEBUG, "ExpireAt=", tostring(value_o.ExpireAt))
4948
local expiryTimeUTC = value.ExpireAtTimestamp or awsDate.convertDateStringToTimestamp(value_o.ExpireAt, true)
5049
local expiryTimeInSeconds = expiryTimeUTC - os.time()
5150
return math.min(DEFAULT_TOKEN_EXPIRATION, expiryTimeInSeconds)

Diff for: src/lua/api-gateway/aws/AWSSTSCredentials.lua

+176
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
--[[
2+
Copyright 2016 Adobe Systems Incorporated. All rights reserved.
3+
4+
This file is licensed to you under the Apache License, Version 2.0 (the
5+
"License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
6+
7+
http://www.apache.org/licenses/LICENSE-2.0
8+
9+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR RESPRESENTATIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
10+
]]
11+
12+
--
13+
-- STS Credentials provider
14+
-- User: ddascal
15+
-- Date: 19/03/16
16+
-- Time: 21:26
17+
--
18+
local cjson = require "cjson"
19+
local SecurityTokenService = require "api-gateway.aws.sts.SecurityTokenService"
20+
local awsDate = require "api-gateway.aws.AwsDateConverter"
21+
local cacheCls = require "api-gateway.cache.cache"
22+
23+
local _M = {}
24+
25+
26+
local IAM_DEFAULT_SECURITY_CREDENTIALS_HOST = "169.254.169.254" -- used by IAM
27+
local IAM_DEFAULT_SECURITY_CREDENTIALS_PORT = "80" -- used by IAM
28+
local IAM_DEFAULT_SECURITY_CREDENTIALS_URL = "/latest/meta-data/iam/security-credentials/" -- used by IAM
29+
30+
---
31+
-- default expiration for the STS token
32+
local DEFAULT_TOKEN_EXPIRATION = 60 * 60 * 1 -- in seconds
33+
34+
-- configure cache Manager for IAM crendentials
35+
local stsCache = cacheCls:new()
36+
37+
local function initStsCache(shared_cache_dict)
38+
local localCache = require "api-gateway.cache.store.localCache":new({
39+
dict = shared_cache_dict,
40+
ttl = function (value)
41+
local value_o = cjson.decode(value)
42+
local expiryTimeUTC = value.ExpireAtTimestamp or value_o.ExpireAt -- ExpireAt is already an epoch time awsDate.convertDateStringToTimestamp(value_o.ExpireAt, true)
43+
local expiryTimeInSeconds = expiryTimeUTC - os.time()
44+
return math.min(DEFAULT_TOKEN_EXPIRATION, expiryTimeInSeconds)
45+
end
46+
})
47+
48+
stsCache:addStore(localCache)
49+
end
50+
51+
---
52+
-- @param o Init object
53+
-- o.role_ARN -- required. The Amazon Resource Name (ARN) of the role to assume.
54+
-- o.role_session_name -- required. An identifier for the assumed role session.
55+
-- o.policy -- optional. An IAM policy in JSON format.
56+
-- o.security_credentials_timeout -- optional. specifies when the token should expire. Defaults to 1 hour.
57+
-- o.external_id -- optional. A unique identifier used by third parties
58+
-- o.iam_user -- optional. iam_user. if not defined it'll be auto-discovered
59+
-- o.security_credentials_host -- optional. AWS Host to read IAM credentials from. Defaults to "169.254.169.254"
60+
-- o.security_credentials_port -- optional. AWS Port to read IAM credentials. Defaults to 80.
61+
-- o.security_credentials_url -- optional. AWS URI to read IAM credentials. Defaults to "/latest/meta-data/iam/security-credentials/"
62+
-- o.shared_cache_dict -- optional. For performance improvements the credentials may be stored in a share dict.
63+
--
64+
function _M:new(o)
65+
o = o or {}
66+
setmetatable(o, self)
67+
self.__index = self
68+
if (o ~= nil) then
69+
-- config options specific to STS
70+
self.security_credentials_timeout = o.security_credentials_timeout or DEFAULT_TOKEN_EXPIRATION
71+
self.role_ARN = o.role_ARN
72+
self.role_session_name = o.role_session_name
73+
self.policy = o.policy
74+
self.external_id = o.external_id
75+
76+
-- config options specific to IAM User
77+
self.iam_user = o.iam_user
78+
self.iam_security_credentials_host = o.iam_security_credentials_host or IAM_DEFAULT_SECURITY_CREDENTIALS_HOST
79+
self.iam_security_credentials_port = o.iam_security_credentials_port or IAM_DEFAULT_SECURITY_CREDENTIALS_PORT
80+
self.iam_security_credentials_url = o.iam_security_credentials_url or IAM_DEFAULT_SECURITY_CREDENTIALS_URL
81+
82+
-- config options for static AWS Credetials
83+
self.aws_region = o.aws_region or "us-east-1" -- TBD: when aws_region is not provided should it go to sts.amazonaws.com ?
84+
self.aws_secret_key = o.aws_secret_key
85+
self.aws_access_key = o.aws_access_key
86+
87+
-- shared dict used to cache STS and IAM credentials
88+
self.shared_cache_dict = o.shared_cache_dict
89+
if (self.shared_cache_dict) then
90+
initStsCache(self.shared_cache_dict)
91+
end
92+
end
93+
return o
94+
end
95+
96+
---
97+
-- Return credentials from AWS' Security Token Service
98+
-- Sample response from STS :
99+
100+
-- {
101+
-- "AssumeRoleResponse": {
102+
-- "AssumeRoleResult": {
103+
-- "AssumedRoleUser": {
104+
-- "Arn": "arn:aws:sts::123456789012:assumed-role/xaccounts3access/s3-access-example",
105+
-- "AssumedRoleId": "AROAJEMMODNR6NAEO0000:s3-access-example"
106+
-- },
107+
-- "Credentials": {
108+
-- "AccessKeyId": "access-key",
109+
-- "Expiration": 1460355675,
110+
-- "SecretAccessKey": "some-secret",
111+
-- "SessionToken": "session-token"
112+
-- },
113+
-- "PackedPolicySize": null
114+
-- },
115+
-- "ResponseMetadata": {
116+
-- "RequestId": "91609a0a-ffa2-11e5-97e6-bbbbbbbbb"
117+
-- }
118+
-- }
119+
--}
120+
121+
function _M:getCredentialsFromSTS()
122+
local sts = SecurityTokenService:new({
123+
shared_cache_dict = self.shared_cache_dict,
124+
security_credentials_host = self.iam_security_credentials_host,
125+
security_credentials_port = self.iam_security_credentials_port,
126+
aws_debug = self.aws_debug,
127+
aws_region = self.aws_region,
128+
aws_conn_keepalive = 60000, -- how long to keep the sockets used for AWS alive
129+
aws_conn_pool = 10 -- the connection pool size for sockets used to connect to AWS
130+
})
131+
local response, code, headers, status, body = sts:assumeRole(self.role_ARN,
132+
self.role_session_name,
133+
self.policy,
134+
self.security_credentials_timeout,
135+
self.external_id)
136+
137+
if (code ~= ngx.HTTP_OK) then
138+
ngx.log(ngx.WARN, "Could not get STS credentials. Response Code=", tostring(code), ", response body=", tostring(body))
139+
return nil
140+
end
141+
142+
local stsCreds = response.AssumeRoleResponse.AssumeRoleResult.Credentials
143+
stsCreds.ExpireAt = tostring(stsCreds["Expiration"])
144+
stsCreds.ExpireAtTimestamp = stsCreds["Expiration"]
145+
-- stsCreds.Expiration is already an epoch time
146+
-- stsCreds.ExpireAtTimestamp = awsDate.convertDateStringToTimestamp(stsCreds.ExpireAt, true)
147+
148+
-- store it in cache
149+
stsCache:put("sts_" .. tostring(self.role_ARN) .. "_" .. tostring(self.role_session_name), cjson.encode(stsCreds))
150+
return stsCreds
151+
end
152+
153+
function _M:getSecurityCredentials()
154+
--1. try to read it from the local cache
155+
local stsCreds = stsCache:get("sts_" .. tostring(self.role_ARN) .. "_" .. tostring(self.role_session_name))
156+
if (stsCreds ~= nil) then
157+
local sts = cjson.decode(stsCreds)
158+
-- check to see if the cached credentials are still valid
159+
local now_in_secs = ngx.time()
160+
local expireAtTimestamp = sts.ExpireAtTimestamp or now_in_secs
161+
if (now_in_secs >= expireAtTimestamp) then
162+
ngx.log(ngx.WARN, "Current STS token expired " .. tostring(expireAtTimestamp - now_in_secs) .. " seconds ago. Obtaining a new token.")
163+
else
164+
return sts.AccessKeyId, sts.SecretAccessKey, sts.SessionToken, sts.ExpireAt, sts.ExpireAtTimestamp
165+
end
166+
end
167+
--2. get credentials from SecurityTokenService
168+
stsCreds = self:getCredentialsFromSTS()
169+
if (stsCreds == nil) then
170+
return "",""
171+
end
172+
return stsCreds.AccessKeyId, stsCreds.SecretAccessKey, stsCreds.SessionToken, stsCreds.ExpireAt, stsCreds.ExpireAtTimestamp
173+
end
174+
175+
return _M
176+

0 commit comments

Comments
 (0)