Merge pull request #231 from andrestoll/207-output-stats

Add optional input parameter to include all results in the output

Author: alexcasalboni

README-INPUT-OUTPUT.md

@@ -23,6 +23,7 @@ The state machine accepts the following input parameters:
 * **discardTopBottom** (number between 0.0 and 0.4, by default is 0.2): By default, the state machine will discard the top/bottom 20% of "outliers" (the fastest and slowest), to filter out the effects of cold starts that would bias the overall averages. You can customize this parameter by providing a value between 0 and 0.4, with 0 meaning no results are discarded and 0.4 meaning that 40% of the top/bottom results are discarded (i.e. only 20% of the results are considered).
 * **sleepBetweenRunsMs** (integer) If provided, the time in milliseconds that the tuner function will sleep/wait after invoking your function, but before carrying out the Post-Processing step, should that be provided. This could be used if you have aggressive downstream rate limits you need to respect. By default this will be set to 0 and the function won't sleep between invocations. Setting this value will have no effect if running the invocations in parallel.
 * **disablePayloadLogs** (boolean) If provided and set to a truthy value, suppresses `payload` from error messages and logs. If `preProcessorARN` is provided, this also suppresses the output payload of the pre-processor.
+* **includeOutputResults** (boolean) If provided and set to true, the average cost and average duration for every power value configuration will be included in the state machine output.
 
 ## State machine configuration (at deployment time)
 
@@ -151,7 +152,8 @@ The state machine will return the following output:
       "executionCost": 0.00045,
       "lambdaCost": 0.0005252,
       "visualization": "https://lambda-power-tuning.show/#<encoded_data>"
-    }
+    },
+    "stats": [{ "averagePrice": 0.0000002083, "averageDuration": 2.9066666666666667, "value": 128}, ... ]
   }
 }
 ```
@@ -164,4 +166,4 @@ More details on each value:
 * **results.stateMachine.executionCost**: the AWS Step Functions cost corresponding to this state machine execution (fixed value for "worst" case)
 * **results.stateMachine.lambdaCost**: the AWS Lambda cost corresponding to this state machine execution (depending on `num` and average execution time)
 * **results.stateMachine.visualization**: if you visit this autogenerated URL, you will be able to visualize and inspect average statistics about cost and performance; important note: average statistics are NOT shared with the server since all the data is encoded in the URL hash ([example](https://lambda-power-tuning.show/#gAAAAQACAAQABsAL;ZooQR4yvkUa/pQRGRC5zRaADHUVjOftE;QdWhOEMkoziDT5Q4xhiIOMYYiDi6RNc4)), which is available only client-side
-
+* **results.stats**: the average duration and cost for every tested power value configuration (only included if `includeOutputResults` is set to a truthy value)

README.md

@@ -18,10 +18,8 @@ Please note that the input function will be executed in your AWS account - perfo
 
 It's pretty simple and you can visually inspect each step in the AWS management console.
 
-
 ![state-machine](imgs/state-machine-screenshot.png?raw=true)
 
-
 ## What results can I expect from Lambda Power Tuning?
 
 The state machine will generate a visualization of average cost and speed for each power configuration.
@@ -36,17 +34,14 @@ How to interpret the chart above: execution time goes from 35s with 128MB to les
 
 How to interpret the chart above: execution time goes from 2.4s with 128MB to 300ms with 1GB, for the very same average cost.
 
-
-## How to deploy the state machine 
+## How to deploy the state machine
 
 There are a few options documented [here](README-DEPLOY.md).
 
-
 ## How to execute the state machine
 
 You can execute the state machine manually or programmatically, see the documentation [here](README-EXECUTE.md).
 
-
 ## State Machine Input and Output
 
 Here's a typical execution input with basic parameters:
@@ -81,7 +76,6 @@ The state machine output will look like this:
 
 Full output documentation [here](README-INPUT-OUTPUT.md#user-content-state-machine-output).
 
-
 ## Data visualization
 
 You can visually inspect the tuning results to identify the optimal tradeoff between cost and performance.
@@ -106,7 +100,6 @@ Power Tuner UI repository: [mattymoomoo/aws-power-tuner-ui](https://github.com/m
 
 [Here](README-ADVANCED.md) you can find out more about some advanced features of this project, its internals, and some considerations about security and execution cost.
 
-
 ## CHANGELOG (SAR versioning)
 
 From most recent to oldest, with major releases in bold:
@@ -157,7 +150,6 @@ From most recent to oldest, with major releases in bold:
 * ***1.0.0*** (2019-05-13): AWS SAM refactor (published on SAR)
 * *0.0.1* (2017-03-27): previous project (serverless framework)
 
-
 ## Contributing
 
 Feature requests and pull requests are more than welcome!

lambda/analyzer.js

@@ -25,7 +25,19 @@ module.exports.handler = async(event, context) => {
         return console.log('[Dry-run] Skipping analysis');
     }
 
-    return findOptimalConfiguration(event);
+    const result = findOptimalConfiguration(event);
+
+    if (!!event.includeOutputResults) {
+        // add stats to final result
+        result.stats = event.stats.map(stat => ({
+            value: stat.value,
+            averagePrice: stat.averagePrice,
+            averageDuration: stat.averageDuration,
+            // totalCost is omitted here
+        }));
+    }
+
+    return result;
 };
 
 const getStrategy = (event) => {

test/unit/test-lambda.js

@@ -1647,6 +1647,73 @@ describe('Lambda Functions', async() => {
 
         });
 
+        it('should output all results if "includeOutputResults" is set to true', async() => {
+            const event = {
+                strategy: 'speed',
+                stats: [
+                    { value: '128', averagePrice: 100, averageDuration: 300, totalCost: 1 },
+                    { value: '256', averagePrice: 200, averageDuration: 200, totalCost: 1 },
+                    { value: '512', averagePrice: 300, averageDuration: 100, totalCost: 1 },
+                ],
+                includeOutputResults: true,
+            };
+
+            const result = await invokeForSuccess(handler, event);
+            expect(result).to.be.an('object');
+
+            expect(result).to.have.property('stats');
+            expect(result.stats).to.eql(event.stats.map(stat => ({
+                value: stat.value,
+                averagePrice: stat.averagePrice,
+                averageDuration: stat.averageDuration
+            })));
+
+            expect(result.stats[0]).to.not.have.property('totalCost');
+
+            expect(result).to.have.property('power');
+            expect(result).to.have.property('cost');
+            expect(result).to.have.property('duration');
+            expect(result.stateMachine).to.be.an('object');
+
+        });
+
+        it('should not output any results if "includeOutputResults" is set to false', async() => {
+            const event = {
+                strategy: 'speed',
+                stats: [
+                    { value: '128', averagePrice: 100, averageDuration: 300, totalCost: 1 },
+                ],
+                includeOutputResults: false,
+            };
+
+            const result = await invokeForSuccess(handler, event);
+            expect(result).to.be.an('object');
+
+            expect(result).to.not.have.property('stats');
+            expect(result).to.have.property('power');
+            expect(result).to.have.property('cost');
+            expect(result).to.have.property('duration');
+            expect(result.stateMachine).to.be.an('object');
+        });
+
+        it('should not output any results if "includeOutputResults" is not set', async() => {
+            const event = {
+                strategy: 'speed',
+                stats: [
+                    { value: '128', averagePrice: 100, averageDuration: 300, totalCost: 1 },
+                ],
+            };
+
+            const result = await invokeForSuccess(handler, event);
+            expect(result).to.be.an('object');
+
+            expect(result).to.not.have.property('stats');
+            expect(result).to.have.property('power');
+            expect(result).to.have.property('cost');
+            expect(result).to.have.property('duration');
+            expect(result.stateMachine).to.be.an('object');
+        });
+
     });
 
     describe('optimizer', async() => {