v1.0.0

New Features:

- **(#172) New SDK convenience function:**
  Add a new helper function `sdk.FromRequest()` allowing to retrieve Sqreen's
  request context and perform SDK calls. It is equivalent to
  `sdk.FromContext(r.Context())`.

- **(#156) Performance monitoring:**
  Monitor the execution time of requests protected by Sqreen. Optionally, it is
  possible to enforce the maximum amount of time Sqreen is allowed to run per
  request: Sqreen's monitoring and protections will only run for the given
  amount of time. This option is disabled by default and should be used with
  caution as it can lead to partially protected requests.
  The resulting performance monitoring diagrams and setting are available at
  <https://my.sqreen.com/application/goto/settings/performance>.
  Note that the execution time diagram cannot be used as a strict Application
  Performance Monitoring diagram as it is based on a lossy representation. It
  gives rough estimates of the actual execution time.

- **(#170) Transparent response writer instrumentation:**
  Make the HTTP response writer instrumentation transparent by providing the
  same set of interfaces as the instrumented HTTP response writer. The set of
  interfaces is currently every optional `net/http` response writer interface,
  along with some relevant `io` interfaces, among which:
    - `http.Flusher`: for HTTP streaming support (multipart, chunked...).
    - `http.Pusher`: for HTTP2 server push support.
    - `http.Hijacker`: for websocket server support (experimental).
    - `io.ReaderFrom`: for optimized copies (eg. file copies)
    - `io.WriteString`: for optimized string copies.

- **(#163) HTTP status code 404 (not found) monitoring:**
  Automatically log a security event when the response status code is 404. This
  event is used by an internal Sqreen backend playbook to detect security scans.

- **(#163) Scalable security event throughput:**
  To be able to handle a higher throughput of security events, the agent can now
  scale its number of goroutines. An extra goroutine is created every time the
  internal event queue is full, up to the number of available CPUs. Note that
  the agent still drops security events when the event queue is full in order to
  avoid slowing down the host application.

- **(#165) Agent errors in the request hot-path:**
  To avoid slowing down request handlers, agent errors happening in the request
  hot path are now logged based on an exponential backoff algorithm.
  This is disabled when the agent log level is `debug`.

Breaking Change:

- **(#168) SDK return values:**
  The SDK function and method return values are no longer pointer values but Go
  interface values. This may break integrations using explicit SDK return types,
  and we recommend to instead use type-inference when possible. This change will
  allow us to transparently change the actual return values without involving
  any further breaking change.
  As of today, the actual return value is a structure small enough to be
  returned by value in order to save memory-allocation and garbage-collection
  time. Returning an interface value allows to hide such implementation detail.

Fixes:

- **(#167) Playbook security response events:**
  Fix playbook security response events (blocking or redirecting a user or ip)
  so that Sqreen's dashboard can properly display them and link them to their
  source playbook.

- **(#169) SQL-injection protection with Elastic APM:**
  Fix the detection of the SQL dialect when the SQL driver is instrumented by
  Elastic's APM tracer. This requires Elastic's Go agent version greater than
  `v1.9.0`.

- **(#164) Echo middleware:**
  Fix the response status code monitoring when Echo's request handlers return an
  error.

- **(#166) Gin middleware:**
  Fix the response content-length monitoring of default responses
  (ie. when the handler does nothing).

Author: Julio Guerra

.github/workflows/agent-tests.yaml

@@ -9,6 +9,9 @@ jobs:
       matrix:
         runs-on: [ macos-latest, ubuntu-latest, windows-latest ]
         go-version: [ 1, 1.15, 1.14, 1.13, 1.12 ]
+        go-test-options:
+          - ""
+          - "-tags sqassert -race"
       fail-fast: false
     runs-on: ${{ matrix.runs-on }}
     steps:
@@ -17,7 +20,7 @@ jobs:
         uses: actions/setup-go@v2
         with:
           go-version: ${{ matrix.go-version }}
-      - run: go test ./...
+      - run: go test ./... # note: do not support large number of goroutines for -race
 
   # Same tests but on the official golang container for linux
   # Docker for Windows is not yet available on Github Actions.
@@ -35,7 +38,7 @@ jobs:
       # Install gcc and the libc headers on alpine images
       - if: ${{ matrix.distribution == 'alpine' }}
         run: apk add gcc musl-dev libc6-compat git
-      - run: go test ./...
+      - run: go test ${{ matrix.go-test-options }} ./...
 
   # debian stretch doesn't have the latest go versions
   golang-debian-stretch-container:
@@ -48,4 +51,4 @@ jobs:
       image: golang:${{ matrix.go-version }}-stretch
     steps:
       - uses: actions/checkout@v2
-      - run: go test ./...
+      - run: go test ${{ matrix.go-test-options }} ./...

.github/workflows/codeql-analysis.yml

@@ -0,0 +1,69 @@
+# For most projects, this workflow file will not need changing; you simply need
+# to commit it to your repository.
+#
+# You may wish to alter this file to override the set of languages analyzed,
+# or to provide custom queries or build logic.
+name: "CodeQL"
+
+on:
+  push:
+    branches: [master, dev]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [feature/*, fix/*, hotfix/*, release/*]
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # Override automatic language detection by changing the below list
+        # Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript', 'python']
+        language: ['go']
+        # Learn more...
+        # https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+      with:
+        # We must fetch at least the immediate parents so that if this is
+        # a pull request then we can checkout the head.
+        fetch-depth: 2
+
+    # If this run was triggered by a pull request event, then checkout
+    # the head of the pull request instead of the merge commit.
+    - run: git checkout HEAD^2
+      if: ${{ github.event_name == 'pull_request' }}
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v1
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file. 
+        # Prefix the list here with "+" to use these queries and those in the config file.
+        # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v1
+
+    # ℹ️ Command-line programs to run using the OS shell.
+    # 📚 https://git.io/JvXDl
+
+    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
+    #    and modify them (or add more) to build your code if your project
+    #    uses a compiled language
+
+    #- run: |
+    #   make bootstrap
+    #   make release
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v1

CHANGELOG.md

@@ -18,64 +18,66 @@ Sqreen provides automatic defense against attacks:
 - Protect with security modules: RASP (Runtime Application Self-Protection),
   in-app WAF (Web Application Firewall), Account takeovers and more.
 
-- Sqreen’s modules adapt to your application stack with no need of configuration.
+- Sqreen’s modules adapt to your application stack with no need of
+  configuration.
 
-- Prevent attacks from the OWASP Top 10 (Injections, XSS and more), 0-days,
-  data Leaks, and more.
-  
-- Create security automation playbooks that automatically react against
-  your advanced business-logic threats.
+- Prevent attacks from the OWASP Top 10 (Injections, XSS and more), 0-days, data
+  Leaks, and more.
+
+- Create security automation playbooks that automatically react against your
+  advanced business-logic threats.
 
 For more details, visit [sqreen.com](https://www.sqreen.com/)
 
 # Quick start
 
 1. Use the middleware function for the Go web framework you use:
-   - [net/http](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqhttp)
-   - [Gin](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqgin)
-   - [Echo](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqecho/v4)
+    - [net/http](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqhttp)
+    - [Gin](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqgin)
+    - [Echo](https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqecho/v4)
 
    If your framework is not listed, it is usually possible to use instead the
-   standard `net/http` middleware. If not, please, let us know by [creating an
-   issue](http://github.com/sqreen/go-agent/issues/new).
+   standard `net/http` middleware. If not, please, let us know
+   by [creating an issue](http://github.com/sqreen/go-agent/issues/new).
 
 1. Compile your program with Sqreen
 
    Sqreen's dynamic configuration of your protection is made possible thanks to
-   Go instrumentation. It is safely performed at compilation time by the following
-   instrumentation tool.
+   Go instrumentation. It is safely performed at compilation time by the
+   following instrumentation tool.
 
    Install the following instrumentation tool and compile your program using it
    in order to enable Sqreen.
 
-   1. Use `go build` to download and compile the instrumentation tool:
-
-      ```console
-      $ go build github.com/sqreen/go-agent/sdk/sqreen-instrumentation
-      ```
-
-   1. Configure the Go toolchain to use it:
-
-      Use the instrumentation tool using the go options
-      `-a -toolexec /path/to/sqreen-instrumentation`.
-
-      It can be done either in your Go compilation command lines or by setting the
-      `GOFLAGS` environment variable.
-      
-      For example, the following two commands are equivalent:
-      ```console
-      $ go build -a -toolexec $PWD/sqreen-instrumentation-tool my-project
-      $ env GOFLAGS="-a -toolexec $PWD/sqreen-instrumentation-tool" go build my-project
-      ```
-    
-1. [Signup to Sqreen](https://my.sqreen.io/signup) to get a token for your app,
-   and store it in the agent's configuration file `sqreen.yaml`:
-   
+    1. Use `go install` to compile the instrumentation tool:
+       ```console
+       $ go install github.com/sqreen/go-agent/sdk/sqreen-instrumentation-tool
+       ```
+
+       By default, the resulting `sqreen-instrumentation-tool` tool is installed
+       in the
+       `bin` directory of the `GOPATH`. You can find it using `go env GOPATH`.
+
+    1. Configure the Go toolchain to use it:
+
+       Use the instrumentation tool using the go options
+       `-a -toolexec /path/to/sqreen-instrumentation-tool`.
+
+       It can be done either in your Go compilation command lines or by setting
+       the `GOFLAGS` environment variable.
+
+       For example, the following two commands are equivalent:
+       ```console
+       $ go build -a -toolexec $(go env GOPATH)/bin/sqreen-instrumentation-tool my-project
+       $ env GOFLAGS="-a -toolexec $(go env GOPATH)/bin/sqreen-instrumentation-tool" go build my-project
+       ```
+
+1. [Signup to Sqreen](https://my.sqreen.io/signup) to get your app credentials:
     ```sh
     app_name: Your Go app name
     token: your token
     ```
-   
+
    This file can be stored in your current working directory when starting the
    executable, the same directory as your app's executable file, or in any other
    path by defining the configuration file location into the environment
@@ -87,9 +89,9 @@ Congratulations, your Go web application is now protected by Sqreen!
 <img width="60%" src="./doc/images/blocking-page-with-gopher.png" alt="Sqreen for Go" title="Sqreen for Go" />
 </p>
 
-
 # Advanced integration
 
-Optionally, use the SDK to perform [user monitoring](https://docs.sqreen.com/go/user-monitoring/)
+Optionally, use the SDK to
+perform [user monitoring](https://docs.sqreen.com/go/user-monitoring/)
 or [custom security events](https://docs.sqreen.com/go/custom-events/) you would
 like to track and possibly block.