How to modify request headers on TLS connection

[sandrich]

Hi,

I have been trying to modify the request headers when using a TLS endpoint. The goal is to modify/annotate headers before it gets forwarded to the downstream connection.

With pure http this works just fine

class FsProxyPlugin(HttpProxyBasePlugin):
    def before_upstream_connection(
            self, request: HttpParser,
    ) -> Optional[HttpParser]:
        request.add_header(b'test-header', b'bla')


        return request

curl -x localhost:8899  http://httpbin.org/get 
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "Test-Header": "bla", 
    "User-Agent": "curl/8.6.0", 
    "X-Amzn-Trace-Id": "Root=1-6663010e-67beef541cb0758024297e78"
  }, 
  "url": "http://httpbin.org/get"
}

vs

curl -x localhost:8899 --cacert ca-cert.pem https://httpbin.org/get
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/8.6.0", 
    "X-Amzn-Trace-Id": "Root=1-66630130-5908034b3c98368153643828"
  }, 
  "url": "https://httpbin.org/get"
}

I am starting with

if __name__ == "__main__":

    with proxy.Proxy(
        input_args=[
            "--threaded",
            "--proxy-config='{}'",
            "--ca-key-file=/Users/chrissandrini/dev/proxy.py/ca-key.pem",
            "--ca-cert-file=/Users/chrissandrini/dev/proxy.py/ca-cert.pem",
            "--ca-signing-key-file=/Users/chrissandrini/dev/proxy.py/ca-signing-key.pem",
            "--ca-file=/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages/pip/_vendor/certifi/cacert.pem"
        ],
        plugins=[FsProxyPlugin]
    ) as _:
        proxy.sleep_loop()

[abhinavsingh]

@sandrich Do you mean modification of request headers is not working under TLS interception mode?

[sandrich]

yes correct @abhinavsingh . I am trying to modify the request headers before forwarding the request.

[abhinavsingh]

I have sent out a PR for example ModifyRequestHeaderPlugin plugin.

Please refer to my comment here https://github.com/abhinavsingh/proxy.py/pull/1420/files#diff-c908d29a70bb1577a9c571fb291e806b0624b0d7f81a9f6ff877e5fec2cae312R22-R26.

[sandrich]

Amazing thanks for that. Is a self signed CA actually required to add headers to the request? Since in our environment self signed CAs are not allowed.