* core: add modular `network_proxy` support
Co-authored-by: @ImpostorKeanu
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* move modules around
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* add caddyfile implementation
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* address feedbcak
* Apply suggestions from code review
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* adapt ForwardProxyURL to use the NetworkProxyRaw
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* remove redundant `url` in log
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* code review
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* remove `.source` from the module ID
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* reverseproxy: allow user to define source address
Closes#6503
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* reverse_proxy: caddyfile support for local_address
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* added health_follow_redirect in active health checks
* chore: code format
* chore: refactore reversproxy healthcheck redirect variable name and description of the same
* chore: formatting
* changed reverse proxy health check status code range to be between 200-299
* chore: formatting
---------
Co-authored-by: aliasgar <joancena1268@mail.com>
* added new modular ca providers to caddy tls HttpTransport
* reverse-proxy, httptransport: added tests and caddyfile support for ca module
---------
Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* reverseproxy: active health check allows configurable health_passes and health_fails
* Need to reset counters after recovery
* rename methods to be more clear that these are coming from active health checks
* do not export methods
* reverseproxy: Add more debug logs
This makes debug logging very noisy when reverse proxying, but I guess
that's the point.
This has shown to be useful in troubleshooting infrastructure issues.
* Update modules/caddyhttp/reverseproxy/streaming.go
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Update modules/caddyhttp/reverseproxy/streaming.go
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* Add opt-in `trace_logs` option
* Rename to VerboseLogs
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* use gofmput to format code
* use gci to format imports
* reconfigure gci
* linter autofixes
* rearrange imports a little
* export GOOS=windows golangci-lint run ./... --fix
* reverseproxy: do not parse upstream address too early if it contains replaceble parts
* remove unused method
* cleanup
* accommodate partially replaceable port
This allows users to, for example, get upstreams from multiple SRV
endpoints in order (such as primary and secondary clusters).
Also, gofmt went to town on the comments, sigh
* reverseproxy: Implement retry count, alternative to try_duration
* Add Caddyfile support for `retry_match`
* Refactor to deduplicate matcher parsing logic
* Fix lint
* Make reverse proxy TLS server name replaceable for SNI upstreams.
* Reverted previous TLS server name replacement, and implemented thread safe version.
* Move TLS servername replacement into it's own function
* Moved SNI servername replacement into httptransport.
* Solve issue when dynamic upstreams use wrong protocol upstream.
* Revert previous commit.
Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb63e66bdcac8e1ed4520c9d135cb011d
* Added SkipTLSPorts option to http transport.
* Fix typo in test config file.
* Rename config option as suggested by Matt
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update code to match renamed config option.
* Fix typo in config option name.
* Fix another typo that I missed.
* Tests not completing because of apparent wrong ordering of options.
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* reverseproxy: New `copy_response` handler for `handle_response` routes
Followup to #4298 and #4388.
This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream.
Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.
To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.
Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.
Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.
* Delete the "Content-Length" header if we aren't copying
Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:
```
curl: (18) transfer closed with 18 bytes remaining to read
```
To replicate:
```
{
admin off
debug
}
:8881 {
reverse_proxy 127.0.0.1:8882 {
@200 status 200
handle_response @200 {
header Foo bar
}
}
}
:8882 {
header Content-Type application/json
respond `{"hello": "world"}` 200
}
```
* Implement `copy_response_headers`, with include/exclude list support
* Apply suggestions from code review
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* reverseproxy: Begin refactor to enable dynamic upstreams
Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU
* Implement SRV and A/AAA upstream sources
Also get upstreams at every retry loop iteration instead of just once
before the loop. See #4442.
* Minor tweaks from review
* Limit size of upstreams caches
* Add doc notes deprecating LookupSRV
* Provision dynamic upstreams
Still WIP, preparing to preserve health checker functionality
* Rejigger health checks
Move active health check results into handler-specific Upstreams.
Improve documentation regarding health checks and upstreams.
* Deprecation notice
* Add Caddyfile support, use `caddy.Duration`
* Interface guards
* Implement custom resolvers, add resolvers to http transport Caddyfile
* SRV: fix Caddyfile `name` inline arg, remove proto condition
* Use pointer receiver
* Add debug logs
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
