Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 63 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Routes requests and builds middleware chains based on configuration.
- **Cache** - In-memory response caching with TTL
- **Rewrite** - URL/header/query parameter manipulation
- **Options** - Handles CORS preflight requests
- **HAR Collector** - Records all request/response pairs to an HTTP Archive (HAR 1.2) file

### Infrastructure (`internal/infra`)

Expand All @@ -52,10 +53,10 @@ Colored logging and request/response formatting.

1. **Client sends request** → UNCORS server
2. **Route matching** - Find mapping by host/port
3. **Middleware pipeline** - Apply options, rewrite, cache, static
3. **Middleware pipeline** - Apply HAR capture → options cache static
4. **Handler selection** - Choose mock, script, or proxy handler
5. **CORS modification** - Add/modify CORS headers
6. **Response** - Return to client
6. **Response** - Return to client (HAR entry is enqueued asynchronously)

Example CORS headers added:

Expand All @@ -76,6 +77,7 @@ uncors/
│ ├── contracts/ # Interfaces (handler, logger, http client)
│ ├── handler/ # Request handlers & middleware
│ │ ├── cache/
│ │ ├── har/ # HAR collector middleware & async writer
│ │ ├── mock/
│ │ ├── proxy/
│ │ ├── script/
Expand All @@ -89,6 +91,65 @@ uncors/
└── tests/ # Integration tests
```

## HAR Collector

The HAR (HTTP Archive) collector records every proxied request/response pair to
a [HAR 1.2](https://w3c.github.io/web-performance/specs/HAR/Overview.html) file.
It is enabled per-mapping via the `har.file` config key and is implemented as a
standalone middleware (`internal/handler/har`).

**Design goals:**
- **Non-blocking** — the middleware never blocks the request goroutine. Entries
are sent over a buffered channel (capacity 4096). If the channel is full the
entry is silently dropped rather than stalling the request.
- **High throughput writes** — a single background goroutine serialises all
disk I/O. After every new entry it atomically replaces the HAR file using a
write-to-tmp-then-rename strategy so the file is always in a valid state.
- **Per-mapping isolation** — each mapping creates its own `Writer` instance and
its own output file, so traffic from different mappings can be captured
independently.
- **Lifecycle management** — `Writer` implements `io.Closer`. The app registers
each writer via `registerCloser`; on shutdown or config reload it calls
`Close()` which drains the channel and flushes outstanding entries before
stopping the background goroutine.

**Configuration:**

The simplest form uses a string shorthand — the value is treated as the output file path:

```yaml
mappings:
- from: http://localhost:3000
to: https://api.example.com
har: ./recordings/api.har
```

For full control, use the object form:

```yaml
mappings:
- from: http://localhost:3000
to: https://api.example.com
har:
file: ./recordings/api.har
capture-secure-headers: true # default: false
```

**Security-sensitive headers:**

By default the following headers are **excluded** from HAR entries to avoid
persisting credentials on disk. Set `capture-secure-headers: true` to include
them.

| Header | Why it is sensitive |
|-----------------------|--------------------------------------------|
| `Cookie` | Session identifiers |
| `Set-Cookie` | Session identifiers set by the server |
| `Authorization` | Bearer tokens, Basic credentials |
| `WWW-Authenticate` | Server auth challenges (reveals scheme) |
| `Proxy-Authorization` | Proxy credentials |
| `Proxy-Authenticate` | Proxy auth challenges |

## Key Design Patterns

**Middleware Pattern** - Composable request/response processing
Expand Down
47 changes: 47 additions & 0 deletions internal/config/har.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
package config

import (
"reflect"

"github.com/mitchellh/mapstructure"
)

// HARConfig defines settings for the HAR (HTTP Archive) collector middleware.
// When File is non-empty, all requests/responses passing through the proxy
// for this mapping will be recorded to the specified HAR file.
type HARConfig struct {
File string `mapstructure:"file"`
CaptureSecureHeaders bool `mapstructure:"capture-secure-headers"`
}

func (h HARConfig) Enabled() bool {
return h.File != ""
}

func (h HARConfig) Clone() HARConfig {
return HARConfig{
File: h.File,
CaptureSecureHeaders: h.CaptureSecureHeaders,
}
}

var harConfigType = reflect.TypeFor[HARConfig]()

// HARConfigHookFunc returns a mapstructure decode hook that allows HARConfig
// to be specified as a plain string in YAML/config files.
//
// Short form: har: ./recordings/api.har
// Full form: har: { file: ./recordings/api.har, capture-secure-headers: true }.
func HARConfigHookFunc() mapstructure.DecodeHookFunc {
return func(f reflect.Type, t reflect.Type, rawData any) (any, error) {
if t != harConfigType || f.Kind() != reflect.String {
return rawData, nil
}

if file, ok := rawData.(string); ok {
return HARConfig{File: file}, nil
}

return rawData, nil
}
}
68 changes: 68 additions & 0 deletions internal/config/har_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package config_test

import (
"testing"

"github.com/evg4b/uncors/internal/config"
"github.com/evg4b/uncors/testing/testutils"
"github.com/mitchellh/mapstructure"
"github.com/spf13/viper"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestHARConfigHookFunc(t *testing.T) {
decode := func(t *testing.T, raw any) config.HARConfig {
t.Helper()

var out config.HARConfig

decoder, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{
Result: &out,
DecodeHook: config.HARConfigHookFunc(),
})
require.NoError(t, err)
require.NoError(t, decoder.Decode(raw))

return out
}

t.Run("string shorthand sets File", func(t *testing.T) {
cfg := decode(t, "./recordings/api.har")

Check failure on line 31 in internal/config/har_test.go

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal "./recordings/api.har" 3 times.

See more on https://sonarcloud.io/project/issues?id=evg4b_uncors&issues=AZz06vP0tslefFn7ycMi&open=AZz06vP0tslefFn7ycMi&pullRequest=71
assert.Equal(t, config.HARConfig{File: "./recordings/api.har"}, cfg)
})

t.Run("map form decoded normally", func(t *testing.T) {
cfg := decode(t, map[string]any{
"file": "./out.har",
"capture-secure-headers": true,
})
assert.Equal(t, config.HARConfig{
File: "./out.har",
CaptureSecureHeaders: true,
}, cfg)
})
}

func TestHARShorthandInMapping(t *testing.T) {
const configFile = "config.yaml"

const yaml = `
from: http://localhost:3000
to: https://api.example.com
har: ./recordings/api.har
`

viperCfg := viper.New()
viperCfg.SetFs(testutils.FsFromMap(t, map[string]string{configFile: yaml}))
viperCfg.SetConfigFile(configFile)
require.NoError(t, viperCfg.ReadInConfig())

actual := config.Mapping{}
require.NoError(t, viperCfg.Unmarshal(&actual, viper.DecodeHook(
config.URLMappingHookFunc(),
)))

assert.Equal(t, "./recordings/api.har", actual.HAR.File)
assert.False(t, actual.HAR.CaptureSecureHeaders)
}
3 changes: 3 additions & 0 deletions internal/config/mapping.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ type Mapping struct {
Cache CacheGlobs `mapstructure:"cache"`
Rewrites RewriteOptions `mapstructure:"rewrites"`
OptionsHandling OptionsHandling `mapstructure:"options-handling"`
HAR HARConfig `mapstructure:"har"`

// Cached parsed URL and its components (not serialized)
fromURL *url.URL `json:"-" mapstructure:"-" yaml:"-"`
Expand All @@ -36,6 +37,7 @@ func (m *Mapping) Clone() Mapping {
Cache: m.Cache.Clone(),
Rewrites: m.Rewrites.Clone(),
OptionsHandling: m.OptionsHandling.Clone(),
HAR: m.HAR.Clone(),
fromURL: m.fromURL, // Share cached URL
fromHost: m.fromHost,
fromPort: m.fromPort,
Expand Down Expand Up @@ -113,6 +115,7 @@ func URLMappingHookFunc() mapstructure.DecodeHookFunc {
data,
&mapping,
StaticDirMappingHookFunc(),
HARConfigHookFunc(),
)

return mapping, err
Expand Down
30 changes: 30 additions & 0 deletions internal/config/validators/har.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package validators

import (
"fmt"
"path/filepath"

"github.com/evg4b/uncors/internal/config"
"github.com/gobuffalo/validate"
)

// HARValidator validates per-mapping HAR collector configuration.
type HARValidator struct {
Field string
Value config.HARConfig
}

func (h *HARValidator) IsValid(errors *validate.Errors) {
if !h.Value.Enabled() {
return
}

file := h.Value.File

if filepath.Ext(file) == "" {
errors.Add(
joinPath(h.Field, "file"),
fmt.Sprintf("%s: HAR file path %q must have a file extension (e.g. .har)", h.Field, file),
)
}
}
54 changes: 54 additions & 0 deletions internal/config/validators/har_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package validators_test

import (
"testing"

"github.com/evg4b/uncors/internal/config"
"github.com/evg4b/uncors/internal/config/validators"
"github.com/gobuffalo/validate"
"github.com/stretchr/testify/assert"
)

func TestHARValidator(t *testing.T) {
t.Run("valid cases", func(t *testing.T) {
cases := []struct {
name string
value config.HARConfig
}{
{
name: "disabled (empty file)",
value: config.HARConfig{},
},
{
name: "valid file path with extension",
value: config.HARConfig{File: "output.har"},
},
{
name: "path with directory and extension",
value: config.HARConfig{File: "/tmp/trace.har"},
},
}

for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
errs := validate.Validate(&validators.HARValidator{
Field: "mappings[0].har",
Value: tc.value,
})

assert.False(t, errs.HasAny())
})
}
})

t.Run("invalid cases", func(t *testing.T) {
t.Run("file path without extension", func(t *testing.T) {
errs := validate.Validate(&validators.HARValidator{
Field: "mappings[0].har",
Value: config.HARConfig{File: "outputfile"},
})

assert.True(t, errs.HasAny())
})
})
}
4 changes: 4 additions & 0 deletions internal/config/validators/mapping.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ func (m *MappingValidator) IsValid(errors *validate.Errors) {
Mapping: m.Value,
Fs: m.Fs,
},
&HARValidator{
Field: joinPath(m.Field, "har"),
Value: m.Value.HAR,
},
))

for i, static := range m.Value.Statics {
Expand Down
48 changes: 48 additions & 0 deletions internal/handler/har/capture_writer.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package har

import (
"bytes"
"io"
"net/http"

"github.com/evg4b/uncors/internal/contracts"
)

// captureWriter wraps a ResponseWriter and tees the response body
// into an internal buffer so the middleware can build a HAR entry
// after the handler returns. It satisfies contracts.ResponseWriter.
type captureWriter struct {
contracts.ResponseWriter

buffer bytes.Buffer
output io.Writer
code int
}

func newCaptureWriter(wrapped contracts.ResponseWriter) *captureWriter {
capture := &captureWriter{
ResponseWriter: wrapped,
code: http.StatusOK,
}

capture.output = io.MultiWriter(&capture.buffer, wrapped)

return capture
}

func (cw *captureWriter) Write(b []byte) (int, error) {
return cw.output.Write(b)
}

func (cw *captureWriter) WriteHeader(statusCode int) {
cw.code = statusCode
cw.ResponseWriter.WriteHeader(statusCode)
}

func (cw *captureWriter) StatusCode() int {
return cw.code
}

func (cw *captureWriter) body() []byte {
return cw.buffer.Bytes()
}
Loading
Loading