fix: sanitize JSON body before storing to Postgres to prevent SQLSTATE 22P05

[hdot123]

Summary

When storing request bodies in Postgres JSONB columns, invalid Unicode escape sequences (e.g., malformed \u sequences) cause SQLSTATE 22P05 errors: unsupported Unicode escape sequence.

This manifests as warnings in logs:

Failed to save request body due to error, retrying with placeholder
ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)

Fix

Use xjson.SafeJSONRawMessage() which:

1. Validates JSON with Go's json.Valid()
2. If invalid, repairs using the jsonrepair library
3. Falls back to empty JSON object {} if repair fails

Changes

• CreateRequest(): sanitize httpRequest.JSONBody with SafeJSONRawMessage() before storing
• CreateRequestExecution(): sanitize channelRequest.JSONBody with SafeJSONRawMessage() before storing

This prevents the SQLSTATE 22P05 error and eliminates the warning logs when clients like Codex send JSON with invalid Unicode escapes.

Testing

Existing tests should pass. The fix is backward compatible - valid JSON is stored unchanged, only invalid JSON is repaired.

[gemini-code-assist]

Code Review

This pull request introduces comprehensive enhancements including secure API key hashing, JWT revocation, encrypted proxy passwords, and improved backup/restore functionality. It also hardens the system with atomic data storage writes, better error handling to prevent internal leakage, and optimized dashboard queries. Feedback focuses on correcting error suppression in migrations, fixing garbage collection logic for soft-deleted records, and addressing performance concerns such as synchronous cleanup in the dumper and per-request database queries in middleware. Additionally, potential data loss in the pass-through stream and inefficient goroutine usage in the SSE decoder were identified as areas for improvement.

[gemini-code-assist]

The migration returns nil when authz.WithSystemBypass fails. This silently suppresses the error and marks the migration as completed in the Migrator tracking, even though the migration logic was never executed. It should return the actual error to allow the migrator to handle the failure correctly.

Suggested change

	ctx, err = authz.WithSystemBypass(ctx, "database-migrate")
	if err != nil {
	return nil
	}
	ctx, err = authz.WithSystemBypass(ctx, "database-migrate")
	if err != nil {
	return err
	}

[gemini-code-assist]

Similar to the v0.3.0 migration, returning nil on a bypass error silently skips the migration while marking it as successful. The error should be propagated.

Suggested change

	ctx, err = authz.WithSystemBypass(ctx, "database-migrate")
	if err != nil {
	return nil
	}
	ctx, err = authz.WithSystemBypass(ctx, "database-migrate")
	if err != nil {
	return err
	}

[gemini-code-assist]

The current implementation of the GC worker will not achieve its goal of hard-deleting records to reclaim space. By removing SkipSoftDelete from the context, the ent framework's soft-delete filter will remain active, meaning the query will only see records that have not been deleted yet. Furthermore, the Delete() operation will likely trigger another soft-delete (an UPDATE) rather than a hard DELETE. To hard-delete records that have already been soft-deleted, you must: 1) Use SkipSoftDelete(ctx) to see all records, and 2) Filter by DeletedAt (e.g., DeletedAtLT(cutoff)) instead of CreatedAt to target records that were soft-deleted past the retention period.

Suggested change

	ctx = ent.NewContext(ctx, w.Ent)
	ctx = schematype.SkipSoftDelete(ctx)
	// F-D72: Do NOT use SkipSoftDelete. GC should respect soft-delete semantics:
	// only hard-delete records that have been soft-deleted past their retention period.
	ctx = ent.NewContext(ctx, w.Ent)
	ctx = schematype.SkipSoftDelete(ctx)

[gemini-code-assist]

Calling d.cleanup() synchronously within the dump methods can significantly increase latency for the caller, especially if the dump directory contains many files or if disk I/O is slow (due to os.ReadDir and sort.Slice). Consider running the cleanup process in a background goroutine to avoid blocking the main execution flow.

Suggested change

	d.cleanup()
	go d.cleanup()

[gemini-code-assist]

The atomicWriteFile function uses a .tmp/ prefix for temporary files. If the underlying filesystem is local and the .tmp directory does not exist, afero.WriteFile will fail. You should ensure the parent directory of the temporary file exists before attempting to write.

Suggested change

	tmpKey := fmt.Sprintf(".tmp/%s-%s", uuid.New().String(), filepath.Base(key))
	if err := afero.WriteFile(fs, tmpKey, data, 0o777); err != nil {
	tmpKey := fmt.Sprintf(".tmp/%s-%s", uuid.New().String(), filepath.Base(key))
	_ = fs.MkdirAll(".tmp", 0o777)
	if err := afero.WriteFile(fs, tmpKey, data, 0o777); err != nil {

[gemini-code-assist]

Performing a database query to verify project access on every request in the middleware can become a performance bottleneck. Consider caching the user's project access list or using a more efficient way to verify permissions (e.g., including project IDs in the JWT claims if applicable).

[gemini-code-assist]

The default case in the select block will cause events to be silently dropped if the rawStreamCh is full or if the consumer has not started reading yet. This leads to data loss in the pass-through stream. If the intention is to ensure all events are captured, the default case should be removed, or the channel buffer should be appropriately sized.

[gemini-code-assist]

Creating a new goroutine for every single SSE event in Next() is highly inefficient and can lead to excessive resource consumption and goroutine leaks. If Next() returns false due to context cancellation, the goroutine will remain blocked on Recv() until the server sends data or closes the connection. Consider using a single background goroutine per stream to read from the SSE source and push to a channel.