Skip to content

Add custom HTTP header support - #17

Closed
infraguard-web wants to merge 1 commit into
mainfrom
fix/custom-headers-rebased
Closed

infraguard-web wants to merge 1 commit into
mainfrom
fix/custom-headers-rebased

Conversation

@infraguard-web

Copy link
Copy Markdown
Collaborator

Summary

  • Add repeatable --header CLI support for outbound HTTP requests.
  • Store configured headers globally via feature.CustomHeaders.
  • Apply custom headers through shared transport handling for common and detector HTTP clients.
  • Route the HuggingFace client through the common instrumented custom transport.

Verification

  • git diff --check origin/main..HEAD
  • go build ./...

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds global, repeatable --header CLI support so custom HTTP headers can be configured once and applied across outbound HTTP clients/transports (including detectors and the HuggingFace source).

Changes:

  • Introduce a global, atomically stored feature.CustomHeaders and parse --header flags into it at startup.
  • Apply globally configured headers in shared HTTP transports (common transport + detector transport).
  • Route HuggingFace HTTP client through the common instrumented/custom transport and update docs/manpage for the new flag.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/sources/huggingface/client.go Switch HF client to use common instrumented + custom transport so global headers are applied.
pkg/feature/feature.go Add CustomHeaders global and AtomicHeader helper for atomic storage.
pkg/detectors/readme/readme.go Use the shared detector HTTP client (so transport-level behavior like custom headers applies).
pkg/detectors/http.go Apply global custom headers in detector transport RoundTrip.
pkg/common/http.go Add ApplyCustomHeaders and apply it in CustomTransport.RoundTrip.
main.go Add repeatable --header flag and parse/store headers globally.
docs/man/trufflehog.1 Document the new --header flag and expected formats.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/feature/feature.go
Comment on lines +52 to +62
// Load returns the current http.Header value; may be nil if unset.
func (ah *AtomicHeader) Load() http.Header {
if v := ah.value.Load(); v != nil {
return v.(http.Header)
}
return nil
}

// Store sets the http.Header value.
func (ah *AtomicHeader) Store(h http.Header) {
ah.value.Store(h)
Comment thread pkg/common/http.go
Comment on lines +99 to 113
// ApplyCustomHeaders adds any globally configured custom headers to req.
func ApplyCustomHeaders(req *http.Request) {
if hdr := feature.CustomHeaders.Load(); hdr != nil {
for k, vals := range hdr {
for _, v := range vals {
req.Header.Add(k, v)
}
}
}
}

func (t *CustomTransport) RoundTrip(req *http.Request) (*http.Response, error) {
req.Header.Add("User-Agent", UserAgent())
ApplyCustomHeaders(req)
return t.T.RoundTrip(req)
Comment thread pkg/detectors/http.go
Comment on lines 76 to 80
func (t *detectorTransport) RoundTrip(req *http.Request) (*http.Response, error) {
req.Header.Add("User-Agent", userAgent())
common.ApplyCustomHeaders(req)
return t.T.RoundTrip(req)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants