Testing Proxy Integrations in CI/CD: A Practical Approach

Proxy-dependent code is some of the hardest infrastructure to test properly in CI, and most teams end up either skipping it entirely (and finding out about breakage in production) or mocking it so thoroughly that the tests stop catching anything real. Here's a middle path that actually works.

Why proxy integrations resist normal testing patterns

Standard CI practice says: mock external dependencies, keep tests fast and deterministic. Proxies break this cleanly because the entire point of testing them is to catch things that only show up with real network behavior — actual latency variance, actual IP-based responses, actual auth handshakes. A fully mocked proxy layer tests your code's logic around a proxy, but tells you nothing about whether the proxy integration itself still works.

A three-tier approach that balances speed and realism

Tier 1 — Unit tests, fully mocked, run on every commit. Test your own logic (retry handling, error classification, rotation decisions) against a mocked proxy client that returns controlled responses. Fast, deterministic, catches logic bugs in your code. Doesn't catch anything about the actual proxy service.

Tier 2 — Integration smoke tests, real proxy, limited scope, run on every PR. A small number of real requests through your actual proxy provider, against a stable, low-risk target (your own test endpoint if you have one, or a target explicitly designed for this like httpbin-style echo services). This catches "the integration itself is broken" — wrong auth format, wrong endpoint, expired credentials — without depending on volume or hitting real production targets from CI.

Tier 3 — Scheduled health checks against real targets, run on a cadence, not per-commit. Periodic (hourly/daily) checks against actual production-relevant targets, run outside the PR pipeline so a target's temporary rate limiting doesn't block unrelated merges. This is where you catch pool degradation and target-specific issues, on a schedule that matches how those things actually manifest — gradually, not per-commit.

What NOT to run per-commit

Anything that consumes meaningful proxy quota, anything that hits a real production scraping target directly, and anything whose pass/fail depends on external service behavior that isn't yours to control. These belong in tier 3, on a schedule, with alerting — not blocking every PR on a flaky external dependency.

Handling credentials in CI safely

Proxy credentials in CI need the same treatment as any other secret: environment-injected via your CI platform's secrets manager, never in the repo, never in a config file that gets committed even temporarily during debugging. A surprising amount of proxy credential leakage happens specifically through CI logs printing full request URLs (including embedded user:pass@ auth) during test failures — make sure your test framework redacts these before they hit build logs.

Making tier-2 tests actually catch what matters

The temptation is to make the integration smoke test trivial (just check you get any 200 response). A more useful version explicitly checks:

• Auth actually succeeded (not just that a request went through)

• The response actually came through the proxy (check for an expected header, or that the reported IP matches your proxy's range, not your CI runner's own IP)

• Basic rotation is working if that's part of your setup (two consecutive requests get different IPs, if rotation is expected)

That second check specifically — verifying the response actually reflects proxy routing rather than the CI runner having somehow bypassed the proxy — catches a class of "tests pass, prod is broken" bugs that a naive status-code check would miss entirely.

This tiered approach is close to what we recommend to teams building on top of SotaProxy — fast mocked tests for logic, a small real smoke test for the integration itself, and scheduled checks for the things that only degrade slowly. If your current setup either skips proxy testing entirely or mocks it into uselessness, this tiering is usually the fix.