Proxy Scanner

A fast and efficient proxy scanner written in Go that tests proxies from multiple sources and maintains a list of working proxies.

• Fetches proxies from multiple sources concurrently
• Tests proxies against a real HTTP fetch (example.com), not a sham loop
• Protocol-specific checks, in order: HTTP CONNECT proxy (http://), HTTPS-to-proxy CONNECT (https://), SOCKS5 (golang.org/x/net/proxy), then SOCKS4 (IPv4 path)
• Maintains state between runs to avoid retesting failed proxies (when you reuse the same workspace)
• Single run by default, or periodic scanning only when -interval / -i is set
• Option to test previously working proxies (-test-working)
• Concurrent testing with rate limiting
• Pretty JSON state file for easy inspection
• Configurable maximum test count before marking proxies as failed
• Working proxies grouped by detected type (http, https, socks for SOCKS4, socks5)

git clone https://github.com/yourusername/proxy-scanner.git
cd proxy-scanner
go build

Run ./proxy-scanner -h (or --help) for the synopsis. It matches:

Usage:
  proxy-scanner [OPTION]...

Description:
  Load proxy-feed URLs from sources.txt (auto-seeded if empty).
  Probe HTTP CONNECT, HTTPS-to-proxy, SOCKS5, SOCKS4 (GET http://example.com, want 200).
  Persist state.json; append working proxies to working_proxies.txt.

Options:
  -i, -interval DURATION repeat every DURATION; omit for single run
  -test-working          also probe entries in working_proxies.txt
  -max-test-count N      give up after N failed probes (default 2)
  -h, -help              show help and exit

The real program aligns the option text with whitespace (see your terminal).

1. Single scan, then exit (no periodic loop):

./proxy-scanner

2. Repeat every hour (same as -interval 1h):

./proxy-scanner -i 1h

3. Repeat every minute and re-test known working proxies:

./proxy-scanner -interval 1m -test-working

4. Custom test count with hourly cadence:

./proxy-scanner -i 1h -max-test-count 3

• main.go: Main program file
• sources.txt: List of proxy source URLs (one URL per line); if missing or empty, defaults are written automatically (see source comments in code / prior README list)
• state.json: JSON file containing proxy states and test history
• working_proxies.txt: List of currently working proxies organized by type

For each host:port (or URL with http://, https://, etc.), the scanner tries real transports until one succeeds:

1. HTTP proxy — http.Transport with Proxy set to http://host:port, then GET http://example.com/.
2. HTTPS proxy — same with https://host:port (TLS to the proxy, then CONNECT as implemented by Go’s Transport).
3. SOCKS5 — SOCKS5 dial via golang.org/x/net/proxy, then HTTP over that tunnel.
4. SOCKS4 — minimal SOCKS4 CONNECT (IPv4 only for the requested host), then HTTP over the tunnel.

A proxy is counted as working only when the probe returns HTTP 200 on http://example.com. Responses are drained and connections are closed explicitly to avoid buildup.

HTTPS destination probes (i.e. https://example.com through the proxy) are not used here; validating CONNECT + HTTP over the tunnel keeps the checker fast while still distinguishing HTTP vs HTTPS proxies vs SOCKS stacks.

Proxies that only support IPv6 end-to-end for the target may fail the SOCKS4 step (expected); SOCKS5 can still succeed if the proxy resolves the target.

• http — reached via an HTTP proxy URL (http://).
• https — reached via an HTTPS proxy URL (https://) to the proxy server.
• socks5 — SOCKS5 per RFC 1928 (via x/net/proxy).
• socks — SOCKS4 (IPv4); stored under [socks] in working_proxies.txt.

The working_proxies.txt file is organized by proxy type with sections:

[http]
1.2.3.4:8080
5.6.7.8:80

[https]
9.10.11.12:443
13.14.15.16:8443

[socks]
17.18.19.20:1080
21.22.23.24:1080

[socks5]
25.26.27.28:1080
29.30.31.32:1080

The program can use several reliable proxy list projects (auto-seeded when sources.txt is empty):

• TheSpeedX/PROXY-List
• ShiftyTR/Proxy-List
• monosans/proxy-list
• clarketm/proxy-list
• sunny9577/proxy-scraper

The program logs:

• Initialization (single run vs repeat interval)
• Number of sources found
• Working proxies with their type and response time
• Errors encountered while fetching sources

Example log output:

Found working http proxy (HTTP CONNECT tunnel): 1.2.3.4:8080 (response time: 1.234s)
Found working socks5 proxy: 5.6.7.8:1080 (response time: 0.876s)

The program maintains state in state.json which includes:

• Proxy status (working / not_working / pass)
• Last test time
• Number of tests performed
• Proxy type (http / https / socks / socks5)

Proxies are marked pass after failing the configured number of tests.

• Go version as specified in go.mod (run go version and go mod download in the project directory)
• Internet connection for fetching proxies and for the example.com probe
• Write permissions in the program directory (for state.json, working_proxies.txt, and optional sources.txt updates)

MIT License - feel free to use this code for your own projects.

Contributions are welcome. Please feel free to submit a Pull Request.