ProxyTools is a package of Perl network utilities designed mainly to assist those whose Internet access is censored, unreliable, or otherwise damaged. Uncensored access is provided to any outside service required (Usenet News, Web browsing, IRC, Socks etc.). Setup requires installation of Perl and some modules; this is doable by even a novice MS Windows user with email instruction, allowing help to be provided to those inside these countries from expert users outside.
In pursuit of this rather non-specific goal, some interesting network utilities have already been produced. We think the code is interesting in itself, useful in other areas, and would welcome contributions to the overall sum of ideas, concepts and ideals expressed in these tools.
The project is of interest to the following groups of people:
The project distribution contains the following tools:
localProxy (screen shot)This is essentially a character switch, with some interest in the content switched. Normally run on the user's computer (but runnable on a Unix ISP shell outside the user's firewall) this application provides a user with localhost services to use which implement censor avoidance strategies in the background. A front end GUI is included which allows convenient control of the localProxy engine. A single 'AutoConfigure' button allows even a novice user to map his local firewall rules and create a configuration file which describes his network environment, specifies proxy capabilities which may be used to enable (by default) uncensored HTTP proxy and Usenet News services; selection of this configuration will cause LP to build and run the configuration. LocalProxy will build a running configuration based on a database of usable hosts and proxies, applying several sources of information relevant to accessible hosts, services, firewall rules and firewalled subnets. LocalProxy runs this configuration and makes connections for the user, learning to use the fastest and most reliable of the censor-avoidance strategies and multiple proxy hosts/chains available. The application maintains (and displays) a 'useful data speed' for each level of the 'user'->'strategy'->'host'->'host'->website/service chain, to assist the user with configuration. The choices made by localProxy in switching each request is based on these speeds, ensuring localProxy avoids non-working strategies/hosts, and ensuring maximum use of the available 'uncensored bandwidth'. Failover capabilities and robustness are built in, so that users need not frequently update the database of hosts whenever one dies, becomes blocked, or is temporarily down.
LocalProxy (excluding Perl modules) is approximately 6000 lines of commented, debugged Perl code now. Worth a look, just for the spectacle!
Implemented strategies so far, are:
statProxyA tool to test various aspects of a proxy's function to allow a user to determine it's usefulness for his purposes. It scans ports on the proxy if required, tests for CONNECT tunneling capability, censoring, anonymity, speed, Socks capability etc. The output format is suitable for use by 'mergeHosts' to merge these test results into the hosts database automatically.
sortProxyA simple sort and merge utility for lists of hosts. A fast, parallel DNS forward and reverse lookup is done on each entry to produce a standard format output. Dupes are recognized by IP address and removed. The output format is suitable for piping to both findProxy or statProxy.
findProxyThis tool scans web pages, local files, bulletin boards, mailing lists etc. for likely looking proxy entries and tests them. This uses LWP, so it's slower than statProxy (which uses socket level code). It's useful for it's extensive proxy entry recognition ability (roughly done with Perl regexps) and it's final test for censoring by the tested proxies.
masterThis tool is intended to analyze the user's network environment and produce an XML configuration file for use by the other tools to aid the user in choices of proxies and strategies. The firewall rules are analyzed by direct testing, DNS servers are verified, localhost Internet interface IP address is established and confirmed and all is written to the configuration file for use by localProxy at build time. All operations are done in portable ways.
The reason for going public with these tools is to ensure a wide distribution to those in need, and to elicit ideas, and comments, from the networking community. These tools have been completely developed in an environment consisting of myself (network/proxy consultant) and a group of naive users. It is likely that the Internet community would be able to add new perspectives to the project.
Comments and feature suggestions, as well as code, are welcome here.
The proxyTools project summary page shows the current status of the project, file downloads, mailing lists, news etc.
This project is an implementation of many of the strategies discussed at this site
Thanks for your time.
Last updated April 4, 2002. Send feedback, comments and criticism to Wayne.