CGIProxy  (see http://www.jmarshall.com/tools/cgiproxy/)
========================================================


CH'CH'CH'CH'CHANGES:
--------------------


2.2.4, released January 26, 2019:
---------------------------------

The top form is no longer inserted into the HTML of each page, which means
it behaves *much* better.  Instead, the top form is now in its own HTML along
with an <iframe> that includes the full visited page.  No longer will the top
form cover up page content, or disappear under page content.

The starting page is now a little easier to use, with language names under
each flag and more obvious button placement.

No longer crashes with Perl 5.22.x .

Closed a couple of potential privacy holes.

Various bug fixes that make most websites work again.



2.2.3, released November 18, 2018:
----------------------------------

There is now an "upgrade" command, which does everything needed to securely
upgrade an installation of CGIProxy to the latest version.  It keeps all
previous configuration settings.  Note that if you are upgrading from version
2.2.2 or earlier, the easiest way is to first run the "uninstall" command, and
then run the "install" command; you can reuse previous config settings by
saving your old cgiproxy.conf and adding the "--old-config my_saved_file"
parameter to the "install" command.  But from 2.2.3 forward, the "upgrade"
command will do everything for upgrading.

There is now an "uninstall" command, to easily uninstall CGIProxy.

There is now an automatic mechanism to *optionally* report simple usage
metrics back to our server:  hourly hit counts per language, hourly traffic
per language, and unique users per month.  We do NOT collect any private info
such as IP addresses or URLs visited.  I think this is safe and uninvasive, but
please email me if you have any concerns.  It helps us a lot, both in seeing
where CGIProxy is used the most, and in our ongoing efforts to get funding for
CGIProxy development.  In any case, you don't have to report these numbers to
us-- it's a question in the installation wizard that sets the $REPORT_USAGE
config variable.

Removed support for -f command-line parameter.

Many bugs fixed and various new technologies supported under the covers,
which makes most complex websites work again.



2.2.2, released July 16, 2017:
------------------------------

The installation wizard has been greatly improved, based on user feedback:
the server's package manager is now used instead of cpan when
installing Perl modules, if available and if installing as root (dnf, yum,
and apt-get are supported); Web server configuration is read better on more
platforms; fewer fatal errors; better messages; at the end, the wizard now
gives the new proxy URL.  Also, the installed script now has one line modified
by the wizard, to set $PROXY_DIR to an absolute path.

$SECRET_PATH is now randomly-generated by default, and is no longer optional.
For CGI and mod_perl installations, it names the subdirectory where the
script file is installed.

All pages are now optimized for mobile usage, as a <meta name="viewport" ...>
is inserted into all of them.

For maximum security, TLS 1.2 is now used if available; if not, successively
older TLS/SSL versions are tried.

Improved caching of both the starting page and the JavaScript library.  Among
other things, the If-Modified-Since: header is respected for those two.

Added a "create-db" command to create the database, e.g. if you change
$DB_DRIVER after installing.

Added a -q ("quiet") option to suppress output when running as FastCGI script.

A SOCKS proxy on localhost is now allowed, even if "127" is set in
@BANNED_NETWORKS .

If the Math::Random::Secure module is installed, that will be used for
randomly-generated strings such as $SECRET_PATH or session cookies.  That
module is not installed by default because loading it adds significant time
to every request, plus it has many dependencies.  But you can install it
if you need cryptographic-quality randomness in those generated strings.

Many bugs and potential privacy holes fixed.



2.2.1, released May 28, 2017:
-----------------------------

CGIProxy now has an installation wizard!  Run "./nph-proxy.cgi install" to
install it (the "init" command is no longer needed or supported).  This
should work on all platforms, and should work if run as either root or
non-root.  PLEASE REPORT ANY PROBLEMS, OR IF ANYTHING IS UNCLEAR!  The more
feedback I get, the better the installation wizard will work on all platforms
in the future.

CGIProxy now has a configuration menu!  Run "./nph-proxy.cgi config" to run
it.  Related to this, CGIProxy now uses an external configuration file,
which will make future upgrades much easier.  There is no longer any need to
edit the script file directly, unless you need to change $PROXY_DIR .

$SECRET_PATH is no longer optional, and is now set to a random alphanumeric
string by the installation wizard.  It is used in all four run methods-- for
CGI and mod_perl installations, a subdirectory is named by it and the script
is copied there during installation.

Added $RUN_AS_GROUP config variable, to go with $RUN_AS_USER.

Many bugs fixed, making many sites work again.



2.1.17, released June 11, 2016:
-------------------------------

Fixed to no longer access Window.prototype, which has been recently disallowed
by most browsers.  This makes *many* websites work again, including Gmail,
YouTube, and facebook.

A few other bug fixes.



2.1.16, released November 19, 2015:
-----------------------------------

Cleaned up installation process.  In particular, $RUN_AS_USER and $PROXY_DIR
should now be set before running "./nph-proxy.cgi init", so that file
ownership and permissions can be set correctly.

Added Dutch localization.



2.1.15, released October 21, 2015:
----------------------------------

Database support is MUCH less error-prone.  The database is created when
running "./nph-proxy.cgi init" rather than when running as a CGI script
or manually, which is really how it should have been done all along.  Better
separation of the various roles and permissions.

Added support for SQLite databases, which means you can now use a database
without running a database engine like MySQL/MariaDB or Oracle.  $DB_DRIVER
now defaults to "SQLite", hopefully making "Bad Request" errors a thing of
the past.

There is now a "./nph-proxy.cgi init" command, which a) creates needed
directories, b) installs all Perl modules (like the "install-modules"
command" has done), and c) creates the database.  "install-modules" still
works, but is usually unnecessary.

Accordingly, the installation instructions at install.html have been
updated.

Added support for Storage objects, mostly.

MANY major fixes to CORS support, XMLHttpRequest support, cookie support,
and other areas, making popular sites work again.



2.1.14, released November 13, 2014:
-----------------------------------

Fixed FastCGI support.  Note that $FCGI_SOCKET is now a listening port number
rather than a Unix-domain socket (which wasn't working well anyway).  Be sure
to fix your setting of $FCGI_SOCKET; more details are in the comments above
it in the configuration section.  Also note that the installation page has
been updated with better FastCGI configuration for nginx and Apache.

Added the $USER_FACING_PORT config variable for when one user-facing web server
calls CGIProxy running under other web server software, or any other case when
the port the user sees isn't the same port that CGIProxy sees.

Many bug fixes, making many more sites work.  Major sites are working again,
and in all major browsers.



2.1.13, released October 6, 2014:
---------------------------------

Complete performance rewrite of the JavaScript-modifying code, i.e. proxify_js()
and related routines.  The delay when viewing YouTube video pages is much shorter,
and the load on the server CPU is much less.  :)

Several bug fixes.



2.1.12, released June 21, 2014:
-------------------------------

Now has fully-tested and working IPv6 support.

Works again with older Socket module.  Note that IPv6 support only works with
Socket 1.94 or later.

Several bug fixes.



2.1.11, released May 31, 2014:
------------------------------

Added CORS support, which among other things makes YouTube work again.

Added a page about CGIProxy security, linked to from the footer on the CGIProxy
start page and all error and similar pages.

Added untested IPv6 support-- please tell me how it works if you have IPv6 access!

Several bug fixes.



2.1.10, released April 5, 2014:
-------------------------------

Added Spanish and Polish message localization.

Added support for LZMA-compressed SWF files.  Note that this requires the
Perl module IO::Compress::Lzma, which will be installed if you run
"./nph-proxy.cgi install-modules" again, like you did when you first
installed CGIProxy.

Added %TIMEOUT_MULTIPLIER_BY_HOST option, to tune performance with certain
sites.

Fixed many bugs, making many sites work better.



2.1.9, released January 27, 2014:
---------------------------------

Added German, Italian, Javanese, and Sundanese message localization.

Content Security Policy (CSP) 1.0 is now supported with Firefox and Chrome.
Other browsers, and CSP 1.1, will be added when they support CSP.

Added $ALERT_ON_CSP_VIOLATION option.

Various bugs fixes and workarounds.



2.1.8, released October 22, 2013:
---------------------------------

Added Chinese, French, and Indonesian message localization.

The full Gmail now works through CGIProxy.

YouTube once again works through CGIProxy.

Can now use a SOCKS 5 proxy, such as Tor (recommended only on same server).
Configured with $SOCKS_PROXY, $SOCKS_USERNAME, and $SOCKS_PASSWORD .

Database initialization now works better.

Can now use a remote database by setting the $DB_SERVER config variable.

Many bugs fixed or worked around, and privacy holes closed.

Now once again runs on Perl 5.6.1 (one statement in 2.1.7 required Perl 5.10.0).

Shuffles HTTP request headers to better avoid detection.

$ANONYMIZE_INSERTION now defaults to 1.



2.1.7, released July 26, 2013:
------------------------------

CGIProxy now has message localization:  The user can choose an interface in
Arabic, English, Farsi, Russian, or Turkish.  If you would like support for
other languages, please consider translating CGIProxy's messages-- see
http://www.jmarshall.com/tools/cgiproxy/translate.html for full details.

The full facebook site now works almost fully through CGIProxy, so it's no
longer redirected to the mobile site by default.  If it's slow for you or your
users, see the comments and suggestions above where %REDIRECTS is set.

Running under FastCGI now works on servers other than just nginx.

Resuming partial downloads is now supported, with partial support of the
Range: header.

The JavaScript library (jslib) is now gzipped when possible, to save bandwidth.
Should have done this a while ago.

Fixed error with "-c" in usage message; sorry about that.

Added support for Content-Security-Policy: header, though it's disabled until
the header is better defined and browsers support it.

Many bugs fixed, making many sites work better.



2.1.6, released February 4, 2013:
---------------------------------

Now can run as a FastCGI script.

Now can run without an external HTTP server, by using its own embedded secure
HTTP server.

Installation is easier, as Perl modules can be automatically installed
(including under your home directory) by running
"./nph-proxy.cgi install-modules" from the command line.  See the $LOCAL_LIB_DIR
config option, if you need to install the modules and you're not root.

Windows support has improved.

Documentation has been improved, especially for installation.

Command-line usage is now documented; run "./nph-proxy.cgi -?" for usage.

There are some new config options, mostly for FastCGI support, the embedded
server, and database support.

Some of the configuration section has been rearranged; most potentially
needed config options are now near the top.

Fixed a bug handling spaces in path when using proxy_encode().



2.1.5b, released November 10, 2012:
-----------------------------------

Added redirection for Gmail to %REDIRECTS ; redirects to HTML-only version.



2.1.5, released October 21, 2012:
---------------------------------

Now optionally uses a server-side database to store cookies, which fixes
"Bad Request" errors when user has too many cookies.  Can use either MySQL
or Oracle.  Configure this with $DB_DRIVER, $DB_USER, $DB_PASS, and
$USE_DB_FOR_COOKIES .

Now supports a simple mechanism to automatically redirect pages that aren't
handled well by CGIProxy.  For example, www.facebook.com is redirected to
m.facebook.com (mobile), until we can get www.facebook.com working better.
This is configured with the %REDIRECTS hash.

17 bugs fixed, mostly in JavaScript support but some in Flash and HTML support too.



2.1.4, released May 8, 2012:
----------------------------

Fixed a bug with chunked responses, making captcha work better.

Closed some privacy holes in JavaScript and Flash support.

Fixed some small bugs, making more pages work better.

Added "Delete all cookies" link to top of cookie management screen.



2.1.3, released April 27, 2012:
-------------------------------

Improved Flash support, including better support for online video.  No delay
with YouTube anymore.

Improved support for Same-Origin Policy (browser security).

Other security fixes.

Other fixes and workarounds, making more pages work correctly.



2.1.2, released April 8, 2012:
------------------------------

Added "download" link to footer.

Fixed "SSL options" bug.

No longer fails in Windows because of getpwuid() call.

Various security fixes.

Various small fixes to make more pages work better.

Internal: rewrote _proxy_jslib_handle() and _proxy_jslib_assign() more cleanly,
using _proxy_jslib_instanceof() instead of the error-prone _proxy_jslib_object_type()
(which was removed).  Still works around many browser bugs.  :P



2.1.1, released January 19, 2012:
---------------------------------

$ENCODE_URL_INPUT is now on by default, and is fixed to work better in all
major browsers.

Link to CGIProxy home page in the footer is now proxified.

Now supports jQuery-based sites better.

Added support for a few non-standard HTTP request headers.

Works better with captcha.

$NO_COOKIE_WITH_IMAGE is now off by default, to support captcha.

Many small fixes and workarounds, including several privacy and security fixes.



2.1, released December 9, 2011:
-------------------------------

Flash 9 and later is now supported, which among other things means YouTube
works through CGIProxy again.

Changed flag segment of full URLs to something less obvious.

Now supports "data:" URIs.

Now supports ECMAScript (JavaScript) version 5.

Many fixes in JavaScript support and elsewhere.

No longer eats memory when connecting to secure servers.

$PROXIFY_SWF is now set by default.

Now uses the Encode module instead of the old utf8:: stuff.

Added two config options $PROXY_DIR and $ALLOW_RTMP_PROXY, though they're not
used yet.



2.1beta19, released December 25, 2008:
--------------------------------------

Fixed a couple of bugs with cookies, so they (including logins) should work
better.

Fixed so that Safari no longer chokes on StorageList handling.

Various other small fixes.



2.1beta18, released August 10, 2008:
------------------------------------

Certain pages were very slow in MSIE due to the way MSIE implements
Array.pop(), so a lot of code was rewritten to avoid using Array.pop().  Those
pages now function at normal speed in MSIE.  Some pages see an improvement
of 10x or better.

proxify_js() is now about 5% faster due to reworking of $div_ok setting.

Now handles "application/xhtml+xml" content correctly.

$REMOVE_SCRIPTS and $HIDE_REFERER both now default to 0 (false).

Various small fixes, workarounds, and cleanup.



2.1beta17, released March 11, 2008:
-----------------------------------

Fixed a couple of bugs with SWF (Flash) support.  In particular, <param> tags
are now proxified correctly when in certain <object>s, and thus youtube.com now
works when using MSIE.



2.1beta16, released March 3, 2008:
----------------------------------

Includes a ton of fixes and workarounds in JavaScript support, making more
sites work through it.

Includes a fair amount of performance improvement too.

Added experimental support for Shockwave Flash (SWF) apps.  If you set
$PROXIFY_SWF=1, the script will proxify the SWF bytecode so that any
network accesses go back through the same script.  It works with many but
not all Flash apps.  Sometimes it can slow down a page, if the page has e.g.
lots of Flash ads.

Fixed port-handling in HTTP Basic authentication support.

Several other small bug fixes, cleanup, and comments.



2.1beta15, released October 26, 2006:
-------------------------------------

Fixed bug handling "javascript:" URLs that was causing some sites to fail.

Fixed bug in _proxy_jslib_cookie_encode() and _proxy_jslib_cookie_decode(),
in the commented-out rot-13 line.  Cookies should now work when using rot-13.

"about:blank" pages no longer generate the "WARNING:" page.

In the start page, the URL entry field now initially has focus (only if JS
is running).

Added support for Element.getElementsByTagName to _proxy_jslib_handle()
(though it's done in the form of "Node.getElementsByTagName").

Now explicitly defaults the first argument of "Document.open()" to text/html.
This is supposed to happen anyway, but not all browsers do it correctly when
writing to a frame.

Proxification of top-level "return" statements should now work better in a
couple of ways.

Several more small fixes and cleanup, making more sites work.



2.1beta14, released October 16, 2006:
-------------------------------------

Better handling of "javascript:" URLs.

Now correctly handles "delete(...)" .

Various browser bugs/crashes are now trapped by try/catch blocks so they're
not fatal.

More erroneous HTML and JavaScript is now worked around.

Many more small fixes and workarounds.



2.1beta13, released September 12, 2006:
---------------------------------------

The $TRANSMIT_HTML_IN_PARTS config variable has been removed; now you only
need to set @TRANSMIT_HTML_IN_PARTS_URLS .  Use of both was redundant.

Now correctly handles UTF-8 content.

Authorization cookies are now associated with the server and port, rather than
just the server.

Worked around MSIE bug that shifts centered content to the right.

Now works around more erroneous HTML, JavaScript, and browser behavior.

Other minor fixes.



2.1beta12, released June 6, 2006:
---------------------------------

Now optionally supports compression (gzip) of message bodies, if the
Compress::Zlib Perl module is installed.  This also helps with certain server
bugs.

To help with pages that are returned in parts with long delays between those
parts, you can now use the $TRANSMIT_HTML_IN_PARTS and
@TRANSMIT_HTML_IN_PARTS_URLS options to have CGIProxy process and return each
piece of HTML as it receives it rather than wait for the whole page.  This
helps with certain library database queries, for example.

Further improved CSS handling, including better display of the top form.

Improved handling of "&#nnn;" HTML entities.

Better updating of the top form when using frames.

When $TEXT_ONLY is set, there are no longer any wasteful attempts to get the
images (unless $RETURN_EMPTY_GIF is set).  Also, $RETURN_EMPTY_GIF now defaults
to 0 .

Now handles JavaScript's deprecated with() statement.

Now handles Document.referrer .

Cookie values are now allowed to have spaces in them, even though that's
technically illegal; unfortunately, some sites require them to work.  If this
causes any problems, please let me know.

Other minor fixes.



2.1beta11, released March 14, 2006:
-----------------------------------

Improved (though not yet perfect) CSS handling.

Now uses a different system to track which elements have been proxified.
(PLEASE tell me if you find any privacy holes, i.e. when the browser
makes a direct connection to the end server when it should be going though
CGIProxy.)

Fixed a hang when using MSIE and IIS.

HTML (e.g. innerHTML) is no longer doubly-proxified when using "+=".

Improved handling of &#nnn; -type entities.

No longer inserts "<!-- resource has been modified by proxy -->"; it was
becoming more trouble than it's worth.

Better handling of data retrieved through XMLHttpRequest.

Cleaner support of Document.domain .

Many other minor fixes, workarounds, and cleanup, mostly in JavaScript support.



2.1beta10, released December 6, 2005:
-------------------------------------

Now supports UTF-16 pages, if using Perl 5.8.0 or later.

Now supports query-only URLs.

Window.open with a relative URL is now handled correctly.

In JavaScript, (non-standard) octal numbers and characters are now handled.

Now allows (erroneous) commas in cookie values, created by at least one server.

Now allows (erroneous) line terminators in JavaScript string literals if
  preceded by a "\", since browsers seem to allow it.

Many other minor fixes, mostly in JavaScript support.



2.1beta9, released October 27, 2005:
------------------------------------

Cleaned up URL handling in proxy_encode(), proxy_decode(), and their JS
counterparts.  Those routines are now back to their old format in which
they contain only user-configurable statements.  The "do not remove" code
has been moved into wrapper functions; anywhere proxy_encode() is called
should now call wrap_proxy_encode() instead, and the same is true for the
three other related routines.

@BANNED_NETWORKS now includes the whole 127.x.x.x subnet.

Cookies are now handled more like browsers handle them (though not as the
spec calls for), in a couple of ways.

In the JavaScript code, reworked handling of "delete", preincrement, and
predecrement.  Those operators are now handled much better.

Other minor fixes.



2.1beta8, released August 23, 2005:
-----------------------------------

Now more properly encodes all "?", "#", and others in default proxy_encode().
This includes further working around aforementioned Apache bug with PATH_INFO.

Now supports Accept-Language: header in requests, which means that pages will
more often be returned in the expected language.

Added support for query-only URLs, i.e. those beginning with "?".  Don't know
why this never came up before.

A couple fixes and workarounds in JavaScript support.

Other minor fixes.



2.1beta7, released June 28, 2005:
---------------------------------

Added a "Report a bug" link in the top form, which will hopefully result
in more bug reports.

Now supports (erroneous) JavaScript that contains the string "</script"
within it, as browsers do.

Worked around an Apache bug (fixed in 2.0.55) that causes problems when
PATH_INFO contains "//".

Fixed a few bugs in JavaScript handling.

Other minor fixes.



2.1beta6, released May 24, 2005:
--------------------------------

JavaScript is now supported, i.e. JavaScript is modified as necessary to
route all network accesses back through the script.

$PROXIFY_SCRIPTS is now on by default.

Unsupported scripts are now removed by default; to not remove them, use
$ALLOW_UNPROXIFIED_SCRIPTS.

Top form has been put into its own one-cell table, with a white background.
This makes it readable on many pages where it wasn't before.

Checkbox text in top form now uses <label> tags, to allow clicking on text.

Now can encode URLs before submitting them, if you set $ENCODE_URL_INPUT.

Now supports external tests for valid user IP address ($USER_IP_ADDRESS_TEST)
and valid destination server ($DESTINATION_SERVER_TEST).  Both may be either
a command-line program, or a CGI script on a remote server.  These features
were added at the request of (and paid for by) the International Broadcasting
Bureau.

No longer times out after 10 minutes, which will help with large files.

Added some more appropriate values to @BANNED_NETWORKS.

Cookies with no path specified now use a path of "/", even though that
violates the spec, because that's how browsers treat them.  If you want
to follow the spec instead, you can set $COOKIE_PATH_FOLLOWS_SPEC.

Illegal cookie domains that contain only two dots but are not in one of
the seven main TLDs are now allowed by default, because that's how browsers
behave.  You can follow the spec instead by setting $RESPECT_THREE_DOT_RULE.

Many other minor fixes, and workarounds for browser bugs and buggy sites.



2.0.1, released November 19, 2002:
----------------------------------

This release improves compatibility and installability in a few environments;
it doesn't really add new features.  In particular:

An SSL proxy is now supported with the $SSL_PROXY option, analogous to
$HTTP_PROXY.  Authentication for it is handled with $SSL_PROXY_AUTH,
analogous to $PROXY_AUTH.

The $RUNNING_ON_WINDOWS option is no longer needed, and so no longer exists--
relevant code now determines the OS automatically when needed, or is handled
another way.

Sockets now work correctly on BSDI and possibly other systems, where before
the script generated messages like "Address family not supported by protocol
family" and possibly others.  It works now because socket address structures
are created with the general pack_sockaddr_in() and inet_aton() functions from
the Socket module, instead of the traditional hard-coded "pack('S n a4 x8')"
method.  The new way is more "correct", given that we're already loading the
Socket module anyway.  It should help for future IPv6 support too.

All page insertions, including the initial "<!-- resource ... -->" comment, are
now inserted *after* any initial <!doctype> declaration, to avoid confusing
MSIE 6.0, which does not allow comments before an initial <!doctype>.  The
problem showed up as subtle errors in page elements (like fonts, etc.) when
using MSIE; some or all of these problems should go away now.

Cache behavior when using a caching HTTP proxy should be correct now, because
Pragma: and Cache-Control: headers (if available) are now passed through to
the outgoing request.  Before, caches would not always refresh as expected,
so page reloads didn't always work.

Fixed a compilation error when using certain older versions of Perl.



2.0, released September 18, 2002:
---------------------------------

This is a MAJOR release, even though most of the changes are internal.  Some
things about the 2.0 script are fundamentally different from the 1.x series.
Here's a list of changes, roughly categorized:


---- Visible new features and changes: ----

Now supports SSL, i.e. can retrieve pages on secure servers.  For this to
work, the separate packages OpenSSL and Net::SSLeay must be installed.  If
they are not installed, then CGIProxy still works but cannot download pages
from secure servers.  Also, it is strongly recommended to run CGIProxy on a
secure server when SSL is supported, or else secure data will be compromised
on the link between the browser and CGIProxy.

The top entry form now has an "UP" link, which links to the parent directory of
the current URL.  The idea comes from the (quite useful) button in Konqueror.

Handling of top insertions has been cleaned up-- FTP directory listings now
include the correct insertions, and other cleaner behavior.


---- New or changed config options: ----

$RUNNING_ON_SSL_SERVER is now a three-way option:  If it's set to '', then
assume an SSL server if and only if port 443 is being used.  Besides being a
good default, this lets you put the script where it can be served by both a
secure server and a non-secure server.

Perl 4 is no longer supported.  CGIProxy should run fine with Perl 5.004 or
later.  Better yet, upgrade to Perl 5.6.1 or later if you can-- future
versions of CGIProxy may require that for some features.

FTP requests now use passive (PASV) mode by default, though you can use
non-passive mode by setting $USE_PASSIVE_FTP_MODE=0.

$HTTP_PROXY and $NO_PROXY are now used instead of $ENV{'http_proxy'} and
$ENV{'no_proxy'}.

For VPN-like installations, there is now a $QUIETLY_EXIT_PROXY_SESSION option
that allows a smooth transition from browsing an intranet through the proxy
to browsing external sites directly, without getting intermediate warning
screens.  It's not meant for any situation where anonymity is important.  See
the comments where it is set for more details.

Set the new $SESSION_COOKIES_ONLY option to make all cookies expire when the
browser closes.

Set the new $MINIMIZE_CACHING option to minimize any caching that may be done
by the browser, by using appropriate HTTP response headers.  Cacheability of
various responses has also been cleaned up in general.

If for some reason you want content (e.g. HTML) inside <!-- --> comments to
be proxified like the rest, set the new $PROXIFY_COMMENTS option.

Improved the sample list of ad servers in @BANNED_IMAGE_URL_PATTERNS.

Cleaner and slightly different handling of $INSERT_HTML and $INSERT_FILE;
see comments in user config section for details.

$FASTER_HTML_LESS_PRIVACY no longer exists, because it's no longer relevant
with the new HTML-parsing structure (see below).


---- For programmers: ----

SSL support was implemented as a package that implements a tied filehandle,
called "SSL_Handle".  The idea came from the Net::SSLeay::Handle module,
which for a couple reasons wasn't suitable to use directly.  The SSL_Handle
package may be useful in other SSL applications, though this version is not
a full implementation.  It does buffer its input, though, which can make it
a much faster alternative to Net::SSLeay's ssl_read_until() routine.

The entire section of code that modifies an HTML response (roughly 20-25% of
the whole program) has been completely rewritten from scratch, and has a new
structure.  It's MUCH cleaner and slightly smaller than before, and is now
encapsulated in the routine proxify_html().  It handles the heterogeneity of
HTML correctly, i.e. non-HTML content that can exist within HTML (like
scripts, stylesheets, comments, or SGML declarations) is correctly separated
out and handled according to type.  Also, tags are fully parsed into
attributes and rebuilt if needed, instead of hacking it with long regular
expressions as before; this removes a family of potential bugs and a lot of
messy code.  Overall, the results are more accurate, and the new structure is
much more solid, flexible, and extensible, and much easier to work with than
before; it should let us solve any new problems in the "right" way when they
arise.  :) :)

Many other sections of code have been partially or entirely rewritten, and
behavior is generally cleaner.

Global variables are now handled much more cleanly, especially regarding
their persistence when using mod_perl.  Variables are now (almost) completely
divided between UPPER_CASE constants which retain their values between runs,
and lower_case variables which are reset for each run.  After the user config
section is a constant initialization section; both are run only during the
first run of the script, and are skipped for efficiency during subsequent
runs under mod_perl.  The config and initialization sections have been
arranged more carefully than before, for clarity and other reasons.  Several
variables have had their semantics clarified.

NOTE: If you modified the code in an earlier version and used certain config
variables, you should review the new code before inserting the same changes.
For example, a variable like $REMOVE_SCRIPTS should normally be replaced by
$e_remove_scripts; the former is the config setting that never changes
anymore, while the latter reflects the value used for this run of the program
(which the user might change via a checkbox).  There are several similar
variables.  Also, avoid modifying UPPER_CASE variables, because those will
retain their values between runs under mod_perl... unless that's what you
want.


---- Other stuff: ----

The HTTP client in CGIProxy now uses HTTP/1.1 if the browser uses HTTP/1.1.
Before, only certain HTTP/1.1 features like the Host: header were supported;
now, all required HTTP/1.1 client features are supported, so CGIProxy is
"conditionally compliant" with HTTP/1.1 regarding its client functions, as
per the HTTP spec.  CGIProxy can't control all server functions, but for
those that it does (such as the Date: header), it complies with HTTP/1.1.

Improved detection of text vs. non-text when supporting $NO_COOKIE_WITH_IMAGE,
which makes some pages behave better.

Many changes to take advantage of Perl 5, such as "use strict", better
regular expressions, references, and cleaner code all over the place.

Various other bugs fixed, privacy holes closed, performance improvements, UI
improvements, code rearrangement, and cleanup.



1.5.1, released February 7, 2002:
---------------------------------

Headers are no longer split on commas, which among other things means that
cookies work again.  :P

A couple other minor bug fixes.



1.5, released November 26, 2001:
--------------------------------

Many changes this time around, some major, some minor.  The code is about
50% larger.  Here's a list of changes, roughly categorized:


---- Most visible new features and changes: ----

A new cookie management screen lets the user view and selectively delete any
cookies being sent through the proxy.

On pages with frames, any insertion (such as the small URL entry form) is now
in its own top frame.

Cookies may now be encoded with the cookie_encode() and cookie_decode()
routines, similar in concept to proxy_encode() and proxy_decode().


---- Making more pages work: ----

Referrer information, required by some servers, is now optionally sent to
the server.

Certain pages with Flash or other embedded objects now work better.

HTTP URLs which include authentication in them (e.g. "username:password")
are now supported.

Links to "javascript:" URLs are handled in a more friendly way.

Inline frames (i.e. <iframe> tags) are now handled like frames with regard
to insertions.


---- New config options: ----

If you're running this on or inside a firewall, use @BANNED_NETWORKS to ban
access to single hosts and whole networks by IP address.  This is more
reliable than banning by hostname.

Set $REMOVE_TITLES to remove titles from HTML pages.

Error pages now link to the starting page, unless $NO_LINK_TO_START is set.


---- For programmers: ----

There's now a simple framework for handling any arbitrary MIME types that
may need modification (e.g. changing embedded links); see proxify_block().
To add handling for a MIME type, add code in proxify_block() and add your
MIME type into @TYPES_TO_HANDLE (and @OTHER_TYPES_TO_REGISTER if needed).

Additionally, a framework to proxify script content is in place:  All script
content will be routed through the MIME type handler if $PROXIFY_SCRIPTS is
set.  You need to add your own code to proxify_block() for this to actually
do anything.

Parsing of the flag segment of PATH_INFO has been encapsulated into the
routines pack_flags() and unpack_flags().


---- Invisible, or bug fixes: ----

Style sheets are handled much better, including <style> elements, "style"
attributes, and external style sheets.  CSS is explicitly handled, but the
framework is in place for other types.

Certain tags in HTML specify the expected MIME type of the resource they
link to; this is now handled better.

The list of script MIME types is now more complete.

Cookies with the "secure" clause are now supported, though it's never
actually used without SSL support.

Strengthened $NO_COOKIE_WITH_IMAGE support by requiring an appropriate Accept:
header, to guard against certain sneaky Web bugs like at zdnet.com.

The script's URL is now generated using the Host: header if available,
instead of SERVER_NAME; this tends to have better results.

Any resource with an unidentified MIME type is now treated as text/html,
like Netscape does.  This is safest.

<a href> attributes containing character entities are now handled better when
proxy_encode()'ing is used.

FTP URLs with spaces in them are now handled better, in a couple of ways.

Various tags, attributes, and HTTP headers that require special treatment
are now handled more correctly.

Various regex fixes, other minor bug fixes, and cleanup.



1.4.1 and 1.4.1-SSL, released March 8, 2001:
--------------------------------------------

CPU load was decreased 15% with two simple changes that I should have thought
of long ago.

Fixed error with <meta> "refresh" tags that caused proxy to loop through
itself.

Fixed problem with user-chosen URL entry form.



1.4-SSL, released February 22, 2001:
------------------------------------

This is a special version that can retrieve pages from SSL servers.  It
is based on version 1.4, and otherwise works pretty identically to that
release.



1.4, released February 10, 2001:
----------------------------------

You can now optionally insert a compact version of the initial entry form
into the top of every downloaded page, by setting $INSERT_ENTRY_FORM=1.  The
form also displays the URL you're currently viewing.  This is selectable by
the user like the three original user-selectable options.  Frames are handled
correctly, i.e. it's not inserted in frames, because that would be really
ugly.  That was the hard part.

You can also insert your own block of HTML if desired: specify it either as a
fixed string in $INSERT_HTML, or name a file to be inserted in $INSERT_FILE.

For consistency, the URL format was changed slightly-- the initial flags in
PATH_INFO are always there and are now five in number.  So any bookmarks
saved through the proxy will have to be converted or recreated.  If there's
enough demand, I can write a simple converter each time I change the URL
format.

The user-entered hostname is now always lowercased, since host names are
case-insensitive.

Fixed a minor bug in 1.3.2 having to do with PATH_INFO encoding.



1.3.2, released February 3, 2001:
---------------------------------

By popular demand, you can now restrict which servers the proxy can access,
like the online demo does.  This is configured with the lists
@ALLOWED_SERVERS and @BANNED_SERVERS.

For FTP transfers, a "Content-Length:" header is now returned when
guessable.  This lets some browsers show you the percentage progress.

Pseudo-headers created by <meta http-equiv> tags are now handled like real
HTTP headers.  Internally, the handling of HTTP headers has been cleaned up.

If you absolutely, truly, can't run NPH scripts on your server, there is
now an option to run as best as possible as a normal non-NPH CGI script.
For this to work, your server MUST support the "Status:" CGI response
header.  All servers are supposed to support it, but not all do.

There is now a $NO_BROWSE_THROUGH_SELF option which prevents the proxy from
calling itself, which is usually a mistake anyway.

Proxy authentication (the "Proxy-Authorization:" request header) is now
supported in a limited way, with $PROXY_AUTH.

Regexes have been improved to match tag attributes better, and a related
privacy hole was fixed.

URLs with spaces (which are a bad idea anyway) are now more likely to be
handled as expected.



1.3.1, released June 6, 2000:
-----------------------------

Script now runs correctly under mod_perl (requires at least Perl 5.004).

Script now runs correctly on an SSL server, if $RUNNING_ON_SSL_SERVER is set.

Main URL-conversion loop runs almost twice as fast (40% less CPU time), with
a fix I should have noticed a long time ago.

Login for HTTP Basic authentication is now submitted with POST instead of
GET, for better security.

Fixed privacy hole when servers didn't return Content-Type: header.



1.3, released April 8, 2000:
-----------------------------

Anonymity has been improved, especially regarding JavaScript or other
script content.  Before it was an afterthought; now it's being implemented
as completely as possible.  If you know any anonymity holes, please tell
me.  I'm especially interested in knowing any MIME types that identify
scripts.

In particular:

. Much more JavaScript is filtered out than before.  As far as I know, all
  of it is removed:  in <script> blocks, in style sheets, in HTML attributes,
  wherever indicated by HTTP headers, and other places.

. By default, $REMOVE_SCRIPTS is set to true.

. A potential privacy hole from a bug in Internet Explorer is protected.

You can now select which servers to allow scripts from, by setting
@ALLOWED_SCRIPT_SERVERS and @BANNED_SCRIPT_SERVERS.


If several people share a proxy, they can customize their own settings if
you set $ALLOW_USER_CONFIG.

Large files and streaming media are now supported, by transmitting the
data from the server as it arrives, rather than receiving the whole
resource before sending it to the client.  This works for both HTTP and
FTP.

HTTP Basic authentication is now supported.  (I sure hope people use it,
because it's the most elaborate and convoluted hack of the whole program.)

There's an experimental load-balancing feature.  If you set @PROXY_GROUP
to a set of URL's of cooperating proxies, they'll randomly distribute the
load among them.  This may help or hinder privacy, and it may have other
uses too.  Let me know if you find those uses.

For those who like to mess with the code, there are some neat new internal
mechanisms.  Cookies have now been extended to handle multiple tasks (had
to for Basic authentication), and there's a new internally-handled URL
scheme "x-proxy" that lets you plug in whatever magic functionality you
want (had to for Basic authentication).

There is no longer a startproxy.cgi.  It was swallowed by the main script.
It was becoming a vanishingly small percentage of the overall code.

More HTML tags are transformed, whichever non-standard tags people
reported to me (thanks!).

A couple of non-standard HTTP headers with URLs are now transformed
correctly.

If you're running on Windows, you can now set a configuration flag, and
CGIProxy will work around a couple problems on that platform.

$SUPPORT_COOKIES has been reversed, and renamed to $REMOVE_COOKIES.  This
makes it more analogous to $REMOVE_SCRIPTS, and each have their
@ALLOWED...  and @BANNED... server lists.

@BANNED_COOKIE_SERVERS and $NO_COOKIE_WITH_IMAGE have changed slightly--
they now take effect even when $FILTER_ADS isn't set.  They're more
associated with the $REMOVE_COOKIES flag now.

The initial URL-entry form may now submit using POST instead of GET,
based on the setting of $USE_POST_ON_START.  This is because some
filters apparently search outgoing URIs, but not POST request bodies.

FTP now follows symbolic links correctly, and another FTP bug or two were
fixed.



1.2, released September 11, 1999:
---------------------------------

The internal structure was rearranged in a big way, to support multiple
protocols more cleanly.  Previously, HTTP was ingrained throughout; now
it's more modular.

FTP is now supported.

@ALLOWED_COOKIE_SERVERS lets you only accept cookies from certain servers.

@BANNED_COOKIE_SERVERS and @ALLOWED_COOKIE_SERVERS are now lists of Perl
patterns (regular expressions) to match, rather than literal host names.
This lets you allow or forbid whole sets of servers rather than listing
each server individually.  For more information on Perl patterns, read the
Perl documentation.  nph-proxy.cgi has a note in the user config section
that may help enough.

You can remove scripts from HTML pages by setting $REMOVE_SCRIPTS=1.  This
helps with anonymity somewhat by removing some JavaScript (but not all!).
It also removes most popup ads.  :)

The HEAD method is now supported more cleanly.

Rare net_path form of relative URL (i.e. like "//host.com/path/etc") is
now supported, for completeness and safety.

The default lists of cookie and ad servers are a bit better.



1.1, released March 9, 1999:
----------------------------

The whole format of the target URL in PATH_INFO was restructured.  It can
be encoded however the user wishes.  This gets around PATH_INFO clashes in
various servers, solving most problems regarding server incompatibilities
I've heard about.

Cookies are now optionally supported (but off by default).

Banner ads can be filtered out.  Only a simple set of URL patterns are
filtered out by default, but it's easy to add more entries to
@BANNED_IMAGE_URL_PATTERNS.

Cookies from ad servers are filtered out (at least the main ones).  Again,
the default list in @BANNED_COOKIE_SERVERS is simple, but you can easily
add more.

Binary files are no longer getting messed up on Windows.

More HTTP headers are fixed to point back through the proxy.

Under some conditions in 1.0, extra processes would hang around for hours
and drag the system.  Alex Freed added a timeout to solve this for now.  I
can't reproduce the problem, so any info is appreciated.  [9-9-1999: It
may be a bug in older Apaches, fixed by upgrading to Apache 1.3.6 or
better.  Julian Haight reports the same problem with other scripts on
Apache 1.3.3, but not with Apache 1.3.6.]

Internally: code was cleaned up, URL-parsing was improved, and relative
URL calculation was redone.



1.0, released August 3, 1998:
-----------------------------

Initial release.


========================================================================