Many people are trying to (mis)use HTTP to create a full-duplex connection between the client and server. However, there are a number of problems in doing so. I will try to explain here what they are, and under what circumstances you can possibly get it to work. At the end is also a method that doesn't actually use HTTP at all, but is related.
The first thing to note is that HTTP uses a request-response paradigm, not a full-duplex streaming paradigm. Let me repeat that: HTTP is a request-response protocol! This means that the client sends a request, and when the complete request has been sent then the server sends the response. This is the case even if so-called keep-alive is used, i.e. multiple requests are sent over the same TCP connection. Because this behaviour is fundamental to the protocol most implementations make certain (valid) assumptions which make it difficult to create a full-duplex connection.
If you are wanting to create a full-duplex connection through HTTP then you must first ask yourself why. Why not just use TCP? After all, that's exactly what TCP gives you. The reasons I usually hear are that 1) you don't want to write a standalone server, but instead want to use the web server that's already running; or 2) the application needs to work through a firewall, and the only way to get through it is by using HTTP. Unfortunately, the work involved to get it working will probably negate any advantages 1) might seem to offer, so that only 2) is a reasonable argument.
OK, down to the details. We'll first discuss the problems when no proxy is involved, and then discuss the added problems a proxy generates.
The problems here are the following.
In summary, you may be able to get your server to provide you with a full-duplex connection, but chances are it would be easier to write your own (non-HTTP) server. Furthermore, using your own server is probably more efficient as HTTP servers are not designed for long requests and responses (you usually tie up a process or thread per connection).
When proxies are involved the above mentioned problems are compounded - now both the proxy (or proxies) and the server must fulfill the necessary requirements. I've especially seen the second point above as the major problem when going through a proxy: the proxy will wait until it has the complete request before forwarding it to the server. Additionally, while you may have control over the server (to the point of being able to write your own), you (generally) don't have any control over the proxy. This means you must assume the worst case for the proxy.
Furthermore, if you are writing an applet and don't sign it then you must use the browser's HTTP client (via java.net.URLConnection - see Applet Network Security Policy for why). Unfortunately, these clients all first buffer the complete request data (i.e. everything written to the stream from URLConnection.getOutputStream()) before even sending the request, thereby preventing you from creating a true client->server stream.
The easiest solution is to change your application to use a request-response paradigm. If you are trying to tunnel through an HTTP proxy and for some reason really can't change your paradigm, then here are a couple ideas.
In either case you will still have either write your own (simple) HTTP server, or then write some server side handler using an API native to the server.
There's an alternate solution that will give you a full-duplex TCP connection and will work through (virtually) all HTTP proxies. It usually requires that your write your own server, though. The trick is to note that when using HTTPS (HTTP over SSL/TLS) a proxy cannot look into the data stream (since it's encrypted), and therefore can't do any HTTP processing, i.e. get in the way. Instead, when you send the proxy the CONNECT request it turns itself into a simple tunnel, which is exactly what we want.
So, instead of using port 80 use port 443. In the case of transparent proxies (where ISP's reroute all port 80 traffic through a proxy of theirs) you'll automatically avoid going through the proxy; in the case of explicit proxies, for each connection you open you need to first send the CONNECT request and process the response, after which you have full-duplex connection to your server. Note that in most (all?) cases you needn't actually run SSL/TLS, as the proxies don't inspect the data stream to see what exactly you're sending through the tunnel.
One drawback remains: this doesn't work well in applets because you can't use URLConnection for this, but instead you need to use a Socket directly. This means it can't be used in unsigned applets at all, and in signed applets you have to somehow figure out what, if any, proxy is being used and what, if any, username/password is need to authenticate with the proxy.Ronald Tschalär / 16. September 2006 / email@example.com.