Perl and Python through a proxy server

BLOG MOVED

Please click the link below to read in new location.

http://goochgooch.co.uk/2014/09/08/perl-and-python-through-a-proxy-server/

 

---

 

Writing scripts from inside a corporate environment to retrieve URLs or to call APIs, you’re bound to hav

e to negotiate a web proxy server/firewall. Whilst it’s not too difficult to set the necessary environment variables, it often means having to wrap your script in another script, which entails deploying more files, writing more instructions, etc. It’s much neater to set up the proxy and associated authentication from inside the script.

Here is the bare bones code I’ve found most successful for Perl and Python, written on my Windows 7 Enterprise work laptop.

Perl 5.18 (Strawberry Perl )

#!/usr/bin/perl -w

use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use Data::Dumper;

# *** PUT YOUR VALUES HERE ***
use constant {
  PROXY      => "xxx.xxx.xxx.xxx",
  PROXY_PORT => "80",
  YOUR_ID    => "id-goes-here",
  YOUR_PW    => "password-goes-here",
  URL        => "http://www.google.co.uk",
};


my $ua = new LWP::UserAgent;
$ua->proxy('http', 'http://'.PROXY.':'.PROXY_PORT.'/');

my $request = HTTP::Request->new(GET => URL);
$request->proxy_authorization_basic( YOUR_ID, YOUR_PW );

# Make the call
my $response = $ua->request($request);

print Dumper $response;

if ($response->is_success) {
  print "SUCCESS\n"
} else {
  print "FAILURE\n"
}

Python 3.4

#!/usr/bin/python

import urllib.request
from urllib.error import URLError

# *** PUT YOUR VALUES HERE ***
PROXY      = "xxx.xxx.xxx.xxx"
PROXY_PORT = "80"
YOUR_ID    = "id-goes-here"
YOUR_PW    = "password-goes-here"
URL        = "http://www.google.co.uk"


proxy = urllib.request.ProxyHandler({
    "http":
    ''.join(["http://", YOUR_ID, ":", YOUR_PW, "@", PROXY, ":", PROXY_PORT])
})

auth = urllib.request.HTTPBasicAuthHandler()

opener = urllib.request.build_opener(proxy, auth, urllib.request.HTTPHandler)
urllib.request.install_opener(opener)

# Make the call
try:
    connection = urllib.request.urlopen(URL)
    print(connection.read())
    print("SUCCESS")
except URLError as e:
    print("FAILURE: %s" % e)