WTF *Nix

Python is an extremely extensible language in that you can add new capabilities by importing modules.

Here are just a slim few, but a widely used all throughout the day to day process of programming within python, so here’s for anyone that needs a WTF Quick Card…

sys – Allows interaction with the Python system:

• exit() – exit!
• argv – access command line arguments
• path – access the system module search path
• ps1 – change the ‘>>>’ python prompt!

os – Allows interaction with the operating system:

• name – the current operating system, useful for portable programs
• system – execute a system command
• mkdir – create a directory
• getcwd – find the current working directory

re – Allows manipulation of strings with Unix style regular expressions

• search – find pattern anywhere in string
• match – find at beginning only
• findall – find all occurences in a string
• split – break into fields separated by pattern
• sub,subn – string substitution

math – Allows access to many mathematical functions:

• sin,cos etc – trigonometrical functions
• log,log10 – natural and decimal logarithms
• ceil,floor – ceiling and floor
• pi, e – natural constants

time – time(and date) functions

• time – get the current time (expressed in seconds)
• gmtime – convert time in secs to UTC (GMT)
• localtime – convert to local time instead
• mktime – inverse of localtime
• sleep – pause program for n seconds

random – random number generators – useful for games programming!

• randint – generate random integer between inclusive end points
• sample – generate random sublist from a bigger list
• seed – reset the number generator key

Remember this is just a SLIM bit of what’s instore for you that Python has, there are literally dozens of modules given too you with Python, then plenty more you can download!

Remember that using these within your code, you need to first IMPORT them as such:

>>>import SimpleXMLRPCServer as s

>>>s.SimpleXMLRPCRequestHandler()

Notice the above example is a shorter way of calling this module rather than typeing out the FULL SimpleXMLRPCServer Just saves us time in typing, unless you have an IDE that does it all for you by autocompletion… I prefer VIM for my IDE… So WTF, get it done and >>>import WTFNix.com

The urlparse module included with Python makes it easy to break down URLs into specific components and reassemble them. This is very useful for a number of purposes when processing HTML documents.

The urlparse(urlstring [, default_scheme [, allow_fragments]]) function takes the URL provided in urlstring and returns the tuple (scheme, netloc, path, parameters, query, fragment). The tuple can then be used to determine things such as location scheme (HTTP, FTP, and so on), server address, file path, and so on.

The urlunparse(tuple) function accepts the tuple (scheme, netloc, path, parameters, query, fragment) and reassembles it into a properly formatted URL that can be used by the other HTML parsing modules included with Python.

The urljoin(base, url [, allow_fragments]) function accepts a base URL as the first argument and then joins whatever relative URL is specified in the second argument. The urljoin function is extremely useful in processing several files in the same location by joining new filenames to the existing base URL location.

Try this example out within your PY environment yourself and see:

import urlparse
parsedTuple = urlparse.urlparse(
"http://www.google.com/search?hl=en&q=urlparse&btnG=Google+Search")
unparsedURL = urlparse.urlunparse((URLscheme, \
        URLlocation, URLpath, '', '', ''))
newURL = urlparse.urljoin(unparsedURL,
"/module-urllib2/request-objects.html")

Well here’s the fix if you keep getting a 404 and issues on a Python VV 2.5.x

Just a few minor tweeks for DEFAULT_VERSION and DEFAULT_URL:

#!python
“”"Bootstrap setuptools installation

If you want to use setuptools in your package’s setup.py, just include this
file in the same directory with it, and add this to the top of your setup.py::

from ez_setup import use_setuptools
use_setuptools()

If you want to require a specific version of setuptools, set a download
mirror, or use an alternate download directory, you can do so by supplying
the appropriate options to “use_setuptools()“.

This file can also be run as a script to install or upgrade setuptools.
“”"
import sys
DEFAULT_VERSION = “0.6c9″
DEFAULT_URL     = “http://pypi.python.org/packages/2.5/s/setuptools/” #% sys.version[:3]

md5_data = {
’setuptools-0.6a10-py2.3.egg’: ‘162d8357f1aff2b0349c6c247ee62987′,
’setuptools-0.6a10-py2.4.egg’: ‘803a2d8db501c1ac3b5b6fb4e907f788′,
’setuptools-0.6a11-py2.3.egg’: ‘d12bf8e13aaeb25c91350c8d77f01a71′,
’setuptools-0.6a11-py2.4.egg’: ‘a95d5bc7a070aa1028bc4dcb5270b133′,
}

import sys, os
try: from hashlib import md5
except ImportError: from md5 import md5

def _validate_md5(egg_name, data):
if egg_name in md5_data:
digest = md5(data).hexdigest()
if digest != md5_data[egg_name]:
print >>sys.stderr, (
“md5 validation of %s failed!  (Possible download problem?)”
% egg_name
)
sys.exit(2)
return data

def use_setuptools(
version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir,
download_delay=15
):
“”"Automatically find/download setuptools and make it available on sys.path

`version` should be a valid setuptools version number that is available
as an egg for download under the `download_base` URL (which should end with
a ‘/’).  `to_dir` is the directory where setuptools will be downloaded, if
it is not already available.  If `download_delay` is specified, it should
be the number of seconds that will be paused before initiating a download,
should one be required.  If an older version of setuptools is installed,
this routine will print a message to “sys.stderr“ and raise SystemExit in
an attempt to abort the calling script.
“”"
was_imported = ‘pkg_resources’ in sys.modules or ’setuptools’ in sys.modules
def do_download():
egg = download_setuptools(version, download_base, to_dir, download_delay)
sys.path.insert(0, egg)
import setuptools; setuptools.bootstrap_install_from = egg
try:
import pkg_resources
except ImportError:
return do_download()
try:
pkg_resources.require(“setuptools>=”+version); return
except pkg_resources.VersionConflict, e:
if was_imported:
print >>sys.stderr, (
“The required version of setuptools (>=%s) is not available, and\n”
“can’t be installed while this script is running. Please install\n”
” a more recent version first, using ‘easy_install -U setuptools’.”
“\n\n(Currently using %r)”
) % (version, e.args[0])
sys.exit(2)
else:
del pkg_resources, sys.modules['pkg_resources']    # reload ok
return do_download()
except pkg_resources.DistributionNotFound:
return do_download()

def download_setuptools(
version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir,
delay = 15
):
“”"Download setuptools from a specified location and return its filename

`version` should be a valid setuptools version number that is available
as an egg for download under the `download_base` URL (which should end
with a ‘/’). `to_dir` is the directory where the egg will be downloaded.
`delay` is the number of seconds to pause before an actual download attempt.
“”"
import urllib2, shutil
egg_name = “setuptools-%s-py%s.egg” % (version,sys.version[:3])
url = download_base + egg_name
saveto = os.path.join(to_dir, egg_name)
src = dst = None
if not os.path.exists(saveto):  # Avoid repeated downloads
try:
from distutils import log
if delay:
log.warn(“”"
—————————————————————————
This script requires setuptools version %s to run (even to display
help).  I will attempt to download it for you (from
%s), but
you may need to enable firewall access for this script first.
I will start the download in %d seconds.

(Note: if this machine does not have network access, please obtain the file

%s

and place it in this directory before rerunning this script.)
—————————————————————————”"”,
version, download_base, delay, url
); from time import sleep; sleep(delay)
log.warn(“Downloading %s”, url)
src = urllib2.urlopen(url)
# Read/write all in one block, so we don’t create a corrupt file
# if the download is interrupted.
data = _validate_md5(egg_name, src.read())
dst = open(saveto,”wb”); dst.write(data)
finally:
if src: src.close()
if dst: dst.close()
return os.path.realpath(saveto)

def main(argv, version=DEFAULT_VERSION):
“”"Install or upgrade setuptools and EasyInstall”"”
try:
import setuptools
except ImportError:
egg = None
try:
egg = download_setuptools(version, delay=0)
sys.path.insert(0,egg)
from setuptools.command.easy_install import main
return main(list(argv)+[egg])   # we’re done here
finally:
if egg and os.path.exists(egg):
os.unlink(egg)
else:
if setuptools.__version__ == ‘0.0.1′:
print >>sys.stderr, (
“You have an obsolete version of setuptools installed.  Please\n”
“remove it from your system entirely before rerunning this script.”
)
sys.exit(2)

req = “setuptools>=”+version
import pkg_resources
try:
pkg_resources.require(req)
except pkg_resources.VersionConflict:
try:
from setuptools.command.easy_install import main
except ImportError:
from easy_install import main
main(list(argv)+[download_setuptools(delay=0)])
sys.exit(0) # try to force an exit
else:
if argv:
from setuptools.command.easy_install import main
main(argv)
else:
print “Setuptools version”,version,”or greater has been installed.”
print ‘(Run “ez_setup.py -U setuptools” to reinstall or upgrade.)’

def update_md5(filenames):
“”"Update our built-in md5 registry”"”

import re

for name in filenames:
base = os.path.basename(name)
f = open(name,’rb’)
md5_data[base] = md5(f.read()).hexdigest()
f.close()

data = ["    %r: %r,\n" % it for it in md5_data.items()]
data.sort()
repl = “”.join(data)

import inspect
srcfile = inspect.getsourcefile(sys.modules[__name__])
f = open(srcfile, ‘rb’); src = f.read(); f.close()

match = re.search(“\nmd5_data = {\n([^}]+)}”, src)
if not match:
print >>sys.stderr, “Internal error!”
sys.exit(2)

src = src[:match.start(1)] + repl + src[match.end(1):]
f = open(srcfile,’w')
f.write(src)
f.close()

if __name__==’__main__’:
if len(sys.argv)>2 and sys.argv[1]==’–md5update’:
update_md5(sys.argv[2:])
else:
main(sys.argv[1:])

Someone  requested for one, and here you have it. Enjoy this python cheat sheet

python_cheat_sheet