Scraping the web with ♥ Python ♥

I have a bunch of reference folders where I dump stuff that inspires me. I use those images as my screensaver which pops up after 3mins or so. Once in a while, I go on an inspiration finding frenzy and fill those up with new stuff.

Here are some of my sources (but not all):
National Geographic POD
Flickr explore last 7 days
Inspire Me Now – Random
Astronomy POD
Earth POD

My favorite must be National Geographic POD. Amazing photography, every single day. But instead of going through manual labor of saving that picture to my ref folder every day (you lazy bastard!), I thought I’d automate it.

Here’s the script I wrote in python that visits the NPOD page each time my PC boots up and saves the image to my reference folder. You’ll need to get the BeautifulSoup python module, which greatly simplifies the code.

import os
import urllib2
import time
from urllib import urlretrieve
from BeautifulSoup import BeautifulSoup

# Make sure we have an internet connection (10 attempts)
#  because we might not be connected when this script runs at startup
for i in range(0, 10):
		print "Connection to interwebs attempt:", i
		urllib2.urlopen("", timeout=2)
		print "...succes!"
		print "...failed! Retrying."

# Find today's image in the html doc
url = ""
soup = BeautifulSoup(urllib2.urlopen(url).read())
npod_image = None
	# Is there a high resolution wallpaper of today's image available?
	npod_image = soup('div', {'class' : 'download_link'})[0].a['href']
	print "Getting wallpaper image failed"
		# No wallpaper, get the picture
		npod_image = soup('div', {'class' : 'primary_photo'})[0].a.img['src']
		print "Getting main image failed"

# Save the image
script_path = os.path.abspath(os.path.dirname(__file__))
npod_image_dest = os.path.join(script_path, os.path.basename(npod_image))
print "Saving", npod_image, "to...\n   ", npod_image_dest
urlretrieve(npod_image, npod_image_dest)

To make it run at start-up, create a batch file in your startup dir that runs the python script.

C:\Python27\python.exe "E:\Waldo\Pictures\Reference\scripts\"

I have a bunch of these scripts that pull stuff in that inspires me every day.

I love python!