Create Dynamic Wallpaper with a Bash Script

Create Dynamic Wallpaper with a Bash Script
Patrick Wheelan Wed, 04/18/2018 - 09:58

Harness the power of bash and learn how to scrape websites for exciting new images every morning.

So, you want a cool dynamic desktop wallpaper without dodgy programs and a million viruses? The good news is, this is Linux, and anything is possible. I started this project because I was bored of my standard OS desktop wallpaper, and I have slowly created a plethora of scripts to pull images from several sites and set them as my desktop background. It's a nice little addition to my day—being greeted by a different cat picture or a panorama of a country I didn't know existed. The great news is that it's easy to do, so let's get started.

Why Bash?

BAsh (The Bourne Again shell) is standard across almost all *NIX systems and provides a wide range of operations "out of the box", which would take time and copious lines of code to achieve in a conventional coding or even scripting language. Additionally, there's no need to re-invent the wheel. It's much easier to use somebody else's program to download webpages for example, than to deal with low-level system sockets in C.

How's It Going to Work?

The concept is simple. Choose a site with images you like and "scrape" the page for those images. Then once you have a direct link, you download them and set them as the desktop wallpaper using the display manager. Easy right?

A Simple Example: xkcd

To start off, let's venture to every programmer's second-favorite page after Stack Overflow: xkcd. Loading the page, you should be greeted by the daily comic strip and some other data.

Now, what if you want to see this comic without venturing to the xkcd site? You need a script to do it for you. First, you need to know how the webpage looks to the computer, so download it and take a look. To do this, use wget, an easy-to-use, commonly installed, non-interactive, network downloader. So, on the command line, call wget, and give it the link to the page:

user@LJ $: wget

--2018-01-27 21:01:39--
 ↪, ...
Connecting to||:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2606 (2.5K) [text/html]
Saving to: 'index.html'

index.html                                  100%
2.54K  --.-KB/s    in 0s

2018-01-27 21:01:39 (23.1 MB/s) - 'index.html' saved [6237]

As you can see in the output, the page has been saved to index.html in your current directory. Using your favourite editor, open it and take a look (I'm using nano for this example):

user@LJ $: nano index.html

Now you might realize, despite this being a rather bare page, there's a lot of code in that file. Instead of going through it all, let's use grep, which is perfect for this task. Its sole function is to print lines matching your search. Grep uses the syntax:

user@LJ $: grep [search] [file]

Looking at the daily comic, its current title is "Night Sky". Searching for "night" with grep yields the following results:

user@LJ $: grep "night" index.html

Night Sky
Image URL (for hotlinking/embedding):

The grep search has returned two image links in the file, each related to "night". Looking at those two lines, one is the image in the page, and the other is for hotlinking and is already a usable link. You'll be obtaining the first link, however, as it is more representative of other pages that don't provide an easy link, and it serves as a good introduction to the use of grep and cut.

To get the first link out of the page, you first need to identify it in the file programmatically. Let's try grep again, but this time instead of using a string you already know ("night"), let's approach as if you know nothing about the page. Although the link will be different, the HTML should remain the same; therefore, always should appear before the link you want:

user@LJ $: grep "img src=" index.html logo

Selected Comics

It looks like there are three images on the page. Comparing these results from the first grep, you'll see that grep. The other two links contain "/s/"; whereas the link we want contains "/comics/". So, you need to grep the output of the last command for "/comics/". To pass along the output of the last command, use the pipe character (|):

user@LJ $: grep "img src=" index.html | grep "/comics/"

And, there's the line! Now you just need to separate the image link from the rest of it with the cut command. cut uses the syntax:

user@LJ $: cut [-d  delimeter] [-f field] [-c characters]

To cut the link from the rest of the line, you'll want to cut next to the quotation mark and select the field before the next quotation mark. In other words, you want the text between the quotes, or the link, which is done like this:

user@LJ $: grep "img src=" index.html | grep "/comics/" |
 ↪cut -d\" -f2


And, you've got the link. But wait! What about those pesky forward slashes at the beginning? You can cut those out too:

user@LJ $: grep "img src=" index.html | grep "/comics/" |
 ↪cut -d\" -f 2 | cut -c 3-

Now you've just cut the first three characters from the line, and you're left with a link straight to the image. Using wget again, you can download the image:

user@LJ $: wget

--2018-01-27 21:42:33--
Resolving, 2a04:4e42:4::67
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 54636 (53K) [image/png]
Saving to: 'night_sky.png'

night_sky.png                               100%
53.36K  --.-KB/s    in 0.04s

2018-01-27 21:42:33 (1.24 MB/s) - 'night_sky.png'
 ↪saved [54636/54636]

Now you have the image in your directory, but its name will change when the comic's name changes. To fix that, tell wget to save it with a specific name:

user@LJ $: wget "$(grep "img src=" index.html | grep "/comics/"
 ↪| cut -d\" -f2 | cut -c 3-)" -O wallpaper
--2018-01-27 21:45:08--
Resolving, 2a04:4e42:4::67
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 54636 (53K) [image/png]
Saving to: 'wallpaper'

wallpaper                                   100%
53.36K  --.-KB/s    in 0.04s

2018-01-27 21:45:08 (1.41 MB/s) - 'wallpaper' saved [54636/54636]

The -O option means that the downloaded image now has been saved as "wallpaper". Now that you know the name of the image, you can set it as a wallpaper. This varies depending upon which display manager you're using. The most popular are listed below, assuming the image is located at /home/user/wallpaper.


gsettings set org.gnome.desktop.background picture-uri
gsettings set org.gnome.desktop.background picture-options


gsettings set org.cinnamon.desktop.background picture-uri
gsettings set org.cinnamon.desktop.background picture-options


xfconf-query --channel xfce4-desktop --property
 ↪/backdrop/screen0/monitor0/image-path --set

You can set your wallpaper now, but you need different images to mix in. Looking at the webpage, there's a "random" button that takes you to a random comic. Searching with grep for "random" returns the following:

user@LJ $: grep random index.html

  • Random
  • Random
  • This is the link to a random comic, and downloading it with wget and reading the result, it looks like the initial comic page. Success!

    Now that you've got all the components, let's put them together into a script, replacing with the new

    wget "$(grep "img src=" index.html | grep /comics/ | cut -d\"
     ↪-f 2 | cut -c 3-)" -O wallpaper
    gsettings set org.gnome.desktop.background picture-uri
    gsettings set org.gnome.desktop.background picture-options

    All of this should be familiar except the first line, which designates this as a bash script, and the second wget command. To capture the output of commands into a variable, you use $(). In this case, you're capturing the grepping and cutting process—capturing the final link and then downloading it with wget. When the script is run, the commands inside the bracket are all run producing the image link before wget is called to download it.

    There you have it—a simple example of a dynamic wallpaper that you can run anytime you want.

    If you want the script to run automatically, you can add a cron job to have cron run it for you. So, edit your cron tab with:

    user@LJ $: crontab -e

    My script is called "xkcd", and my crontab entry looks like this:

    @reboot /bin/bash /home/user/xkcd

    This will run the script (located at /home/user/xkcd) using bash, every restart.


    The script above shows how to search for images in HTML code and download them. But, you can apply this to any website of your choice—although the HTML code will be different, the underlying concepts remain the same. With that in mind, let's tackle downloading images from Reddit. Why Reddit? Reddit is possibly the largest blog on the internet and the third-most-popular site in the US. It aggregates content from many different communities together onto one site. It does this through use of "subreddits", communities that join together to form Reddit. For the purposes of this article, let's focus on subreddits (or "subs" for short) that primarily deal with images. However, any subreddit, as long as it allows images, can be used in this script.


    Figure 1. Scraping the Web Made Simple—Analysing Web Pages in a Terminal

    Diving In

    Just like the xkcd script, you need to download the web page from a subreddit to analyse it. I'm using for this example. First, check for images in the HTML:

    user@LJ $: wget && grep
     ↪"img src=" index.html
    --2018-01-28 20:13:39--
    Connecting to||:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 27324 (27K) [text/html]
    Saving to: 'index.html'
    index.html                                  100%
    26.68K  --.-KB/s    in 0.1s
    2018-01-28 20:13:40 (270 KB/s) - 'index.html' saved [169355]
    a community ↪for ↪....Forever and ever...... --- SNIP ---

    All the images have been returned in one long line, because the HTML for the images is also in one long line. You need to split this one long line into the separate image links. Enter Regex.

    Regex is short for regular expression, a system used by many programs to allow users to match an expression to a string. It contains wild cards, which are special characters that match certain characters. For example, the * character will match every character. For this example, you want an expression that matches every link in the HTML file. All HTML links have one string in common. They all take the form href="LINK". Let's write a regex expression to match:


    Now let's break it down:

    • href=" — simply states that the first characters should match these.

    • () — forms a capture group.

    • [^] — forms a negated set. The string shouldn't match any of the characters inside.

    • + — the string should match one or more of the preceding tokens.

    Altogether the regex matches a string that begins href=", doesn't contain any quotation marks or hashtags and finishes with a quotation mark.

    This regex can be used with grep like this:

    user@LJ $: grep -o -E 'href="([^"#]+)"' index.html
    --- SNIP ---

    The -e options allow for extended regex options, and the -o switch means grep will print only patterns exactly matching and not the whole line. You now have a much more manageable list of links. From there, you can use the same techniques from the first script to extract the links and filter for images. This looks like the following:

    user@LJ $: grep -o -E 'href="([^"#]+)"' index.html | cut -d'"'
     ↪-f2 | sort | uniq | grep -E '.jpg|.png'

    The final grep uses regex again to match .jpg or .png. The | character acts as a boolean OR operator.

    As you can see, there are four matches for actual images: two .jpgs and two .pngs. The others are Reddit default images, like the logo. Once you remove those images, you'll have a final list of images to set as a wallpaper. The easiest way to remove these images from the list is with sed:

    user@LJ $: grep -o -E 'href="([^"#]+)"' index.html | cut -d'"'
     ↪-f2 | sort | uniq | grep -E '.jpg|.png' | sed /redditstatic/d

    sed works by matching what's between the two forward slashes. The d on the end tells sed to delete the lines that match the pattern, leaving the image links.

    The great thing about sourcing images from Reddit is that every subreddit contains nearly identical HTML; therefore, this small script will work on any subreddit.

    Creating a Script

    To create a script for Reddit, it should be possible to choose from which subreddits you'd like to source images. I've created a directory for my script and placed a file called "links" in the directory with it. This file contains the subreddit links in the following format:


    At run time, I have the script read the list and download these subreddits before stripping images from them.

    Since you can have only one image at a time as desktop wallpaper, you'll want to narrow down the selection of images to just one. First, however, it's best to have a wide range of images without using a lot of bandwidth. So you'll want to download the web pages for multiple subreddits and strip the image links but not download the images themselves. Then you'll use a random selector to select one image link and download that one to use as a wallpaper.

    Finally, if you're downloading lots of subreddit's web pages, the script will become very slow. This is because the script waits for each command to complete before proceeding. To circumvent this, you can fork a command by appending an ampersand (&) character. This creates a new process for the command, "forking" it from the main process (the script).

    Here's my fully annotated script:

    DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
     ↪# Get the script's current directory
    mkdir $DIR/downloads
    cd $DIR/downloads
    # Strip the image links from the html
    function parse {
    grep -o -E 'href="([^"#]+)"' $1 | cut -d'"' -f2 | sort | uniq
     ↪| grep -E '.jpg|.png' >> temp
    grep -o -E 'href="([^"#]+)"' $2 | cut -d'"' -f2 | sort | uniq
     ↪| grep -E '.jpg|.png' >> temp
    grep -o -E 'href="([^"#]+)"' $3 | cut -d'"' -f2 | sort | uniq
     ↪| grep -E '.jpg|.png' >> temp
    grep -o -E 'href="([^"#]+)"' $4 | cut -d'"' -f2 | sort | uniq
     ↪| grep -E '.jpg|.png' >> temp
    # Download the subreddit's webpages
    function download {
    rname=$( echo $1 | cut -d / -f 5  )
    tname=$(echo t.$rname)
    rrname=$(echo r.$rname)
    cname=$(echo c.$rname)
    wget --load-cookies=../cookies.txt -O $rname $1
     ↪&>/dev/null &
    wget --load-cookies=../cookies.txt -O $tname $1/top
     ↪&>/dev/null &
    wget --load-cookies=../cookies.txt -O $rrname $1/rising
     ↪&>/dev/null &
    wget --load-cookies=../cookies.txt -O $cname $1/controversial
     ↪&>/dev/null &
    wait # wait for all forked wget processes to return
    parse $rname $tname $rrname $cname
    # For each line in links file
    while read l; do
       if [[ $l != *"#"* ]]; then # if line doesn't contain a
     ↪hashtag (comment)
            download $l&
    done < ../$linksFile
    wait # wait for all forked processes to return
    sed -i '/' temp # remove reddit pics that
     ↪exist on most pages from the list
    wallpaper=$(shuf -n 1 temp) # select randomly from file and DL
    echo $wallpaper >> $DIR/log # save image into log in case
     ↪we want it later
    wget -b $wallpaper -O $DIR/wallpaperpic 1>/dev/null # Download
     ↪wallpaper image
    gsettings set org.gnome.desktop.background picture-uri
     ↪file://$DIR/wallpaperpic # Set wallpaper (Gnome only!)
    rm -r $DIR/downloads # cleanup

    Just like before, you can set up a cron job to run the script for you at every reboot or whatever interval you like.

    And, there you have it—a fully functional cat-image harvester. May your morning logins be greeted with many furry faces. Now go forth and discover new subreddits to gawk at and new websites to scrape for cool wallpapers.