Jag brukar ju vanligtvis skriva på svenska, men eftersom detta inlägget riktar sig mot en internationell publik tänker jag ta tillfället i akt och skriva det på engelska.
The 4chan download script
This is an update of the 4chan download script written by Daniel Triendl, http://blog.pew.cc/blog/4chan+download+script/
The modified script downloads every image file in a 4chan thread, preserving the original file names (not the incrementing numbers given by 4chan). Perfect for downloading entire sets of pictures or other original content. Tested on a few different boards but should theoretically work on all. Anyway, here it is:
#!/bin/bash
# A bash script for downloading all images in a 4chan thread to their original
# filenames. Updates every 30 seconds until canceled or the thread disappears.
#
# Copyright 2008, 2010 Daniel Triendl, Anton Eliasson
# http://blog.pew.cc/blog/4chan+download+script/
# http://antoneliasson.wordpress.com/2010/08/03/4chan-download-script/
#
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
if [ "$1" = "" ]; then # no arguments
echo "Usage: `basename $0` <4chan thread url> [optional: download directory]"
exit 1
fi
if [ "$2" = "" ]; then # only one argument
LOC=$(echo "$1" | egrep -o '([0-9]*)$' | sed 's/\.html//g' ) # find out the thread number
else
LOC=$2 # use download dir specified by user
fi
echo "4chan downloader"
echo "Downloading to \"$LOC\" until canceled or 404'd"
if [ ! -d $LOC ]; then
mkdir -- $LOC
fi
cd -- $LOC # new directory named after the thread number
while [ "1" = "1" ]; do
TMP=`mktemp` # TMP is the html thread
TMP2=`mktemp` # TMP2 will be a list of all image addresses
TMP3=`mktemp` # TMP3 will be a list of all original file names
# get thread
echo "Updating..."
wget -q -O "$TMP" "$1"
if [ "$?" != "0" ]; then
echo "Update failed, exiting"
date
rm $TMP $TMP2 $TMP3
exit 1
fi
# get file list, space separated
grep -E -o 'http://images.4chan.org/[a-z0-9]+/src/([0-9]*).(jpg|png|gif)' "$TMP" | uniq | tr "\n" " " > "$TMP2"
# get original file name list, space separated (spaces in filenames changed to underlines)
sed 's/ /_/g' "$TMP" | grep -E -o '<span_title=\"[[:alnum:]_-]*.*(jpg|png|gif)' | awk -F \" '{print $2}' | tr "\n" " " > "$TMP3"
COUNT=`cat $TMP3 | wc -w` # total number of files/names
for ((i=1; i<=$COUNT; i++)); do
wget -nv -nc -O `cut -d ' ' -f $i $TMP3` `cut -d ' ' -f $i $TMP2` # now download all files, one by one
done
rm $TMP $TMP2 $TMP3
echo "Waiting 30 seconds before next run"
sleep 30
done;
This should run on any Linux-based OS using the bash shell. Feel free to contact me if you find any bugs and/or improve the script.
