wenku8_fetch

From NAT, 6 Years ago, written in Bash, viewed 724 times.
URL https://code.nat.moe/view/60f6e1e4 Embed
Download Paste or View Raw
  1. #!/bin/bash
  2. # A tool to download all novels on wenku8.com
  3.  
  4. STORE_PATH=./save/
  5. FETCH_URL='http://www.wenku8.com/modules/article/articlelist.php?page='
  6. DOWNLOAD_TYPE="utf8"
  7. DOWNLOAD_URL="http://dl.wenku8.com//txt$DOWNLOAD_TYPE/__K/__ID.txt"
  8. TEMP=temp.tmp
  9. FROM=1
  10. TO=93
  11.  
  12. for page in $(seq $FORM $TO)
  13. do
  14.         echo "Starting page $page..."
  15.         curl $FETCH_URL$page 2> /dev/null > $TEMP
  16.         cat $TEMP | iconv -f gbk -t utf-8 | grep 'font-size:13px;' | sed -e 's/.*book\///g; s/.htm">/ /g; s/<\/a><\/b>//g;' > title$TEMP
  17.         _ids=$(cat $TEMP | iconv -f gbk -t utf-8 | grep 'font-size:13px;' | sed -e 's/.*book\///g; s/\.htm.*//g')
  18.         for novel in $_ids
  19.         do
  20.                 echo "Downloading $(cat title$TEMP|grep $novel)"
  21.                 _this_url="$(echo $DOWNLOAD_URL|sed -e "s/__K/1/; s/__ID/$novel/;")"
  22.                 _this_save="$STORE_PATH/$(cat title$TEMP|grep $novel|tr ' ' '_')"
  23.                 curl $_this_url > $_this_save 2> /dev/null
  24.                 [[ ! -z $(cat $_this_save | grep '404 Not Found') ]] && _this_url="$(echo $DOWNLOAD_URL|sed -e "s/__K/2/; s/__ID/$novel/;")" && curl $_this_url > $_this_save 2> /dev/null
  25.         done
  26. done

Replies to wenku8_fetch rss

Title Name Language When
wenku8_fetch_se NAT bash 6 Years ago.

Reply to "wenku8_fetch"

Here you can reply to the paste above

captcha