digitalFAQ.com Forum

digitalFAQ.com Forum (https://www.digitalfaq.com/forum/)
-   Website and Server Troubleshooting (https://www.digitalfaq.com/forum/web-tech/)
-   -   Software for downloading complete websites? (https://www.digitalfaq.com/forum/web-tech/4967-software-downloading-complete.html)

naripeddi 02-22-2013 03:56 AM

Software for downloading complete websites?
 
One of the websites host plenty of historical news papers (archive). I would like to be able to download the enitre site so I can read the papers in offline mode.

Is there a free software that lets me do it?

Thanks.

kpmedia 02-22-2013 06:02 AM

What's the site? It may or may not be possible, depending on what the site uses.

jmac698 02-22-2013 07:12 AM

This isn't the easiest way, but I've used wget, which is a traditional utility for this purpose. It's command line only, but there must be some GUI tool to do what you want somewhere.
http://freecode.com/projects/wget

You have to figure out the options, like level of recursion etc., but I was able to download an entire archive of radioplays from a website.

Here's a sample command line I came up with:
Code:

wget -nd -t 0 -T 15 "http://movies.apple.com/movies/fox/avatar/avatar2009aug0820a-tsr_h1080p.mov" -U "QuickTime/7.6 (qtver=7.6;os=Windows NT 6.0Service Pack 1)" --header "Host: movies.apple.com"
You could also try flashget.
It will grab all files off one page.

thecoalman 02-23-2013 12:23 AM

When you use a tool like that you're abusing the resources of that site. You might find yourself on the other side of a firewall depending on the site. Site scraping is not something webmasters appreciate whether it's legitimate user or not and many monitor for such activity.

naripeddi 02-25-2013 03:33 AM

Thanks to all for the answers.


All times are GMT -5. The time now is 12:04 AM

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.