[go: up one dir, main page]

Menu

Tree [9a0e51] 0.20 /
 History

HTTPS access


File Date Author Commit
 .cvsignore 2003-10-04 hsn <> [e3499d] *** empty log message ***
 FSPInputStream.java 2004-04-09 hsn <> [9a0e51] 0.20 release
 FSPStreamHandler.java 2004-04-09 hsn <> [9a0e51] 0.20 release
 FSPURLConnection.java 2004-04-09 hsn <> [9a0e51] 0.20 release
 FSPpacket.java 2004-04-09 hsn <> [9a0e51] 0.20 release
 FSPsession.java 2004-01-07 hsn <> [a38996] updated FSP stack
 FSPstat.java 2004-01-07 hsn <> [a38996] updated FSP stack
 FSPutil.java 2004-01-07 hsn <> [a38996] updated FSP stack
 HTUU.java 2001-11-24 hsn <> [ec96b3] Initial revision
 Makefile 2004-04-09 hsn <> [9a0e51] 0.20 release
 NEWS 2004-04-09 hsn <> [9a0e51] 0.20 release
 README 2004-04-09 hsn <> [9a0e51] 0.20 release
 dmachine.cnf 2004-01-07 hsn <> [a38996] updated FSP stack
 dmachine.java 2004-04-09 hsn <> [9a0e51] 0.20 release
 downloadfactory.java 2003-11-22 hsn <> [ed5e89] Added FSP protocol support
 fspfactory.java 2003-11-22 hsn <> [ed5e89] Added FSP protocol support
 ftpfactory.java 2003-11-22 hsn <> [ed5e89] Added FSP protocol support
 qfile.java 2003-09-28 hsn <> [c98dfa] removing downloaded files from queue
 queuefile.java 2003-10-08 hsn <> [7172fd] version 0.18
 regexp.java 2003-09-23 hsn <> [955cf5] *** empty log message ***
 util.java 2003-09-28 hsn <> [c98dfa] removing downloaded files from queue

Read Me

				Download Machine
				
	  simple, non interactive, non graphical, batch download tool
	      for FTP, FSP and HTTP downloads with resume support
		     Licensed as Open Source Software (GPL)

			 Radim Kolar (hsn/at/netmag.cz)
			http://dmachine.sourceforge.net/             

How to use:
===========

1. Configure it by editing dmachine.cnf.
Normally not needed.

2. Run the background download daemon:
java dmachine

3. Put some URLs into queue file:

Open queue file (default name is queue) with any ASCII editor and place
required URL to it, one per line. I use copy & paste between Netscape
and text editor. 

Smart Cache proxy server can be configured to wrote download requests to the
queue file. see http://scache.sourceforge.net

4. Save queue file and wait :)

5. Queue file(s) is checked by daemon, so you can add more URL to it and
save it.

6. DM has also drop directory support. This is a special directory; any
   file in this directory will be processed as queue and then deleted.

Special commands in queue file
==============================

1. Specifying an alternative URLs for file

Download Machine uses only filename part of URL. If you are place:

 http://www.server.com/file.zip (and)
 http://mirror.server.com/mirrored/file.zip
 
 into queue file, DM will thinks that there are alternative URLs for the
 same file.
 
 If the same file is on the net with different filename you can use:
 http://www.server.com/stuff/file-1.2.3.zip#file123.zip
 http://www.server.org/stuff/file-123.zip#file123.zip

2. Custom Referer header

 Some servers checks Referer header. Download Machine sends fake Referer
 header, which works in most time, but sometimes you will need a custom
 Referer header.

 If you place into queue file:
 Referer http://216.42.31.135/ http://www.zephyr.com/
 every request to http://216.... will gets second (zephyr) URL as Referer.

3. Custom cookie header
  
  Some servers needs Cookie header to be sent. Cookies are often used for
  authorization purposes and sometimes server can refuse a download if
  no valid cookie was found in request. First argument is server name
  and second is cookie value in format name=value[; ...]

 cookie http://www.fileplanet.com/ auth=542345123456
 every request to http://216.... will gets second (zephyr) URL as Referer.
 
4. Check also other directory if file is already downloaded

  check_also <regexp> <directory>

  example:
  check_also *.mp3 /home/hsn/mp3/Krishna_Das/

5. Save file to alternate directory

  save_to <regexp> <directory>

  example:
  save_to *.mp3 /home/hsn/mp3/Krishna_Das/
  
Limitations:
============

Access to Password-protected HTTP/FTP files fails for sure because
Java 1.1 can't parse URL with passwords.
Send me your reports about that if you want...

Future:
=======

Make file info's data persistent between dm runs
Support lastmod for FTP
Caching open FTP connections
Support for scheduled downloads
Support for min,max CPS

C/