FTPDIFF Look for differences in the remote site file system |
Top Previous Next |
Note Robo-FTP offers numerous commands for monitoring for files and iterating over directory structures on local and remote servers. There is significant overlap in capabilities between these commands, but there are also important differences that often make one command better suited to a particular task than the others. Please see the chapter Monitoring for Files and Iterating over Directory Structures for a description of these commands and recommendations for which commands to use in a particular situation.
The FTPDIFF and FTPGETDIFF commands work together to identify and process changes to files on a remote server.
The FTPDIFF command detects changes in the portion of the file system specified by the [ path ] argument using a small database file specified by the [ dbfile ] argument. This command adds a record to a "snapshot database" for each file listed in directory listings provided by the remote site. Empty folders are ignored. The FTPDIFF command then compares the new database records that represent the current state of the file system to the old records that represent the previous state of the file system and notes all differences. The differences are available to be processed, one at a time, by the FTPGETDIFF command.
If the file specified by the [ dbfile ] argument does not exist it will be created. The time required to create a new database depends on how many files are present. Since new snapshot databases do not contain any rows representing the previous state of the file system, all files added to a new database will be recorded as "new" files. If you don't want FTPGETDIFF to identify all files as new on the first execution of your command script then you can run this command once in Robo-FTP's main console window to establish a baseline snapshot database. This way, when you first run your command script only file changes since the baseline will be detected.
Note: The fact that all files are considered "new" in a freshly created snapshot database can be exploited to process folders with dynamic contents.
The total number of differences found in the remote site file system is saved in the %ftpdifffiles internal variable.
Consider the following example which compares the state of the current remote folder with the previous state of the remote file system as preserved in the default “snapshot_site.sql” file and then only downloads the changed files:
FTPDIFF "*" :loop FTPGETDIFF IFERROR= $ERROR_READ_EOF GOTO done ;; skip download if the "change" is that file is gone! IFNUM= %ftpdifffileid $DIFF_FILE_NOT_FOUND GOTO loop RCVFILE %ftpdifffilename GOTO loop :done
DIFF is the equivalent command for detecting differences in the local file system.
Note: Snapshot database files are small SQLite database files created by Robo-FTP to support the DIFF and FTPDIFF family of script commands.
Important When called without the optional [ dbfile ] argument, FTPDIFF stores its data in a database called "snapshot_site.sql". If multiple scripts use this particular form of FTPDIFF, problems can arise because this database may end up containing data for what would seem to be the wrong directory or even the wrong site. Data will remain from any previous usage in a different script or even a different context in the same script. This can cause problems when processing the changed files with the associated FTPGETDIFF command. For example, a file could appear new when it is not, or a file could appear to have been deleted when that was not the case. Moreover, a script may function correctly on its own with its FTPDIFF usage, but when another script is created much later that uses the FTPDIFF command in this way, one may find that running the second script causes previously unseen issues with the first. To avoid these conflicts, it is strongly recommended that (1) a [ dbfile ] argument should be provided for each usage of FTPDIFF and FTPGETDIFF, and (2) that the [ dbfile ] filename itself should differ from script to script.
Important The success of this command depends on Robo-FTP's ability to automatically read and understand the directory listings returned by the remote site. Most HTTP/HTTPS sites do not return listings in a supported format and many return no raw directory listings in response to HTTP GET requests based on slash-terminated URLs. Some servers respond to HEAD requests and are partially compatible. Some supported HTTP listing types like APACHE_DEFAULT include file sizes rounded to the nearest megabyte or kilobyte and are therefore not fully compatible with this command. Contact technical support if you have an urgent need related to a raw directory listing format that is currently unsupported.
Related command(s): FTPGETDIFF, FTPDIFFREWIND, DIFF, FILECOMEPAREFROM, SYNC See also: Using the built-in database engine, Processing Dynamic Folders, Alternate Default Path |