FTP sync is a process where, through File Transfer Protocol, files are synchronized to or from an FTP or SFTP server so that they can be backed up, shared with certain parties, or fed to a back-end system promptly.
Benefits of FTP and SFTP Sync with TurboFTP Client
- Choose one of the flexible FTP sync schemes to do a selective sync or simply mirroring.
- Protect your data in transit with a secure FTP/SFTP channel protected with industrial-strength cryptography.
- The Built-in Open PGP feature offers the option to add an extra layer of protection to your data in transit and even at rest. Open PGP encryption is a much more reliable way to ensure the safety of data stored in the public or private cloud.
- Relentless attempts to automatically retry interrupted transfers with checkpoint restart in any hostile network environment — critical to ensure delivery of your valuable data on schedule and complete.
- Email notification upon successful delivery or error.
How to Synchronize Data?
With the TurboFTP client, synchronizing data is easy to set up and test drive with the GUI without scripting. An FTP or SFTP server such as TurboFTP Server must be running to allow TurboFTP to sync files against. The following guide will help you to sync data on FTP/SFTP properly.
1. Concepts
Source and Destination
The source and destination folders are defined based on the synchronization direction. For example, if the sync direction is Download, the remote folder is the source folder, and the local folder is the destination folder.
2. Synchronization Schemes
Synchronization Schemes define how TurboFTP chooses files to transfer after the file comparison is done. We will explain each of the available Synchronization Schemes in TurboFTP by examples.
2.1 Add only new files to the destination
Only transfer new source files.
EXAMPLE 1: Only transfer new source files. No file timestamp comparison is involved.
2.2 Update only existing files at the destination
Transfer files that overwrite old copies in the destination folder.
EXAMPLE 2: TurboFTP will compare local and remote file timestamps under this scheme. In this example, the local file root.txt exists locally and remotely, and the local copy is newer. After the synchronization, the local copy of root.txt is uploaded, while test.php is not.
2.3 Update and add new files to the destination
Transfer new and updated source files that overwrite old copies in the destination folder.
EXAMPLE 3: the local copy of root.txt is newer, and after synchronization, it is uploaded. The local folder has test.php, which the remote destination folder doesn't have; therefore, test.php is uploaded as a new file.
Before action:
After action:
2.4 Update, add new, and remove unique files at the destination
Transfer new and updated source files that overwrite old copies in the destination folder and remove orphan files in the destination folder. Unique/orphan files exist at the destination but not at the source.
EXAMPLE 4: Note that 'have.rar' at remote doesn't exist locally and is seen as a unique file. Therefore it is removed after the action.
Before action:
After action:
2.5 Transfer files added or updated since the last sync
Compare time stamps of source files with the last run time of the task; if it is newer, transfer them to the destination. With this scheme, file size comparison is not applicable. The first time the job runs, all source files will be transferred since there is no valid 'last runtime'.
EXAMPLE 5: After the 1st execution of the task:
Then root.txt is modified, and the task is executed again. This time the file 'test.php' is uploaded because its timestamp is newer than the last run time.
2.6 Remove duplicate files at the destination
Find out files that exist both locally and remotely and remove the copies at the destination.
Before action:
After action:
2.7 Transfer all source files
Transfer all files in the source folder, overwriting any existing files at the destination.
EXAMPLE 6: all files are transferred regardless of whether the destination folder has them or not.
Before action:
After action:
2.8 Transfer all source files and remove unique files at the destination
Transfer all files in the source folder, overwriting any existing files at the destination. Remove orphan files in the destination folder. This is an option for 100% mirroring without caring about the result of file comparisons between both sides in terms of file time or file size.
EXAMPLE 7: Before action: the orphan file 'have_no.rar' at the destination is removed after the action.
After action:
3. Other Options for Synchronization
3.1 Include subfolders: whether synchronize subfolders as well.
3.2 Ignore file name case: when comparing local and remote files, ignore file name case difference.
3.3 Compare file size rather than file time: TurboFTP compares file time stamps during synchronization by default. Due to the limitation of regular FTP, where time in remote file listing is within the accuracy of a minute, you might want to decide which files to transfer based on the file size difference in bytes.
3.4 The last two options only relate to the visual comparison of Folder Synchronization in TurboFTP and will be ignored when doing automated synchronizations.
4. Execute external programs during synchronization
TurboFTP allows you to interface your workflow processes with its synchronization action. External programs can be executed before the task, after the task, and against each downloaded file (locally) or uploaded file.
4.1 Execute Programs before Task
EXAMPLE 8: Pack local files into an archive, and upload the archive to a remote server.
archive.bat
"C:\Program Files\WinRAR\Rar.exe" a "C:\123\test\test.rar" "C:\123\test\test.php"
As we only want to upload archives, we set up an inclusion file filter:
After it runs, the local files are packed into test.rar and uploaded:
4.2 Run program after task
EXAMPLE 9: Download a remote archive and extract it into a local folder.
expand.bat
"C:\Program Files\WinRAR\UnRAR.exe" e "C:\123\test\test.rar" "C:\123\test\"
4.3 Execute external program against each downloaded file
EXAMPLE 10: Add each downloaded file to an archive. We have an archive back.rar and will add each downloaded file to it.
addarchive.bat
"C:\Program Files\WinRAR\Rar.exe" a "C:\backup\backup.rar" %1
4.4 Execute a program against each uploaded file
EXAMPLE 11: Extract a backup archive and upload extracted files to a remote folder.
expand.bat
"C:\Program Files\WinRAR\UnRAR.exe" e "C:\backup\backup.rar" "C:\123\test\"
While the task runs, we delete each uploaded local file using del.bat.
