Introduction: Syncing Folders With Python

By

This instructable will show you how to keep two folders (and all folders within them) in sync so one is a direct copy of the other. Ideal for backing up work both locally, to a cloud/network server or a USB drive. No experience with programming is necessary to complete this tutorial. Please note this only works with Windows although the process is similar on Mac and Linux.

I have an old windows computer that I have setup acting as a server for various functions, one of which is a cheap network attached storage which acts as both a media server and backup to all my family's computer data.

As my family all like to work locally on their PCs, we needed an easy way to backup our data regularly. I also needed a way to double down on the backup by implementing a RAID like approach on two hard drives on the server PC. I also did not want to pay for any software (yes I know stingy me). As a consequence of this, all free software tends to come with pop ups or even slowed down the PC with long sync times and big CPU usage, which was just annoying.

Therefore with a small bit of effort, I wrote a python script which would sync the folders we all needed. I could then customise and distribute this to the family as an executable which I could set windows to run on a regular basis in the background. The computer user had to be none the wiser.

Step 1: Installing Python and Dependencies

To create the distributable program you need to install python on your computer. To do this follow the web link here https://www.python.org/downloads/ and scroll down until you find the version you want. Note: Unfortunately, as of writing the module pyinstaller which we will use does not work yet with python 3.8 so you will need to use a compatible version of python (3.5-3.7).

Follow the installation through making sure to tick the "Add to PATH" checkbox.

Once installed open up a command prompt and install the python modules we need, to do this type the following and follow through with any prompts as required:

pip install pyinstaller

pip install dirsync

Step 2: The Python Script

The script is nice and simple, only two lines, copy and paste the following into either IDLE (installed with Python) or notepad and save as "DirectorySync.py":

from dirsync import sync

sync('C:\\FOLDER_A', 'E:\\FOLDER_B', 'sync', purge = True)

Make sure to change the two folders above with the two folders you wish to sync. The double backslash is required in the path name as the backslash is an escape character in Python.

The first line imports the dirsync module we installed previously.

The second performs the sync. The first folder is the source folder and the second is the target, the 'sync' is the telling the sync function what syncing mode to implement. Adding purge=True tells the function to delete anything in the target folder which is no longer in the source folder. There are other options which can be implemented depending on your needs.

The way it is configured above will also sync all the folders within the top level folder too, this can be stopped if required. If you want to sync more than one top level folder, simply add more lines to your code with the sync() function. For more options and help see the following link:

https://pypi.python.org/pypi/dirsync/2.2.2

Step 3: Creating .exe

To create the executable we now need to move back to the command prompt.

Type the following replacing the folder path with the route to the DirectorySync.py script we create in the last step:

pyinstaller -F -w C:/Route_to_your_folder/DirectorySync.py

In the folder location of your python script this creates a series of folders: __pycache__, build, dist and two other files. In the dist folder is now a file called DirectorySync.exe, running this will perform the sync in the background. This file can be distributed as is to anyone and they can then run a sync without having to have python installed on the computer.

To explain what is going on in the BOLD text which was typed in the command prompt:

'pyinstaller' tell the computer to use the module pyinstaller which we downloaded earlier

'-F' is an option which tells pyinstaller to only generate one executable and not a series of folders that would also have to be distributed.

'-w' is an option which tells the computer to not display a command prompt every time it runs the script.

The path is the path to the python script.

For more options and help see the below link:

https://pyinstaller.readthedocs.io/en/stable/usage...


Step 4: Running Automatically

You are now ready to sync any folders without having to copy, paste and delete repeatedly by just double clicking on the executable. But we want to go a step further than that and have Windows run the process automatically so you do not need to worry.

To do this we will use the Task Scheduler program that comes with Windows, this process is based on Windows 10 but is almost identical on other Windows platforms.

  1. Open up the Task Scheduler from the start menu.
  2. On the right hand side select 'Create Task' from the menu.
  3. Give it a name and description and at the bottom make sure it is configured for the right operating system.
  4. On the 'Triggers' tab, create a new trigger by clicking on 'New' on the bottom left, on the new pop up select the configuration you want, I chose to begin the task at Log on and repeat every hour so I know I have a backup of my work every hour. Click OK.
  5. On the 'Actions' tab create a new action in the same way. The action we need is to start a program which is the default. Browse to the executable we created earlier and select. NOTE :- if you move the executable after creating the task, the task and therefore the sync will not complete.
  6. On the 'Conditions' tab de-check the power settings so it will run on battery as well as plugged in.
  7. Click OK and you have now created your task.

Restart the computer and after a while check the target folder location and see that the sync worked, please note if you have a large folder, the sync may take a while to copy all the folders across the first time.

That is the tutorial complete, I hope you find it useful, any questions, let me know.

Step 5: UPDATE 15 JAN 2020 - Download Program

I still use this program which has served me well However I have been frequently asked by people to do one off syncing jobs for different folders manually etc. Therefore I thought I would update this Instructable with a link to a program I created to do the job. It provides a user interface so that people can easily do one of sync jobs. The program can be downloaded from Github .