corrupted dbase

[kovidgoyal]

Note that the save to disk operation can be automated with the

calibredb export --all --to-dir backup_path

command.

[ccayer]

1. In a perfect world (sigh) a back up would occur of the NEW books I added that day (with covers) along with a daily metadata.

Microsoft's ROBOCOPY is a free command line replacement for XCOPY that is designed for reliable mirroring of directories or directory trees. It's a great tool when you're trying to backup a folder to another location. The switches can be set to remove files in the destination when they are removed from the source. Probably the most useful feature is to ignore files that haven't changed; the initial copy may take a while, but subsequent copies will finish much quicker.

Here's a sample command line I'd use to backup the Calibre library:

robocopy "c:\path\to\Calibre\Library" "e:\Backup\Location" /s /purge /xo /r:1 /w:1

This will simply copy all files from the Library to the Backup location, including all subdirectories (/s). It will delete destination files and directories that no longer exist in the source (/purge). If destination file exists and is the same date or newer than the source, don't bother to overwrite it (/xo). If a file can't be read for some reason, it will retry 1 time (/r:1), at a 1 second interval (/w:1)

[itimpi]

Microsoft's "RichCopy" is a newer (free) alternative to RoboCopy that is meant to be faster.

[ccayer]

Microsoft's "RichCopy" is a newer (free) alternative to RoboCopy that is meant to be faster.

RichCopy is a great utility; it make it very simple to set up a copy job, including all of the extra features like purging deleted files, excluding certain file types, and only copying files that have changed. It will even show you the corresponding command line for the options you have chosen so it can be easily integrated into a batch file.

The command line to copy only changed files, purging anything that was deleted from the source, would look like this:

richcopy "C:\path\to\Calibre\Library" "e:\Backup\Location" /FSD /P /PF

[Debby]

Wow! So many choices..... thank you, I really appreciate the code examples, it really helps.

Here is the batch file I created last night and tested. Although it worked, is my syntax correct? I am using a mapped network drive.

xcopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /D /S /Y

I like the idea of using kovidgoyal code ....
calibredb export --all --to-dir backup_path

But, when I do run "save to disk" I always get the following errors (see below). Is this normal? I normally hit the x in the right corner to make it go away but it happens at least 15 times during the save to disk. If I am not there to hit the x would it matter?

About syncing - can I have the libraries sync with a program such as good sync?

For the future may I suggest that Calibre consider some type of automation for backing up? Or maybe start a dedicated thread for backup and saving? Would this be something a plugin would do? I will search that forum to see of there is any information.


Calibre-parallel.exe has stopped working
Problem signature:
Problem Event Name: APPCRASH
Application Name: calibre-parallel.exe
Application Version: 0.6.52.0
Application Timestamp: 4be457d1
Fault Module Name: podofo.dll
Fault Module Version: 0.7.0.0
Fault Module Timestamp: 4ac4c55c
Exception Code: c0000005
Exception Offset: 0005fa7d
OS Version: 6.1.7600.2.0.0.256.1
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

Read our privacy statement online:
http://go.microsoft.com/fwlink/?link...8&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt

[ccayer]

Here is the batch file I created last night and tested. Although it worked, is my syntax correct? I am using a mapped network drive.

xcopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /D /S /Y

The syntax looks perfect. Most people forget the quotes around the source & destination paths. It looks like it should only copy the files and folders that have been updated, without prompting you to overwrite existing files. If you remove something in the source, it won't remove it from the backup folder though. If you rename something that has already been backed up, the original AND the renamed file will be in the backup folder after you run the xcopy.

You can add the /V switch to verify that the destination file, once written, can be read. Most of the time this isn't necessary, but since it's a backup the switch is an extra bit of insurance.

[Starson17]

Originally Posted by Debby

Here is the batch file I created last night and tested. Although it worked, is my syntax correct? I am using a mapped network drive.

xcopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /D /S /Y

It looks good to me, assuming your directory paths are correct. It relies on the date on the computer being correct, but it normally would be. You can precede it with:

z:
cd \Debby\My Documents\Calibrelibrary\Debs
if exist metadata.db.6 del metadata.db.6
if exist metadata.db.5 ren metadata.db.5 metadata.db.6
if exist metadata.db.4 ren metadata.db.4 metadata.db.5
if exist metadata.db.3 ren metadata.db.3 metadata.db.4
if exist metadata.db.2 ren metadata.db.2 metadata.db.3
if exist metadata.db.1 ren metadata.db.1 metadata.db.2
if exist metadata.db copy metadata.db metadata.db.1

and it will always keep the last 6 copies of your metadata.db.

I like the idea of using kovidgoyal code ....
calibredb export --all --to-dir backup_path

That's a different type of backup, as discussed here. It doesn't keep the metadata.db, recipes, book conversion settings, etc. found in that file. It also will export all books each time it runs. ccayer had some valid comments on XCOPY, and they would apply to doing this type of export. Old files would not be removed. That would make it harder to import later.

IMHO, RichCopy or another sync program would be a better option.

[Debby]

I have now successfully added Robocopy and have the following running in Task Scheduler (If I remove the quotes it won't run). Everything is tested and running! Thank you!

robocopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /s /purge /xo /r:1 /w:1

I also created another batch file to back up the metadata data using Starson17's code.

@echo off

REM metafile copy documents
Z:
cd\Debby\My Documents\Calibrelibrary\Debs
if exist metadata.db.6 del metadata.db.6
if exist metadata.db.5 ren metadata.db.5 metadata.db.6
if exist metadata.db.4 ren metadata.db.4 metadata.db.5
if exist metadata.db.3 ren metadata.db.3 metadata.db.4
if exist metadata.db.2 ren metadata.db.2 metadata.db.3
if exist metadata.db.1 ren metadata.db.1 metadata.db.2
if exist metadata.db copy metadata.db metadata.db.1

move metadata.db.* "Z:\Debby\My Documents\Calibre Saved Libraries\Metafile BU

[Starson17]

Originally Posted by Debby

I have now successfully added Robocopy and have the following running in Task Scheduler

Great!

(If I remove the quotes it won't run).

That's because you have some directory names with spaces in them.

I also created another batch file to back up the metadata data using Starson17's code.

You could have put the code into the same batch file. Either have robocopy copy off the multiple metadata.db backups when it runs, or you can just change the locations of the metadata.db.N source and destination files to c:\some\place\else\metadata.db.N.

I'm glad it works for you, and thanks for posting your final setup - it may help others.

[Debby]

You could have put the code into the same batch file. Either have robocopy copy off the multiple metadata.db backups when it runs, or you can just change the locations of the metadata.db.N source and destination files to c:\some\place\else\metadata.db.N.

I'm glad it works for you, and thanks for posting your final setup - it may help others.[/QUOTE]

I think it would make more sense if it was in one file. So, should it look like this?

@echo off

REM metafile copy documents
Z:
cd\Debby\My Documents\Calibrelibrary\Debs
if exist metadata.db.6 del metadata.db.6
if exist metadata.db.5 ren metadata.db.5 metadata.db.6
if exist metadata.db.4 ren metadata.db.4 metadata.db.5
if exist metadata.db.3 ren metadata.db.3 metadata.db.4
if exist metadata.db.2 ren metadata.db.2 metadata.db.3
if exist metadata.db.1 ren metadata.db.1 metadata.db.2
if exist metadata.db copy metadata.db metadata.db.1

move metadata.db.* "Z:\Debby\My Documents\Calibre Saved Libraries\Metafile BU

robocopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /s /purge /xo /r:1 /w:1




Sorry - I don't understand the syntax

c:\some\place\else\metadata.db.N.

[Starson17]

Originally Posted by Debby

I think it would make more sense if it was in one file.

I was thinking like this:

@echo off

REM metafile copy documents
Z:
cd \Debby\My Documents\Calibrelibrary\Debs
REM The default location for the batch file is now is inside your library with the metadata.db file

if exist metadata.db.6 del metadata.db.6
if exist metadata.db.5 ren metadata.db.5 metadata.db.6
if exist metadata.db.4 ren metadata.db.4 metadata.db.5
if exist metadata.db.3 ren metadata.db.3 metadata.db.4
if exist metadata.db.2 ren metadata.db.2 metadata.db.3
if exist metadata.db.1 ren metadata.db.1 metadata.db.2
if exist metadata.db copy metadata.db metadata.db.1

REM Now you have the last 6 copies plus the master of metadata.db in your library folder.
REM Every time the batch file runs, it will throw away the oldest of the 6 and keep the newest.

REM You don't really need the next line, so I remarked it out.
REM move metadata.db.* "Z:\Debby\My Documents\Calibre Saved Libraries\Metafile BU"


REM when the next line runs, it will put the 6 current backups of your metadata AND all your ebooks into your backup folder.
REM so you really don't need to keep them separately in "Metafile BU"

robocopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU" /s /purge /xo /r:1 /w:1

REM I didn't check your options, but I assume they will move the contents of "Calibrelibrary\Debs" into "Calibre Saved Libraries\BatchFile BU"

REM You could have just used
REM  robocopy "Z:\Debby\My Documents\Calibrelibrary\Debs" "Z:\Debby\My Documents\Calibre Saved Library" /s /purge /xo /r:1 /w:1

Was this clear?
If you need a backed up metadata.db.1, you just find it in:
Z:\Debby\My Documents\Calibre Saved Libraries\BatchFile BU
or in:
Z:\Debby\My Documents\Calibre Saved Library
(If you used the shorter one I suggested).

Sorry - I don't understand the syntax

c:\some\place\else\metadata.db.N.

I meant this:

if exist Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db.6 del Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db.6
if Z:\Debby\My Documents\Calibrelibrary\Debs\exist metadata.db.5 ren Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db.5 Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db.6
...
if exist Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db copy Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db Z:\Debby\My Documents\Calibrelibrary\Debs\metadata.db.1

It was just ugly, but it can be run anywhere instead of requiring you to do this first:
Z:
cd \Debby\My Documents\Calibrelibrary\Debs

[Debby]

Thank you! Hopefully one last question - the files say always say "Running" in Task Scheduler. But, they have completed the task and never switch to "ready". Is that a problem?

[Starson17]

Originally Posted by Debby

Thank you! Hopefully one last question - the files say always say "Running" in Task Scheduler. But, they have completed the task and never switch to "ready". Is that a problem?

I have a backup task that I started 4 years ago. It still says "Running" on XP's Scheduled Tasks list.

Depending on the size of your library, that batch will take a while to run the first time. If you are concerned, just go to the run box, type "cmd" to get a terminal/command prompt and then cd to your batch and run it manually - see if it ends properly. I suspect it will.

One last comment. I looked at your robocopy options. You might want to take off the /purge option, and keep another copy of your batch file with that option. You could manually run the second batch file whenever you wanted to recover some extra space.

Here's why:
The backups of metadata.db contain links to ebooks in a specific folder. Most of us don't delete a lot of books, but we do fix metadata, which may change author or title. If you change author or title on a book, Calibre will change the folder name where the book is stored, and change the link in metadata.db to that folder. Running robocopy with the /purge option will remove the old folder name and the old book from your backup, but that's the folder name you'd like to keep because it matches the metadata.db backups.

Basically, turning off the purge option will keep a few older copies of ebooks in your backup that match the older metadata.db backups. They won't hurt the newer backups, except to the extent that a bit of extra space is used.

If you leave purge always on, the older metadata.db backups can still be used, but you might lose the links to a book if you changed title or author. The book would still be in your backup, but it would be under its newer folder name. You could probably find it if necessary.

[Debby]

Thank you! I really appreciate your taking the time to help!