Corrupted library won't restore

[Geta]

Hi all. I have a sizeable library that I've been slowly organising.

It's stored on an external HDD along with Calibre Portable.

Recently I noticed a bunch of strange looking folders with garbled names on the hard drive, and without thinking I deleted them. (I know, I know)

Next time I loaded Calibre it told me the database was corrupt, and it took a good few hours to go through the repair process and then failed. I've tried to restore the deleted folders but with no luck.



This is the error message:

calibre, version 2.83.0
ERROR: Failed to repair library: Restoring database failed, click Show details to see details

Traceback (most recent call last):
File "site-packages\calibre\db\restore.py", line 127, in run
File "site-packages\calibre\db\restore.py", line 280, in replace_db
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: u'E:\\eBooks\\metadata.db'


I'd greatly appreciate any help in getting it sorted. I thought I may just create a whole new library, but when doing a test run on one author the metadata was all over the place (some swapped author/titles, missing metadata and covers etc) so it seems Calibre hasn't been updating the files themselves as I'd hoped, and if I re import the whole library I will lose all of my work in cleaning it up.

Thanks

[kovidgoyal]

Run the restore from the command prompt, with calibre closed, like this:


calibredb restore_database -r

[Geta]

Thanks Kovid. It didn't work - unfortunately I think I've stuffed things up irreversibly by trying to fix the problem myself rather than coming here first and asking for help from the experts.

Edit: I've managed to get a backup of the database from another hard drive I had this synced to. It's a month since I last synced them so I've lost some work but I'm not back to square one at least.

I now have a library sitting in a folder that contains more books than are included in the library.

What is the best way to add the new books while skipping over the ones that are already in the library?

[kovidgoyal]

You want one book per directory not multiple books per directory to restore.

[Geta]

Thanks mate, you're a life saver. Thanks for taking the time to help me out

[Geta]

Unfortunately it wasn't so easily solved....

I copied the database file from the other HDD and started importing the folder to bring in the 'new' books.

It got a bit of the way through before giving an error - something about the import robot(?) failing. Clicking details showed me which book it happened on, and since it wasn't a book I cared about much I simply deleted it, and tried again.

The same thing happened once more on a different book.

I closed Calibre down and restarted it, and then when I tried to go into the library it gave a database corrupt error.

Trying to restore it got most of the way through, until it failed towards the end with this message:

calibre, version 2.83.0
ERROR: Failed to repair library: Restoring database failed, click Show details to see details

Traceback (most recent call last):
File "site-packages\calibre\db\restore.py", line 127, in run
File "site-packages\calibre\db\restore.py", line 280, in replace_db
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: u'E:\\eBooks\\metadata.db'

This is the message it shows when I try to open it now:

calibre, version 2.83.0
Corrupted database: The library database at E:\eBooks appears to be corrupted. Do you want calibre to try and rebuild it automatically? The rebuild may not be completely successful.

Traceback (most recent call last):
File "site-packages\calibre\gui2\ui.py", line 648, in library_moved
File "site-packages\calibre\db\legacy.py", line 78, in __init__
File "site-packages\calibre\db\cache.py", line 350, in init
File "site-packages\calibre\db\backend.py", line 1239, in read_tables
File "site-packages\calibre\db\tables.py", line 110, in read
File "c:\cygwin64\home\kovid\sw\build\apsw-3.8.2-r1\src\cursor.c", line 231, in resetcursor
CorruptError: CorruptError: database disk image is malformed

I tried to run the repair function from the command line again, but it gives this error:

E:\eBooks\1 - Calibre Portable\Calibre>calibredb restore_database -r
Qt: Untested Windows version 10.0 detected!
Starting restoring preferences and column metadata ... 0%
Cannot restore preferences. Backup file not found. ... 100%
Restoring database failed with error:
Traceback (most recent call last):
File "site-packages\calibre\db\restore.py", line 127, in run
File "site-packages\calibre\db\restore.py", line 281, in replace_db
File "shutil.py", line 83, in copyfile
IOError: [Errno 2] No such file or directory: u'C:\\Users\\danie\\OneDrive\\Documents\\Calibre Library\\metadata.db'

Possibly as the library isn't in the default directory? (I can move it there if it will help)

I can delete the corrupted database file and restore it with the one from the other HDD again but I can only imagine the same thing will happen again so I'm not sure where to go from here.

[kovidgoyal]

Umm if you are getting repeated database corruption then you have bigger problems. Either the disk is failing or you have put the calibre lirbary on some kind of non-native filesystem, such as a network share, dropbox folder or the like.

Move your calibre library to an NTFS internal disk and then add your old books.

[Geta]

I did a clean install of Calibre on my laptop, copied the books over and the database file from my back up HDD and then added the other books.

All worked perfectly so I guess either the HDD is dodgy, or a little 2.5" portable HDD just can't run a library of 10k plus books...

Anyway, thanks for your help and for making such a great program.

[KrautHolg]

Originally Posted by kovidgoyal

Run the restore from the command prompt, with calibre closed, like this:


calibredb restore_database -r

I have several libraries, but only one of them I can't restore. How would I go about restoring only that particular one (D:\Calibre eBook Library 002) instead of the currently active one which doesn't have any problems?

I am afraid I might bollix things up some more if I simply move these libraries elsewhere for the time being...

BTW, thank you so much for everything!

[theducks]

Originally Posted by Geta

I did a clean install of Calibre on my laptop, copied the books over and the database file from my back up HDD and then added the other books.

All worked perfectly so I guess either the HDD is dodgy, or a little 2.5" portable HDD just can't run a library of 10k plus books...

Anyway, thanks for your help and for making such a great program.

USB HD are fine. If yours has a second USB plug, be sure it is connected as well.
You may need to use a POWERED USB Hub to get the extra ports AND enough power.
Be sure to exit Calibre, then wait for the activity LED on the drive to stop blinking (the buffer may be flushing) before shutting down.