Database Rebuild... lost books

[madelonw]

I got the error asking if I wanted to rebuild my Calibre database... I said Yes.

I had 13,000+ books cataloged. Now I have 810. I am using version 2.0 (I know there is an update, just haven't gotten around to installing it.) I am running 64-bit Calibra on a Windows 8.1 laptop. My Calibre folder lives on an external hard drive. I would add that I have added several custom columns to Calibre, all of which are still there.

I still seem to have all the folders in my Calibre folder, and I see the following files:

.driveinfo.calibre
.metadata.calibre
metadata.db
metadata_db_prefs_backup.json (which appears to be editable with a text editor)
metadata_pre_restore.db

Short of starting all over with a totally new database and adding all the books from my Calibre folder back in, is there a way to get back to where I was.

[eschwartz]

calibre should not delete anything. Are the folders still there? Try using Check Library to locate the non-indexed book folders. You can then move them and add them manually.

One possible problem is that calibre will only rebuild one book per id. Was there an error message when you rebuilt the database? If so, the log might prove enlightening.

[theducks]

Originally Posted by madelonw

I got the error asking if I wanted to rebuild my Calibre database... I said Yes.

I had 13,000+ books cataloged. Now I have 810. I am using version 2.0 (I know there is an update, just haven't gotten around to installing it.) I am running 64-bit Calibra on a Windows 8.1 laptop. My Calibre folder lives on an external hard drive. I would add that I have added several custom columns to Calibre, all of which are still there.

I still seem to have all the folders in my Calibre folder, and I see the following files:

.driveinfo.calibre
.metadata.calibre
metadata.db
metadata_db_prefs_backup.json (which appears to be editable with a text editor)
metadata_pre_restore.db

Short of starting all over with a totally new database and adding all the books from my Calibre folder back in, is there a way to get back to where I was.

The RED should not be there, those are device only files. Someone has been messing beneath the skirts and probably done additional harm.

The green is normal (and is the heart of the Library GUI)
Author folders with titles (index_number) inside those

The last is probably from a restore attempt (not usually leftover)

[BetterRed]

Originally Posted by theducks

The RED should not be there, those are device only files. Someone has been messing beneath the skirts and probably done additional harm.

The green is normal (and is the heart of the Library GUI)
Author folders with titles (index_number) inside those

The last is probably from a restore attempt (not usually leftover)

Maybe someone accidentally connected to the Library folder in the Connect/Share->connect to folder. I've no idea what would happen if one did that - I shall go away and do it, after I do a backup

BR

[eschwartz]

On its own it does nothing but index the books there. The danger is connecting to the folder and maybe deleting stuff. Or sending stuff and overwriting files or clogging the library with junk.

[BetterRed]

Originally Posted by BetterRed

Maybe someone accidentally connected to the Library folder in the Connect/Share->connect to folder. I've no idea what would happen if one did that - I shall go away and do it, after I do a backup BR

Well if you do that then you'll end up with a .driveinfo.calibre and a .metadata.calibre library in your library folder. See first attachment.

Then if you send some books to that folder you'll end up with a bit of shambles see second attachment. The backup is on the left, the operational library is on the right

So I'm postulating that the OP made a genuine usage mistake rather than breaking the 'golden rule'.

@madelonw - does the above make any sense, might you have inadvertently selected your library folder in a Connect/Share->connect to library. I used a Test library, the 9 books I sent to the Connected folder disappeared from the Library, as can be seen in the second attachment there are less author folders on the right than there are on the left.

IMO you need to build a new library by importing books from the 'broken' one - but I'm unsure of the best way to do that such that your metadata is retained. If the library copies have current metadata embedded - eg via Polish, Modify or Embed then its easy to set up the Add options to extract metadata from the books, if not then ideally you would want to get the metadata from the opf files, I don't know how to do that.

Oh, to retain your custom columns in the new library, you should be able to created the new library based on the existing one.

Aside : maybe there should be a warning - "Do you really want to connect to this library folder" ie a folder with a metadata.db.

BR

[eschwartz]

Originally Posted by BetterRed

IMO you need to build a new library by importing books from the 'broken' one - but I'm unsure of the best way to do that such that your metadata is retained. If the library copies have current metadata embedded - eg via Polish, Modify or Embed then its easy to set up the Add options to extract metadata from the books, if not then ideally you would want to get the metadata from the opf files, I don't know how to do that.

Oh, to retain your custom columns in the new library, you should be able to created the new library based on the existing one.

It should be enough to use Library Maintenance. Check library will allow you to clean up any extraneous files (anything the database doesn't say is there).

Unless files were deleted or overwritten, in which case they don't exist to be imported into the new library.

[eschwartz]

Instead of using Check library to list and optionally delete files not in the library, to move/reimport/discard them, one could create a new library (check the box to use the current library structure, if you wish to preserve layout/custom columns), and use the second option in the Add books dropdown, to import books from a directory, including subdirectories, assue each directory is a separate book where each file is is the same book in a different format.

It will import metadata from the opf, and the cover.jpeg -- I am informed (today) that apparently it will not import the cover unless there is an opf. But that shouldn't matter here.

[BetterRed]

@e..w - answered my query

[eschwartz]

Excellent point. See my cross-post. (cross-posting = fun )

Just giving general theories here (and it's difficult on my Kindle Touch to keep track of more than one or two posts tiny screen).

[BetterRed]

Originally Posted by eschwartz

Excellent point. See my cross-post. (cross-posting = fun )

Just giving general theories here (and it's difficult on my Kindle Touch to keep track of more than one or two posts tiny screen).

I have the same problem on a 24" 1920x1200 :lol:

BR

[madelonw]

Originally Posted by theducks

The RED should not be there, those are device only files. Someone has been messing beneath the skirts and probably done additional harm.

The green is normal (and is the heart of the Library GUI)
Author folders with titles (index_number) inside those

The last is probably from a restore attempt (not usually leftover)

Thanks so much for the info. I have my original database back again... I took out the red, took out the new metadata.db, renamed the pre_backup to metadata.db and all looks fine now.