Guide to Collections Management with calibre (Touch / Glo / Mini /Aura)

[Mrs_Often]

Thanks for your replies David!

Originally Posted by davidfor

That is expected. The way the Kobo readers work is to process a book the first time it is seen. It adds the book to its database. When doing this it extracts the title, authors, description, some other metadata and the TOC. But, if you replace the book later, it doesn't reprocess it for updates. This is good for performance, but annoying for this sort of thing.

That would mean that it is better to delete the book and process content first, before adding the book to the device again, right? Well, I learned that the hard way anyway.

I'm glad you like it. I nearly didn't add the option. In fact, I think one of the beta testers advised against it. But, when I looked at the code, it was a simple change and an obvious option. But, personally, I don't use any of the cover options. I'll rely on you to tell me if it is broken

Oh my!! Am I glad that you did add it anyway!

I haven't checked if the image size have changed with the new firmware. That might explain this. Also, I don't have a Glo, so I don't know if they use different sizes.

I don't have a Glo either (yet... hmmm), and not on 2.1.4 either (I'm back to 2.0.0 after yesterday! Agh! I miss 2.1.1). Funnily enough, the three books that still had margins on either side of their thumbnail covers fixed themselves at a certain point. I don't know what I changed or what I did, but first and second time I looked at them they were ugly, after that they were perfect... I'll make sure to check what happens and exactly what I do next time I put some books on.

[Mrs_Often]

I'm not sure if it's related to calibre and the lovely new kobo touch driver, but I'm having database issues now. I've done nothing except use calibre to load books, sync touch over wifi and switch kobo off and on. (The off and on switching was after the issues started, otherwise I wouldn't have switched off and on.) So there might be a link between the new driver and my problems. I posted about it here:

https://www.mobileread.com/forums/sho...4789&page=4#57

[davidfor]

Originally Posted by Mrs_Often

That would mean that it is better to delete the book and process content first, before adding the book to the device again, right? Well, I learned that the hard way anyway.

Generally yes. But, if you delete using calibre and send a new copy over it works OK.

I don't have a Glo either (yet... hmmm), and not on 2.1.4 either (I'm back to 2.0.0 after yesterday! Agh! I miss 2.1.1). Funnily enough, the three books that still had margins on either side of their thumbnail covers fixed themselves at a certain point. I don't know what I changed or what I did, but first and second time I looked at them they were ugly, after that they were perfect... I'll make sure to check what happens and exactly what I do next time I put some books on.

If you detect a pattern tell me and I'll see if I can handle it in the driver.

[Mrs_Often]

Originally Posted by davidfor

If you detect a pattern tell me and I'll see if I can handle it in the driver.

Thanks David. I checked a few books and all image sizes seem to be perfect now. I'll keep an eye out with other books too. See my next post in a few minutes.

[Mrs_Often]

I just want to officially state that there is absolutely no link between my problems and the Kobo Touch/Glo/Mini driver for Calibre.

The driver works perfectly and was not the cause of my problems.

After I got my Touch working (7 hours total now), I retraced my steps that I thought caused the problems last night, but nothing happened. All is well.

I did find one oddity though. I'll include all my steps from after I got the Touch fixed in the spoiler.

Spoiler:

NB. 1. "Disconnecting" always means ejecting from calibre, then unplugging from computer; "Kobo" or "device" means "Kobo Touch" with 2.0.0 firmware; 3. I'm using a MacBook Air running OS 10.7.5 with Calibre 0.9.2

- Connected Kobo Touch
- Deleted all sideloaded books that calibre could see from calibre.
- Disconnected, processed content.
- On Kobo, deleted all sideloaded books calibre didn't see (weird)
- Synced Kobo via Wifi
- Attached Kobo to computer
- Sent one book from calibre to kobo
- Disconnected and processed content
- Book was fine
- Reconneced to calibre
- Sent 7 books from one series to the kobo
- Disconnected, processed content
- Books were fine and included the series info in the titles as per my settings in the calibre kobo driver
- Reconnected to calibre
- Calibre did not recognise that the 7 books were the same as the books I had just sent

Elaboration: the books did appear in the device library with the titles including the series and series number, but calibre didn't show the "on device" check mark next to those books in the calibre library. Odd, because the same formatting was used on the first book that was sent (also a book from a series, though a different series), and that does show a check mark for "on device" in the calibre library. The first book does not show the series information in its title in the device library of calibre.

- Disconnected without doing anything
- Books still on device.

- Synced Kobo via wifi.
- Reconnected to calibre
- Calibre suddenly does recognise the 7 series books properly

Elaboration: the 7 books show a check mark in "on device" within the Calibre library.

- In the Device library in calibre, the titles of the 7 series books changed to exclude the series letters and number.

- Disconnected without changing anything
- Checked library on the Kobo: series info is present in titles.

To retrace my steps from last night when things went wrong:
- Reconnected to calibre
- Deleted the 7 series books from calibre
- Deleted all the Kobo welcome and guide books from calibre
- Disconnected and processed content
- Did a wifi sync from the Kobo
- Reconnected to calibre
- Everything looked fine
- Checked Kobo disk with Disk utility
- No problems at all
- Everything perfect.

Update: there is one difference between the above success and last night's catastrophe: I'm using firmware 2.0.0 now, while last night I was on 2.1.1. So 2.0.0 works fine with the driver, 2.1.1 I can't say anything about.

[davidfor]

Originally Posted by Mrs_Often

I just want to officially state that there is absolutely no link between my problems and the Kobo Touch/Glo/Mini driver for Calibre.

The driver works perfectly and was not the cause of my problems.

After I got my Touch working (7 hours total now), I retraced my steps that I thought caused the problems last night, but nothing happened. All is well.

Glad to hear you have it fixed. I'm happy it wasn't the driver, but it would be good to know what it was.

- Books were fine and included the series info in the titles as per my settings in the calibre kobo driver
- Reconnected to calibre
- Calibre did not recognise that the 7 books were the same as the books I had just sent

Elaboration: the books did appear in the device library with the titles including the series and series number, but calibre didn't show the "on device" check mark next to those books in the calibre library. Odd, because the same formatting was used on the first book that was sent (also a book from a series, though a different series), and that does show a check mark for "on device" in the calibre library. The first book does not show the series information in its title in the device library of calibre.

- Disconnected without doing anything
- Books still on device.

- Synced Kobo via wifi.
- Reconnected to calibre
- Calibre suddenly does recognise the 7 series books properly

Elaboration: the 7 books show a check mark in "on device" within the Calibre library.

- In the Device library in calibre, the titles of the 7 series books changed to exclude the series letters and number.

- Disconnected without changing anything
- Checked library on the Kobo: series info is present in titles.

I get this occasionally as well. The matching is done in a couple of different ways. And it is most likely to fail for titles with series info added. But, then the next time you connect it works OK. Very annoying to try and debug.

Update: there is one difference between the above success and last night's catastrophe: I'm using firmware 2.0.0 now, while last night I was on 2.1.1. So 2.0.0 works fine with the driver, 2.1.1 I can't say anything about.

It should be fine with 2.1.1. The database is the same, and nearly every action on it is done by the firmware. Deletes from calibre are safe. The only other updates are for shelves. The real work, adding a book to the database, is done by the device when it is disconnected. And there is no way I would attempt to do this from calibre. The only thing I can think of going wrong is disconnecting the device before the database is saved. If you do that the database could be corrupt, but the disk will probably need to be fixed as well.

[Mrs_Often]

Originally Posted by davidfor

Glad to hear you have it fixed. I'm happy it wasn't the driver, but it would be good to know what it was.

Tanks! Me too... yes. I'm still quite confused at what happened.

I get this occasionally as well. The matching is done in a couple of different ways. And it is most likely to fail for titles with series info added. But, then the next time you connect it works OK. Very annoying to try and debug.

I can imagine! Luckily it's nothing of consequence though, so oh well.

It should be fine with 2.1.1. The database is the same, and nearly every action on it is done by the firmware. Deletes from calibre are safe. The only other updates are for shelves. The real work, adding a book to the database, is done by the device when it is disconnected. And there is no way I would attempt to do this from calibre. The only thing I can think of going wrong is disconnecting the device before the database is saved. If you do that the database could be corrupt, but the disk will probably need to be fixed as well.

That's reassuring thank you. I don't think I disconnected before the database was saved. I never pulled out the usb cable before safely ejecting, and my Mac doesn't let me eject if the disk is still in use. It doesn't even attempt to disconnect, just says "Nope, it's busy, leave it alone." But of course, you never know. Something must have gone wrong somewhere. The only other thing I can think of is that my Touch might not be suitable for 2.1.1 yet after all and I do need the 2.0.4 first, but by some magic I got 2.1.1 working anyway, with a messed up database. Might not have noticed the mess up until I started changing things. I was happily using 2.1.1 for a while before I started changing things, and that's exactly when things went wrong.

[John F]

I'm thinking about trying this. I used to use Calibre on my Sony 505 with collections and tags and got the results I wanted. If I'm using Calibre can I ...

Say I have books Book_A, Book_B, Book_C. I want a collection (Collection_A) to have books in order Book_A, Book_B, Book_C. I want another collection (Collection_B) to have books in order Book_C, Book_B.

And I also want to be able to order by (original) book title order. I want only one copy of a book on the device.

Can I do this with Calibre and the procedures outlined so far?

The reasons I ask:

1) I want to be able to do something similar with real books.
2) I'm having a conversation with a fellow forum member who says it is possible (I think) and I have my doubts from what I've read
3) I tried editing the database to fiddle with the date added/creation date and was unable to get what I wanted

Thank you for the help.

[PeterT]

I thnk the answer is no... Books can be on multiple shelves, but the ordering within a shelf is by Title / Author / File Size / File Type / Recent Reads / Date Added

And I think that the date added is the date added to the Kobo, not to the shelf.

[davidfor]

Originally Posted by John F

3) I tried editing the database to fiddle with the date added/creation date and was unable to get what I wanted

As Peter said, there are a set of sorts for the shelves. None match exactly what you want. But, the "Date added" is on the column "DateModified" in the table "ShelfContent". Changing that might get what you want.

[John F]

Originally Posted by davidfor

As Peter said, there are a set of sorts for the shelves. None match exactly what you want. But, the "Date added" is on the column "DateModified" in the table "ShelfContent". Changing that might get what you want.

Thanks, I thought I tried that (along with the date fields in "Contents", and the file timestamps). I'll try it again.

[davidfor]

Originally Posted by John F

Thanks, I thought I tried that (along with the date fields in "Contents", and the file timestamps). I'll try it again.

I suppose I should have thought of this earlier, but if you are using calibre, you could use "series" as a shelf column and send the books one at a time in reverse order. The driver does set this column in the when putting the book on the shelf. So, if there is second difference, it will probably do what you want.

It might also work if you display the whole series, sort in reverse order, select them all and send them to the device together. I don't know if calibre will process them in the selected order or some other. If it sends them in the selected order, the timestamps will increase with each, but probably won't be different by a second. But, because of the insertion sequence into the table it might give the desired order in the list. It will depend on exactly the query the device uses to build the shelf list.

[kiltedtwizzles]

Thank you so much for this! I didn't know about the metadata management options to make this work.

A silly question from a novice though: do I need to re-send to device to get the shelves to go on there? And will this overwrite the copy on my Kobo Touch or will it delete the one one there & add a new one? I only ask because I'd like to preserve my read/unread/currently x% through counts.

Thanks again!

[Urdu]

Thank you David for all the work you put into updating the Kobo driver. It's really a great experience to create shelves and covers with such ease.

[davidfor]

Originally Posted by kiltedtwizzles

Thank you so much for this! I didn't know about the metadata management options to make this work.

A silly question from a novice though: do I need to re-send to device to get the shelves to go on there? And will this overwrite the copy on my Kobo Touch or will it delete the one one there & add a new one? I only ask because I'd like to preserve my read/unread/currently x% through counts.

Not a silly question at all.

If you have the metadata management set to auto, the shelves are maintained when you connect. They are also updated when you send a book. When you send the book it updates the shelves for all books in the library that are on the device.

If you resend the book, it just replaces the file on the device. The reading state doesn't get changed. There are some problems with this. Different firmware versions react differently to replacing the file. If you simple copy the same file over unchanged, it should be OK. But, if there are changes, different things happen. FW 1.9.17 didn't care and took the changed book OK. FW 2.0.0 tended to delete any changed book. 2.0.3 and later generally accept the changes.

There can be problems after the changed book is on the device. If the changes were simple, say spelling or style changes, it should read OK. If there were changes to the file structure in the book, it will generally open but navigation via the TOC might not work. I have had a few occasions when opening the book cause a restart. In each case there had been file structure changes the place the current bookmark pointed to didn't exist anymore.

One last thing, if you change the title or authors in calibre, or the save template, sending the book again will put it in a different place or with a different name. The device will see this as a different book when it loads it.