Guide to Collections Management with calibre (Touch / Glo / Mini /Aura)

[PeterT]

PLEASE NOTE: I AM LEAVING THIS THREAD HERE FOR DISCUSSION AND HISTORY.

ALL FUTURE UPDATES WILL BE MADE ON THE MOBILEREAD WIKI Kobo_Shelves_and_Collections

ADDITIONALLY PLEASE NOTE THAT YOU THE READER CAN UPDATE THE WIKI ENTRY AS WELL

** NOTE: THIS ALSO APPLIES TO USE OF THE KOBO TOUCH EXTENDED DRIVER ***

ALSO SEE THE END OF THIS POST FOR TIPS ON REMOVING DUPLICATED SHELVES




With the new Kobo driver introduced with calibre 0.9.1, there has been some confusion over how to automate the management of shelves.

The steps I use are the following

Driver / calibre Configuration
This column is just what I made. If you have an existing column (either custom or builtin that YOU want to use, there is NO need to make one.

1. Added a custom column to calibre
   This column is just what I made. If you have an existing column (either custom or builtin that YOU want to use, there is NO need to make one.
   • Preferences | Change Calibre Behaviour | Add your own columns
   • Select Add custom column
   • Select Tags in Quick create
   • I used Lookup Name myshelves and Column heading My Shelves
     Do make sure this column is of type Comma separated text, like tags, shown in the tag browser.
2. Configured either the Kobo Touch or the Kobo Touch Extended driver depending on which you are using.
   • Preferences | Advanced | Plugins
   • Expand Device Interface plugins section and look for KoboTouch or Kobo Touch Extended driver
   • With KoboTouch or Kobo Touch Extended highlighted, select Customize plugin
   • In the area for shelf info, type #myshelves, series.
     (Any time a custom column is used within calibre, it HAS to be referenced with a # symbol in front of it.)
     (Please note that #myshelves is just what I used. As I said at the start "These are the steps I used". Nothing prevents you from using an existing column in YOUR calibre setup.)
   • Select any other relevant options. It is suggested that at a minimum you select the Create Bookshelves and Delete Empty Bookshelves options. The other options are not shelf specific.

Now, when you edit the metadata for a book, look for the Custom Metadata area (where it is found depends on the view you are using). In the field My Shelves type in whatever shelves you want the books to be placed.

If you do as I did and also add series to the driver options, a shelf will be created automatically for each book series, and the books added to that shelf.

Configuring calibre's Metadata Management options

calibre has three options as to when it updates Metadata on the device. This also affects when the shelves are managed on the Kobo.

This option is managed as follows:

• Preferences | Import/Export | Sending books to devices
• The Metadata management selection has three options
  • Manual management: Calibre updates the metadata and adds collections only when a book is sent. With this option, calibre will never remove a collection.
  • Only on send: Calibre updates metadata and adds/removes collections for a book only when it is sent to the device.
  • Automatic management: Calibre automatically keeps metadata on the device in sync with the calibre library, on every connect
• As can be seen from the descriptions, for full functionality of the Kobo driver, set the Metadata management to either Only on send or Automatic management. From my experience, I have seen the easiest option to be Automatic management.
  
  If the option is set to Manual management, shelves will NEVER be removed.

Changing Book Titles to include Series Info

One probably also wants to change book titles to include Series info. That gets done via what is called a "metadata plugboard". Through these you are able to modify items like the book's title or author as it gets transferred to your device.

Goto Preferences | Import/Export | Metaboard plugboards

It looks complex, but is not.

On the line that reads Add new plugboard do the following

• In the box Format (choose first) select epub
• in the Device (choose second) box select KOBOTOUCH
• In the Source Template the magic happens . Here you have a lot of options
  • To modify the book's title to <Series> - <Number> - <Title> type

    {series}{series_index:0>2s| - | - }{title}

  • To modify the book's title to <Number> - <Title> type

    {series_index:0>2s|| - }{title}

  • To modify the book's title to <First Letter of Each word in Series> - <Number> - <Title> type

    {series:re(([^\s])[^\s]+(\s|$),\1)}{series_index:0>2s| - | - }{title}

• Finally in Destination Field select title

Once all that is done, take a DEEP breath and hit the Save Plugboard button and then the Apply checkmark at the top of the screen.

You will find a lot more discussion of these over in the calibre Library Management forum with more examples in this thread.

For those people who don't care about syncing of shelves between their device and the cloud (and in turn with other devices), the ability exists to override shelf syncing.

Originally Posted by davidfor

While we are on the topic of shelves and syncing, there is a way to prevent shelves from syncing. Add the following line into your "Kobo eReader.conf" file in the "ApplicationPreferences":

SyncShelves=false

To enable shelf syncing again, either remove the line or change the value to "true".

I'm not sure which firmware level this was added, but it is present in 2.8.1 at least.

CLEANING UP DUPLICATE SHELVES

Occasionally, due to issues with the Kobo infrastructure, a sync with Kobo will have the unfortunate side-effect of restoring ALL previously deleted shelves (including the possibility of myriads of duplicated shelf names).

These are a PAIN (if not impossible) to remove from the device itself, but luckily CAN be handled via calibre. Remember, this ONLY effects the shelves on the device; the definitions within calibre stay intact, and shelves WILL be correctly re-added and populated by calibre when the device is next attached.

1. calibre has available for it a plugin called Kobo Utilities.
   
   With this plugin installed, and your Kobo attached, launch the plugin. (This will be dependent on where YOU selected to have the icon for the plugin added when you installed it).
   
   From the plugins menu, select Database and then Fix Duplicate Shelves. The plugin will then display a list of shelves and how many occurences of each shelf are found. Select which shelves to delete, and then indicate whether to keep the oldest or newest shelf of each duplicated one. (I *think* that normally, you would want to keep the newest instance).
   
   Next, decide whether or not to Purge the shelves from the database. If you decide to Purge, then the records are totally dropped from the Kobo; however, they will remain on the Kobo Server, and, unless the second bullet is done, will reappear. If you decide NOT to purge, then the next sync of your device to Kobo will also remove the duplicate records from their infrastructure.
   
   Finally, press the OK button and watch the magic happen.
2. The option also exists to ensure that neither shelves nor their conrents (only Kobo format books) EVER sync to the Kobo servers.
   
   To do this, with your Kobo attached to your computer, browse through the directories on it and find the .kobo\Kobo directory. (Note that under Linux and OS/X environments, the .kobo directory will be hidden due to the leading period). Within this directory will be a couple of files; the one you are concerned with is the Kobo eReader.conf file. This file contains various options that are used to control the device.
   
   Open this file in a text editor (on Windows for instance, use Notepad, NOT Word or Write), and look for a line [ApplicationPreferences]. After this line, add a new line SyncShelves=false. Save the file, and disconnect the Kobo as usual.
   
   The addition of this line will ensure that shelves do NOT get synced to the Kobo Cloud.

[voidoid]

Thankyou PeterT. Your explanation has enabled me to finally get this working properly. Going to save lots of time. Thanks again.

[Mrs_Often]

Thanks Peter! I would have had a hard time figuring out all that on my own. I started adding series first letters and numbers to the titles manually aaages ago and gave up because it's just a hassle. This is exactly what I need!

[rainio]

Thanks for this.
I can change a books shelf and send it to main memory where it will appear in that shelf, but when sending to Card A the shelf isn't recorded.
Is this a known problem?
Thanks.

[davidfor]

Originally Posted by rainio

Thanks for this.
I can change a books shelf and send it to main memory where it will appear in that shelf, but when sending to Card A the shelf isn't recorded.
Is this a known problem?

Just did a check and it is working here. I tried a few different combinations, and it worked each time.

Just to check what you have done:

- Configure shelves as above
- Set the shelf for a book
- Connected the device
- Sent the book to the main memory
- Ejected the device and the book was on the shelf
- Deleted the book
- Connected the device
- Sent the book to the SD card
- Ejected the device and the book was not on the shelf

Is that what you did? Or something different?

[rainio]

Thanks, davidor, for taking the time to look into it.
I tried it again, and the selheves worked when using calibre on the original computer I transferred the first shelves to it a few days ago, but this morning when I tried copying 100 books on various shelves on different computer the shelves didn't work.
So it's something to do with using two separate calibres to transfer the books.
Should this be a problem?

Thanks.

[vice-versa]

It worked for me

[buffaloseven]

The instructions worked great. Love being able to change book titles to <Series> - <##> - <Title> as they're transferred to the device; makes it significantly easier to browse on the device. Regarding shelves, Calibre seems to be creating shelves with no problem, but won't delete empty ones. Has anyone else had this problem?

[vice-versa]

Originally Posted by buffaloseven

Regarding shelves, Calibre seems to be creating shelves with no problem, but won't delete empty ones. Has anyone else had this problem?

same problem for me, but as I'm to make a factory reset anyway... I'll start with a brand new database and will put my books back "cleanly" on my SD card.

[buffaloseven]

Originally Posted by vice-versa

same problem for me, but as I'm to make a factory reset anyway... I'll start with a brand new database and will put my books back "cleanly" on my SD card.

Let me know how it goes; I'd be interested to see if it fixes it or if there's just a small bug in the (new) driver.

[vice-versa]

Originally Posted by buffaloseven

Let me know how it goes; I'd be interested to see if it fixes it or if there's just a small bug in the (new) driver.

after a factory reset and a brand new database, the shelves without a book in it aren't deleted... I had to remove them manually

[rainio]

Thanks.
A quick update: the reason the shelves weren't updated on a different computer was that in the devicereferences the shelves name #bookshelves was different between computers.

[davidfor]

Originally Posted by vice-versa

after a factory reset and a brand new database, the shelves without a book in it aren't deleted... I had to remove them manually

I have to ask: You have checked the "Delete Empty Bookshelves" option in the driver configuration?

As I can't break this here, I need a little debugging help. Could you:

- Using your reader, create a shelf but don't put a book on it.
- Start calibre and then restart in debugging mode.
- Connect your reader and let calibre sync.
- Close calibre.
- When it closes, a log file will be displayed. Send that to me in a PM or put it somewhere I can download.
- Disconnect your reader and check if the shelf is there.

To restart calibre in debugging mode, click on the arrow next to the preferences button and select "Restart in debug mode".

[vice-versa]

Originally Posted by davidfor

I have to ask: You have checked the "Delete Empty Bookshelves" option in the driver configuration?

Yes, I did it

Originally Posted by davidfor

As I can't break this here, I need a little debugging help. Could you:

- Using your reader, create a shelf but don't put a book on it. done
- Start calibre and then restart in debugging mode. Tried all buttons to look for this feature, found it (I'd rather not play with buttons for wich I don't know the future behaviour when hit )
- Connect your reader and let calibre sync. Can't find a button for that
- Close calibre. Done
- When it closes, a log file will be displayed. Send that to me in a PM or put it somewhere I can download. (PM sended or nearly)
- Disconnect your reader and check if the shelf is there. It is still there but as I couldn't figure how to sync the device... I think it's normal

To restart calibre in debugging mode, click on the arrow next to the preferences button and select "Restart in debug mode". Ooops I should not have run the process step by step

[davidfor]

Originally Posted by vice-versa

Thanks, but, the log wasn't complete. Or at least it didn't contain anywhere near as many lines of output as I expected.

I'm hoping the problem is the "let calibre sync" bit. There isn't a button, it happens automatically. When calibre detects a USB device it checks if it is a reader it knows about. If it is, calibre does the sync. The sync involves running two background jobs and then calibre updating the library list and menubar. From the log it looks like you stopped calibre at the moment the first background job started.

So, can you try again? This time watch the bottom right corner were it says "Jobs". The number after it will go up to 2 when calibre starts syncing the device. When it has finished, it drops back to 0. Then close calibre and see if there is more in the debug log.