The Kobo Utilities Plugin adds extra function to calibre for working with Kobo e-ink eReaders. This will allow changes on the ereaders that either cannot be done or will make them easier to do. The current functions of the plugin are:
The plugin supports the WiFi, Touch, Glo, Mini, Aura HD, Aura and Aura H20. When a WiFi is being used, the options are limited options it supports. The plugin will probably work with the original Kobo eReader, but this has not been tested.
Known problems:
Almost all function of this plugin directly update the database on the Kobo device. If anything goes wrong, the database could get corrupted. It shouldn't happen, but it could. Before using any function of the plugin, it is recommended that you backup the database.
The plugin has been created by reverse engineering the function of the Kobo eReaders. This has been done by examining the database on the Touch and Glo and experimenting with it and the interface. There is a good chance that my conclusions from this are wrong. This is unlikely to damage the device, but it could result in the need to reset it. Again, if you cannot accept this, please do not try this plugin.
A large chunk of the code is borrowed from the plugins written by kiwidude. A lot of the rest came from looking at other plugins and the general calibre code.
A lot of people on MobileRead have helped. Some by beta testing new versions, some with ideas for new functions and some by answering questions from other users.
And of course a big thank you to Kovid Goyal for calibre.
The plugin is installed through the calibre preferences dialog. During the installation, you will be prompted on which toolbar or menu to put the plugin button. As there is no available function when no devices are connected, put it on the "Main toolbar when a device is connected" or "The menubar when a device is connected". After the plugin is installed, restart calibre before attempting to use it.
To use the plugin, it needs to be configured. This can be done through the calibre preferences, or from the "Customise plugin" option on the plugins menu. There are two areas that need to be configured: the columns to be used by the plugin and the action to be taken when the plugin button is pressed.
The configuration dialog has three tabs:
The "Profiles" tab is used to create one or more profiles for different Kobo devices. Each profile has a set of custom columns used for storing the reading positions of books. If you have different devices use by different people, this allows them to store their reading status for each book in the same calibre library. The custom columns needed are described in "Columns for the Plugin".
The "Device" tab has the list of know Kobo devices and the configuration needed for them. The options are for backing up the database and checking for firmware updates. These can be configured separately for each device or a common configuration used.
The known devices will be available to create a profile for on the "Profiles" tab.
The "Other" tab has options that are common for the plugin for all devices and libraries. This includes setting the function of the toolbar button, keyboard shortcuts and viewing the preferences as stored.
The plugin can use several columns to store the current reading position of ePubs on the eReader. If you do not want to use this part of the plugin, they do not need to be configured.
The columns and types needed are:
To add a custom column to calibre:
Once you have created the columns and restarted calibre, you can set the columns to be used by the plugin. To do this:
The following section details each function available and how to use them. Some functions are only available for the library list, some only on the device list and some work for both. Most functions act on the books that are selected in the current list.
For most of the functions available, the way to use them are:
When the reader settings are changed in the device, they are stored for that book and for any future book opened. This allows the user to change these setting for one or more books. And, for some of the settings, it allows finer grade options than the device settings dialog.
This option works the same for both the library list and the device list.
The options on the dialog are:
Once the settings are how you want them, press the OK button to save the settings and update the database on the device.
The new home screen on the Aura HD and Glo uses as set of tiles to display recent activities. These can be pressed to do the same activity or dismissed. The tiles displayed change over time as different as the device is used.
This function is available for both the library and device lists.
When "Dismiss Tiles from Home Screen" is selected, a dialog is displayed. This has:
When you have selected the options, press the OK button. This updates the device and displays a message when finished.
Notes:
The Glo HD does not have as many Extras as the other devices. And Kobo had also removed Chess from the devices. But, if there is a tile on the home screen for the Extras, they can be used. This function can add tiles for the Extras to the home screen.
This function is available for both the library and device lists.
When "Display extras tiles" is selected, a dialog is displayed. This has:
When you have selected the options, press the OK button. This updates the device and displays a message when finished.
Notes:
The reader settings are stored in a database for each book. This option removes the settings for all books selected. It will prompt you to continue by pressing "Yes" or stop by pressing "No".
This function is available for both the library and device lists. It will work for all book types that store the settings.
The metadata displayed on the device is read from sideloaded books the first time the device sees them. If the book and metadata is changed and sideloaded again, the metadata is not reread. There is also some standard metadata that is not read from ePubs.
For KePubs synced from the Kobo server, the metadata is populated from the server. It might get changed at later syncs.
This option only works from the library list.
The options on the dialog are:
This allows the reading status of books in the device that are not in the calibre library to be changed. An intended use is to reset an accidentally opened book to unread. It can also be used to mark books as finished after a factory reset or other reason for reloading books.
The function on dialog is shared with the "Reading Status" section of the "Update metadata in device library" function.
Note: When the OK button is pressed, the changes are made on the device. But, the reading status options are not save. The next time the dialogs is opened, all options will be reset. This is deliberate. If this function is used, the decision of what changes to make must be made each time.
This allows you to change the series information for books already on the device. It allows you to set the series name and number for books that you do not have in your calibre. This can change the series information for KePubs.
The interface and most of the function has been taken from the "Manage Series" plugin written by kiwidude. If you are familiar with that plugin, this works in a very similar way, but is restricted to books on the device. If you want to manage series for books in your calibre library, please use the "Manage Series" plugin. The Kobo Touch driver can update the device with this information.
If you find a problem with the "Manage Series" function of the Kobo Utilities plugin, report it to the developer of the Kobo Utilities plugin. DO NOT report it as a bug in the original "Manage Series" plugin.
To manage series on the device, from the device list:
This allows the current reading position in an ePub to be stored in the calibre library. This can then be sent back to the device or another Kobo device to restore the reading position. As well, the rating set on the device can also be stored and restored.
The device store the reading position in the database on the device. This is stored in several parts:
These can all be retrieved from the device and stored in custom columns in the calibre library.
This is only available in the library list.
To use this function, you need to:
The options on the dialog are:
The "Show Reading Position Changes" dialog shows the list of books whose reading position has changed. Each book is listed with the percent read and last reading date that is on the device and in the calibre library.
The changes can be accepted or rejected. Each book has is ticked. If a book is not selected, the reading position will not be updated in the library when the OK button is pressed. To reject all changes, press the "Cancel" button.
The changes are shown compared to the values in the library for the profile used.
The dialog is shown when the storing the reading status is run as background job. This could be when started manually or when it is run automatically when the device is connected.
When new books are added to Kobo devices, an import process is run. Details from the new books are extracted from the book and added to the database. The database is then used when the book is displayed in the library on the device.
One of the things extracted from the book during the import is the Table of Contents (ToC). This is added to the database and is used when the book is being read. The ToC displayed when reading is from the database, not the book. As well, any navigation done using the ToC or the next/previous buttons is based on the database version of the ToC. If the book is replaced with a new copy using calibre, it can be replaced without the import being run again. In this case, the ToC in the database will be different to the ToC in the book when it is read. But, the ToC in the database will still be used. This can cause problems when attempting to navigate in the book, or even when opening the book.
This function of the plugin can be used to update the ToC in the database when books have been changed. It only needs to be used then the ToC changes. This means either the ToC entries have changed, or the internal file structure has changed. If there are other changes, such as to the stylesheet, or the text of the book, this funcition is not needed.
This funtction is used from the calibre library when the device is connected. To use it:
With this, you can change the cover on the device to the current cover in the calibre library.
To use it, select the books that you want to change the cover for, and choose the desired options. Then press the OK button to upload the covers.
The options on the dialog are:
This will remove saved covers for the selected books from the device. This will allow the device to regenerate covers for these books when they are next needed.
To use it, select the books that you want to change the cover for, and choose the desired options. Then press the OK button to remove the covers.
The options on the dialog are:
This function is available for both the library and device lists. It will work for all book types that store the settings.
This looks at the images directory on the device and looks for any cover images that are not for books currently in the device database. The cover images can be removed or simply listed.
To use it, select the menu option and choose the desired options. Then press the OK button to check and clean the images directory. The clean is run as a background job.
The options on the dialog are:
When the job is finished, how many extra cover image files were found and deleted is displayed. Pressing the "Show Details" button will list the extra cover image files that were found.
Using the KoboTouch driver, shelves can be created for each book series on the device. The books are then added to the shelf. But, there is no way to sort these shelves into the series order. This option allows uses the "Date Added" sort to show the books in the series order.
To use it, select the menu option. A dialog with the list of series shelves will be displayed. The shelves listed are those that have the same name as for series that are on the device. The number of books on each shelf will be displayed.
Select the options and press the OK button to order the shelves. The options on the dialog are:
When the shelves are ordered, a message box will be displayed with the number of shelves whose order was set. If the shelf only had one book on it, the shelf order will not be changed.
To see the books in the series order, select "Date Added" when viewing the shelf on the device.
A few notes about the ordering:
This function is available for both the library and device lists.
As part of the details of each book, there is a "Related book" section. Kobo populates this for books synced from the Kobo server with a list of books that are in some way related to the current book. The related book could be from the same series, the same author or in a similar genre. Usually, it can be seen why the book is there, but occasionally it is not obvious.
To see the related books, go to the library list on the device and long press on a book to display the menu. On the menu, press "Details". The details of the book will be displayed. At the bottom is a section that can be changed. Select "Related books" in this section to see the books.
This also works for sideloaded books. But, the Kobo server does not populate this for sideloaded books. This function can. It allows you to set the related books as either all the books in the same series, or all the books by the same author. This is an alternative to using a series or author shelf as a way of grouping books.
To set the related books for sideloaded books:
The related books will be set for all the names in the list. If there was only one book with the name, nothing will be done. This will also replace all current related books. This can take some time and a message will be displayed when finished.
If the "Delete All" button is pressed on the options dialog, all current related book entries for sideloaded books will be removed from the device database.
The function uses the device database for determining the related books. This means it can operate on all sideloaded books on the device, not just those in the current calibre library. It also means that the series name or author needs to be correct and consistent.
This will retrieve any annotations for the selected books and display them in a dialog. The annotations can be copied and save elsewhere. The annotations are retrieve using the same code as the "Fetch annotations".
Annotations for ePubs are stored on the device in two places: the internal database and files. The files have type "ANNOT" and are stored within the directory Digital Editions\Annotations. These files are contain XML that define the annotations. They can be copied to another Kobo device, or with a little fiddling, used with ADE.
Selecting this option will prompt for a directory to copy the files to. Either enter the directory name, or press the "..." button to select a directory. When the directory has been entered, press the OK button to copy the files.
Annotations on the device are stored in two places. For all books, they are stored in the device database. For ePubs and PDFs that are in the main memory of the device, the annotations are also stored in a file. This file is created by the Adobe RMSDK. This is used by the firmware for reading ePubs and PDFs.
There are two problems related to the annotations files.
Because of the above problems, it may be a good idea to remove the annotations files if you are reading ePubs in the main memory.
As the annotations files are created by the RMSDK, they are in the directory, "Digital Editions/Annotations". Within this directory, there is one file for each ePub or PDF that has been opened. The path and name of the annotations file is the same as for the book file with ".annot" added as an extensions.
To use this function:
When the processing has been finished, a message will be displayed indicating how many files where removed.
There are several circumstances where calibre will think a book is on the device, but it is not in the device database. This usually means the book has been sent to the device by calibre, but the book has not been processed yet.
When selected, a list of books is displayed in a separate window. This is simply a list of the books and has no function other than to display the books.
Note: This does not show any books that are on the device and calibre does not know about and are not in the device database. If books have been sideloaded in some other way and are not in the device database, they will not be displayed.
This function is only available when viewing the device list.
When this is used, the books on the device will be reread. This is done in the same way as when the device is first connected. The main use for this is it will cause any changes for shelves an series information to be sent to the device.
Due to some errors with syncing to the Kobo server, the shelves occasionally appear to be duplicated. There are several things that can happen, but in one case old shelves that have been deleted are recreated.
To use it, select the menu option. A list of all shelves with the following information is displayed:
To fix the duplicates, choose the options and press the OK button. The options are:
When the OK button is pressed, the duplicate shelves will be marked for deletion. The version of each shelve that is kept will be the one with either the newest or oldest date, depending on the options. After this is done, you should disconnect the device and sync to the Kobo server. This should remove all the duplicates from the server and should solve the problem.
If the Purge duplicate shelves option is selected, the duplicates will be deleted from the device database. If this is done and a sync to the Kobo server is then done, it is possible that the server will send all the shelves to the device again. But, if you don't sync the shelves in the future, this probably isn't important.
This function is available for both the library and device lists.
The Kobo devices collect information about actions taken on the device. This is stored in a database table called "AnalyticsEvents". There is evidence that this information is sent to Kobo during a sync. It is unknown how Kobo store or use this.
To use it, select the menu option and choose the desired options. Then press the OK button. The entries already in the AnalyticsEvents table will be deleted and a database trigger installed to prevent further entries from being created.
The options on the dialog are:
This function is available for both the library and device lists.
Unfortunately, there are circumstances where the database on the device can become corrupted. When this happens, the database usually has to be replaced. This means loss of annotations and reading status for any sideloaded books. With this option, the status of the database can be checked.
One of the signs of a corrupt database is an error from calibre when the device is connected. This error is "DatabaseError: database disk image is malformed". When using the device, the symptoms can include things like the loss of reading position or the same books being processed multiple times.
Selecting this option will run PRAGMA integrity_check against the database and display the output in a window. If there are no problems with the database, "ok" will be displayed in the window. If the database is corrupt, the errors found in the database will be displayed.
If there is an error in the database, there is generally not much that can be done to fix it. If the problem is only with the indexes, then compressing the database might fix it. But, most errors will only be fixed restoring a database backup, or signing out of the Kobo account on the device and signing back in. If this doesn't work, a factory reset might be needed.
As books are added or removed from the device, their details are added or removed from the device database. This causes the size of the database file to increase. This option will compress the database file to clear unused space.
Selecting this option will run VACUUM against the database. When finished the uncompressed and compressed sizes of the database are displayed. If there is an error, the error message will be displayed.
It is recommended that you backup the device database before compressing it.
This will copy the device database to somewhere on your computer. This will allow it to be restored if the database is ever corrupted.
When started, you will be prompted for a directory where to backup the database and a name for the backup. Once these are selected, pressing "Save" will copy the database from the device.
If you would like to run the backup when the device is connected to calibre, see "Automatic Device Backup".
The plugin can automate some actions when the device is connected. These will be run when calibre first sees the device and has run the initial device jobs to get the list of books from the device.
The following sections describe the actions that can be automated.
A backup of the device database can be run when the device is connected. This can be done either the first time the device is connected each day, or every time it is connected.
When the database backup is run, some configuration files are also backed up.
Configuring the automatic backup is done on the plugin configuration. To do this:
When the device is connected, a background job is started in calibre to do the backup. The first step of this job is to determine if a backup is needed. If it is not, then nothing is done. If a backup is needed, the following steps are done:
The backup files are named to show when the backup was done and which device. The format of the name is:
KoboReader-<device model>-<device serial number>-<timestamp>.[zip|sqlite]
Where:
This displays the configuration dialog. The details are in "Configuration" and the following sections.
This shows the usual about dialog for the plugin.
If you problems, you can find me on the MobileRead forums. The best way to get support is by posting a message in the thread for this plugin. It is Kobo Utilities. I monitor the thread and as long as I am not away or sick or something, I will see it within a few hours. And if I am not around, someone else is likely to try and help. And it is likely that there is already discussion for any common problems.
You can also send a PM (private message) to me. My profile is davidfor. I will respond to PMs, but my response will be no faster than if you post in the thread.
When posting about a problem, please give as much information as possible. In general, there is no such thing as to much information when trying to debug remotely. And if you think "he won't need that", it is almost guaranteed to the most important bit.
When a problem is reported, I will respond as quickly as I can. But, remember, it will take time to read the report, think about what could be going on and then write a response. And I am probably in a different timezone to you. Plus, this is done in my spare time. I work, have a family and like to use my Kobo devices for their intended purpose, reading. If I know I am going to be away for a few days, I will try to post in the thread or inform someone of what is happening.
After hearing of a problem, I might ask you to run calibre in debugging mode and send me the output produced. If I do, these are the steps: