Creating Catalogs: Start here

[GRiker]

Overview

Calibre's catalog feature generates a file summarizing selected books from your library in one of several formats: AZW3, BIB, CSV, EPUB, MOBI or XML. BIB is a bibliography format (BiBTeX) used by LaTeX; CSV and XML output is intended to be manipulated in another program; EPUB and MOBI are intended as output formats to be read on a reader device.

Getting started

• In Preferences|Conversion|Common Options|Page Setup, select an Output profile matching your device.
• Click Convert books↓Create catalog of books in your calibre library to open the Generate catalog dialog window.
• Catalog format specifies the output format for the generated catalog. If you are generating a catalog to be read on a device, select the appropriate output format. For example, Kindle users would select 'MOBI', iPad and Sony users would select 'EPUB'. If you want to print out a list of books with selected metadata fields, choose 'CSV', export the generated file to your desktop, then open it with a spreadsheet program.
• Catalog title will be the name of the generated catalog in your calibre library.
• The Send catalog to device automatically checkbox indicates that you want calibre to download the generated catalog (enabled for EPUB/MOBI only) to your device upon completion.
• The default settings will generate a useful catalog. If you want to customize the generated catalog, read on.

Controlling which books are exported to your catalog

Spoiler:

What you see is what you get:

• If you want all of your collection exported to your catalog (using the exclusions noted above), remove any search criteria by clearing the Search: field. With a single book selected (or no books selected), all books in your collection will be candidates for export to the generated catalog. Books matched with Exclusion criteria will not be added to the catalog.

• If you want only some of your collection exported to the generated catalog, you have two options:

  • Select the books you want exported. With more than one book selected, only the selected books will be exported.
  • Use the Search: field to filter the displayed books. With a single book selected (or no books selected) in the filtered list, only the displayed books will be exported.

Catalog options

Spoiler:

The E-book options tab provides customization control depending upon the selected catalog format:

• BibTex: Choose which database fields will be saved in the generated catalog file. Standard columns and custom columns you have added to your database are available for inclusion. After catalog generation, you will be prompted for a save location.
• CSV/XML: Choose which database fields will be saved in the generated catalog file. Standard columns and custom columns you have added to your database are available for inclusion. After catalog generation, you will be prompted for a save location.

• AZW3/EPUB/MOBI: The E-book options tab provides greater control over catalog content.
  
  
  
  Callouts
  1. Presets:
     • Load/save catalog settings, including Catalog title and Catalog format from the Catalog options tab
  2. Included sections:
     • Authors - all books, sorted by author, presented in a list
     • Titles - all books, sorted by title, presented in a list
     • Series - all series books, sorted by series, presented in a list
     • Genres - all enabled genres, with individual genres presented in a list sorted by Author and Series. The default source for the Genres section is the Tags column. Certain custom columns types may also be used as a source for Genres:
       • Comma separated text, like tags, shown in the tag browser
       • Text, but with a fixed set of permitted values
       • Text, column shown in the tag browser
     • Recently Added - all included books sorted in reverse chronological order by date added to your library over the last 30 days, plus a month-by-month listing of added books. Presented in a list.
     • Descriptions - detailed descriptions of each book, including a cover thumbnail and Comments. Description pages are sorted by author, with regular titles followed by series titles.
  3. Prefix rules specify matching criteria for books to be shown in the catalog with a special prefix.
     The checkbox in the first column enables the rule.
     Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be marked with the prefix from the Prefix column.
     Three prefix rules have been specified in the example above:
     • Rule #1, 'Read book', specifies that a book with any date in a custom column 'Last read' will be prefixed with a checkmark.
     • Rule #2, 'Wishlist item', specifies that any book with a 'Wishlist' tag will be prefixed with an X.
     • Rule #3, 'Library books', specifies that any book with a value of True (or Yes) in a custom column 'Available in Library' will be prefixed with a double arrow.
     The first matching rule supplies the prefix. Disabled or incomplete rules will be ignored.
     
  4. Excluded books. Exclusion rules specify criteria for books that will not be included in the generated catalog.
     The checkbox in the first column enables the rule.
     Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be excluded from the generated catalog.
     Two exclusion rules have been specified in the example above:
     • Rule #1, 'Catalogs' specifies that any book with a 'Catalog' tag will be excluded from the generated catalog.
     • Rule #2, 'Archived books', specifies that any book with a value of 'Archived' in the custom column 'Status' will be excluded from the generated catalog.
     All rules are evaluated when deciding to exclude books. Disabled or incomplete rules will be ignored.
     
  5. Excluded genres. A regex pattern describing tags to exclude as genres.
     Tags are used to create the genre sections in your catalog. You may be using certain tags that you don't want included as genres in the catalog. For example, you might use a tag [Amazon freebie], but you don't want that tag included as a genre in your catalog. The default exclusion pattern \[.+\]|\+ excludes any tags of the form [tag] as genres, as well as excluding +, the default tag for read books.
     Results of regex displays which tags will be excluded as genres when the catalog is built.
     
  6. Other options.
     • Author cross-references specifies whether to create additional book listings in the Authors section for books with multiple authors.
     • Catalog cover specifies whether to generate a new cover for the catalog, or use the existing one.
     • Extra Description note specifies a custom column's contents to be inserted into the Description header, opposite the cover. For example, you might want to display the date you last read a book using a Last Read custom column described earlier. For advanced use of the Description note feature, see this post.
     • Thumbnail width specifies a width preference for cover thumbnails included in the Descriptions section. The catalog generator caches these thumbnails for performance reasons; changing this value invalidates the cache. If you want to experiment with different widths, try building a catalog with just a few books until you've found your optimal width, then build your entire catalog. The first time a catalog is generated with a new thumbnail width will be slower, but subsequent builds will take advantage of the cache.
     • Merge with Comments specifies a custom column whose content will be non-destructively merged with the Comments field during catalog generation. For example, you might have a custom colum Author Bio that you'd like to append to the Comments section. You can choose to insert the custom column contents before or after the Comments section, and optionally separate the appended content with a horizontal rule. Eligible custom column types include text, comments, and composite.

Customizing the Descriptions template and Section lists

Spoiler:

If you wish to change the default appearance of the Description pages or Section lists, you can create a local copy of the template and CSS files which will override the default layout. This requires familiarity with HTML and CSS.

• Open your calibre configuration directory - Preferences|Advanced|Miscellaneous|Open calibre configuration directory.
• Within this directory, create a subdirectory resources (if it doesn't already exist).
• Within the newly created resources directory, create a subdirectory catalog.
• Open a new file system window, navigate to the default catalog resources folder:
  • OSX: Applications/calibre/Contents/Resources/resources/catalog
  • Windows:Program Files\Calibre2\resources\catalog
• Copy all of the files from the default catalog resources folder to the newly created resources/catalog directory within your calibre configuration directory.
• Edit template.xhtml and stylesheet.css to your preferences. If you want to start over, copy from the default template and CSS files again. If you want to discard your customizations, delete the catalog subfolder from your calibre configuration resources directory.

Customizing the Description page. Available fields in template.xhtml:

• {author} - Book author(s)
• {author_prefix} - Checkmark if read, 'X' if wishlist, else blank
• {comments} - Contents of the book's Comments field
• {formats} - List of installed formats
• {genres} - List of genres
• {note_content} - Note content as specified in options
• {note_source} - Note source field as specified in options
• {pubdate} - Month/Year published
• {publisher} - Publisher
• {pubmonth} - Month published
• {pubyear} - Year published
• {rating} - Graphical rating
• {series} - Series name
• {series_index} - Series index
• {thumb} - Cover thumb url
• {title} - Book title

Example: Changing year of publication in Description header

• Default: <td class="date">{pubyear}</td>
• Removed: <td class="date"></td>
• Augmented: <td class="date">Year of publication: {pubyear}</td>
• Modified: <td class="date">{pubmonth} {pubyear}</td>

Customizing the Section lists. Templates controlling the display of book titles in the various Section lists (Books by Author, Books by Title, etc) may be edited to taste.
Available fields in section_list_templates.py:

• {title} - Title of the book
• {series} - Series name
• {series_index} - Number of the book in the series
• {rating} - 0-5 stars
• {rating_parens} - (0-5 stars)
• {pubyear} - Year the book was published
• {pubyear_parens} - (Year the book was published)
• Any other column, custom or otherwise, defined for the book

Example: Changing Books by Author to remove year of publication:

• Default:
  
  

  by_authors_normal_title_template = '{title} {pubyear_parens}'
  by_authors_series_title_template = '[{series_index}] {title} {pubyear_parens}'

• Year of publication removed:
  
  

  by_authors_normal_title_template = '{title}'
  by_authors_series_title_template = '[{series_index}] {title}'

• Rating added:
  
  

  by_authors_normal_title_template = '{title} {rating}'
  by_authors_series_title_template = '[{series_index}] {title} {rating}'

Tips for experimenting with customization:

• Work with a small subset of your catalog, 5-10 books
• If you are experimenting with Section list templates, disable Descriptions in E-book options - catalog generation will be much faster.
• If you are experimenting with CSS, build an EPUB version of your catalog, explode it with the Tweak EPUB feature to find the class of the element you want to change.

Known Issues

Spoiler:

• Depending upon the size of your library, and the Sections you include when generating a catalog, it is possible to create a very large ebook which can overwhelm your ereader. If you observe sluggish performance or even crashes when browsing your generated catalog, consider reducing the size of the generated catalog by eliminating Sections or splitting up your library into smaller catalogs.
• Kindle 1 is not supported.
• Users have reported that the Kindle for PC app does not use the same font as the Kindle2 and KindleDX, so the checkmark for a read book does not display properly. The Kindle Previewer app does display correctly.
• MS Excel does not import some csv files correctly, it strips the leading quotes. Open the csv file in a text editor and copy/paste into MS Excel.

Creating catalogs from the command line

Spoiler:

All catalog generation functionality is available through calibre's CLI interface. You will need to install the command line tools: Preferences|Advanced|Miscellaneous|Install command line tools before running these commands.

• Type 'calibredb catalog --help' for general options
• Type 'calibredb catalog .[csv|epub|mobi|xml] --help' for options specific to a output format

Tagging Tips

Spoiler:

By following a few simple guidelines, your database will be easier to keep organized and the Catalog generation process will run more quickly.

• Use a simple, consistent scheme for tagging. It's better to have separate Fiction, Romance and Suspense tags than a single Fiction-Romance-Suspense tag.
• Use the Edit metadata dialog to add/edit tags. Use existing tags whenever possible.
• Learn to use the bulk metadata editing feature to change tags on groups of books at a time.
• You can easily preview how your genre groups will be assembled by using the Search feature. Type tag:Fiction in the Search: box to see all books tagged with Fiction. To list books without a particular tag, e.g. Fiction, type not tag:Fiction.
• Take a look at the log generated in the Jobs dialog. It will show you all of the tags in your database, plus those actually used in your generated catalog.

Tips for Kindle users

Spoiler:

The catalog builder has some special TOC optimizations for Kindle users.

• I can't see my catalog on my Kindle
  Because your catalog is formatted as a periodical, it will only be visible on the Home page if you are viewing 'Subscriptions' or 'All My Items'. If you 'Books' or 'Personal Docs' is displayed at the top of your Kindle's Home page, your catalog will not be visible on the Home page.
• To explore your catalog through the Kindle TOC:
  • From the Home screen, move the highlight to your catalog, then toggle right.
  • Move the highlight down to 'Go to Section...', then click to open the Sections List.
  • Highlighting and clicking on a Section name (Descriptions, Authors, Titles, Recently Added, Genres) will take you to that Section.
  • Highlighting and clicking on the number in parentheses (right of the Section) will take you to a high-level overview of that Section in your catalog. Try it.
  • If you end up somewhere you didn't expect to be, just press the Back button on your Kindle to return to the previously displayed page.

If You Have Problems

Spoiler:

• Review the Catalogs subforum for possible solutions
  If someone else has had a similar problem, you may find the solution in this subforum. If you don't find the answer you're looking for, please post your problem in this subforum.
• Update to most recent version of calibre
  Problems that have been reported are usually fixed in the next release of calibre. It's a good policy to stay current with the most recent release.
• I can't see my catalog on my Kindle
  Because your catalog is formatted as a periodical, it will only be visible on the Home page if you are viewing 'Subscriptions' or 'All My Items'. If you are showing 'Books' or 'Personal Docs' at the top of your Kindle's Home page, your catalog will not be visible on the Home page.
  
• No books found to catalog
  
  
  Books are shown in the main window, but they are all excluded. Check Tags to exclude and Column/Value in the Excluded books section of the E-book options dialog.
  
• Inconsistent Author Sort values
  
  
  Having a single author with different Author Sort values among different books will terminate catalog generation. To fix this, type author:<author name> in the Search box to isolate all books by the author. Select all of the displayed books, then click the Edit metadata icon, or type E. Click the Automatically set author sort checkbox, then the OK button.
  
  A quicker approach is to use the Manage authors dialog to reset all Author Sort values in the database at once. Open the Tag Browser, right-click on Authors, select Manage authors.
• Invalid cover file warning
  
  
  This warning occurs when a cover associated with a book cannot be read as an image file. The catalog is built with a default cover substituted for the problem cover.
  
• Tag problems. The quick way to confirm this is to change the regex expression in the EPUB/MOBI options tab. The default value ignores tags of the form [<tag>]. By changing the regex to . (a single period) all tags will be ignored as genres. Build the catalog again; if that works, then you need to so some housekeeping on your tags. Review Tagging Tips above.
• Creating a bug report
  • Create a debug log
    • Click the Jobs: icon in the lower right corner of calibre's main window
    • In the calibre - jobs dialog window, click on the most recent (closest to the top) Generate catalog job
    • Click the Show job details button
    • Select all of the text in the 'Generate catalog' window, and copy it to your clipboard
    • Save that text to DEBUG.TXT
  • Go to calibre's project management site to create a ticket.
  • Attach the DEBUG.TXT file to the ticket.

Creating a printed catalog

Spoiler:

To get a printed catalog of your books, you'll simply need to create a catalog in CSV format. This is a format recognized by spreadsheet applications, so you can easily open the catalog in one, format it, and print.
For example, if you open the catalog in OpenOffice.org Calc, you'll need to do the following:

• You'll first get a dialog asking you for the import settings. Select UTF-8 as character set and make sure that 'comma' is selected as a separator. There's a preview in the lower part of the window. If it looks good, click OK to confirm.
• You'll get an unformatted list of the books in your catalog. Format it to your liking using the programs' tools.
• Print.

Implementation history

Spoiler:

• 0.6.35: initial implementation supporting CSV, EPUB, XML for Kindle 2 and DX.
• 0.6.36: add support for MOBI
• 0.6.37: add 'Recently Added' section, title sorting option, improved debug diagnostics
• 0.6.38: 'Titles' and 'Recently Added' sections optional, support for formatting markup in descriptions (<br/>, <b>, <i>), display Series information
• 0.6.42: add 'Last 30 days' to Recently Added Section, change title display style in Titles Section when book is part of series, various bug fixes and improvements.
• 0.6.43: add 'Apply' button to Catalog dialog, applying changes to catalog build criteria without initiating build process. Introduce *experimental* feature 'Recently Read' (Kindle only). If Kindle is connected when initiating catalog build and 'Recently Added' is enabled, bookmarks are scanned and listed in a 'Recently Read' section, including a progress meter.
• 0.7.18: add 'Series' section, similar to Genres section. Many revisions to layout in EPUB/MOBI catalogs. Add 'ondevice' field in CSV/XML catalogs.
• 0.7.29: Added 'Wishlist' feature, thumbnail caching, revised description page layout.
• 0.7.32: Revised method for recognizing read books.
• 0.7.36: New UI for EPUB/MOBI E-book options, added support for custom columns, variable thumbnail width, customizable Description template, improved widow/orphan management.
• 0.7.37: Bug fixes for Windows users.
• 0.7.39: Author with differing author_sort values will cause catalog generation to terminate. Offending authors are written to the Job log.
• 0.7.41: Added Section list templates, enabling customization of title format. Added diagnostic warning and error dialogs for Author Sort mismatches, corrupt cover images. All Section lists are now optional.
• 0.8.64: Added Exclusion rules and Prefix rules, support for AZW3 output.
• 0.8.68: Added live regex preview, Help button, option to use existing cover.
• 0.9.6: Added Author cross-referencing option, relisting books with multiple authors as additional entries in Authors section.
• 0.9.8: Added custom column support for Genres section.
• 0.9.24 Added preset load/save support for GUI and CLI