kindlegen plugin for Sigil

[Doitsu]

[Plugin] KindleGen - Simple KindleGen wrapper

Updated: August 8, 2023
Current Version: "0.6.9"

This plugin is a very simple kindlegen wrapper for converting epubs to mobi files with kindlegen. (If you're a macOS user or using Sigil 0.9.8 or higher, please download KindleGenQt_v0.6.9.zip, otherwise download KindleGen_v0.5.5.zip.)

Disclaimer: Even though it's unlikely that the plugin will somehow corrupt the epub that you're working on or crash Sigil, you might want to save your ePub in Sigil before running this plugin.

Credits: This plugin uses code originally written by KevinH, nickredding and other KindleUnpack developers. It also uses jhowell's Calibre KFX Output plugin.

System requirements

This plugin requires the Kindlegen binary or Kindle Previewer, which comes with KindleGen. If you uncheck the Use Bundled Python option, you'll also need to install Python, bs4, lxml and Pillow.

Linux users will need to download the kindlegen binary from the Internet Archive and unzip it to a /bin folder.

Installation

1. Select Manage Plugins from the Plugins menu. Select Use Bundled Python to use the bundled interpreter.
If you haven't updated to Sigil 0.9.0 (or higher) click one of the Auto buttons to detect the path or Set to manually select the Python interpreter path. You also might need to install bs4, Pillow and lxml.
2. Click Add Plugin and select KindleGen_v0.6.8.zip. This will install the kindlegen plugin, which you can select via Plugins > Output > KindleGenQt.
If you want to generate KFX files, you'll also need to install Kindle Previewer and jhowell's Calibre KFX Output plugin.
Since the Calibre KFX Output plugin doesn't support Linux, you can't generate KFX files with the Linux version of Sigil.

When you run the plugin for the first time, you'll need to select the kindlegen location, unless you have installed Kindle Previewer in the default folder. Select either kindlegen.exe (Windows) or kindlegen (macOS, Linux).

Settings

By default, the plugin will generate .mobi files using the same parameters as Kindle Previewer (master mobi file, no compression, attached source files).

There are two groups of settings:

1. KindleGen settings that are passed on to the KindleGen binary and allow you to:

• Prevent KindleGen from attaching the source files.
• Change the compression factor (0 = no compression, 2 = maximum compression).
• Display verbose messages.
• Force Windows-1252 output.
• Convert .jpg images to .gif files.
• Change the language used to display KindleGen messages.

2. Postprocessing options that allow you to:

• Add a fake ASIN number (using DualMetaFix).
• Split the master mobi file into azw3 and mobi7 components (using KindleUnpack).
• Generate a KFX file with jhowell's Calibre KFX Output plugin (Windows und macOS only).
• Generate a thumbnail image for eink Kindles.
• Change the output folder (default = Desktop).

If you select any of the post-processing options, you can't upload the AZW3/MOBI7/KFX files generated by the plugin to KDP!

The plugin will also check for the presence of the following recommended guide/landmarks/metadata items:

• SRL = Start Reading Location*
• TOC = HTML TOC
• Cover = cover metadata entry

Advanced Settings

* According to the Kindle Publishing Guidelines, the SRL has been deprecated. (If you don't use the SRL, edit KindleGen.json and change "check_srl": true to "check_srl": false.)

License: GNU General Public License v3 (GPL-3)

[KevinH]

Hi Doitsu,

Already 19 downloads .... Great Work.

Thank you!

KevinH

[dhdurgee]

I am trying this out on my linux mint 17.1 rebecca x64 system, but unfortunately am getting errors:
================================================== ==
Status: success

Running KindleGen ... please wait

************************************************** ***********
Amazon kindlegen(Linux) V2.9 build 1028-0897292
A command line e-book compiler
Copyright Amazon.com and its Affiliates 2014
************************************************** ***********

Info:I9006ption: -c1: Standard DOC compression
Info:I9014ption: -verbose: Verbose output
Info(prcgen):I1047: Added metadata dc:Title ""Sweet Young Things""
Info(prcgen):I1047: Added metadata dcate "2015-11-03"
Info(prcgen):I1047: Added metadata dc:Creator "Kristine Kathryn Rusch"
Info(prcgen):I1047: Added metadata dc:Contributor "calibre (1.25.0) [http://calibre-ebook.com]"
Info(prcgen):I1047: Added metadata dc:Source "Free Fiction Monday 2014/11/17"
Info(prcgen):I1052: Kindle support cover images but does not support cover HTML. Hence using the cover image specified and suppressing cover HTML in content. URL: /OEBPS/Text/cover.xhtml
Info(prcgen):I1002: Parsing files 0000001
Info(prcgen):I1003: Parsing file URL: part0000_split_002.html
Info(cssparser):I10004: @rules other than @import, @charset and @font-face are not supported.
Info(prcgen):I1015: Building PRC file
Info(prcgen):I1006: Resolving hyperlinks
Info(prcgen):I1010: Writing hyperlinks
Info(prcgen):I1049: Building table of content URL: /OEBPS/toc.ncx
Error(prcgen):E24010: Hyperlink not resolved in toc (One possible reason can be that the link points to a tag with style display:none):/OEBPS/Text/titlepage.xhtml#
Error(prcgen):E24001: The table of content could not be built.
Info(prcgen):I1016: Building enhanced PRC file
Info(prcgen):I1007: Resolving mediaidlinks
Info(prcgen):I1011: Writing mediaidlinks
Info(prcgen):I1009: Resolving guide items
Info(prcgen):I1038: MOBI file could not be generated because of errors!

Kindlegen failed.

Please click OK to close the Plugin Runner window.

================================================== ==

What do I need to do to get this working?

Dave

[HarryT]

The error seems pretty clear; there's an invalid hyperlink in the TOC.

[dhdurgee]

Originally Posted by HarryT

The error seems pretty clear; there's an invalid hyperlink in the TOC.

I never created a TOC for this epub. It is a short-story and as such a TOC is unnecessary. All I really want in this epub is the cover and the text.

I ran the validator and get the following errors:

OEBPS/Text/titlepage.xhtml N/A This OPS document is reachable but not present in the OPF <spine>. "Reachable" means that a reference of some kind that points to this resource exists in the epub.

OEBPS/Text/titlepage.xhtml N/A This resource is reachable but not present in the OPF <manifest>. "Reachable" means that a reference of some kind that points to this resource exists in the epub.

OEBPS/toc.ncx 17 This <content> element's "src" attribute value is "Text/titlepage.xhtml", but that file does not exist.

I don't see a way to remove the TOC or to modify the OPF anywhere.

I was able to use ebook-convert from calibre to create a .mobi from this file, so I am surprised that kindlegen has problems with it.

Dave

[HarryT]

All those errors seem to be saying the same thing: that you're referencing a file called "titlepage.xhtml", but the file doesn't exist.

[eschwartz]

calibre is a better piece of software than kindlegen -- and you're surprised to hear that?

[dhdurgee]

Originally Posted by HarryT

All those errors seem to be saying the same thing: that you're referencing a file called "titlepage.xhtml", but the file doesn't exist.

Correct, a file that is not needed for a short story does not exist. How can I change whatever needs to be changed to remove these references to an unnecessary file?

Dave

[eschwartz]

Regenerate your Toc.ncx (semantic ToC) -- or manually edit it -- to get rid of the reference to that nonexistent file...

[dhdurgee]

Originally Posted by eschwartz

Regenerate you Toc.ncx (semantic ToC) -- or manually edit it -- to get rid of the reference to that nonexistent file...

Duh! Somehow I never expected it to be that easy. I was able to use kindlegen without problems after doing as you suggested.

The .mobi kindlegen made is substantially larger than the one ebook-convert made, over twice the size. Perhaps I need to tweak the kindlegen.ini settings to deal with this. I have both a K3 WiFi and a KT2, so I need a file that both can read, so I would appreciate some pointers.

Dave

[eschwartz]

kindlegen is creating a dual-MOBI.
I assume you mean it is bigger than a dual-MOBI that calibre creates. (Rather than being bigger than a standalone AZW3 which is just duh. )



Try using the undocumented

-dont_append_source

switch to kindlegen -- by default it creates a MOBI that has both mobi7 && KF8... and also the original source files, because reasons.
For kindlegen.ini that would be the "donotaddsource = False True" mentioned in the OP.

...

Also consider creating the standalone AZW3 and MOBI, and sideloading each one as and when needed.
Unless you intend to use Amazon PDOCs of course.

[dhdurgee]

Originally Posted by eschwartz

kindlegen is creating a dual-MOBI.
I assume you mean it is bigger than a dual-MOBI that calibre creates. (Rather than being bigger than a standalone AZW3 which is just duh. )



Try using the undocumented

-dont_append_source

switch to kindlegen -- by default it creates a MOBI that has both mobi7 && KF8... and also the original source files, because reasons.
For kindlegen.ini that would be the "donotaddsource = False True" mentioned in the OP.

...

Also consider creating the standalone AZW3 and MOBI, and sideloading each one as and when needed.
Unless you intend to use Amazon PDOCs of course.

I changed the donotaddsource and azw3_only settings and now I get a .mobi file smaller than that generated by ebook-convert and a .azw3 file that is larger. I will confirm that these look good on my kindles and then will start using the plugin.

On a related topic, is there a way to improve the file name created? I left the mobi_file setting blank, so it used the title as a starting point. As this was a short story, the title was in double-quotes which were translated to underscores. The embedded blanks in the title were kept in the file name. My preference in this matter is to elide the double-quotes and translate the embedded spaces to underscores in the file name. It would be nice if this could be automated by the plug-in. I also normally convert embedded apostrophes to hyphens in the file name. I imagine a more general translate table might be needed for foreign languages titles to file names.

At this point I will simply rename the files manually, but it would be nice to automate it.

Dave

[eschwartz]

I don't know, doesn't sound like it.


I usually use calibre's metadata template language for that. In calibre.

[Doitsu]

Originally Posted by dhdurgee

I was able to use ebook-convert from calibre to create a .mobi from this file, so I am surprised that kindlegen has problems with it.

Calibre has extremely robust conversion settings that allow users to convert invalid ePubs to invalid Mobi files. These files can be opened with Kindles and Kindle apps, but some links/styles might not work.
You may want to make sure that all books that you want to convert with KindleGen pass ePubCheck and/or FlightCrew, because books that fail validation usually also fail KindleGen. Also note that azw3 and mobi7 files generated by this plugin can't be used for KDP uploads.

If you want to convert multiple epubs you're better off using Calibre.

Originally Posted by dhdurgee

On a related topic, is there a way to improve the file name created? [...] My preference in this matter is to elide the double-quotes and translate the embedded spaces to underscores in the file name.

If you're familiar with regular expressions, you can change the generated file name by editing line #285 in plugin.py.

            title = re.sub('[/|\?|<|>|\\\\|:|\*|\||"|\^]+', '_', dc_title.group(1))

You could also press F8 in Sigil and modify the title before running the KindleGen plugin. (You don't have to save the ePub after the change; the plugin will always use the current version of the files as they're displayed in Sigil.)

[dhdurgee]

Originally Posted by Doitsu

Calibre has extremely robust conversion settings that allow users to convert invalid ePubs to invalid Mobi files. These files can be opened with Kindles and Kindle apps, but some links/styles might not work.
You may want to make sure that all books that you want to convert with KindleGen pass ePubCheck and/or FlightCrew, because books that fail validation usually also fail KindleGen. Also note that azw3 and mobi7 files generated by this plugin can't be used for KDP uploads.

If you want to convert multiple epubs you're better off using Calibre.



If you're familiar with regular expressions, you can change the generated file name by editing line #285 in plugin.py.

            title = re.sub('[/|\?|<|>|\\\\|:|\*|\||"|\^]+', '_', dc_title.group(1))

You could also press F8 in Sigil and modify the title before running the KindleGen plugin. (You don't have to save the ePub after the change; the plugin will always use the current version of the files as they're displayed in Sigil.)

Thank you for the pointer to the appropriate code. I have tweaked it a bit to suit my preferences as follows:;

            # replace unwanted characters in book title
            title = re.sub('[/|\?|<|>|\\\\|:|\*|\||"|\^|\'|\s]+', '_', dc_title.group(1))
            title = title.strip("_")

I edited the comment to be a little more accurate, added the apostrophe and white space to the list to be replaced with underscores and stripped leading/trailing underscores to deal with titles in quotes.

There might even be a few more unwanted characters that ought to be added to the replacement list that could cause problems in a file name. I will see if I run into any and add them as I encounter them.

The white space is a preference of mine to avoid the need of quoting filenames or using escape codes while entering file names. I find the underscores an acceptable replacement avoiding that problem.

Dave