Fixed layout ePub converted from AZW3 displays black pages

[tomsem]

I am trying to convert a manga AZW3 to ePub. The conversion is 'successful', and calibre viewer displays something (but with issues), but Apple Books and Thorium Reader only show black pages.

It seems perfectly okay when converted to KEPub for Clara Colour, but sending it 'as is' to my PB InkPad Color 3, it includes a black rectangle with the image overlaid on top of it (resulting in 2 pages) and I don't know what accounts for that (if I convert to PDF with calibre this black rectangle shows in the same way). It also has black rectangles in calibre viewer (resulting in 2 pages).

(I edited the AZW3 to add ToC entries before conversion.)

I have tried both calibre direct conversion and also KindleUnpack's, with the same symptoms.

I am able to produce a 'working' ePub using Kindle Comic Converter with extracted images, but page order is all messed up because the image numbering in the AZW3 does not follow the page sequence.

The HTML in the AZW3 is different than that from the other manga AZW3's that convert well: the latter have 2 top-level <div> tags, each of which contains <img> tag for the page image in each. The AZW3 that converts to 'black rectangle' ePub has only the first of these <div> sections.

I have 2 ideas for workaround:

1. Rename image files using a script to read the manifest in metadata.opf to determine order and generate ePub with KCC. (but need to match spine id to an html file and read image filename from the <img> tag there, or just trust that html filenames sort into the correct order)

2. convert to KePub and back to ePub. <-did not work

[kovidgoyal]

Either the inmages are in a format apple books and thorium dont like or there is some css they dont like. do an image conversion to png for one of the internal images and see if it works and delete the css stylesheet and see if it works.

[tomsem]

Originally Posted by kovidgoyal

Either the inmages are in a format apple books and thorium dont like or there is some css they dont like. do an image conversion to png for one of the internal images and see if it works and delete the css stylesheet and see if it works.

Yes, those are just worst case, where black is 'foreground'. Calibre viewer and PocketBook also display black 'background' rectangles and I cannot account for where those come from. The CSS seems pretty basic. The images themselves are not the issue.

I think it is some styling in the HTML files, but there are 200 of them. So it looks like it is either going to involve updating all of the HTML files, or renaming the images so they are in the proper sort order, and then building an ePub from those.

I'll try to create a minimal example that demonstrates the issue.

[DNSB]

Last time I edited an ebook with over 100 stylesheets, they were identical and short. All the real styling was done using a mix of styles declared at the start of each text file and inline styles.

Here's what there were 173 stylesheets worth of:

@page {
  padding: 0pt;
  margin: 0pt;
}

body {
  text-align: justify;
  padding: 0pt;
  margin: 0pt;
}

[tomsem]

Attached is a two page book (AZW3 & ePub converted by calibre) that demonstrates the issue.

I created new AZW3 with calibre Edit, copied the (only) CSS file from the AZW3 for the book I've been trying to convert, and took one of the HTML pages, updated 'page' and pointed it at the 'cover' and 'page' image files PS beta generated for me. The image dimensions are exactly the same as those in the original AZW3.

[kovidgoyal]

Its the CSS your best bet is to extract the actual images files into a CBZ and convert that. Fixed layout is not actually supported in calibre.

[DNSB]

For what it may be worth, I took your .azw3 and unpacked it to epub with the KindleUnpack plugin. I then looked at your cover and page1 files and modified the code to display them full page with an svg wrapper. This may be of some help to you if you don't want to use a CBZ file. Basically, the only change needed between the files was the name of image file to display as long as all the image files have the same dimension (1279x1920 pixels for your test images.) so I used a regex search/replace to do the dirty work.

The '<meta name="primary-writing-mode" content="horizontal-rl"/>' was preserved so the epub reads from back to front. The 'rightspread" and "leftspread" were not preserved so a single image per page.

[tomsem]

Originally Posted by kovidgoyal

Its the CSS your best bet is to extract the actual images files into a CBZ and convert that. Fixed layout is not actually supported in calibre.

I understand calibre doesn't support fixed layout, but often it 'just works', or KindleUnpack can create one that does. Or Kindle Comic Creator can create one.

The problem is that the image filenames are numbered, but the sequence of those numbers is not in page order sequence (and not even close to being so). So it is not just a matter of extracting them and creating a CBZ; I have to figure out how to re-number them, and I don't want to do it manually.

[tomsem]

Originally Posted by DNSB

For what it may be worth, I took your .azw3 and unpacked it to epub with the KindleUnpack plugin. I then looked at your cover and page1 files and modified the code to display them full page with an svg wrapper. This may be of some help to you if you don't want to use a CBZ file. Basically, the only change needed between the files was the name of image file to display as long as all the image files have the same dimension (1279x1920 pixels for your test images.) so I used a regex search/replace to do the dirty work.

The '<meta name="primary-writing-mode" content="horizontal-rl"/>' was preserved so the epub reads from back to front. The 'rightspread" and "leftspread" were not preserved so a single image per page.

Thanks, I will have a look at this!

[tomsem]

I managed to write a script to make a copy of the image files with the proper sequential order, from which I was able to create ePub & CBZ with KCC, and KFX (with added ToC, facing pages, and panel view) with Kindle Create and KFX Output plugin.

Direct conversation to KFX didn't produce a KFX that supported facing pages in landscape. Maybe @jhowell can add that for fixed layout ePub conversion; I'll ask about this on the KFX output plugin forum.

[tomsem]

Originally Posted by tomsem

I managed to write a script to make a copy of the image files with the proper sequential order, from which I was able to create ePub & CBZ with KCC, and KFX (with added ToC, facing pages, and panel view) with Kindle Create and KFX Output plugin.

Direct conversation to KFX didn't produce a KFX that supported facing pages in landscape. Maybe @jhowell can add that for fixed layout ePub conversion; I'll ask about this on the KFX output plugin forum.

Ironically I think the cleanest workflow for pure image content will start with KFX, when available. Then use From KFX to generate CBZ, import those images to Kindle Create, add ToC, add panel view, make sure there are no facing pages issues, export to KPF, convert back to KFX and then From KFX to ePub, CBZ, or PDF. Save the original KFX in ZIP just in case.

Kindle Previewer (as used by KFX plugins) seems to preserve everything in the KPF, so one winds up with more functional KFX than Amazon delivers (typically there is no real ToC, existence of panel view is not a given, page flip does not always include multi-thumbnail view, etc). And the ePubs it creates seem to do well in the wild.

At least in my experiments so far, there is no particular advantage to CBZ, except that my PocketBook insists on a 1/8" margin for ePub. It's nice to have a ToC as well. The CBZ's created with From KFX don't have proper page turn direction, but I assume it's easy to fix that. Clara Colour displays KePub edge to edge. But neither seem to support facing pages in landscape mode.

Most of the comics/manga I've imported to calibre started as AZW3, but I may need to go back and check to see just what they and the converted ePub look like, and for many of those I did not bother creating ToC.