NickelSeries: Integrated EPUB/KEPUB series metadata support

[geek1011]

Originally Posted by mbovenka

I could live with that I'm not using a custom column now (just a template putting everything from the title after the first colon in the subtitle), but I could easily do that.

@geek1011, please pretty please?

(Edit: now putting the same info in a custom column called (surprise) 'subtitle' and using the template '{#subtitle}' in the driver config.)

Can I see an example of the OPF meta tag for that? Based on skimming the Calibre source code, I think I'll need to parse calibre:user_metadata:#subtitle as JSON, then get the string #value#, but I want to confirm that.

I'll implement that and EPUB3 subtitle metadata afterwards.

@davidfor, do you know if there is a reason why Calibre doesn't seem to support standard EPUB3 metadata at all (even if it's in addition to it's own format)?

[DNSB]

Originally Posted by geek1011

Can I see an example of the OPF meta tag for that? Based on skimming the Calibre source code, I think I'll need to parse calibre:user_metadata:#subtitle as JSON, then get the string #value#, but I want to confirm that.

Here's some samples, the first two are from epub2 files, the next two are from an epub3. It could have been harder to read if someone had really tried.

Spoiler:

<meta content="{&quot;rec_index&quot;: 29, &quot;name&quot;: &quot;Subtitle&quot;, &quot;colnum&quot;: 8, &quot;is_multiple2&quot;: {}, &quot;kind&quot;: &quot;field&quot;, &quot;label&quot;: &quot;subtitle&quot;, &quot;column&quot;: &quot;value&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;search_terms&quot;: [&quot;#subtitle&quot;], &quot;is_category&quot;: true, &quot;is_editable&quot;: true, &quot;is_csp&quot;: false, &quot;is_multiple&quot;: null, &quot;display&quot;: {&quot;description&quot;: &quot;&quot;, &quot;use_decorations&quot;: 0}, &quot;#extra#&quot;: null, &quot;#value#&quot;: &quot;Under the Ocean for Sunken Treasure&quot;, &quot;table&quot;: &quot;custom_column_8&quot;, &quot;link_column&quot;: &quot;value&quot;, &quot;is_custom&quot;: true, &quot;datatype&quot;: &quot;text&quot;}" name="calibre:user_metadata:#subtitle" />

<meta name="calibre:user_metadata:#subtitle" content="{&quot;table&quot;: &quot;custom_column_8&quot;, &quot;display&quot;: {&quot;description&quot;: &quot;&quot;, &quot;use_decorations&quot;: 0}, &quot;label&quot;: &quot;subtitle&quot;, &quot;column&quot;: &quot;value&quot;, &quot;colnum&quot;: 8, &quot;#value#&quot;: &quot;A Sector General Omnibus&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;datatype&quot;: &quot;text&quot;, &quot;is_editable&quot;: true, &quot;rec_index&quot;: 29, &quot;search_terms&quot;: [&quot;#subtitle&quot;], &quot;is_multiple2&quot;: {}, &quot;link_column&quot;: &quot;value&quot;, &quot;#extra#&quot;: null, &quot;is_category&quot;: true, &quot;is_multiple&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;name&quot;: &quot;Subtitle&quot;, &quot;is_custom&quot;: true, &quot;is_csp&quot;: false}" />

  "#subtitle": {
    "#extra#": null, 
    "#value#": "An Oral History of the Zombie War", 
    "category_sort": "value", 
    "colnum": 10, 
    "column": "value", 
    "datatype": "text", 
    "display": {
      "description": "", 
      "use_decorations": 0
    }, 
    "is_category": true, 
    "is_csp": false, 
    "is_custom": true, 
    "is_editable": true, 
    "is_multiple": null, 
    "is_multiple2": {}, 
    "kind": "field", 
    "label": "subtitle", 
    "link_column": "value", 
    "name": "Subtitle", 
    "rec_index": 29, 
    "search_terms": [
      "#subtitle"
    ], 
    "table": "custom_column_10"
  }

    <dc:title id="subtitle">An Oral History of the Zombie War</dc:title>

[jackie_w]

Originally Posted by geek1011

Can I see an example of the OPF meta tag for that? Based on skimming the Calibre source code, I think I'll need to parse calibre:user_metadata:#subtitle as JSON, then get the string #value#, but I want to confirm that.

I'll implement that and EPUB3 subtitle metadata afterwards.

@davidfor, do you know if there is a reason why Calibre doesn't seem to support standard EPUB3 metadata at all (even if it's in addition to it's own format)?

I don't know if this will help or hinder, but anyway ...

• #value# may not always be a string. Looking at my own calibre custom columns I have examples of all of these - null, string, integer, boolean, list. This may not be exhaustive. Contents of #value# is going to depend on what type of custom column has been defined.
• There is no guarantee that the custom column(s) being used to derive Kobo subtitle is #subtitle (e.g. mine is #awards). But perhaps your're saying that, for simplicity, you only intend to support this simple scenario.

Which standard EPUB3 metadata is calibre not supporting?

[geek1011]

Thanks, @DNSB.

Originally Posted by jackie_w

#value# may not always be a string. Looking at my own calibre custom columns I have examples of all of these - null, string, integer, boolean, list. This may not be exhaustive. Contents of #value# is going to depend on what type of custom column has been defined.

That makes sense. I won't be supporting anything but a string, but I wanted to confirm it wasn't inside another object or something. I can check the type while parsing it.

There is no guarantee that the custom column(s) being used to derive Kobo subtitle is #subtitle (e.g. mine is #awards). But perhaps your're saying that, for simplicity, you only intend to support this simple scenario.

Yes, that's correct. I'm only going support a custom column named "#subtitle", as it appears to make the most sense to me out of all the ways of having non-standard Calibre metadata.

Which standard EPUB3 metadata is calibre not supporting?

• All EPUB3 metadata around meta[property][refines="#..."]. Series metadata isn't supported at all (it only supports calibre:series), you can't refine the role on a dc:author (it only supports EPUB2 opf:role), etc.
• I don't think it supports EPUB3 nav documents.
• I'm not 100% certian about this one, but I don't think it supports [properties="cover-image"] on manifest items.

[jackie_w]

Originally Posted by geek1011

• All EPUB3 metadata around meta[property][refines="#..."]. Series metadata isn't supported at all (it only supports calibre:series), you can't refine the role on a dc:author (it only supports EPUB2 opf:role), etc.
• I don't think it supports EPUB3 nav documents.
• I'm not 100% certian about this one, but I don't think it supports [properties="cover-image"] on manifest items.

Calibre handles Series & Series_index like this in EPUB3:

    <meta property="belongs-to-collection" id="id-3">Harry Bosch</meta>
    <meta refines="#id-3" property="collection-type">series</meta>
    <meta refines="#id-3" property="group-position">1</meta>

I suspect "#id-3" isn't necessarily the id that will always be used.

The equivalent for calibre EPUB2 is:

    <meta content="Harry Bosch" name="calibre:series"/>
    <meta content="1.0" name="calibre:series_index"/>

The NAV file and cover image file in my EPUB3 books looks like this:

<manifest>
    ...
    <item href="nav.xhtml" id="nav" media-type="application/xhtml+xml" properties="nav"/>
    <item href="images/cover.jpeg" id="cover" media-type="image/jpeg" properties="cover-image"/>
    ...
</manifest>

I suspect filenames nav.xhtml and cover.jpeg are calibre defaults rather than must-haves.

[NiLuJe]

There's a fair bit of ePub 3.something OPF features supported: https://github.com/kovidgoyal/calibr...tadata/opf3.py

I can definitely vouch for the Cover & Series stuff, since that was what I double-checked the last time I looked at this ;p.

[geek1011]

Thanks for the info, @jackie_w, @NiLuJe.

Originally Posted by NiLuJe

There's a fair bit of ePub 3.something OPF features supported: https://github.com/kovidgoyal/calibr...tadata/opf3.py

I can definitely vouch for the Cover & Series stuff, since that was what I double-checked the last time I looked at this ;p.

I seem to have been a bit behind on this (the last time I did anything major with Calibre was in 2016). See 7e1d8a9 and the Debian package I was using from stretch.

I guess most ebooks I've seen which were created/modified with Calibre were EPUB2 (either actually EPUB2 or a book which declared a EPUB3 books as EPUB2 for ... reasons), so it didn't write/read those.

I should probably take another look at Calibre sometime soon to have a better idea of its current state and to get familiar with how it currently works.

___

I'll have subtitle support (calibre #subtitle column, EPUB3 refined title) finished by next week at the latest (first, I'm going to finish my work on NickelHook).

[mbovenka]

Originally Posted by geek1011

I'll have subtitle support (calibre #subtitle column, EPUB3 refined title) finished by next week at the latest (first, I'm going to finish my work on NickelHook).

Much appreciated! Thanks.

[geek1011]

Originally Posted by mbovenka

Much appreciated! Thanks.

Does this work for you: KoboRoot.tgz?

[NiLuJe]

@geek1011: Ah, the joys of Debian.... .

(Jokes aside, Calibre is notoriously awfully handled by downstream distributions. To make a long-story short, if you're not running Arch, use Kovid's binary package unless you know what you're doing).

[davidfor]

Originally Posted by geek1011

I'll have subtitle support (calibre #subtitle column, EPUB3 refined title) finished by next week at the latest (first, I'm going to finish my work on NickelHook).

In calibre the column should be of type, "Text, column shown in the Tab browser". In the OPF, this will have:

- "datatype": "text"
- "is_multiple": null

And "#value#" will either be a string, or "null".

I don't think any other column type makes sense.

Except...

Maybe the tags. I think some people are putting the tags in the subtitle. That makes them visible and searchable on the device. That would mean reading all the dc:subject entries in the OPF.

[mbovenka]

Originally Posted by geek1011

Does this work for you: KoboRoot.tgz?

Them weeks sure are gettin' short

Works like a charm! Thanks a lot.

For the record, my setup is as follows:

My Calibre 'title' column is formatted as <maintitle>:<subtitle>

I use a plugboard to fill the Kobo Title with '{title:list_item(0,: )}'
I have a custom column '#subtitle' which is 'built from other columns' with '{title:list_item(1,: )}' as a template.
I use '{#subtitle}' as a template to fill the Kobo Subtitle in the extended driver.

NickelSeries does the rest

[ps67]

Originally Posted by geek1011

Does this work for you: KoboRoot.tgz?

It works (I tried a Kepub), I see the subtitle after sending the book without needing of connecting again the kobo to calibre.

You are a genius! Thank You.

[geek1011]

Originally Posted by mbovenka

Them weeks sure are gettin' short

I did mean it when I said a week before it's finished and released . You just happen to be testing a bunch of other in-progress work at the same time too (NickelHook), although I do expect that NickelHook is stable and essentially complete at this point.

Works like a charm! Thanks a lot.

Originally Posted by ps67

You are a genius! Thank You.

No problem.

[konstantinus]

This is great news! Surely many have been waiting for this functionality for a long time. But for some reason it doesn't work for me: the metadata does not appear, although SeriesMeta works fine with the same books.