KoboTouch driver & SeriesID

[ownedbycats]

On Kobo devices, the SeriesID field in the content table determines which books appears in each series. KoboTouch driver never seems to update the SeriesID once it's set, both sideloads and purchased books.

Sometimes series can get split apart due to differing SeriesIDs:

• Some publishers split up series into multiple smaller series for no good reason. If you purchase a lot of Pocket Books and load them from the store, you'll likely end up with half a dozen Star Trek series in your series tab.
  
• Sideloaded books have a SeriesID set to the series name, but Kobo-loaded books have a GUID. This results in series with mixed sideloads and Kobo-loaded getting split into two separate series.

As per this thread, changing the SeriesID to all the same value (SQLite query) merges them together. I've not noticed any detrimental effects when tampering it manually, but sometimes after a sync it gets reset and splits apart again. Plus you have to do it every time you add a new book to the series.

So, my thought: Could the KoboTouch driver be modified to set the SeriesID to the series name during automatic metadata management?

[kovidgoyal]

I am not familiar with the kobo driver, so I could be wrong, but a quick inspection of the code indicates it does update the series, see line 3339 in kobo/driver.py and line 2650

[ownedbycats]

3339:

            if series_changed or series_number_changed \
               or not kobo_series_id == series_id \
               or not kobo_series_number_float == newmi.series_index:
                update_values.append(series_id)
                set_clause.append('SeriesID')
                update_values.append(newmi.series_index)
                set_clause.append('SeriesNumberFloat')
                if show_debug:
                    debug_print(f"KoboTouch:set_core_metadata Setting SeriesID - new_series='{new_series}', series_id='{series_id}'")

The only time I ever see it update though is when the SeriesID for a newly-imported sideload is empty - upon a second connect to Calibre, it then populates with the series name.

[DNSB]

Rather odd. I just checked a series where I picked up the first book as a freebie, cleaned it up and edited it and then sent it to my Kobos. I liked the book so I picked up the next 3 books in the series from Kobo Plus. All four books show up with the same series name and the correct series number.

I checked all my Kobo and Kobo Plus books and none of them showed what looked like a GUID in the Series information.

[ownedbycats]

Originally Posted by DNSB

I liked the book so I picked up the next 3 books in the series from Kobo Plus. All four books show up with the same series name and the correct series number.

I'm specifically referring to the SeriesID field in the content table. Try sending some of those books to the archive, then sideloading them. You'll notice the Series page now shows the series split apart into two.

The only way to merge them together (without sideloading everything) is directly edit the database to set the SeriesID field of those books so they're all the same. KoboTouch doesn't change it once it's set, nor does Kobo Utilities. (And yes, I have it set to update purchased book.)

[DNSB]

Sorry but I edit my Kobo purchases so they are sideloaded with cleaned up metadata. I don't edit Kobo Plus books due my quasi-religious opinions about them not being my property. Though as near as I can tell, when I archive a book synced from Kobo, I no longer see it on my Kobo ereader and a sideloaded edited version is treated as a new book. I did some playing with this and I can change the series name and number in calibre and it will update the information for a purchased book if I have Update purchased books enabled and I have matched books between device and calibre.

Edit: I went back and looked at when I changed the series name for Mercedes Lackey's Valdemar series books to Valdemar : Subseries_name. Once I made those changes, I used KoboUtilies update metadata tool to update the metadata on the device and the series names changed (along with a couple of numbers that were incorrect to start with).

[ownedbycats]

SeriesID doesn't affect that, it's behind-the-scenes and affects how the books group together on the Series page. Is possible to have books with same Series but different SeriesIDs. They'll show up as separate on the Kobo.

EDIT: Two recent acquisitions give an opportunity to demonstrate the accursed Pocket Books issue - 40 books I already set to the same SeriesID with the SQLite query, so they're a single series.

But the two other books, which I haven't merged in yet, have it set to different values (GUIDs provided by the Kobo store). So they show up as two separate series of 1 book each, even with the same name.

[Terisa de morgan]

Ok, now I get Series Id. I've got a specific query that remove all the seriesid associates only to one book, so I clean the series page.

[chaley]

To repeat what you said, the problem seems to be caused by having books that are both sideloaded and not. Looking at line 3330, the SeriesID is updated only if the book is sideloaded.

        if self.supports_series_list and book.is_sideloaded:

    @property
    def is_sideloaded(self):
        # If we don't have a content Id, we don't know what type it is.
        return self.contentID and self.contentID.startswith("file")

This means that the seriesID checks aren't done so the field won't be updated.

Does the problem fix itself if you send again from calibre the non-sideloaded books? It might not if the resend doesn't change enough to cause the book to be considered sideloaded (apparently where the book is stored). If resending the book does fix it then we have a no-risk solution.

A possible fix would be to always update the series stuff, even for non-sideloaded books. I don't know why non-sideloaded books are excluded so I'm not willing to make this change without someone testing it. The change would be to (again) line 3300:

        if self.supports_series_list:

Run from source, change that line, and let us know what happens.

[ownedbycats]

Yes, sideloaded books set the SeriesID to the series name. But re-sending a store book to device just results in a duplicate book, lol.

I made the change to line 3300 and tested with a newly-purchased store book, unfortunately it doesn't seem to update. The SeriesID is still 7385b960-7cb7-5146-853b-968a87f2f8a0 rather than the series name.

[chaley]

Originally Posted by ownedbycats

Yes, sideloaded books set the SeriesID to the series name. But re-sending a store book to device just results in a duplicate book, lol.

I made the change to line 3300 and tested with a newly-purchased store book, unfortunately it doesn't seem to update. The SeriesID is still 7385b960-7cb7-5146-853b-968a87f2f8a0 rather than the series name.

Try changing line 3339 as well

            if series_changed or series_number_changed or not book.is_sideloaded \

This should force updating of any non-sideloaded book.

[chaley]

Looking at the code, I think it is wrong. The block of code at line 3339 should be

            if series_changed or series_number_changed \
               or kobo_series_id != series_id \
               or kobo_series_number_float != newmi.series_index:

This might be the root cause of the problem.

[ownedbycats]

I was curious about whether line 3340 could've had anything to do with it.

Alas, still no change to 7385b960-7cb7-5146-853b-968a87f2f8a0

[chaley]

Did you also change line 3339, adding " or not book.is_sideloaded"?

[chaley]

Try this for the code from lines 3330 to 3347. The change to line 3331 forces the series_id to the series name if the book is not sideloaded.

        if self.supports_series_list:
            series_id = new_series if not book.is_sideloaded else self.kobo_series_dict.get(new_series, new_series)
            try:
                kobo_series_id = book.kobo_series_id
                kobo_series_number_float = book.kobo_series_number_float
            except Exception:  # This should mean the book was sent to the device during the current session.
                kobo_series_id = None
                kobo_series_number_float = None

            if not book.is_sideloaded or series_changed or series_number_changed \
               or kobo_series_id != series_id \
               or kobo_series_number_float != newmi.series_index:
                update_values.append(series_id)
                set_clause.append('SeriesID')
                update_values.append(newmi.series_index)
                set_clause.append('SeriesNumberFloat')
                if show_debug:
                    debug_print(f"KoboTouch:set_core_metadata Setting SeriesID - new_series='{new_series}', series_id='{series_id}'")