Kobo locations

[olof-ynwa]

Hello!, Kindle user here.
I'm currently working on a annotations parser that parses the kindle annotations and then pairs them into note-highlight pairs. (Everytime a note is created in your kindle a highlight is created at the same location and at the same time). The note is then parsed into list of commands that govern if the highlight should be changed (adding text with metadata to it, search and replace, basic formatting), and where it should be written (add to line/header in markdown file) etcetera. It works fine but I want to implement some extra functionality.

I want to be able to implement a command that allows the user to extract specific images, or tables from an unpacked version of the kindle file.
Whenever a image is highlighted in the kindle, the content of the saved highlight is just an empty string as such:

Note highlight pair example from MyClippings.txt:


Highlight:

==========
Nils Holgerssons underbara resa genom Sverige. 1 (Selma Lagerlöf)
- Your Highlight on page 29 | Location 431-432 | Added on Monday, January 31, 2022 12:28:12 PM

Associated note containing commands:

==========
Nils Holgerssons underbara resa genom Sverige. 1 (Selma Lagerlöf)
- Your Note on page 29 | Location 432 | Added on Monday, January 31, 2022 12:28:12 PM

x image

The x image command above says: this is an image, Extract it from the html of the unpacked version of this book by using information about the location (preferable) or page. Replace the actual highlight content with image content (markdown image embed in this case) and copy the image source file to markdown attachements folder.

I had issues implementing this with the kindle because of the mobi format. Even when unpacking the format, I can't find any pattern connecting the locations as specified in the kindle, with the unpacked version and the unpacked html. Perhaps its easier on the kobo?

The actual questions

Can you implement something like this on the kobo?
Would it possible to pair a highlight and a note? How does locations work on the kobo. Is it possible to know the corresponding "kobo location" of the html content in the epub in some way so that you can find the html of an annotation based on location?

Thank you in advance.

[davidfor]

Firstly, you have completely misunderstood how the "My Clippings.txt" file works. It is a simple log of the annotations you made using the device. The location is just an approximation and should match the page number shown when you make the annotation, Trying to match that back to the book is not going to work except by doing text search. There is another file on the device that has location details that are used to show the highlight when reading. There are some details about this somewhere in the Kindle forum. I see you posted the question in the Kindle forum, so someone will probably come along with the details there.

For Kobo devices, the annotations are stored in the database. This includes the location in the book. It isn't hard to use that to work out where in the book it is. But, it only works for that edition. If the book is formatted differently, then you will probably have to resort to a text search again to find the location.

There are a few things around that will read the annotations from the database. Such a a function in calibre, the Annotations plugin and my Kobo Utilities plugin. There are other scripts around to do it. I have posted a list of all the methods I know about occasionally. A search should find it and I don't think anything has changed. Looking at any of those will show how it works.

[olof-ynwa]

I'm guessing that the sdr folder contains that info in the kindle, which is out of reach then I suppose. But if I understood you correctly, it seems then that the locations in the kindle are basically used to narrow down where the locations are while text search is used to find the actual highlighted text?
So the approximations of the locations are probably difficult to reproduce? But in that case you could just use pages instead. (for the most common case where there's only one image on one page, and then highlighting text next to the image, and text search, for the other rarer cases).

This doesn't really belong to the kobo forum but the unpacked kindle toc.ocx doesn't contain the correct paging.
When parsing the html I get that my example book has 465 pages while the kindle device shows that it has over 600 (on the web it says that it has 458). Which probably means that i can't use pages either on the kindle :/?? (Have to ask on the other forum). I'm guessing that the same issues doesn't exist when using epub on the kobo? But perhaps they appear if you start converting the ebook back and forth betweemn different formats?

[olof-ynwa]

Just got a nice answer on the other forum that solves most of my problems

[davidfor]

Originally Posted by olof-ynwa

I'm guessing that the sdr folder contains that info in the kindle, which is out of reach then I suppose.

Yes, the SDR folder or files is what I understand. From memory, there is an explanation of the file and a script to extract the annotations somewhere in the Kindle forum.

But if I understood you correctly, it seems then that the locations in the kindle are basically used to narrow down where the locations are while text search is used to find the actual highlighted text?
So the approximations of the locations are probably difficult to reproduce? But in that case you could just use pages instead. (for the most common case where there's only one image on one page, and then highlighting text next to the image, and text search, for the other rarer cases).

This doesn't really belong to the kobo forum but the unpacked kindle toc.ocx doesn't contain the correct paging.
When parsing the html I get that my example book has 465 pages while the kindle device shows that it has over 600 (on the web it says that it has 458). Which probably means that i can't use pages either on the kindle :/?? (Have to ask on the other forum). I'm guessing that the same issues doesn't exist when using epub on the kobo? But perhaps they appear if you start converting the ebook back and forth betweemn different formats?

The "My Clippings.txt", it is a transaction log of annotations made on that particular device. The device does not read it. It writes to it as annotations are made, and nothing else. The location is something calculated for the book. I have no idea how a Kindle does this, but, other than as an approximation of how far through the book an annotation is, it is useless. And really, the only use for the "My Clippings.txt" is to make it a little bit easier to copy a quote from the device.