![]() |
#16 | |
Member
![]() Posts: 16
Karma: 10
Join Date: Sep 2021
Device: Kindle PW5 SE (2021)
|
Quote:
I'm finding that, for example, b64 decoding the kfx position ATwIAABnAAAA:177266 yields 0x013c08000067000000. It seems that the first 4 bytes, 0x013c0800, are the eid, and the last 4 bytes, 0x67000000, are the offset, is that right? But there is an byte between them: 0x013c08000067000000 What is this extra byte for? ![]() Also, do you happen to know what template is? Code:
"annotation.personal.bookmark": [ { "startPosition": "AVoDAAAAAAAA:9430", "endPosition": "AVoDAAAAAAAA:9430", "creationTime": "2019-08-11T15:24:03.088000", "lastModificationTime": "2019-08-11T15:24:03.088000", "template": "0\ufffc0" }, Last edited by gesundheit; 02-24-2022 at 09:05 PM. |
|
![]() |
![]() |
![]() |
#17 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,663
Karma: 85000003
Join Date: Nov 2011
Location: Charlottesville, VA
Device: Kindles
|
Quote:
The eid labels a section of KFX content. It is the KFX equivalent to an HTML id attribute. The offset counts positions from the start of that eid. The position following the colon is an overall position within the book and is another way to refer to the same content. Positions in KFX count Unicode characters of visible text with each image also counted as one position. I have no idea what that is for. It appears to be the same in all cases I have seen. Unicode FFFC is the Object Replacement Character. |
|
![]() |
![]() |
Advert | |
|
![]() |
#18 |
Member
![]() Posts: 16
Karma: 10
Join Date: Sep 2021
Device: Kindle PW5 SE (2021)
|
I see, I got it now. Thanks for the explanation.
|
![]() |
![]() |
![]() |
#19 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,637
Karma: 26963509
Join Date: Apr 2009
Location: USA
Device: iPhone 15PM, Kindle Scribe, iPad mini 6, PocketBook InkPad Color 3
|
Looks like this needs to be updated for Scribe annotation type:
Code:
annotation.personal.sticky_note = 11 Will check Print Replica next and supply data files, if not updated code. Also got 'excess values' exception ('val==[0]', i.e. not empty as expected, probably because of this extra annotation?). In meantime, I've patched to handle exceptions so I can look at what's working. Last edited by tomsem; 06-09-2023 at 05:55 PM. |
![]() |
![]() |
![]() |
#20 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
Maybe something like a github repository is in order.
|
![]() |
![]() |
Advert | |
|
![]() |
#21 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,637
Karma: 26963509
Join Date: Apr 2009
Location: USA
Device: iPhone 15PM, Kindle Scribe, iPad mini 6, PocketBook InkPad Color 3
|
I was able to fix it up to handle the two new annotation types (subject to code review).
Main changes: Code:
ANNOT_CLASS_NAMES = { 0: "annotation.personal.bookmark", 1: "annotation.personal.highlight", 2: "annotation.personal.note", 3: "annotation.personal.clip_article", # value not verified 10: "annotation.personal.handwritten_note", 11: "annotation.personal.sticky_note", } (made this a class attribute to consolidate duplicated strings) Code:
elif name in self.ANNOT_CLASS_NAMES.values(): obj["startPosition"] = self.decode_position(val.pop(0)) obj["endPosition"] = self.decode_position(val.pop(0)) obj["creationTime"] = datetime.datetime.fromtimestamp(val.pop(0) / 1000.0).isoformat() obj["lastModificationTime"] = datetime.datetime.fromtimestamp(val.pop(0) / 1000.0).isoformat() obj["template"] = val.pop(0) if name == "annotation.personal.note": obj["note"] = val.pop(0) elif name == "annotation.personal.handwritten_note": obj["handwritten_note_nbk_ref"] = val.pop(0) elif name == "annotation.personal.sticky_note": obj["sticky_note_nbk_ref"] = val.pop(0) I also could not keep myself from fixing linter complaints about 'unreferenced variable i' (by replacing with '_'). Scribe also seems to have another piece of data in font_prefs structure (value 0). I added 'unknown2' to absorb it. I checked another Kindle and it does not have this. Code:
"font.prefs": { "typeface": "_INVALID_,und:bookerly", "lineSp": 1, "size": 3, "align": 0, "insetTop": 64, "insetLeft": 200, "insetBottom": 8, "insetRight": 200, "unknown1": -1, "bold": 1, "userSideloadableFont": "", "customFontIndex": -1, "mobi7SystemFont": "", "mobi7RestoreFont": false, "readingPresetSelected": "", "unknown2": 0 }, Not sure what I'm going to do with this, but excited to have discovered it. Thanks @jhowell & @j.p.s.! Last edited by tomsem; 06-10-2023 at 12:42 AM. |
![]() |
![]() |
![]() |
#22 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,663
Karma: 85000003
Join Date: Nov 2011
Location: Charlottesville, VA
Device: Kindles
|
Quote:
Not worth the bother for me to do it. If someone else wants to create that I will change the first post of this thread to link to it. |
|
![]() |
![]() |
![]() |
#23 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
Quote:
I would also want to add j-howell and tomsem's github account (if he has one) as contributors. I assume that it would be tomsem making all or most of any future commits. |
|
![]() |
![]() |
![]() |
#24 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,663
Karma: 85000003
Join Date: Nov 2011
Location: Charlottesville, VA
Device: Kindles
|
Quote:
|
|
![]() |
![]() |
![]() |
#25 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
I'm inclined to create a new organization for this. It turns out KRDS (unrelated) is already taken.
How about KRDS-parse, KRDS-utils or some such? Suggestions welcome. The repository itself can be KRDS. ETA: K-R-D-S is available, which is my new preferred org name. Last edited by j.p.s; 06-17-2023 at 05:17 PM. Reason: "_" not valid in github org names |
![]() |
![]() |
![]() |
#26 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
I ended up having to restore the original white space to tomsem's version in order to see what the actual coding changes were. I ran jhowell's original version, tomsem's version, and my final version against the yj* files tomsem uploaded for The Experience Machine and the sodoku books. The JSON files exactly matched for tomsem's and my versions.
All versions generated the non fatal error: "ERROR: Unknown data structure whisperstore.migration.status" for both yjf files. |
![]() |
![]() |
![]() |
#27 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
The KRDS repository is up at:
https://github.com/K-R-D-S/KRDS with releases available as tar.gz and zip at: https://github.com/K-R-D-S/KRDS/tags |
![]() |
![]() |
![]() |
#28 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 6,663
Karma: 85000003
Join Date: Nov 2011
Location: Charlottesville, VA
Device: Kindles
|
Quote:
|
|
![]() |
![]() |
![]() |
#29 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 5,418
Karma: 99236514
Join Date: Apr 2011
Device: pb360
|
You're welcome.
I hope it will be useful to anyone developing KRDS further. |
![]() |
![]() |
![]() |
#30 |
Junior Member
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 7
Karma: 123456
Join Date: Aug 2023
Location: BC, Canada
Device: Kindle Scribe
|
Full serializer and deserializer for KRDS
I decided to learn how to use the popular Rust library serde.rs by making a full deserializer and serializer library for the KRDS format (this is sort of a practice run before I attempt a full KPF and KFX serializer/deserializer.) You just have to define the format you are trying to read as a Rust structure. I have already done this for yjr, yjf, azw3r and azw3f files. It should be very easy to add support for any other KRDS files that come up as all that needs to be done is define field names.
The code for this is published in a GitHub repo. https://github.com/willemml/krds_serde There is a discrepancy in yjr and azw3r files in that the Kindle (in my case the Scribe) serializes empty strings as "0x03 0x00 0x00 x00" whereas yjf (and azw3f) files serialize them as "0x03 0x01" I have chosen to use this shorter format for all files (and the Kindle does not complain, I have tested editing annotations and it works.) Other than that binary output of the serializer is identical to an input binary. The code is currently rough and undocumented, but it works and has tests. |
![]() |
![]() |
![]() |
Tags |
krds |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Where does Reader for PC store files? | redbees | Sony Reader | 13 | 01-22-2021 09:07 AM |
Lost Kindle Data files | Alastair_Lack | Amazon Kindle | 5 | 06-28-2013 03:31 PM |
Where does the Android Kindle app store ebook files? | bfollowell | Amazon Kindle | 3 | 03-12-2013 05:04 PM |
Where does Connect Reader store ebook files | 1490peter | Sony Reader | 1 | 08-12-2009 03:45 AM |
Old RSS data on Sony Store | JeffASonyReader | Sony Reader | 3 | 01-14-2008 06:20 PM |