[GUI Plugin] Annotations - Page 8

Aldous04 · 11-15-2014, 02:48 PM

Quote:

Originally Posted by davidfor

I just checked the logging, that line is the last book that worked. There should have been another line with a path to the book. I think you might have the wrong version for the test, but, I've lost track of which is which. Can you test with the attached the correct version? I have updated the version to 1.4.4 to make sure we get the right one.

I'm using Calibre on a new system and I'm still getting the same error, so the problem is almost certainly caused by the Kindle.

I ran version 1.4.4 of the test but I'm not getting the path for the book:

Spoiler:

davidfor · 11-15-2014, 09:55 PM

Quote:

Originally Posted by Aldous04

I'm using Calibre on a new system and I'm still getting the same error, so the problem is almost certainly caused by the Kindle.

I ran version 1.4.4 of the test but I'm not getting the path for the book:

Spoiler:

I think you loaded the wrong version. The message says 1.4.3 not 1.4.4. Plus it is missing the logging I added in 1.4.4. The version you need is in this post.

And, it is something on the device. The error is that the plugin is trying to read a book to get metadata. It is trying to read it as a mobi (or related format). But, when the calibre routines check the format by looking at a particular spot in the file, it is not finding the expected value. It is finding what is shown in the last line of the log.

Aldous04 · 11-16-2014, 08:31 AM

Quote:

Originally Posted by davidfor

I think you loaded the wrong version. The message says 1.4.3 not 1.4.4. Plus it is missing the logging I added in 1.4.4. The version you need is in this post.

That's the version I'm using and the one I used to generate the last log I posted.

davidfor · 11-17-2014, 12:31 AM

Quote:

Originally Posted by Aldous04

That's the version I'm using and the one I used to generate the last log I posted.

Hmm, I don't know what is going on. The version I pointed to downloads for me as 1.4.4. I've attached another version that should be 1.4.5. Before installing it, can you delete the directory "annotations_resources" that is in the plugins directory?

Aldous04 · 12-05-2014, 01:10 PM

Quote:

Originally Posted by davidfor

Hmm, I don't know what is going on. The version I pointed to downloads for me as 1.4.4. I've attached another version that should be 1.4.5. Before installing it, can you delete the directory "annotations_resources" that is in the plugins directory?

Ok, it's correctly identified as 1.4.5 and I'm now getting the directories too:

Code:

calibre Debug log
calibre 2.10  isfrozen: True is64bit: False
Windows-8-6.2.9200 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '8', '6.2.9200')
Python 2.7.8
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && K4PC, K4Mac, Kindle Mobi and Topaz DeDRM && Annotations && Marvin XD && iOS reader applications
Starting up...
DEBUG:    0.0  ~~~~~~~~~~~~~~~~~~~~~~ Annotations plugin 1.4.5 (Windows) ~~~~~~~~~~~~~~~~~~~~~~
DEBUG:    0.0 libiMobileDevice:__init__() 
DEBUG:    0.0 libiMobileDevice:load_library(Windows) 
DEBUG:    0.0   libimobiledevice loaded from 'libimobiledevice.dll'
DEBUG:    0.0   libplist loaded from 'libplist.dll'
DEBUG:    0.0 AnnotationsAction:init_options(disable_caching: False) 
DEBUG:    0.0 AnnotationsDB:connect(db_version: 1) 
DEBUG:    0.1 AnnotationsAction:load_dynamic_reader_classes() 
DEBUG:    0.1   loading built-in class 'GoodReader'
DEBUG:    0.1   loading built-in class 'Kindle'
DEBUG:    0.1   loading built-in class 'KoboTouch'
DEBUG:    0.2   loading built-in class 'ParseKindleMyClippingsTxt'
DEBUG:    0.2 AnnotationsAction:inflate_dialog_resources() 
DEBUG:    0.2 CompileUI:compile_ui() 
DEBUG:    0.2   recompiling C:\Users\Alexander\AppData\Roaming\calibre\plugins\annotations_resources\dialogs\cc_wizard.ui
DEBUG:    0.2   recompiling C:\Users\Alexander\AppData\Roaming\calibre\plugins\annotations_resources\dialogs\new_destination.ui
Started up in 6.01 seconds with 947 books
DEBUG:    7.0 AnnotationsAction:launch_library_scanner(updating library index) 
DEBUG:    9.1 AnnotationsAction:library_index_complete() 
DEBUG:   10.9 AnnotationsAction:on_device_connection_changed(Amazon Kindle) 
Job: 1 Get device information started
DeviceJob: 1 Get device information done, calling callback
DeviceJob: 1 Get device information callback returned
Job: 2 Set library information started
DeviceJob: 2 Set library information done, calling callback
DeviceJob: 2 Set library information callback returned
Job: 3 Get list of books on device started
DEBUG:   11.1 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= None
DEBUG:   11.1 USBMS: dirs are: D:\ documents
DEBUG:   11.2 USBMS: scan from root False D:\documents
Job: 1 Get device information finished
No details available.
Job: 2 Set library information finished
No details available.
DEBUG:   11.6 USBMS: count found in cache: 41, count of files in metadata: 41, need_sync: False
DEBUG:   11.6 USBMS: Finished fetching list of books from device. oncard= None
DEBUG:   11.6 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= carda
DEBUG:   11.6 USBMS: Fetching list of books from device. Device= KINDLE2 oncard= cardb
DeviceJob: 3 Get list of books on device done, calling callback
DeviceJob: metadata_downloaded: Starting set_books_in_library
DeviceJob: set_books_in_library: books to process= 41
DeviceJob: set_books_in_library finished: time= 0.0
DeviceJob: metadata_downloaded: updating views
DeviceJob: metadata_downloaded: syncing
DeviceJob: metadata_downloaded: refreshing ondevice
DeviceJob: metadata_downloaded: sending metadata_available signal
DeviceJob: 3 Get list of books on device callback returned
Job: 3 Get list of books on device finished
No details available.
DEBUG:   28.2 AnnotationsAction:launch_library_scanner(updating library index) 
DEBUG:   30.0 AnnotationsAction:get_annotated_books_on_usb_device() 
DEBUG:   30.0  Kindle:get_installed_books()
DEBUG:   30.0 KindleReaderApp:get_path_map() 
DEBUG:   30.0  ids found = 947
DEBUG:   31.8  entries in path_map = 947
DEBUG:   32.3  _get_metadata - path="D:\documents\Seven Discourses on Art_B0082YWBVC.azw"
DEBUG:   32.3 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.3  _get_metadata - path="D:\documents\Oxford Dictionary of English_B0053VMNYW.azw"
DEBUG:   32.4 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.4  _get_metadata - path="D:\documents\Chartism_B005PZN2QO.azw"
DEBUG:   32.4 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.4  _get_metadata - path="D:\documents\Institutes of Oratory_B0032UYFII.azw"
DEBUG:   32.4 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.6 AnnotationsAction:library_index_complete() 
Job: 3 Get list of books on device finished
No details available.
DEBUG:   32.6  _get_metadata - path="D:\documents\Durant, Will\Story of Civilization, Volume III_ Caesar and Christ, The - Will Durant.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: The Story of Civilization, Volume III: Caesar and Christ
DEBUG:   32.7  _get_metadata - path="D:\documents\Tomlinson, H. M. (Henry Major)\Sea and the Jungle, The - H. M. Tomlinson.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: The Sea and the Jungle
DEBUG:   32.7  _get_metadata - path="D:\documents\Waugh, Evelyn\Vile Bodies - Evelyn Waugh.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: Vile Bodies
DEBUG:   32.7  _get_metadata - path="D:\documents\Waugh, Evelyn\Black Mischief - Evelyn Waugh.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: Black Mischief
DEBUG:   32.7  _get_metadata - path="D:\documents\Waugh, Evelyn\Scoop - Evelyn Waugh.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: Scoop
DEBUG:   32.7  _get_metadata - path="D:\documents\Waugh, Evelyn\Brideshead Revisited - Evelyn Waugh.mobi"
DEBUG:   32.7 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.7  Book on device title: Brideshead Revisited
DEBUG:   32.7  _get_metadata - path="D:\documents\Chartism_B005PZN2QO.azw"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: Chartism
DEBUG:   32.8  _get_metadata - path="D:\documents\Marcuse, Herbert\Eros and Civilization - Herbert Marcuse.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: Eros and Civilization
DEBUG:   32.8  _get_metadata - path="D:\documents\White, Gilbert\Natural History of Selborne, The - Gilbert White.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: The Natural History of Selborne
DEBUG:   32.8  _get_metadata - path="D:\documents\Popper, Karl R_\Open Society and Its Enemies, The - Karl R. Popper.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: The Open Society and Its Enemies
DEBUG:   32.8  _get_metadata - path="D:\documents\Durant, Will\Story of Civilization, Volume II_ The Life of Greece, The - Will Durant.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: The Story of Civilization, Volume II: The Life of Greece
DEBUG:   32.8  _get_metadata - path="D:\documents\Evans, Richard J_\Third Reich at War_ How the Nazis Led Germany From Conquest to Disaster, The - Richard J. Evans.mobi"
calibre 2.10  isfrozen: True is64bit: False
Windows-8-6.2.9200 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '8', '6.2.9200')
Python 2.7.8
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && K4PC, K4Mac, Kindle Mobi and Topaz DeDRM && Annotations && Marvin XD && iOS reader applications
Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 333, in fetch_usb_device_annotations
  File "calibre_plugins.annotations.action", line 522, in get_annotated_books_on_usb_device
  File "c:\users\alexan~1\appdata\local\temp\calibre_annotations_plugin\Kindle.py", line 168, in get_installed_books
  File "c:\users\alexan~1\appdata\local\temp\calibre_annotations_plugin\Kindle.py", line 265, in _get_metadata
  File "site-packages\calibre\ebooks\metadata\mobi.py", line 493, in get_metadata
  File "site-packages\calibre\ebooks\mobi\reader\headers.py", line 273, in __init__
  File "site-packages\calibre\ebooks\mobi\reader\headers.py", line 301, in identity
MobiError: Unknown book type: cÌÆ}ÝÝ›Å

DEBUG:   57.5 AnnotationsAction:shutting_down()

davidfor · 12-07-2014, 01:50 AM

Quote:

Originally Posted by Aldous04

Ok, it's correctly identified as 1.4.5 and I'm now getting the directories too:

Code:

calibre Debug log
calibre 2.10  isfrozen: True is64bit: False
Windows-8-6.2.9200 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '8', '6.2.9200')

DEBUG:   32.8  _get_metadata - path="D:\documents\Popper, Karl R_\Open Society and Its Enemies, The - Karl R. Popper.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: The Open Society and Its Enemies
DEBUG:   32.8  _get_metadata - path="D:\documents\Durant, Will\Story of Civilization, Volume II_ The Life of Greece, The - Will Durant.mobi"
DEBUG:   32.8 KindleReaderApp:_get_metadata(Finish!!!!) 
DEBUG:   32.8  Book on device title: The Story of Civilization, Volume II: The Life of Greece
DEBUG:   32.8  _get_metadata - path="D:\documents\Evans, Richard J_\Third Reich at War_ How the Nazis Led Germany From Conquest to Disaster, The - Richard J. Evans.mobi"
calibre 2.10  isfrozen: True is64bit: False
Windows-8-6.2.9200 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '8', '6.2.9200')
Python 2.7.8
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && K4PC, K4Mac, Kindle Mobi and Topaz DeDRM && Annotations && Marvin XD && iOS reader applications
Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 333, in fetch_usb_device_annotations
  File "calibre_plugins.annotations.action", line 522, in get_annotated_books_on_usb_device
  File "c:\users\alexan~1\appdata\local\temp\calibre_annotations_plugin\Kindle.py", line 168, in get_installed_books
  File "c:\users\alexan~1\appdata\local\temp\calibre_annotations_plugin\Kindle.py", line 265, in _get_metadata
  File "site-packages\calibre\ebooks\metadata\mobi.py", line 493, in get_metadata
  File "site-packages\calibre\ebooks\mobi\reader\headers.py", line 273, in __init__
  File "site-packages\calibre\ebooks\mobi\reader\headers.py", line 301, in identity
MobiError: Unknown book type: cÌÆ}ÝÝ›Å

DEBUG:   57.5 AnnotationsAction:shutting_down()

That tells me the culprit is "D:\documents\Evans, Richard J_\Third Reich at War_ How the Nazis Led Germany From Conquest to Disaster, The - Richard J. Evans.mob". Could you get a copy off the device and send it to me? Either put it somewhere I can download it and PM the location, or PM me for an email address.

And then can you delete the book and try again? If it gets past that, it will show that this is the problem.

Aldous04 · 12-07-2014, 04:32 PM

Quote:

Originally Posted by davidfor

That tells me the culprit is "D:\documents\Evans, Richard J_\Third Reich at War_ How the Nazis Led Germany From Conquest to Disaster, The - Richard J. Evans.mob". Could you get a copy off the device and send it to me? Either put it somewhere I can download it and PM the location, or PM me for an email address.

And then can you delete the book and try again? If it gets past that, it will show that this is the problem.

Success! It appears the above book is corrupted... Thanks for your patience.

davidfor · 12-07-2014, 07:52 PM

Quote:

Originally Posted by Aldous04

Success! It appears the above book is corrupted... Thanks for your patience.

Good to hear. As the problem is the book type in the file, a mobi to mobi conversion will probably fix it. Though calibre might raise an error that it has the wrong book type.

JaneD · 01-05-2015, 02:19 AM

I just found this plug-in today and I could cry with joy. It's everything I've wanted to capture annotations. Thank you so much for creating and maintaining it, very useful!

richo · 01-15-2015, 09:06 AM

I have noticed that from certain update of Kindle Paperwhite the parsing of myclippings.txt file for position of the highlight in book is failing. I have done little digging and found out that location text in MyClippings has been changed as follows:

example of the old text:
Your Highlight Location 93-96

example of the new text:
Your Highlight at location 206-206

I looked at the code in ParseKindleMyClippingsTxt.py and found out that very little change is needed to make this work again:

starting at line 96 (underlined line is added)

Code:

_LOCATION_REGEX = {
    'en': (r"\sLocation\s*%s",
           r"\slocation\s*%s",
           r"\sLoc\.\s*%s",),
    'de': (r"\sPosition\s*%s",),
    'es': (r"\sPosición\s*%s",),
    'fr': (r"\sEmplacement\s*%s",),
    'it': (r"\sPosizione\s*%s",),
    'jp': (r"\s位置No.\s*%s",),
    'pt': (r"\sPosição\s*%s",),
    'ch': (r"\s位置\s*%s",),
}

davidfor · 01-15-2015, 09:28 AM

Thanks for that. I'll try and prepare an update over the weekend.

richo · 01-15-2015, 09:28 AM

Hi,

first I am talking about Kindle Paperwhite. It has notes and highlights stored in file MyClippings.txt This file can be, and it is, removed when you do reboot or sometimes when software is updated. I had made in past few copies of this file. I like to read them using the annotation plug-in.

I have found in developer_notes.txt file in the plugin's zip following note:

Quote:

There are three user-initiated paths to importing annotations:
1.*User drags a .MRV, .MRVI or .TXT file to the plug-in icon. If the dropped file has the proper mime content, drop_event() fires do_drop_event(), which passes the dropped file to each of the installed reader classes until it is successfully parsed, or informs the user that no installed class could parse the file.

2. User invokes the dropdown menu item 'Fetch annotations from <connected device>' (when a supported device is connected).

3.*User invokes the dropdown menu item 'Import annotations from <reader class>'.

I was wondering should dragging and dropping of myClippings.txt file on the icon of the annotation plugin work like it is described in point 1?

At this moment it doesn't work for me and I am not sure:

Did I misunderstand the note in point 1
Is MyClippings.txt file not the expected format
or is something broken?

Thank you for helping.

richo · 01-15-2015, 09:29 AM

Quote:

Originally Posted by davidfor

Thanks for that. I'll try and prepare an update over the weekend.

Thank you David!. I wonder why the end location of the note/highlight is not stored in database even it is parsed in code (at least for paperwhite)

davidfor · 01-17-2015, 06:42 AM

Quote:

Originally Posted by richo

Hi,

first I am talking about Kindle Paperwhite. It has notes and highlights stored in file MyClippings.txt This file can be, and it is, removed when you do reboot or sometimes when software is updated. I had made in past few copies of this file. I like to read them using the annotation plug-in.

I have found in developer_notes.txt file in the plugin's zip following note:

I was wondering should dragging and dropping of myClippings.txt file on the icon of the annotation plugin work like it is described in point 1?

At this moment it doesn't work for me and I am not sure:

Did I misunderstand the note in point 1
Is MyClippings.txt file not the expected format
or is something broken?

No, you don't misunderstand anything, and there isn't anything broken. The problem is that dropping a file on the button triggers the import function. This will read a file to get the annotations, but there is no code to handle importing from this type of file.

The plugin has two ways to get annotations. "Fetch" is used to get annotations from the connected device. "Import" is used to get the annotations from a file or pasted into a prompt. In both cases, there needs to be some code to handle the format. For import, the only known format is "Goodreader" which is an iOS reading app.

It might be possible to create the import code for the myClippings.txt file. But, can you copy it back to the Kindle and do a fetch? Maybe backup the current file, put your old version on, fetch the annotations and the put the current file back.

richo · 01-17-2015, 02:50 PM

Quote:

Originally Posted by davidfor

It might be possible to create the import code for the myClippings.txt file. .

The function ParseKindleMyClippingsTxt.py does the parsing of the files, It may be not that difficult to use it for action triggered by dropping file. (It is funny that cursor icon indicates that drag and drop is not supported)

I tried your idea to put old file back and run Fetch annotation from Kindle. Interesting is that it doesn't fetch all annotations only some and I suspect only those whose books are still loaded in Kindle. Why is that? Is it done in step when progress bar is displayed with text "Getting installed books fro Kindle"?
All the info needed for import is in the MyClippings.txt file, so not really need to scan books on kindle.
That means that I am not able to fetch annotations from the books which are still in calibre but not on Kindle anymore.

01-15-2015, 09:06 AM	#115
richo Connoisseur Posts: 68 Karma: 10 Join Date: Sep 2014 Device: KOReader on: Kindle Paperwhite and Android	I have noticed that from certain update of Kindle Paperwhite the parsing of myclippings.txt file for position of the highlight in book is failing. I have done little digging and found out that location text in MyClippings has been changed as follows: example of the old text: Your Highlight Location 93-96 example of the new text: Your Highlight at location 206-206 I looked at the code in ParseKindleMyClippingsTxt.py and found out that very little change is needed to make this work again: starting at line 96 (underlined line is added) Code: _LOCATION_REGEX = { 'en': (r"\sLocation\s%s", r"\slocation\s%s", r"\sLoc\.\s%s",), 'de': (r"\sPosition\s%s",), 'es': (r"\sPosición\s%s",), 'fr': (r"\sEmplacement\s%s",), 'it': (r"\sPosizione\s%s",), 'jp': (r"\s位置No.\s%s",), 'pt': (r"\sPosição\s%s",), 'ch': (r"\s位置\s%s",), }

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	495	10-19-2024 07:06 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 09:39 AM
[GUI Plugin] Annotations (closed)	GRiker	Plugins	68	06-17-2014 07:11 AM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 01:27 PM

01-05-2015, 02:19 AM	#114
JaneD Addict Posts: 322 Karma: 1057749 Join Date: May 2010 Location: LA, CA Device: Kindle Paperwhite 2013	I just found this plug-in today and I could cry with joy. It's everything I've wanted to capture annotations. Thank you so much for creating and maintaining it, very useful!

01-15-2015, 09:28 AM	#116
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303822 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Thanks for that. I'll try and prepare an update over the weekend.

Advert

Advert