[GUI Plugin] WordDumb

[xxyzz]

You're using 64bit python? Can you try the 32bit python or 64bit version of calibre? You need to delete the 64bit python and the 'C:\Users\sam\AppData\Roaming\calibre\plugins\word dumb-numpy1.20.1' folder if you wanna install the 32bit python.

calibre and python should use the same architecture, both 64 or 32.

[sammayor]

Originally Posted by xxyzz

You're using 64bit python? Can you try the 32bit python or 64bit version of calibre? You need to delete the 64bit python and the 'C:\Users\sam\AppData\Roaming\calibre\plugins\word dumb-numpy1.20.1' folder if you wanna install the 32bit python.

calibre and python should use the same architecture, both 64 or 32.

It worked. Thanks for the plugin

[booklover2111]

Hi xxyzz, I updated your plugin yesterday to version 3.4.0. In the past, I used version 2.1.1, it usually took about 10-15 seconds to generate Word Wise for books about 200-300 pages. The new version took for ages to do same tasks (A 300-page book takes me 3'16 seconds).


I'm using calibre 5.12(64bit) and python 3.9 (64bit)

[xxyzz]

The plugin needs to download some nltk data at first run. It also needs to download numpy and more nltk data if x-ray is enabled. Create word wise alone usually takes a few seconds but create x-ray needs more time because it have to download data from Wikipedia for each x-ray term. You can disable x-ray in the plugin settings.

For comparison, the GitHub action takes 1m18s to create word wise and x-ray for book 1984 and it contains the time for testing and downloading all the files. It's highly affected by your network speed. Maybe it can be optimized but I can't find any yet. Welcome any suggestions and pull requests!

[booklover2111]

Originally Posted by xxyzz

The plugin needs to download some nltk data at first run. It also needs to download numpy and more nltk data if x-ray is enabled. Create word wise alone usually takes a few seconds but create x-ray needs more time because it have to download data from Wikipedia for each x-ray term. You can disable x-ray in the plugin settings.

For comparison, the GitHub action takes 1m18s to create word wise and x-ray for book 1984 and it contains the time for testing and downloading all the files. It's highly affected by your network speed. Maybe it can be optimized but I can't find any yet. Welcome any suggestions and pull requests!

I checked the plugin in calibre and I couldn't find the settings for it. About programming, I'm know nothing about it. The only thing I want is the generating WordWise speed.

[xxyzz]

Originally Posted by booklover2111

I checked the plugin in calibre and I couldn't find the settings for it.

calibre preference -> plugins -> search worddumb -> click 'customize plugin' -> disable x-ray -> click ok

[astro70.1]

Hello, today I installed your plugin and encountered the error below:

calibre, version 5.12.0 (win32, embedded-python: True)
Conversion error: Failed: Generating Word Wise

Starting job: Generating Word Wise
Job: "Generating Word Wise" failed with error:
Traceback (most recent call last):
File "calibre\gui2\threaded_jobs.py", line 83, in start_work
File "calibre_plugins.worddumb.parse_job", line 26, in do_job
File "calibre_plugins.worddumb.unzip", line 51, in install_libs
File "calibre_plugins.worddumb.unzip", line 88, in download_numpy
File "subprocess.py", line 359, in check_call
File "subprocess.py", line 340, in call
File "subprocess.py", line 854, in __init__
File "subprocess.py", line 1307, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified

Called with args: (<calibre.db.cache.Cache object at 0x000002A098032130>, [6]) {'notifications': <queue.Queue object at 0x000002A09CE483D0>, 'abort': <threading.Event object at 0x000002A09CE48970>, 'log': <calibre.utils.logging.GUILog object at 0x000002A09CE48640>}


I'm windows x64 user and have 64 bit python and calibre.

[booklover2111]

Originally Posted by xxyzz

calibre preference -> plugins -> search worddumb -> click 'customize plugin' -> disable x-ray -> click ok

Thanks. It works great. It never runs that fast before. It's super fast exactly as you said after I disabled X-Ray. It takes only2-3 seconds to generate WW for 300-page books. Fanstastic!!!!!!!! Keep up the good work.

[xxyzz]

Originally Posted by astro70.1

Hello, today I installed your plugin and encountered the error below:

When you install Python, you have to select that 'Add Python to PATH' option for other programs to find Python.

Or maybe you don't want x-ray then you don't need to install Python, just disable x-ray as motioned before.

[xxyzz]

Originally Posted by booklover2111

Thanks. It works great. It never runs that fast before.

I should did this in the first place and trust the speed of Python... Anyway, enjoy your reading!

[astro70.1]

Originally Posted by xxyzz

When you install Python, you have to select that 'Add Python to PATH' option for other programs to find Python.

wow appreciate for your quick reply. It works! thank you so much.

[xxyzz]

v3.5.0

- add Accept-Encoding: gzip header to requests

- pipe titles to Wikipedia query requests

- exclude files in docs and tests folder from zip file

This version reduces x-ray creation time from about 7m to 1m on the 1984 novel.

[PoP]

Just to report that I consistently get this X-Ray problem with version 3.5.0

calibre, version 5.12.0 (win32, embedded-python: True)
Erreur lors de la conversion: Échoué: Generating Word Wise

Starting job: Generating Word Wise 
Job: "Generating Word Wise" failed with error: 
Traceback (most recent call last):
  File "calibre\gui2\threaded_jobs.py", line 83, in start_work
  File "calibre_plugins.worddumb.parse_job", line 43, in do_job
  File "calibre_plugins.worddumb.parse_job", line 124, in find_named_entity
  File "calibre_plugins.worddumb.x_ray", line 113, in search
  File "calibre_plugins.worddumb.x_ray", line 91, in insert_entity
  File "calibre_plugins.worddumb.x_ray", line 51, in search_wikipedia
  File "calibre_plugins.worddumb.x_ray", line 31, in insert_wiki_intro
  File "calibre_plugins.worddumb.database", line 193, in insert_x_entity_description
sqlite3.IntegrityError: UNIQUE constraint failed: entity_description.entity
 
Called with args: (<calibre.db.cache.Cache object at 0x00000254150E0760>, [6022]) {'notifications': <queue.Queue object at 0x000002541BE5B220>, 'abort': <threading.Event object at 0x000002541BE5B430>, 'log': <calibre.utils.logging.GUILog object at 0x000002541BE5B5E0>}

[xxyzz]

Originally Posted by PoP

Just to report that I consistently get this X-Ray problem with version 3.5.0

Can you try 3.5.1?

[PoP]

3.5.1 works fine.