[GUI Plugin] WordDumb

[steakhutzeee]

Originally Posted by xxyzz

- send a private message of the book download link to me in this forum

- or create a new issue at GitHub and upload the book there

- or post a screenshot, the book title and the x-ray file, I'll try to find the book

Thanks for this. I just created the github issue. Hope it's ok.

[steakhutzeee]

After a book is sent to the kindle via WordDumb, is it possible to remove the word wise and/or x-ray? I saw that nothing is added into the calibre library but only on the kindle. Correct?

[xxyzz]

Originally Posted by steakhutzeee

After a book is sent to the kindle via WordDumb, is it possible to remove the word wise and/or x-ray? I saw that nothing is added into the calibre library but only on the kindle. Correct?

Created files are in the same folder of the book file, they will be moved to Kindle when Kindle is connected and you click the plugin. Otherwise they just stay there wait to be sent.

[steakhutzeee]

Originally Posted by xxyzz

Created files are in the same folder of the book file, they will be moved to Kindle when Kindle is connected and you click the plugin. Otherwise they just stay there wait to be sent.

Strange, i used WordDumb with 2 books. In Calibre Library folder i have the following files for the books:

-cover.jpg
-metadata.opf
-book.epub
-book.kfx

On the kindle i see also the x-ray files etc.

EDIT: Oh ok got your point, if kindle is not connected the files sit there. When i connect the kindle the files are moved to the kindle and deleted from the Calibre folder.

[xxyzz]

v3.8.0

Search X-Ray people entities, support more languages and some improvements on X-Ray

- Add an option to control whether seach people on Wikipedia

- Use spaCy to find named entities, it should be faster and more accurate. Another benifit is more lanuages are supported.

- Detect Wikipedia title got normalized then redirected

- Fix X-Ray entity length

This version can create X-Ray file for books written in the following languages: Dansk, Deutsch, English, Español, Français, Italiano, Lietuvių, Nederlands, Norsk bokmål, Polski, Português, Română, Ελληνικά, Русский, 中文, 日本語.

I only tested the last three languages.

[xxyzz]

v3.9.0

Support Chinese Wikipedia variants and hide new process window on Windows

- Add option to select variant of Chinese Wikipedia

- Don't create window for each pip process on Windows

[steakhutzeee]

Hi, if i'm not wrong characters are searched on wikipedia to generate the x-ray.

I see v3.8.0 has an option to select Wikipedia. Which other choices are there? Will this update improve the results for fiction characters?

Thanks!

[xxyzz]

Originally Posted by steakhutzeee

I see v3.8.0 has an option to select Wikipedia. Which other choices are there? Will this update improve the results for fiction characters?

The "search people" option is for nonfiction books and novels that have many character pages on Wikipedia like A Song of Ice and Fire and The Lord of the Rings. You should search the characters on Wikipedia in a browser then to decide whether to enable this option. And the plugin may not find the correct page if the search result is a disambiguation page.

Currently the plugin only searches Wikipedia but in theory any website uses MediaWiki and has the TextExtracts extension can be added easily.

[xxyzz]

v3.9.1: reuse HTTP connection and restore compression header when sending requests to Wikipedia API

v3.9.2: use senter component to find sentence boundaries

v3.9.3:

- add menus for create Word Wise and X-Ray individualy

- add preferences and donate menu

- fix import click error on Arch Linux

v3.9.4: update spacy to 3.1.0 and add support of Catalan language

[AlaVander]

This error is always encountered when creating X-ray. It seems to be caused by the network agent, but you know, I need to access wiki through the network agent.

calibre, version 5.22.1 (win32, embedded-python: True)
subprocess.run failed: ERROR: Exception: Traceback (most recent call last): File "D:\Program Files\Python\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in _main status = self.run(options, args) File "D:\Program Files\Python\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper return func(self, options, args) File "D:\Program Files\Python\lib\site-packages\pip\_internal\commands\install.py", line 318, in run requirement_set = resolver.resolve( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 103, in resolve r = self.factory.make_requirement_from_install_req( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 429, in make_requirement_from_install_req cand = self._make_candidate_from_link( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 200, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 306, in __init__ super().__init__( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 151, in __init__ self.dist = self._prepare() File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 234, in _prepare dist = self._prepare_distribution() File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 317, in _prepare_distribution return self._factory.preparer.prepare_linked_requirement( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 508, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 550, in _prepare_linked_requirement local_file = unpack_url( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 239, in unpack_url file = get_http_url( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 102, in get_http_url from_path, content_type = download(link, temp_dir.path) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\download.py", line 144, in __call__ resp = _http_get_download(self._session, link) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\download.py", line 125, in _http_get_download resp = session.get(target_url, headers=HEADERS, stream=True) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 555, in get return self.request('GET', url, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\session.py", line 449, in request return super().request(method, url, *args, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\cachecontrol\adapter.py", line 53, in send resp = super(CacheControlAdapter, self).send(request, **kw) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\adapters.py", line 439, in send resp = conn.urlopen( File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connectionpool.py", line 696, in urlopen self._prepare_proxy(conn) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connectionpool.py", line 964, in _prepare_proxy conn.connect() File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connection.py", line 359, in connect conn = self._connect_tls_proxy(hostname, conn) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connection.py", line 500, in _connect_tls_proxy return ssl_wrap_socket( File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\util\ssl_.py", line 432, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\util\ssl_.py", line 474, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "D:\Program Files\Python\lib\ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "D:\Program Files\Python\lib\ssl.py", line 997, in _create raise ValueError("check_hostname requires server_hostname") ValueError: check_hostname requires server_hostname 

Starting job: Generating X-Ray for The Call of the Wild 
Job: "Generating X-Ray for The Call of the Wild" failed with error: 
Traceback (most recent call last):
  File "calibre\gui2\threaded_jobs.py", line 83, in start_work
  File "calibre_plugins.worddumb.parse_job", line 17, in do_job
  File "calibre_plugins.worddumb.unzip", line 31, in install_libs
  File "calibre_plugins.worddumb.unzip", line 104, in pip_install
  File "subprocess.py", line 512, in run
subprocess.CalledProcessError: Command '['py', '-m', 'pip', 'install', '-t', WindowsPath('C:/Users/Alafande/AppData/Roaming/calibre/plugins/worddumb-libs/en_core_web_sm_3.1.0'), '--no-deps', 'https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl']' returned non-zero exit status 2.
 
Called with args: ((1486, 'KFX', 'BB2Z3HAZBJ', 'E:\\Calibre 书库\\Jack London\\The Call of the Wild (1486)\\The Call of the Wild - Jack London.kfx', <calibre.ebooks.metadata.book.base.Metadata object at 0x0000029D0689D310>, {'spacy': 'en_core_web_sm', 'wiki': 'en'}), False, True) {'notifications': <queue.Queue object at 0x0000029D0692B220>, 'abort': <threading.Event object at 0x0000029D0692B580>, 'log': <calibre.utils.logging.GUILog object at 0x0000029D0692B670>}

[xxyzz]

Originally Posted by AlaVander

This error is always encountered when creating X-ray. It seems to be caused by the network agent, but you know, I need to access wiki through the network agent.

You should also copy and paste the real error message in the error dialog window, it's above the traceback log you posted.

[AlaVander]

Originally Posted by xxyzz

You should also copy and paste the real error message in the error dialog window, it's above the traceback log you posted.

If I understand it correctly, it's in the second line of the code box, a very long error message without a newline.

[xxyzz]

Originally Posted by AlaVander

If I understand it correctly, it's in the second line of the code box, a very long error message without a newline.

Oh.. I didn't see it, sorry about that.

The failed command was downloading spacy model from GitHub. Check your anti-censorship tool.

Maybe you don't need the proxy to download that file, try to run the command in a terminal.

[AlaVander]

calibre, version 5.22.1 (win32, embedded-python: True)
subprocess.run failed: ERROR: Exception: Traceback (most recent call last): File "D:\Program Files\Python\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in _main status = self.run(options, args) File "D:\Program Files\Python\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper return func(self, options, args) File "D:\Program Files\Python\lib\site-packages\pip\_internal\commands\install.py", line 318, in run requirement_set = resolver.resolve( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\resol ver.py", line 103, in resolve r = self.factory.make_requirement_from_install_req( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\facto ry.py", line 429, in make_requirement_from_install_req cand = self._make_candidate_from_link( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\facto ry.py", line 200, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candi dates.py", line 306, in __init__ super().__init__( File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candi dates.py", line 151, in __init__ self.dist = self._prepare() File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candi dates.py", line 234, in _prepare dist = self._prepare_distribution() File "D:\Program Files\Python\lib\site-packages\pip\_internal\resolution\resolvelib\candi dates.py", line 317, in _prepare_distribution return self._factory.preparer.prepare_linked_requirement( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 508, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 550, in _prepare_linked_requirement local_file = unpack_url( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 239, in unpack_url file = get_http_url( File "D:\Program Files\Python\lib\site-packages\pip\_internal\operations\prepare.py", line 102, in get_http_url from_path, content_type = download(link, temp_dir.path) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\download.py", line 144, in __call__ resp = _http_get_download(self._session, link) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\download.py", line 125, in _http_get_download resp = session.get(target_url, headers=HEADERS, stream=True) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 555, in get return self.request('GET', url, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_internal\network\session.py", line 449, in request return super().request(method, url, *args, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\cachecontrol\adapter.py", line 53, in send resp = super(CacheControlAdapter, self).send(request, **kw) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\requests\adapters.py", line 439, in send resp = conn.urlopen( File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connectionpool.py", line 696, in urlopen self._prepare_proxy(conn) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connectionpool.py", line 964, in _prepare_proxy conn.connect() File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connection.py", line 359, in connect conn = self._connect_tls_proxy(hostname, conn) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\connection.py", line 500, in _connect_tls_proxy return ssl_wrap_socket( File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\util\ssl_.py", line 432, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls) File "D:\Program Files\Python\lib\site-packages\pip\_vendor\urllib3\util\ssl_.py", line 474, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "D:\Program Files\Python\lib\ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "D:\Program Files\Python\lib\ssl.py", line 997, in _create raise ValueError("check_hostname requires server_hostname") ValueError: check_hostname requires server_hostname

Starting job: Generating X-Ray for The Call of the Wild
Job: "Generating X-Ray for The Call of the Wild" failed with error:
Traceback (most recent call last):
File "calibre\gui2\threaded_jobs.py", line 83, in start_work
File "calibre_plugins.worddumb.parse_job", line 17, in do_job
File "calibre_plugins.worddumb.unzip", line 31, in install_libs
File "calibre_plugins.worddumb.unzip", line 104, in pip_install
File "subprocess.py", line 512, in run
subprocess.CalledProcessError: Command '['py', '-m', 'pip', 'install', '-t', WindowsPath('C:/Users/Alafande/AppData/Roaming/calibre/plugins/worddumb-libs/en_core_web_sm_3.1.0'), '--no-deps', 'https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl']' returned non-zero exit status 2.

Called with args: ((1486, 'KFX', 'BBH8Y26D2K', 'E:\\Calibre 书库\\Jack London\\The Call of the Wild (1486)\\The Call of the Wild - Jack London.kfx', <calibre.ebooks.metadata.book.base.Metadata object at 0x00000133D622A400>, {'spacy': 'en_core_web_sm', 'wiki': 'en'}), False, True) {'notifications': <queue.Queue object at 0x00000133D8AE4310>, 'abort': <threading.Event object at 0x00000133D8AE41F0>, 'log': <calibre.utils.logging.GUILog object at 0x00000133D8AE4130>}

[xxyzz]

No matter how you download the .whl file from GitHub, as long as you have the file in your PC, you can run the command in a terminal by replacing the url with the .whl file path or leave it if you can download it directly.

py -m pip install -t C:/Users/Alafande/AppData/Roaming/calibre/plugins/worddumb-libs/en_core_web_sm_3.1.0 --no-deps https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl

Please also check out this comment: https://github.com/pypa/pip/issues/9...ment-741836058

And you should remove all the soft hyphens from your book, it will affect spacy's ner component, i.e., less X-Ray entities.