use module in metadata plugin

[dandman]

i am trying to use fuzzywuzzy in my metadata plugin, but as soon as i try to deploy it it will say:

 calibre, version 7.10.0
שגיאה: Unhandled exception: <b>ModuleNotFoundError</b>:No module named 'fuzzywuzzy'

i saw the thread: https://www.mobileread.com/forums/sh...d.php?t=349814

and did everything that is written there (the empty txt file, copying the fuzzywuzzy code from github into my zip ...)

then i got:

calibre, version 7.10.0
שגיאה: Unhandled exception: <b>ModuleNotFoundError</b>:No module named 'calibre_plguins'

how can i overcome this ?

[BeckyEbook]

Typo?

calibre_plguins => calibre_plugins

[dandman]

the spell issue was in the referenced post i mentioned, thanks for that, i fixed it, but now i get:

calibre, version 7.10.0
שגיאה: Unhandled exception: <b>ModuleNotFoundError</b>:No module named 'fuzzywuzzy'

calibre 7.10  embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: he
EXE path: C:\Program Files\Calibre2\calibre.exe
Successfully initialized third party plugins: Library Genesis (1, 2, 0) && Reading List (1, 15, 4)
Traceback (most recent call last):
  File "calibre\gui2\preferences\plugins.py", line 322, in add_plugin
  File "calibre\customize\ui.py", line 525, in add_plugin
  File "calibre\customize\ui.py", line 76, in load_plugin
  File "calibre\customize\zipplugin.py", line 306, in load
  File "importlib\__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "calibre\customize\zipplugin.py", line 205, in exec_module
  File "calibre_plugins.evrit2.__init__", line 9, in <module>
  File "calibre\customize\zipplugin.py", line 205, in exec_module
  File "calibre_plugins.evrit2.fuzzywuzzy.fuzz", line 14, in <module>
  File "calibre\customize\zipplugin.py", line 205, in exec_module
  File "calibre_plugins.evrit2.fuzzywuzzy.utils", line 5, in <module>
ModuleNotFoundError: No module named 'fuzzywuzzy'

my zip structure is:

evrit2/
│
├── __init__.py
├── readme.txt
├── fuzzywuzzy/
│   ├── __init__.py
│   ├── fuzz.py
│   └── ... (more files)

and my code import looks like this:

import json
import re
import time
import unicodedata
from collections.abc import Iterable, Mapping
from queue import Empty, Queue
from threading import Event
from typing import Callable
from calibre_plugins.evrit2.fuzzywuzzy import fuzz
from calibre.ebooks.BeautifulSoup import BeautifulSoup
from calibre.ebooks.metadata.book.base import Metadata
from calibre.ebooks.metadata.sources.base import Source
from calibre.utils.logging import Log
from calibre.utils.date import parse_date, utcnow

...

[dandman]

ok i got it, in the fuzzywuzzy code there is a file named utils.py
and in line 5 it has import that tries to import:

from fuzzywuzzy.string_processing import StringProcessor

i had to change it to:

from calibre_plugins.evrit2.fuzzywuzzy.string_processing import StringProcessor

at least now it installs, i will check if it works as well

Note:
this is a very strange manner to manage packages in python (rather then use the built in delivery system of pip), the only downside i can think of is if the user has a firewall blocking pip from installing requirements / dependencies

in my view it is a flawed architecture and practice

[dandman]

admin, please add SOLVED! as a prefix to this post title (and delete this request)

10x

[DNSB]

Originally Posted by dandman

Note:
this is a very strange manner to manage packages in python (rather then use the built in delivery system of pip), the only downside i can think of is if the user has a firewall blocking pip from installing requirements / dependencies

in my view it is a flawed architecture and practice

calibre is supposed to be used with it's internal python install. This helps to ensure that everything needed for calibre to function is present and that you don't run into issues where your shiny new Python install deprecates items that calibre uses. calibre would be almost impossible to support if it depended on the end user having the correct Python version and add-ons.

If you look at the number of times Linux users have installed their distro version of calibre and then run into issues due to Python which are cured by using the install procedure from calibre's website. I.e., one recent poster complaining that his calibre install had problems with Python 3.12.

[BetterRed]

Originally Posted by DNSB

calibre is supposed to be used with it's internal python install. This helps to ensure that everything needed for calibre to function is present and that you don't run into issues where your shiny new Python install deprecates items that calibre uses. calibre would be almost impossible to support if it depended on the end user having the correct Python version and add-ons.

If you look at the number of times Linux users have installed their distro version of calibre and then run into issues due to Python which are cured by using the install procedure from calibre's website. I.e., one recent poster complaining that his calibre install had problems with Python 3.12.

In other words: "calibre isn't a walled garden, it's a closed shop"

BR

[dandman]

doesn't change the fact that this is a bad practice, there are tons of products successfully managing requirements.txt with min or max versions,

if the problem is dependency collision, then Caliber should have been the one to include and reference all the external libraries it needs (and not the plugins or users), and it should have load the user plugins in a separate runtime, allowing each plugin to run based on the requirements it needs (without the user having to deal with it or take care of it),

but as i said before ... i guess i am not the first to notice these cavities and not here to change the product, just to understand how to make it work

[DNSB]

Originally Posted by BetterRed

In other words: "calibre isn't a walled garden, it's a closed shop"
BR

More that since calibre is open source, the developer is not able to guess which add-ons someone developing a plugin will need. There have been several occasions where help has been given to include new functions and to add aliases in calibre to allow older bits to access functions where the name/functionality has been changed.

Unlike in my career in IT where we could control updates, what software was installed and manage requirements at the domain level, that type of control is not available on a personal computer. We had a fairly homogeneous environment with end users using about 80% Windows, 20% MacOS and a few Linux installs. Even there, we ran into issues where program updates were iffy. The Linux boxes were more or less "managed" by ignoring them.