[GUI Plugin] Library Codes

[alutian]

Count Pages, Extract ISBN, Find Duplicates, Quality Check, View Manager and Modify ePubs all install and appear in the toolbar. They all seem to be working and customisable via Preferences/Plug-ins.

Calibre path is C:\Program Files\Calibre2\calibre.exe as expected, but

C:\Users\Alutian\AppData\Roaming\calibre\plugins\L ibrary Codes.json does not exist. Like jems007 with the 32-bit portable version, I have the .zip file but no .json. Can I assume the .json file will only be produced when the plug-in is customised?

I have exactly the same problem with the 32-bit version, but I have yet to try the portable version.

Alutian.

[DaltonST]

Originally Posted by alutian

Count Pages, Extract ISBN, Find Duplicates, Quality Check, View Manager and Modify ePubs all install and appear in the toolbar. They all seem to be working and customisable via Preferences/Plug-ins.

Calibre path is C:\Program Files\Calibre2\calibre.exe as expected, but

C:\Users\Alutian\AppData\Roaming\calibre\plugins\L ibrary Codes.json does not exist. Like jems007 with the 32-bit portable version, I have the .zip file but no .json. Can I assume the .json file will only be produced when the plug-in is customised?

I have exactly the same problem with the 32-bit version, but I have yet to try the portable version.

Alutian.

Library Codes is not customizable via Preferences\Plugins. All of the customization is done within the menu choice for that purpose. If you click the Preferences\Plugins\Customize button, a little dialog tells you that.


Attached is a .json file for it. Put it where it belongs and see what happens. Rename it from .json.txt to just .json. Mobileread.com does not allow .json files to be uploaded.


DaltonST

[DaltonST]

@alutian:

The only thing I can think of which is not a standard installation of Windows 10 64-bit is that my temp and tmp folders are not on the same drive as the Windows installation. Windows is on C:, temp files are on A:, a RAMDrive.

Just for grins, temporarily reset the temp and tmp folders to the default settings. Keep the Calibre 64 version. Manually delete the Library Codes.zip file. Run Calibre 64. Manually load LC from the .zip file you manually downloaded. Restart Calibre. See what happens.


DaltonST

[alutian]

Originally Posted by DaltonST

@alutian:



Just for grins, temporarily reset the temp and tmp folders to the default settings. Keep the Calibre 64 version. Manually delete the Library Codes.zip file. Run Calibre 64. Manually load LC from the .zip file you manually downloaded. Restart Calibre. See what happens.


DaltonST

You beat me to it!!

Problem cured, but not problem solved.

I now have Library Codes up and running, more by luck than judgement. (History...) I have Windows 64bit set up normally on the C: drive. I also have drive A:, which is an 8GB RAM Drive, originally installed, ironically, for Calibre - it really speeds up the mamipulation of large libraries. (Calibre seems to be very 'safe' with files but at the cost of reading and writing an awful lot...)

Windows keeps its Temp folders on A: (saves wear and tear on the SSD C: drive).

No problem until Library Codes refused to play ball. I thought I might cure the problem by deleting the RAM Drive, but it made things worse - now Calibre failed to start:

Python function terminated unexpectedly [Error 3] The system cannot find the path specified: u'A:\\Temp\\calibre_ywrgmh' (Error Code: 1)

Only cure was to set the Windows Temp to the C: drive and uninstall and re-install Calibre. Now it works!

First thoughts: I have deleted an essential file in the A:\Temp folder, but it's a 'path' that cannot be found and... Caliber will start, with Library Codes working, if the Windows Temp directory is on the C: drive, even though the directory is empty. If you look in the Temp directory before Calibre starts, the 'path specified' does not exist. Something in Calibre (Library Codes?) writes to the Windows Temp directory, then reads from C:\Temp. Fine if the Temp directory is C:\Temp, but not if Windows Temp -> A:\Temp. It's not a RAM Drive problem as I can reproduce it by setting Windows Temp -> D:\Temp, D: being a standard HD.

That's my best guess at the problem!


Alutian

[theducks]

did you try starting Calibre with the commandline switch
CALIBRE_TEMP_DIR - sets the temporary directory used by calibre
(would allow you to keep yor Windows Temp)

[BetterRed]

Originally Posted by theducks

did you try starting Calibre with the commandline switch
CALIBRE_TEMP_DIR - sets the temporary directory used by calibre
(would allow you to keep yor Windows Temp)

and if that works you should be able to use the CALIBRE_TEMP_DIR Environment variable to effect a permanent re-assignment.

BR

[alutian]

OK. RAMDrive A: back online.

Windows Temp -> A:\TEMP, CALIBRE_TEMP_DIR -> C:\TEMP

Fires up Calibre and it works... but no Library Codes

Windows Temp -> A:\TEMP, CALIBRE_TEMP_DIR -> A:\TEMP

Fires up Calibre and it works... but no Library Codes

Windows Temp -> C:\TEMP, CALIBRE_TEMP_DIR -> A:\TEMP

Fires up Calibre and it works - and so does Library Codes!!

I canna be bothered with the last combination - we all know both on the C: drive works.

So, back to my earlier idea...

Something in Library Codes writes to the Windows Temp directory, then reads from C:\Temp. Fine if the Temp directory is C:\Temp, but not if Windows Temp -> A:\Temp.

Alutian

[DaltonST]

Originally Posted by alutian

Something in Library Codes writes to the Windows Temp directory, then reads from C:\Temp. Fine if the Temp directory is C:\Temp, but not if Windows Temp -> A:\Temp.

Alutian

@Alutian:

Yes, Library Codes does its magic by downloading data from multiple places on the internet to get different data to use to derive various Library Codes. It stores downloaded data temporarily in a temporary file, processes it, deletes it, and uses the processed results to update metadata.db table identifiers and custom columns for DDC and LCC.

Standard, generic Python functionality for OS temporary files is used.

In short, yes, Library Codes creates a generic OS (very) temporary file for each book it is processing.

The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the TMPDIR, TEMP or TMP environment variables

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

1. The directory named by the TMPDIR environment variable.
2. The directory named by the TEMP environment variable.
3. The directory named by the TMP environment variable.
4. A platform-specific location:
   • On Windows, the directories C:\TEMP, C:\TMP, \TEMP, and \TMP, in that order.
   • On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.
5. As a last resort, the current working directory.

DaltonST

[BetterRed]

FWIW : Library Codes, being part of Calibre, should surely be reading and writing its temporary files to the location referenced by CALIBRE_TEMP_DIR

I think CALIBRE_TEMP_DIR defaults to the System User TEMP location, so if System User TEMP is referencing A:\TEMP and CALIBRE_TEMP_DIR isn't set, then I would have thought Library Codes should be reading and writing to A:\TEMP.

BR

[DaltonST]

@BR:

Wrong.

Calibre.exe has security that Library Codes does not have. LC cannot write files to everywhere that Calibre can write files to.

LC uses add-on Python packages that Calibre does not use.

The user must set the Environment Variables used by and documented in Python, and stated below.

The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the TMPDIR, TEMP or TMP environment variables

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

The directory named by the TMPDIR environment variable.
The directory named by the TEMP environment variable.
The directory named by the TMP environment variable.
A platform-specific location:
On Windows, the directories C:\TEMP, C:\TMP, \TEMP, and \TMP, in that order.
On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.
As a last resort, the current working directory.

DaltonST

[BetterRed]

Originally Posted by DaltonST

@BR:

Wrong.

If you say so, but I would point out that the EPUBSplit PI reads and writes it's temporary files to the location referenced by CALIBRE_TEMP_DIR. There may be other PI's that do likewise.

But my post was wrong on another count - I mentioned System TEMP, that should have been User TEMP of course

But the question stands - why isn't the PI working when the User TEMP location isn't the Default - %USERPROFILE%\AppData\Local\Temp

BR

[BetterRed]

Long shot - if TEMP is on drive A:, could something be thinking FAT (as in floppy and diskette). And that something needs something that FAT can't handle - like hard or symbolic links or alternate data streams (file forks) and such.

Added : I raised this because when I had a RAM drive I made it X:, I have a vague memory of being 'warned' off using A: or B:

BR

[DaltonST]

Originally Posted by alutian

OK. RAMDrive A: back online.

Windows Temp -> A:\TEMP, CALIBRE_TEMP_DIR -> C:\TEMP

Fires up Calibre and it works... but no Library Codes

Windows Temp -> A:\TEMP, CALIBRE_TEMP_DIR -> A:\TEMP

Fires up Calibre and it works... but no Library Codes

Windows Temp -> C:\TEMP, CALIBRE_TEMP_DIR -> A:\TEMP

Fires up Calibre and it works - and so does Library Codes!!

I canna be bothered with the last combination - we all know both on the C: drive works.

So, back to my earlier idea...

Something in Library Codes writes to the Windows Temp directory, then reads from C:\Temp. Fine if the Temp directory is C:\Temp, but not if Windows Temp -> A:\Temp.

Alutian

@Alutian:

Did you update your Environment Variables that Python uses to force temporary files to go where you want them to go? Especially TMPDIR, since that is the first place it tries before moving down its list of places to check.

Also, since you use Windows, and per the Python documentation, it would be a good idea for you to physically delete c:\temp and c:\tmp. That would force it into '\temp' and '\tmp'.

The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the TMPDIR, TEMP or TMP environment variables

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

• The directory named by the TMPDIR environment variable.
• The directory named by the TEMP environment variable.
• The directory named by the TMP environment variable.

• A platform-specific location:
  On Windows, the directories C:\TEMP, C:\TMP, \TEMP, and \TMP, in that order.

• On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.

• As a last resort, the current working directory.

CALIBRE_TEMP_DIR only is used by plugins that use only Calibre APIs and native Calibre Methods.

Some people naively think that all plug-ins can be lumped together as "a plug-in", and the same rules apply to all of them. Not so.

My Library Codes, QuarantineAndScrub, Multi-Column Search, English Noun Frequenies, CALM, Author Book Count, and Author Book Count Hierarchy add-ons and plug-ins do not fall into that category. At all. To the extent they use Calibre APIs and native Calibre Methods, of course they automatically use CALIBRE_TEMP_DIR. Otherwise, generic Python rules are in play. Hence, TMPDIR etc. as documented above (and soon to be in the Original Posts of all of my plug-ins that create temporary files other than through native Calibre Methods).



DaltonST

[BetterRed]

Windows has two sets of Environment variables - System and User, and both normally have entries for TMP and TEMP, viz:



BR

[jemsoo7]

Did for a test copied the working LC Calibre2 (64bit) folder C:\Program Files\Calibre2 and folder settings C:\Users\jemsoo7\AppData\Roaming\calibre\plugins into 2 versions of calibre portable. ONE from portableappz.blogspot (which date 2014) but is supposed to work fine with latest version of both 64bit and 32bit (as stated on the site), the other in calibre-portable version (default calibre website 2016 version).

The portableappz.blogspot version works fine it launches with LC running fine with all other plugins. The other one simply don't show LC ONLY.... other plugins works fine.

these 2 compiled version of calibre-portable.exe from calibre default website and calibrePortable.exe from portableappz.blogspot treat the 2 folders differently and allows LC plugin to run fine in one and not the other.... strange

Portableappz.blogspot uses an ini file in the folder of portable softs this one has this:

[CalibrePortable]
CheckRegWrite=true
AllowMultipleInstances=false
Library=Users\jemsoo7\Downloads\CalibrePortable\1_ 2\Library
DisableSplashScreen=false
SplashScreenName=

if i put Library=C:\Users\jemsoo7\Downloads\CalibrePortable \1_2\Library instead i get the python error "u' :" stated above by Alutian here and it does not load....

Hope this can add some light to this mystery.... but so far i can use the portableappz.blogspot version its working fine.... The folder structure are somewhat different as shown in the photo below!

not working for me: has change environment variable in user and added portable path in PATH as proposed by alutian.