06-22-2016, 12:13 PM
|
#1 |
|
Deviser
  
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
[GUI Plugin] Zotero Metadata Importer
[GUI Plugin] Zotero Metadata Importer
Summary: You must use Zotero to use ZMI, which both imports files from Zotero into Calibre, and exports files from Calibre for import into Zotero.
What is Zotero? Spoiler:
Workaround for File Attachment Issues: Spoiler:
Requires Minimum Calibre Version: 6.0.0 Desktop Version Only Requires Minimum Zotero Version: 6.0.15 Desktop Version Only Old & Deprecated versions of ZMI: https://www.mobileread.com/forums/sh...d.php?t=327472 Other Useful Calibre Plugins to Consider:
Version History: Spoiler:
Last edited by DaltonST; 02-06-2023 at 01:03 PM. Reason: Version 1.0.78 |
|
|
|
06-22-2016, 12:13 PM
|
#2 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
Reserved for future use.
|
|
|
|
| Advert | |
|
|
|
06-22-2016, 12:53 PM
|
#3 |
|
Connoisseur
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
|
Thanks for the plugin!
I just tried it and I am getting following errors after pressing the first button: calibre, version 2.59.0 ERROR: ZMI: GENERATION OF CUSTOM COLUMNS TERMINATED PREMATURELY........ERROR: >>>>'module' object has no attribute 'STARTUPINFO' calibre, version 2.59.0 ERROR: Automatically Add Custom Columns: Fatal error experienced in calibredb. If you use the Apple OSX operating system, then it is probable that you have not already installed the Calibre OSX 'Command-Line' Utilities, which include calibredb. Unless you install them, or if you do not use OSX, you will have to add the Custom Columns manually. |
|
|
|
|
06-22-2016, 01:43 PM
|
#4 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
@excaliber:
I use Windows 10, and it works perfectly every time. What OS do you have? Did you read the Original Post section about Special OS Notes? DaltonST |
|
|
|
|
06-22-2016, 01:55 PM
|
#5 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
@excaliber:
I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can. I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig. Thanks. DaltonST |
|
|
|
| Advert | |
|
|
|
06-22-2016, 02:11 PM
|
#6 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
New version posted with a subprocess module change for OSs that are not Windows. Windows still works perfectly, but I cannot test Linux or OSX.
DaltonST n.b. Newly uploaded versions are not visible in Calibre's 'Plugin Updater' until after the bottom of the hour plus a few minutes or so. ZMI is new, so that will be problematic for a while. To get it immediately, you must manually download it from the OP and import it from that .zip file via Preferences > Plugins > Import from File Last edited by DaltonST; 06-22-2016 at 02:43 PM. |
|
|
|
|
06-22-2016, 06:29 PM
|
#7 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts
Version 1.0.2 - 2016-06-22 Changes for Linux & OSX; New auto-run Validation of Custom Columns after ZMI starts.
If you do not see it in your Calibre 'Plugin Updater', you will need to download it manually. If anyone has Linux or OSX, I would like to know if the automatic generation of Zotero Custom Columns works for you viz a viz spawning calibredb in subprocesses to create the new Custom Columns. Windows 10 works perfectly, of course, because I developed it on Windows 10. DaltonST |
|
|
|
|
06-22-2016, 08:20 PM
|
#8 |
|
Connoisseur
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
|
@DaltonST:
I bet you use Linux. Based on quick research, it appears Linux cannot handle STARTUPINFO like Windows can. I will upload a theoretical fix, but I do not have Linux so I cannot test it. You can be the guinea pig. Right. I use mostly Linux, I have mentioned this before. I am now testing now your new version. I get no more the aforementioned errors! Automatic generation of Zotero Custom Columns works! (I don't understand what do you mean by "spawning calibredb in subprocesses") I can update title and author of a selected book (ISBN and ISSN haven't tried) I can manually update calibre metadata for a single book. I don't get what is really happening at step 2 and 3. How can I update several books automatically? The books must be first imported into calibre? I thought calibre would import all books and their metadata after reading the csv file. Besides that I would like to make two suggestions: - Apart from the title and author there could be even more entries which coincide, so the user could be offered the option to associate them and overwrite them too - Maybe in a table? - An option to make the new Zotero entries appear in the details view of Calibre would be nice. |
|
|
|
|
06-22-2016, 09:27 PM
|
#9 | |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
Answers: Part 1 of 2
Quote:
[2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths. [3] Yes. Only Calibre imports books into Calibre. [4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin. The ToolTips for each button and grey area in the ZMI dialog box explain how ZMI works. DaltonST Last edited by DaltonST; 06-22-2016 at 09:34 PM. |
|
|
|
|
|
06-22-2016, 09:32 PM
|
#10 | |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
Answers: Part 2 of 2
Quote:
[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming. DaltonST |
|
|
|
|
|
06-23-2016, 10:06 AM
|
#11 |
|
Connoisseur
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
|
[3] Yes. Only Calibre imports books into Calibre.
[4] No. Never will happen. Use Calibre to import books into Calibre, regardless of their origin. Is not possible for Calibre to import files if their path is known? I mean, If there is a file attached in Zotero then this is reflected in the "File Attachments" column. [2] What appears in the Calibre details view is a basic Calibre function, not ZMI. Simply go to Calibre > Preferences > Look & Feel > Book Details. One more thing learned  . Did that but any ideas why I'm getting 2 columns and why in the 2nd column only the values are showing up (screenshot attached)?[1] What other entries should go from the CSV to where in Calibre? Give me a list. That requires specific Python programming. If it is possible, I would suggest to only hard code the fields that are equal in both Zotero and Calibre (default fields) like Title and Authors and give the user the option to do the matching for other fields. For example I have installed the ISBN extraction plugin and Words count plugin or I had already in Calibre a custom column "Entry Type" which corresponds now to the ZItemType. I have attached a file to give you an idea of what I mean. [1] Have you read the ToolTips by hovering your mouse over each object in the ZMI window? They explain everything. Also, have you analyzed the zmi_overview.jpg attachment to the Original Post? It also explains everything simultaneously. [2] You can update 10,000+ books simultaneously and automatically. You just have to select a CSV file that has all 10,000+ Zotero book metadata, and select all of their corresponding 10,000+ Calibre books. If you select "all Calibre books", only those with a corresponding entry in the Zotero CSV file will (or can) can be updated. If you read the various ToolTips, you will see that changing the Calibre Author or Calibre Title prior to finalizing the automatic Zotero metadata update process has negative consequences because of the profound changes in the Calibre ebook paths. Yes, I have read it all but still does't work. I don't know what I am doing wrong. I am proceeding like this: I have a folder with books and I import them into Calibre as well as into Zotero.
The same result when I import the books not from a random folder but from Zotero's storage folder. So what am I doing wrong? How does the matching between the Zotero entries and the Calibre entries work? With the help of which criterion? How does it find the right entries in Calibre? Last edited by excaliber; 06-23-2016 at 10:14 AM. |
|
|
|
|
06-23-2016, 11:12 AM
|
#12 | |||
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
@excaliber:
Quote:
The criterion is Zotero's Key in the CSV being matched against Calibre's Custom Column #zotero_key a.k.a. ZKey. If you mess with the Calibre ZKey column value for a book, it will no longer match the CSV...not good. The next release will add missing books by looking for Zotero Keys in the CSV that are not represented anywhere in the current Calibre Library. It will then invoked Calibre's calibredb command-line utility to add the files represented in the CSV field "File Attachments". See note #1 below. Quote:
Quote:
Perhaps you should create a new Calibre Library, and start from scratch importing the books from Zotero, and then running ZMI before touching the books at all. You can then move the books from your old Library to the new, 'uncorrupted' Library. Just be sure to set your Calibre > Preferences for merging duplicates properly first. Note #1: for the auto-add missing Keys to work, all of your Zotero books must be in the same Calibre Library for the same logical Zotero "library" equivalent. Otherwise, books would be deemed as missing even though you had previously added them and then moved them to another Library. Not good. DaltonST Last edited by DaltonST; 06-23-2016 at 11:50 AM. Reason: Changes |
|||
|
|
|
|
06-23-2016, 01:04 PM
|
#13 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
@excaliber:
The use of the upcoming auto-add via calibredb will result in a 100% match between the CSV and Calibre, since I fully control what calibredb does. It will add the Zotero Key as a Calibre Identifier for each book as it is added. The automatic metadata update will match on that if it exists. The Identifier zkey:xxxxxx will not exist for legacy books already in your Calibre Library. For the latter, the automatic matching still has to match on filenames being exactly the same. Unfortunately, Calibre truncates filenames when they get too long, which means that those books will have to be manually matched to the CSV file. There are also differences between Windows and Linux as regards Calibre filename restrictions. Truncation of the filename for non-auto-added books results in a 100% failure rate for automatic matching for updating the metadata from the CSV. Truncation is irrelevant when books are added via the upcoming auto-add via calibredb, since the books will be matched by the new Identifer zkey:xxxxxx which will be added by calibredb automatically and immediately. You might seriously consider "starting fresh" with a new Calibre Library for Zotero, and populating it exclusively using the upcoming auto-add process DaltonST |
|
|
|
|
06-23-2016, 04:48 PM
|
#14 |
|
Deviser
Posts: 2,265
Karma: 2090983
Join Date: Aug 2013
Location: Texas
Device: none
|
Version 1.0.3 Posted
Version 1.0.3 - 2016-06-22 New: Detects Keys for Zotero books that do not exist anywhere in Calibre in the ZKey (#zotero_key) Custom Column and also not anywhere in the Identifiers table, and offers to run calibredb to auto-add them. Calibredb then adds the book while also adding an Identifier of zkey with a value of the CSV key to the Calibre book. Automated matching of the Identifier zkey with the CSV to automatically update metadata from Zotero is then possible.
See the attached explanatory image. An important note: The Zotero CSV file field "file attachments" must correspond exactly to the path of your Zotero books (file attachments). If you get virtually a zero % success rate in automatically adding the Zotero books to Calibre, you should strongly suspect that the data in the CSV field "file attachments" does not exactly equal the path of your Zotero books in your OS. Check for non-displayable characters in the CSV that of course do not exist in the paths of the books, making finding the books via the CSV file attachments field impossible. DaltonST |
|
|
|
|
06-23-2016, 07:47 PM
|
#15 |
|
Connoisseur
Posts: 59
Karma: 10
Join Date: Nov 2013
Device: Samsung Galaxy Tab 2 10.1 P5110
|
@DaltonST:
It matches by finding a Calibre book Path with a book filename that is identical to the Zotero file attachment filename. It ignores everything in the Paths except the final filename. If you change the Calibre filename in any way, it will no longer match. Hence, the strong warning about not changing Calibre Author or Title prior to completing the ZMI update. Calibre is renaming the files when I import them. That's why it didn't work. After your hint I tried this: Imported some files into Calibre. I then imported the associated files the Library directory of Calibre (they were already renamed) into Zotero. Then I I followed the procedure I mentioned before and it worked. Then I did something I didn't know it was possible - I did drag'n'drop some files from Calibre to Zotero. This has also worked and and it's much easier... It doesn't bother me if it's working like this, I only found weird that I have first to import files into Calibre and then into Zotero and then back to Calibre. Is this the normal procedure? By definition those 2 plug-ins are hardcoded to do only a single thing (ISBN and Word Count/Pages) by reading the .pdf text, and parsing it. OK. But why are you mentioning this? Did you see the attached screenshots? I mentioned these plugins as an example - a user can have already other fields and columns which could theoretically then assigned to the Zotero fields. In the case of these plugins: ISBN and Pages. I had already other custom columns in my Calibre configuration before I start using Zotero which could then be assigned to Zotero fields. If there is no option to unite them then they can appear twice in the Calibre columns. To me it doesn't make sense but I'm happy with this version. Again, it's only a suggestion... The fact that I'm getting 2 columns in the details view is because of Calibre or ZMI? Or why in the 2nd column only the values are showing up? Last edited by excaliber; 06-23-2016 at 08:10 PM. |
|
|
|
 |
| Tags |
| ris, zotero |
«
Previous Thread
|
Next Thread
»
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| [GUI Plugin] KindleUnpack - The Plugin | DiapDealer | Plugins | 495 | 10-19-2024 07:06 AM |
| [GUI Plugin] Consolidate All Library Metadata (CALM) | DaltonST | Plugins | 333 | 12-15-2023 09:40 PM |
| [GUI Plugin] OPML Importer (RSS feeds from feedly/google reader) | fem | Plugins | 14 | 07-19-2014 11:41 AM |
| Getting custom metadata field in GUI plugin | SauliusP. | Development | 7 | 02-11-2012 02:31 AM |
| [GUI Plugin] Plugin Updater **Deprecated** | kiwidude | Plugins | 159 | 06-19-2011 01:27 PM |
All times are GMT -4. The time now is 06:50 PM.