[GUI Plugin] Consolidate All Library Metadata (CALM)

[DaltonST]

Version 2.0.1 - 2015-11-20 Metadata Tools "Delete Non-ISBN/ISSN Identifiers" function changed to also not delete any Identifiers created by the Library Codes plugin (currently: viaf_author_id,lc_authority_name,lccn,isni, oclc, oclc-owi). The pushbutton and other text were changed to say "Non-Library Codes" instead of "Non-ISBN/ISSN Identifiers" .

As I enhance my 'Library Codes' plugin in the future, I will make corresponding changes to CALM.

I have not made corresponding changes to my 'QuarantineAndScrub' add-on, as you should totally finish Scrubbing and have copied 'Work to Real' prior to using 'Library Codes' to derive any new Identifiers. Until totally Scrubbed, the Authors and Titles are not worthy of being used by 'Library Codes' to derive DDC/LCC/etc. using Author/Title, nor are the ISBNs properly converted from ISBN-10 to ISBN-13, properly formated, and properly located in the Identifiers column rather than as part of the Author, Title,Tags, Series or whatever other columns contained them prior to running Book Level Scrubbing. So, Q&S will continue to delete non-ISBN/ISSN/OCLC-OWI Identifiers without regard to 'Library Codes' Identifiers. Preferably, move your newly scrubbed books to your Pristine Library prior to using 'Library Codes'.




DaltonST

[DaltonST]

Version 2.0.2 - 2015-12-02 Menu function to delete "Non-Library Codes Identifiers" changed to also not delete new LC Identifier "loc_lccn".


If you use 'CALM' and also use 'Library Codes', you will want to upgrade sooner rather than later. The loc_lccn Identifier is used to hot-link to the Library of Congress 'permalink' for a book, which is a wonderful source of both author and book information.

Example: http://lccn.loc.gov/2010013227



DaltonST

[elricks]

Good afternoon. My issue = how do I add a newly created library to my exsiting CALM library?

I have looked for the 'add all' button, but cannot find it. I 'must' have found it when I originally set CALM up, but I can't find it now.

I tried deleting everything to start again (I am mostly using it to make sure I don't have a book in another library before adding it - I have not started to look into the editing between libraries yet, but I want to). The button 'Delete the entire library directory' seems to work, but I still have the original CALM library intact.

Thanks

SHIRLEY

[Terisa de morgan]

@DaltonST, I have a question. I'm trying to manually activate/deactivate columns but I don't find the way to do it, because I cannot check/uncheck the square and changing the value from 1/0 and the other way does nothing.

[DaltonST]

Originally Posted by elricks

Good afternoon. My issue = how do I add a newly created library to my exsiting CALM library?

I have looked for the 'add all' button, but cannot find it. I 'must' have found it when I originally set CALM up, but I can't find it now.

@elricks:

Once you have 'Generated Custom Columns', the list of your chosen "Source Libraries" on the "Source Library" Tab is frozen.

If you want to change the list of "Source Libraries" once you have 'Generated Custom Columns', you must click the "Extract a Fresh Copy of the Target Library" pushbutton on the "Target Library" Tab.

The pushbuttons come-and-go based upon what is valid to do at a specific time. What is valid to do is based upon the "Consolidation Target Library Status" groupbox information on the "Target Library" Tab.

In summary, a particular CALM Target Library is temporary and highly disposable. If you want to change the "Source Libraries" to be Consolidated into the Target Library, or change the "Source Custom Columns" that you want mapped into the Target Library, you must first extract a fresh copy of the Target Library via the appropriate "Target Library" Tab pushbutton.


DaltonST

[DaltonST]

Originally Posted by Terisa de morgan

@DaltonST, I have a question. I'm trying to manually activate/deactivate columns but I don't find the way to do it, because I cannot check/uncheck the square and changing the value from 1/0 and the other way does nothing.

@Terisa de morgan:

Easy to do. You must "Select" the row to be changed by clicking it. It will change color. Change the Yes/No value, then immediately click the button-icon in the middle of the left side to "Add" the change to the current list of changes to be made after you "Save and Exit". Then, click the button-icon at the bottom to save the changes and exit.


DaltonST

[Terisa de morgan]

Thank you, I was missing a point: change the value, leave the row so the new value is fixed, come back and then press the icon. And it isn't changed at the list till you save and exit.

It's working now, thank you.

[elricks]

Thanks - worked a treat.

SHIRLEY

[Tanjamuse]

Can I have more than one CALM library?

[DaltonST]

Originally Posted by Tanjamuse

Can I have more than one CALM library?

@Tanjamuse:

Yes and No. Depends on the definition of "more than one", which really boils down to a timing issue.

Yes: You can create CALM Target Library #1 for up to 50 different Source Libraries, consolidate it, and then create CALM Target Library #2 with a different path, with up to 50 Source Libraries, and then consolidate it. You can quick-switch between #1 and #2. And so forth and so on.

No: You can have only one (1) CALM Target Library at a time that you are actively Generating and Consolidating.

Warning: if you use the Library Tools for the identical Source Library in both CALM #1 and CALM #2, you may shoot yourself in the foot. Don't do that. The Library Tools functionality keeps track of what it has done, but only for its own CALM Target Library.

I recommend having only a single "active" CALM Target Library. Since it holds up to 50 Source Libraries, that should not be a limitation.

In other words, you can have a lot of old "read-only" CALM Target Libraries, but only one (1) in which you are actively changing the data. Just use those old "read-only" CALM Target Libraries for research and review, but don't run the CALM Plugin while in them as that would only be to make changes via Metadata Tools and Library Tools.

By the way, you do not have to actually consolidate all of the Source Libraries at one time. Just because you Generated them, and hence made them available for Consolidation, does not mean that you have to consolidate them. You cannot consolidate a Source Library that was not Generated, but nothing makes you consolidate everything that was already Generated.

Finally, remember that all CALM Target Libraries are temporary, transient and highly disposable. Just Refresh and start over. Easy come, easy go.


DaltonST

[rolandt99]

Hello and good day.

Before I jump into the fray, I would like to say thank you DaltonST for all of your great plugins.

I have multiple libraries and would really like to use CALM to have a consolidated view of a group of those libraries. However, I have encountered a few data import and generation issues that mean I cannot use CALM (as is).

All of my libraries have a consistent set of 10 custom columns defined.

The first problem 1) I encounter is when I attempt to activate and generate using custom columns from more than one library. I can do a successful generation when I have any or all custom columns activated for a SINGLE library. However, generation fails every time when I activate the custom columns for two or more libraries.

The second and third problems are with data that gets imported (if that is the correct term) into the generated custom columns.

2) Three of my custom columns are type "Column built from other columns". ALL data imported for all of those columns seems to be taken from the standard ISBN column. So in the CALM library I have multiple columns with the same ISBN value and not the values that should be there. One of my custom columns is for book formats - and an ISBN is certainly not a format type....

3) Another of my custom columns is type "Text column for keeping series-like information" which I use to track publisher sequence numbers. The correct p-series text name value is imported into the CALM library however all p-series index values are set to 1.

I will gladly provide whatever test/diagnostic type information needed to help debug these issues.

Thank you in advance for your help resolving these issues (if that is possible),
Roland

[DaltonST]

@rolandt99:

Version 2.0.5 Answer to your Questions #1 and #2:

You are probably getting Template Errors when you switch to the CALM Target Library that you just Generated. I assume that is what you mean by "generation fails". Composite Custom Columns are problematic because of their Templates. Working on that issue for the next release of CALM is on my to-do list.

2 Possible Immediate Work-Around Solutions to choose from:

[Choice #1] There is a pushbutton on the "Source Custom Columns" tab named "Mass Deactive All Composite Custom Columns (Only)". You need to click that prior to Generating. Then, manually create the desired Composite Custom Columns directly in the CALM Target Library. Since they are "columns built from other columns", their underlying real data will already be there.

[Choice #2] Manually change the templates for all of the Generated Composite Custom Columns directly in the CALM Target Library. I suggest you do this AFTER Generation BUT BEFORE Consolidation. You will get few-to-no error messages about Template Errors, since there will be no "books" in the CALM Target Library to cause them. Then, run a Consolidation.


As I mentioned, this issue is on my to-do list.


Version 2.0.5 Answer to your Questions #3:

The Custom Column type "Text column for keeping series-like information" requires special handling for the index, which I forgot because my many test libraries I used for developing CALM did not have one of that special type. Sorry about that. I will add that for the next release.


DaltonST

[rolandt99]

DaltonST,

Thank you for the quick response.

No, by 'generation fails' I mean the generation process completely fails/stops. Usually with this error:
calibre, version 2.46.0
ERROR: CALM: GENERATION OF CUSTOM COLUMNS TERMINATED PREMATURELY........ERROR: >>>>[Error 6] The handle is invalid

This error was when I attempted to test the first step of your [Choice #1] and run the Generate process after "Mass Deactivate All Composite Custom Columns (Only)". So the Generate process was "Running Calibredb to Create 7 New Custom Column(s)" from two source libraries with NO composite columns. Note - the follow-on error message states to attach a debug log. Where would I find this debug log?

I look forward to testing the next release with the series-like custom column index.

thanks,
Roland

[DaltonST]

@rolandt99:

Are you using OSX? Or Windows 7 as a standard user? See the Original Post. They are the only reason I know of that calibredb will fail while creating new custom columns.

DaltonST

[rolandt99]

Hmm. Yes, I run Windows 7. My error - since my login ID has administrator rights I erroneously assumed that gave the required permissions. The Generation process of 7 custom columns for 2 libraries runs to completion starting calibre "as administrator".

Interestingly, doing the Generation process with the exact same set of custom columns but just one source library runs correctly without needing the UAC permissions.

Roland