09-12-2024, 03:36 AM | #1 |
Zealot
Posts: 108
Karma: 2029154
Join Date: Sep 2013
Location: Pacific Northwest
Device: iPad Mini, iPhone 12, Kindle Paperwhite 3
|
Template to search notes?
I realize this is probably a really niche thing, but is there any way to search a notes field (e.g., notes attached to an author) for specific text? I want to build a custom column that shows me a check mark if particular text is included in an author note.
I have a number of custom columns built from other columns that do something similar, but since the notes aren't fields with lookup names, I'm not even sure if it's possible to do what I want. Currently I'm using a workaround that involves adding a specific tag to every book by an author whom I want to flag, but that's both time-consuming and easy to forget about when adding a new book, whereas if I can just make a single note for the author it would be so much easier. TIA! |
09-12-2024, 05:31 AM | #2 | |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
Quote:
Currently the way I see to do it is using the Notes Browser with its FTS-like search syntax, like this: In that example I've searched for authors born in 9/11. Default shortcut is Ctrl + Shift + N. However I can't tell how to bring all results from that search into "main Calibre", so that I could bulk add the "Born in 9/11" tag, for instance. |
|
Advert | |
|
09-12-2024, 07:49 AM | #3 | |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
You can search a note using a template like this one, suitably modified. Code:
program: first_author = sublist($authors, 0, 1, '&'); note = get_note('authors', first_author, '1'); if 'something' in note then ans = 'Yes' else ans = 'No' fi; ans
Depending on what you want to do, there are two ways to proceed.
Last edited by chaley; 09-12-2024 at 08:25 AM. |
|
09-12-2024, 08:16 AM | #4 | |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
Quote:
I get "EXCEPTION: string indices must be integers, not 'str'" in the Template Editor. I'm trying with: Comparison type - Text, and pasting the code in field Template, while leaving the field Template Value empty. |
|
09-12-2024, 08:24 AM | #5 | |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Code:
get_note('authors', first_author, '') |
|
Advert | |
|
09-12-2024, 08:54 AM | #6 | |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Code:
get_note('authors', first_author, 1) |
|
09-12-2024, 09:29 AM | #7 |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
It works in the template editor, showing "Yes" for a given value when a book whose author's notes contain the string is currently selected.
However when I paste the template in the search bar, it returns no results, so I must be doing something the wrong way Last edited by Comfy.n; 09-12-2024 at 09:32 AM. |
09-12-2024, 09:42 AM | #8 | |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Template: Code:
program: first_author = sublist($authors, 0, 1, '&'); note = get_note('authors', first_author, '1'); if 'a' in note then ans = 'Yes' else ans = '' fi; ans Note that the template search generated by the dialog contains more than just the template. Last edited by chaley; 09-12-2024 at 09:44 AM. |
|
09-12-2024, 10:45 AM | #9 | |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
Quote:
Now I'm trying to fix my advanced search issue, hopefully I can come up later with more detailed info. I think my notes database has grown too much lately, and when I do the search as indicated in your screenshot for Advanced Search, Calibre enters some seemingly infinite database lookup. This has happened other times and I solved it by recreating my Bio column. (deleting it, then re-adding). Today this didn't seem to solve the issue. Process Monitor shows this: |
|
09-12-2024, 06:53 PM | #10 | |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
This python template is optimized for the search problem. When it is first called (the first book) it gets all the note values and caches them using more optimized API calls. Then for each subsequent book it checks the cache, not the database. It would be best done as a stored template so the field and value could be passed as arguments. I didn't bother to do that because I don't know if this solves your problem. The template: Code:
python: def evaluate(book, context): # Set these to what you want field_name = 'authors' search_value = 'aaa' db = context.db.new_api # check if we have already cached the notes note_items = context.globals.get('items_with_notes', None) if note_items is not None: # We have. Get the cached note values note_values = context.globals['note_values'] else: # We haven't. Cache the note item ids and their values # First get all the item ids with notes and cache the result note_items = db.get_all_items_that_have_notes(field_name) context.globals['items_with_notes'] = note_items # Now get the note values for each item id with a note note_values = {} for note_item in note_items: note = db.notes_data_for(field_name, note_item) if note: # Get the plain text of the note note = note['searchable_text'].partition('\n')[2] # Put the value of the note into the cache. note_values[note_item] = note # Write the cached values to the globals context.globals['note_values'] = note_values # Check if this book is a match -- the field has a note containing the right text # get the item_id for the value of the desired field fv = book.get(field_name) # if the field is multi-valued, use the first value if isinstance(fv, list): fv = fv[0] # Now get the internal ID of the value in field_name item_id = db.get_item_id(field_name, fv) # Return the empty string if the item doesn't have a note if item_id not in note_items: return '' # Get the note value from the cache val = note_values.get(item_id, None) if val is None: # This shouldn't happen, but ... return '' # use a case insensitive compare to check if the search value is in the note from calibre.utils.icu import primary_contains return 'Yes' if primary_contains(search_value, val) else '' |
|
09-12-2024, 08:34 PM | #11 | |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
Quote:
Thanks for taking your time to try and help! I had thought of asking for a stored template like "item_has_notes", but I think this is not the case at the moment. |
|
09-13-2024, 07:40 AM | #12 |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Moderator Notice
Moved to it's own thread |
09-13-2024, 07:45 AM | #13 |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
I've thought of a way to significantly speed up the search template, calculating the text search result once per author instead of for every book. I'll post the new version later today.
|
09-13-2024, 11:18 AM | #14 |
Grand Sorcerer
Posts: 12,097
Karma: 7908993
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Here is the improved template. I'd be interested if you think it is faster.
Code:
python: def evaluate(book, context): # Set these to what you want field_name = 'authors' value_in_note = 'a' db = context.db.new_api # check if we have already cached the notes note_items = context.globals.get('items_with_notes', None) if note_items is not None: # We've already fetched which items have notes. # Get the cached search result values note_search_results = context.globals['note_search_results'] else: # First time. Get the items with notes and initialize # the search value cache. note_items = db.get_all_items_that_have_notes(field_name) context.globals['items_with_notes'] = note_items note_search_results = {} context.globals['note_search_results'] = note_search_results # Check if this book is a match -- that the field has a note containing # the desired text. # We must first get the item_id for the value in the field to be checked. field_value = book.get(field_name) # if the field is multi-valued, use the first value if isinstance(field_value, list): field_value = field_value[0] # Now get the internal ID of the value in field_name item_id = db.get_item_id(field_name, field_value) # Does the item have a note? If not, give up now. if item_id not in note_items: return '' # The item has a note. Have we already checked it? if item_id not in note_search_results: # Item has a note but we haven't seen it before. Do the compare # on the plain text version of the note. result = '' # Get the note. note = db.notes_data_for(field_name, item_id) if note: # Get the plain text of the note. note = note['searchable_text'].partition('\n')[2] if note: # use a case insensitive compare to check if the search value is in the note from calibre.utils.icu import primary_contains result = 'Yes' if primary_contains(value_in_note, note) else '' # Cache the result of the comparison note_search_results[item_id] = result context.globals['note_search_results'] = note_search_results # Return the cached value return note_search_results.get(item_id, '') |
09-13-2024, 12:20 PM | #15 |
want to learn what I want
Posts: 1,353
Karma: 6874872
Join Date: Sep 2020
Device: Calibre E-book viewer
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Template Search: Exact matching | ownedbycats | Library Management | 3 | 04-03-2022 06:01 PM |
Template: Converting a search & replace into a template | ownedbycats | Library Management | 11 | 03-26-2021 05:32 AM |
Nova Pro : Notes Template | MachinaCarnis | Onyx Boox | 0 | 01-30-2020 02:57 PM |
template or search feature question | bulldogmo | Calibre | 2 | 08-06-2014 07:34 PM |