Full-Text Search Not Returning Results for Specific Phrases

Cwluc · Yesterday, 12:14 PM

Hello all, I’ve intentionally added everything to my digital library—it’s all in one library, with no virtual libraries. I currently have almost 100,000 books in it. I’ve been using the full-text search for quite some time, but today I found that something in my library wasn’t searchable via text.

I was searching for a very specific phrase from a specific work, and the search results came up empty. The phrase I was searching for is something exceptionally unique that I know only exists in one work, such as “purple skies at midnight over electric seas.” When I search for it, it doesn’t show up, but when I open the work, I can see it right there.

I’m trying not to stress, but now I’m wondering how often this happens. Is there a limitation with full-text search that I’m not aware of? I’m running the latest version of Calibre on Windows 11.

Has anyone else experienced this issue before? Is there any way to resolve it?

Thank you for your help.

kovidgoyal · Yesterday, 12:22 PM

There is no limitation most likely there is some specific reason that book or phrase could not be indexed, hard ot say without access to the book. Typically it would be a PDF file with images of text.

theducks · Yesterday, 02:44 PM

Would soft hyphens muck with the search?

ownedbycats · Yesterday, 03:19 PM

Quote:

Originally Posted by kovidgoyal

Typically it would be a PDF file with images of text.

OCR'd PDFs can also be a problem due to the OCR process being less-than-perfect (0/O, I/l).

DNSB · Yesterday, 04:26 PM

If I remember correctly, Kovid modified FTS to remove soft hyphens back in July, 2022.

Cwluc · Yesterday, 04:38 PM

Okay, thank you all for the replies. Apologies, I forgot to mention the file types. The files in question are all EPUBs, so the full-text search shouldn’t be causing any issues. Most of my library is EPUBs, followed by MOBI/AZW. The other formats (PDF, TXT, HTML, etc.) make up less than 1%, and I haven’t really used them in years.

I have the file and can share it, but I don’t want to be shamed for it. May I PM you the file, Kovid?

ownedbycats · Yesterday, 06:45 PM

On the Launchpad tracker there's an option to flag a new ticket as 'security vulnerability' to make it private. I've done it when reporting bugs related to copyrighted files.

BetterRed · Yesterday, 07:02 PM

@Cwluc - if you wanted a "second opinion" you could install an EPUB IFilter and search for the same term in Windows Search.

See Content Search EPUB

BR

kovidgoyal · Yesterday, 10:19 PM

Opena bug report, mark it private and attach your file there.

Cwluc · Today, 12:32 AM

Well, this is awkward. I noticed the issue this morning before work, so I tried closing and reopening the app, but it still wasn’t working. That’s why I posted about it and updated throughout the day. Now that I’m back home, everything’s working fine, like nothing was ever wrong. So, at the moment, I’m all good. Thanks for the replies and assistance. If the issue pops up again, I’ll pick up from there.

ownedbycats · Today, 03:02 AM

Was the book newly-added? Perhaps the indexing stalled for some reason.

Yesterday, 12:14 PM	#1
Cwluc Member Posts: 17 Karma: 10 Join Date: Dec 2009 Device: Various	Full-Text Search Not Returning Results for Specific Phrases Hello all, I’ve intentionally added everything to my digital library—it’s all in one library, with no virtual libraries. I currently have almost 100,000 books in it. I’ve been using the full-text search for quite some time, but today I found that something in my library wasn’t searchable via text. I was searching for a very specific phrase from a specific work, and the search results came up empty. The phrase I was searching for is something exceptionally unique that I know only exists in one work, such as “purple skies at midnight over electric seas.” When I search for it, it doesn’t show up, but when I open the work, I can see it right there. I’m trying not to stress, but now I’m wondering how often this happens. Is there a limitation with full-text search that I’m not aware of? I’m running the latest version of Calibre on Windows 11. Has anyone else experienced this issue before? Is there any way to resolve it? Thank you for your help.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Feature Request: Text search highlight results	Tenga	KOReader	0	12-09-2023 02:12 PM
Enhance Full Text Search Results + Questions	mikhail_fil	Library Management	2	07-24-2023 09:08 PM
Calibredb --search ondevice:true not returning any results	Meelibomin	Library Management	2	10-10-2022 06:08 PM
Text Search results list not scrolling	BetterRed	Calibre	2	07-18-2022 07:17 PM
Google Book Search to search full-text books online	Bob Russell	Deals and Resources (No Self-Promotion or Affiliate Links)	1	08-19-2006 12:13 PM

Yesterday, 12:22 PM	#2
kovidgoyal creator of calibre Posts: 44,319 Karma: 23661992 Join Date: Oct 2006 Location: Mumbai, India Device: Various	There is no limitation most likely there is some specific reason that book or phrase could not be indexed, hard ot say without access to the book. Typically it would be a PDF file with images of text.

Yesterday, 02:44 PM	#3
theducks Well trained by Cats Posts: 30,355 Karma: 58032210 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Would soft hyphens muck with the search?

Yesterday, 04:26 PM	#5
DNSB Bibliophagist Posts: 39,293 Karma: 153111226 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos	If I remember correctly, Kovid modified FTS to remove soft hyphens back in July, 2022.

Yesterday, 04:38 PM	#6
Cwluc Member Posts: 17 Karma: 10 Join Date: Dec 2009 Device: Various	Okay, thank you all for the replies. Apologies, I forgot to mention the file types. The files in question are all EPUBs, so the full-text search shouldn’t be causing any issues. Most of my library is EPUBs, followed by MOBI/AZW. The other formats (PDF, TXT, HTML, etc.) make up less than 1%, and I haven’t really used them in years. I have the file and can share it, but I don’t want to be shamed for it. May I PM you the file, Kovid?

Yesterday, 06:45 PM	#7
ownedbycats Custom User Title Posts: 9,259 Karma: 63194753 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	On the Launchpad tracker there's an option to flag a new ticket as 'security vulnerability' to make it private. I've done it when reporting bugs related to copyrighted files.

Yesterday, 07:02 PM	#8
BetterRed null operator (he/him) Posts: 20,920 Karma: 27620686 Join Date: Mar 2012 Location: Sydney Australia Device: none	@Cwluc - if you wanted a "second opinion" you could install an EPUB IFilter and search for the same term in Windows Search. See Content Search EPUB BR

Yesterday, 10:19 PM	#9
kovidgoyal creator of calibre Posts: 44,319 Karma: 23661992 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Opena bug report, mark it private and attach your file there.

Today, 12:32 AM	#10
Cwluc Member Posts: 17 Karma: 10 Join Date: Dec 2009 Device: Various	Well, this is awkward. I noticed the issue this morning before work, so I tried closing and reopening the app, but it still wasn’t working. That’s why I posted about it and updated throughout the day. Now that I’m back home, everything’s working fine, like nothing was ever wrong. So, at the moment, I’m all good. Thanks for the replies and assistance. If the issue pops up again, I’ll pick up from there.

Today, 03:02 AM	#11
ownedbycats Custom User Title Posts: 9,259 Karma: 63194753 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	Was the book newly-added? Perhaps the indexing stalled for some reason.