Download Metadata failing

[JohnnyBook]

I reciently added about 25 ebooks to my calibre library, and added placeholders for about 50-60 books that are on my want to buy list.

Thus adding about 70-80 entries in Calibre. When I started going through them and getting metadata for them everything was going fine, then maybe half way through, I stopped getting any data for any book entries.

Despite the books being listed on Amazon and Fantastic Fiction.

Here is an example of a failed search:
The book was "CASTLE HILLS FALL 2" by Stephen Mezzomo asin:B0CN2JLHZH

[EDIT: Apparently I has the asin for the book #1 in the search (B0CNN62B4W), NOT the second book (B0CN2JLHZH), but that should not have stopped the author/title search.]

calibre, version 7.16.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{'title': 'Castle Hills Fall', 'authors': ['Stephen Mezzomo'], 'identifiers': {'mobi-asin': 'B0CNN62B4W'}, 'timeout': 30}
Using plugins: Fantastic Fiction (1, 7, 0), Google (1, 1, 1), Amazon.com (1, 3, 9)
The log from individual plugins is below

****************************** Fantastic Fiction (1, 7, 0) ******************************
Found 0 results
Downloading from Fantastic Fiction took 0.7562577724456787
Querying: https://www.fantasticfiction.com/dbs...Cdb%2Cimageloc
JSON Result: b'{"status":{"rid":"no329pMy6LAxCtRBow==","time-ms":1},"hits":{"found":1,"start":0,"hit":[{"id":"w1321412","fields":{"booktype":"1","title": "Castle Hills Fall 2","pfn":"series/in-the-october-fall-world/castle-hills-fall-2.htm","year":"2024","authorsinfo":"|Stephen Mezzomo|146740|FF","seriesinfo":"In the October Fall World|2|in-the-october-fall-world","db":["FF"]}}]}}'
FantasticFiction url: 'https://www.fantasticfiction.com/series/in-the-october-fall-world/castle-hills-fall-2.htm'
No authors
Could not find title/authors/Fantastic Fiction id for 'https://www.fantasticfiction.com/series/in-the-october-fall-world/castle-hills-fall-2.htm'
Fantastic Fiction: 'series/in-the-october-fall-world/castle-hills-fall-2' Title: 'Castle Hills Fall 2' Authors: None

************************************************** ******************************

****************************** Google (1, 1, 1) ******************************
Found 0 results
Downloading from Google took 3.433234691619873
Making query: https://books.google.com/books/feeds...ewability=none
No results found, doing a web search instead
Making query: https://www.google.com/search?q=Cast...ezzomo&tbm=bks
Plugin Google failed
Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\identify.py", line 47, in run
File "calibre\ebooks\metadata\sources\google.py", line 536, in identify
File "calibre\ebooks\metadata\sources\google.py", line 441, in identify_via_web_search
File "contextlib.py", line 155, in __exit__
File "calibre\ebooks\metadata\sources\search_engines.py ", line 58, in rate_limit
File "calibre\ebooks\metadata\sources\search_engines.py ", line 100, in query
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 429: Too Many Requests

************************************************** ******************************

****************************** Amazon.com (1, 3, 9) ******************************
Found 0 results
Downloading from Amazon.com took 6.381892204284668
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
Server: auto
Making google query: https://www.google.com/search?q=B0CN...zon.com&nfpr=1
Got too many requests error from Google, trying via DuckDuckGo
Making ddg query: https://duckduckgo.com/html/?q=B0CNN...azon.com&kp=-1
No search engine results for terms: B0CNN62B4W
No matches found with identifiers, retrying using only title and authors. Query: ['B0CNN62B4W']
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
Server: auto
Making google query: https://www.google.com/search?q=Cast...www.amazon.com
Got too many requests error from Google, trying via DuckDuckGo
Making ddg query: https://duckduckgo.com/html/?q=Castl...azon.com&kp=-1
No search engine results for terms: Castle Hills Fall Stephen Mezzomo
No matches found with query: ['Castle', 'Hills', 'Fall', 'Stephen', 'Mezzomo']

************************************************** ******************************
The identify phase took 6.42 seconds
The longest time (6.381892) was taken by: Amazon.com
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds


For FF if I use the https search line used for the search it DOES come to the correct page, but the plugin returns no author.

For Amazon: using the same https search line DOES come back to valid pages.
It does say: Got too many requests error from Google

IS there a limit on requests it allows from Calibre?

Same with DuckDuckGo. Is there a request limit?

Thanks,
Johnny

P.S. As I finished typing this an hour after the failure, (took a meal break) The search worked, and I found the book. Really seems like I was hitting some sort of search request limiter?

[kovidgoyal]

The google servers have blocked your computer for making too many requests. Just leave it for a few hours and try again and it will work. Or use a vpn to change your IP address.

[DNSB]

I gave up on bulk metadata since all too often the results were incorrect and even when the data was correct, multiple requests over a short period would get you a suspected bot tag and your IP address being blocked for 15 minutes and up. For that, your series of Too Many Requests errors are a red flag. My workaround is to use Edit Metadata Individually and not being in a rush.

[JohnnyBook]

Originally Posted by DNSB

I gave up on bulk metadata since all too often the results were incorrect and even when the data was correct, multiple requests over a short period would get you a suspected bot tag and your IP address being blocked for 15 minutes and up. For that, your series of Too Many Requests errors are a red flag. My workaround is to use Edit Metadata Individually and not being in a rush.

I never use bulk, because the few times I did use it years ago, there were to many errors.
I only do it individually

[DNSB]

In that case, go to my comment about not being in a rush since I have managed to trigger an IP block even in individual mode by being in too much of a rush.