SSL errors when downloading metadata

[mbovenka]

For a while now metedata download is very spotty, an looking at the logs I see loads of there errors:

[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

A bit of googling tells me that has something to do with the Python certificate store missing certain certs or certs being invalid, but all of the solutions assume I'm running a full Python, which in the case of Calibre I'm obviously not.

Anyone having an idea about how to to fix this?

[kovidgoyal]

Assuming you are on windows visit the site in question using the edge browser to get the certificates installed.

[mbovenka]

Thanks, Kovid. I'm on Windows, yes. I should have mentioned that, sorry. I visited my metadata sources (BOL NL, Amazon.com, Fantastic Fiction & Goodreads) with Edge. and restarted calibre afterwards. BOL & Amazon work, but both Goodreads & Fantastic Fiction still give me the following error:

Spoiler:

****************************** Fantastic Fiction (1, 7, 1) ******************************
Found 0 results
Downloading from Fantastic Fiction took 0.33136820793151855
FantasticFiction url: 'https://www.fantasticfiction.com/a/avery-aames/to-brie-or-not-to-brie.htm'
Failed to make details query: 'https://www.fantasticfiction.com/a/avery-aames/to-brie-or-not-to-brie.htm'
Traceback (most recent call last):
File "mechanize\_urllib2_fork.py", line 1236, in do_open
File "http\client.py", line 1286, in request
File "http\client.py", line 1332, in _send_request
File "http\client.py", line 1281, in endheaders
File "http\client.py", line 1041, in _send_output
File "http\client.py", line 979, in send
File "http\client.py", line 1458, in connect
File "ssl.py", line 517, in wrap_socket
File "ssl.py", line 1108, in _create
File "ssl.py", line 1379, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre_plugins.fantastic_fiction.worker", line 42, in get_details
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 287, in _mech_open
File "mechanize\_opener.py", line 193, in open
File "mechanize\_urllib2_fork.py", line 425, in _open
File "mechanize\_urllib2_fork.py", line 414, in _call_chain
File "calibre\utils\browser.py", line 29, in https_open
File "mechanize\_urllib2_fork.py", line 1240, in do_open
urllib.error.URLError:

************************************************** ******************************

[kovidgoyal]

Goodreads is having issues see the goodreads plugin thread about it. No clue about bol.nl. You can bypass the windows certificate store by setting the SSL_CERT_FILE env var to point to the mozilla-ca-certs.pem from the calibre installation folder. This will cause OpenSSL and hence python to use certificates from that file (which is the set of certificates Firefox uses) instead of the OS cerificate store.

[mbovenka]

Curiouser and curiouser... I set the variable (I'm seeing it set in both CMD and the GUI) but I'm still getting the same error for Goodreads and Fantastic Fiction. I'm using Portable, if it matters.

[kovidgoyal]

Works for me with

SSL_CERT_FILE=C:/Program\ Files/Calibre2/app/resources/mozilla-ca-certs.pem calibre-debug -c "from calibre import browser; br = browser(); print(br.open('https://goodreads.com').geturl())"

[Comfy.n]

For what it may be worth, I did some searching and found a bunch of recent reports of the same SSL error 1006 over on the web:

https://learn.microsoft.com/en-us/an...1-got-sslerror

https://issuetracker.google.com/issues/369237337

https://issuetracker.google.com/issues/355359279

https://www.reddit.com/r/Calibre/com...n_not_working/

Also, not sure if related:

https://www.ibm.com/support/pages/se...e-2024-37891-0

https://exchange.xforce.ibmcloud.com...ilities/295053

[mbovenka]

Originally Posted by kovidgoyal

Works for me with

SSL_CERT_FILE=C:/Program\ Files/Calibre2/app/resources/mozilla-ca-certs.pem calibre-debug -c "from calibre import browser; br = browser(); print(br.open('https://goodreads.com').geturl())"

Ah, ' CERT', not ' CERTIFICATE'. Yep, it works now. Thanks!

[mbovenka]

Originally Posted by mbovenka

Ah, ' CERT', not ' CERTIFICATE'. Yep, it works now. Thanks!

And I found the problem: corrupted system files. SFC /SCANNOW found errors and fixed them. Afterwards it worked without the environment variable, using the Windows cert store.

[kovidgoyal]

Too many people are getting affected by this, the windows cert store seems very fragile. SO next release I will have calibre default to using its bundled certs with an env var to get it to use system certs instead. It already defaults to its own certs on macos