[GUI Plugin] FanFicFare

[JimmXinu]

We don't know what constitutes 'too many', 'too quickly', or how long it takes to cool down--I would assume day(s), not just hours.

It's also possible it's someone else 'nearby' in your network. Some people have had success rebooting routers to get new outside facing IP addresses or using VPNs.

I don't recall if FFF's Browser Cache feature works with AO3 or not.

And worst case, you can always download an epub or other ebook format from AO3, unlike a lot of other sites.

[ownedbycats]

Depending on your ISP, it's posisble that you got an IP address previously used by a bad actor. Also depending on your ISP, unplugging your modem for a bit and then reconnecting might get you a new one and help.

Some users did have some success with the cache, but that was when the site was being DDoS'd.

[vikarti]

How to fix issue with ficbook.net?
regular connection via CLI:
C:\Users\Vikarti Anatra>calibre-debug.exe --run-plugin FanFicFare https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4
Traceback (most recent call last):
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\fetcher_requests.py", line 128, in request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\requests\models.py", line 943, in raise_for_status
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\debug.py", line 289, in main
File "calibre_plugins.fanficfare_plugin.__init__", line 146, in cli_main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 346, in main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 322, in dispatch
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 437, in do_download
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 325, in getStoryMetadataOnly
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 429, in doExtractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\adapter_ficbooknet.py", line 80, in extractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\requestable.py", line 119, in get_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\requestable.py", line 111, in get_request_redirected
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\base_fetcher.py", line 133, in get_request_redirected
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\cache_basic.py", line 122, in fetcher_do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\decorators.py", line 102, in fetcher_do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\base_fetcher.py", line 106, in do_request
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\fetchers\fetcher_requests.py", line 149, in request
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF '403 Client Error: Forbidden for url: https://ficbook.net/readfic/018aa4e6...ad5d4'(403)

attempt to use flareslolver with modifications in personal.ini

use_flaresolverr_proxy:true
#run via docker desktop using docker run -d --name=flaresolverr -p 8191:8191 -e LOG_LEVEL=info flaresolverr/flaresolverr:latest
results in flaresolverr seeing icoming request, saying version_main cannot be converted to an integer and "challenge not detected"

results in
C:\Users\Vikarti Anatra>calibre-debug.exe --run-plugin FanFicFare https://ficbook.net/readfic/018aa4e6...9-8d2beb0ad5d4
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images
FFF: WARNING: 2024-06-01 12:37:42,287: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver
Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\debug.py", line 289, in main
File "calibre_plugins.fanficfare_plugin.__init__", line 146, in cli_main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 346, in main
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 322, in dispatch
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\cli.py", line 437, in do_download
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 325, in getStoryMetadataOnly
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\base_adapter.py", line 429, in doExtractChapterUrlsAndMetadata
File "C:\Users\Vikarti Anatra\AppData\Roaming\calibre\plugins\FanFicFare. zip\fanficfare\adapters\adapter_ficbooknet.py", line 237, in extractChapterUrlsAndMetadata
TypeError: 'NoneType' object is not subscriptable

browser doesn't see any challenges on ficbook.net

usign vpn doesn't help (same set of errors as before)
any advice would be anticipated.

starting flaresolver as docker run -d --name=flaresolverr -p 8191:8191 -e LOG_LEVEL=debug -e LOG_HTML=true flaresolverr/flaresolverr:latest
and connecting to containter for logs shows ficbook's html IS being here


using flaresloverr with fanfiction.net via fanficfare detects challenge correctly

[JimmXinu]

Originally Posted by vikarti

How to fix issue with ficbook.net?

Looks like the site has changed.

I've opened an issue (#1076) asking dbhmw, the dev who worked on it last, if they can look at it.

[JimmXinu]

New Version Released

Version 4.35.0 - 2024-06-01
- Add download_finished_signal for Action Chains #1073
- New Site: touchfluffytail.org thanks dbhmw (#1071)
- adapter_wattpadcom: Improve error reporting when story not found / connection refused.
- base_xenforo2 better detect whether logged in.
- adapter_deviantartcom: Streamline login vs watchers vs mature See #1070
- adapter_inkbunny: Fix author & category thanks dbhmw (#1069)
- base_xenforo2forum: Prefix tag collecting too much.
- datetime.utcnow() deprecated in more recent py3 versions
- Skip invalid images, detect img types thanks dbhmw (#1068)
- Fix for WebToEpub firefox cache key changing
- QQ: Doesn't need reader_posts_per_page:30 anymore
- adapter_forumquestionablequestingcom:Switch to BaseXenForo2ForumAdapter
- adapter_ficbooknet: Remove py3 string handling that breaks on py2
- Update bundled certifi cacert.pem and version, not core.py

Note that ficbook.net has changed and isn't fixed yet in this version.

[JimmXinu]

Originally Posted by JimmXinu

Looks like the site has changed.

I've opened an issue (#1076) asking dbhmw, the dev who worked on it last, if they can look at it.

Turns out I was wrong about the site changing.

dbhmw pointed out in #1076 that it has to do with using a proxy vs direct or using Browser Cache.

[julo]

For SOL FFF doesn't add 'Details' any more. Can FFF be modified to add it again or if that fails, can that be added via personal.ini and if so how?

[JimmXinu]

No idea what you mean by "doesn't add 'Details' any more". Looks like it's reading metadata to me.

[vikarti]

btw, is it possible to make FanFicFare use Ficbook.net's premium subscription if user have it?(site does allow direct epub downloading in user does have subscription but it's not very convinient to check for updates, click, re-add to calibre and so on)

[JimmXinu]

No, the adapter for ficbook doesn't know how to login.

[julo]

Originally Posted by JimmXinu

No idea what you mean by "doesn't add 'Details' any more". Looks like it's reading metadata to me.

At the beginning of an FFF download the 'Details' included title, author, genre, etc. with tile and author linked, see (typical) "epub start-old". Now that part is missing in epubs, see "epub start-now". The part which is not picked up any more is shown on "Details".
Also SOL now adds an ugly cover image see "TitlePage". An option to not add it would be appreciated.

[JimmXinu]

Originally Posted by julo

At the beginning of an FFF download the 'Details' included title, author, genre, etc. with tile and author linked, see (typical) "epub start-old". Now that part is missing in epubs, see "epub start-now". The part which is not picked up any more is shown on "Details".
Also SOL now adds an ugly cover image see "TitlePage". An option to not add it would be appreciated.

Are you sure you haven't turned of title page? Because I'm seeing metadata on the title page as usual and your image looks like it skips from cover directly into the chapter text.

Another possibility: If you include metadata entry 'notice' in titlepage, it seems to include the cover image. If you're looking at the bottom of the titlepage, it might look like that.

I don't read the site, so I've no idea if what 'notice' includes has changed. The site did change recently.

As for the generated cover, my extremely small sample set shows real covers are named '999999-c' and generated ones '-t' or '-w'. Try this to exclude it:

[storiesonline.net]
# ...
add_to_cover_exclusion_regexp:|.*-[tw]$

[julo]

I recently switched one computer from Windows 10 to 11. I wonder if there is a connection, but knowingly I changed nothing. AFAIR I just copied the personal.ini over.
I never kept track of which computer I used for downloads. Now have to keep track of for a while, I guess.

[JimmXinu]

New Test Version Posted

2024-06-03
- adapter_ficbooknet: Fix breakage for proxies & add covers #1077 - #1076, thanks dbhmw

[ownedbycats]

Question: Is there a storyUrl config that can be used to remove empty paragraphs (<p> </p>) from the story body? I have an editor chain to clean them up but I thought if I could do it direcrly that be easier.