[GUI Plugin] FanFicFare

[ownedbycats]

Gremlin, then. I was having some temporary cache errors with this one so I'm guessing something went weird there.

[Da Man]

Here we go, worked again today. Slow down still set to 99seconds, still basically no pause between requests. Here's the debug file:

Spoiler:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
EpubMerge: DEBUG: 2024-03-21 08:20:28,244: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:/home/chr/.config/calibre/plugins/fanficfare_macmenuhack.txt
FFF: DEBUG: 2024-03-21 08:20:28,248: calibre_plugins.fanficfare_plugin.fff_plugin(215): Plugin FanFicFare macmenuhack file_path:/home/chr/.config/calibre/plugins/fanficfare_macmenuhack.txt
calibre Fehlerdiagnoseprotokoll
calibre 7.7  embedded-python: True
Linux-6.5.0-26-generic-x86_64-with-glibc2.38 Linux ('64bit', 'ELF')
('Linux', '6.5.0-26-generic', '#26-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar  5 21:19:28 UTC 2024')
Python 3.11.5
Interface language: de
EXE path: /opt/calibre/bin/calibre-debug
Successfully initialized third party plugins: EpubMerge (2, 17, 0) && FanFicFare (4, 32, 4) && Find Duplicates (1, 10, 9) && Reading List (1, 15, 2)
calibre 7.7  embedded-python: True
Linux-6.5.0-26-generic-x86_64-with-glibc2.38 Linux ('64bit', 'ELF')
('Linux', '6.5.0-26-generic', '#26-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar  5 21:19:28 UTC 2024')
Python 3.11.5
Interface language: de
EXE path: /opt/calibre/bin/calibre-debug
Successfully initialized third party plugins: EpubMerge (2, 17, 0) && FanFicFare (4, 32, 4) && Find Duplicates (1, 10, 9) && Reading List (1, 15, 2)
QPA platform: xcb
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 127.66492146596858 x 127.59069767441859
[0.00] Starting up...
[0.00] Showing splash screen...
[0.04] splash screen shown
[0.04] Initializing db...
[0.49] db initialized
[0.49] Constructing main UI...
[1.85] GUI main window shown
[1.90] main UI initialized...
[1.90] Hiding splash screen
QAbstractItemView::commitData called with an editor that does not belong to this view
QAbstractItemView::closeEditor called with an editor that does not belong to this view
FFF: DEBUG: 2024-03-21 08:23:54,516: calibre_plugins.fanficfare_plugin.fff_plugin(1150): FanFicFare v4.32.4
FFF: INFO: 2024-03-21 08:23:54,629: calibre_plugins.fanficfare_plugin.prefs(216): Using default settings
FFF: DEBUG: 2024-03-21 08:23:54,636: story.py(738): use_flaresolverr_proxy:true
FFF: DEBUG: 2024-03-21 08:23:54,639: configurable.py(1055): use_flaresolverr_proxy:true
FFF: WARNING: 2024-03-21 08:23:54,639: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false
FFF: WARNING: 2024-03-21 08:23:54,639: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images
FFF: WARNING: 2024-03-21 08:23:54,639: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver
FFF: DEBUG: 2024-03-21 08:23:54,639: fetcher_flaresolverr_proxy.py(42): using FlareSolverr_ProxyFetcher
FFF: DEBUG: 2024-03-21 08:23:54,640: configurable.py(1080): use_browser_cache:
FFF: DEBUG: 2024-03-21 08:23:54,640: configurable.py(1100): use_basic_cache:true
FFF: DEBUG: 2024-03-21 08:23:54,640: adapter_fanfictionnet.py(111): URL: https://www.fanfiction.net/s/8275357/1/Battlestar-Victorious-Volume-4-When-Fates-Collide
FFF: DEBUG: 2024-03-21 08:23:54,640: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/8275357/1/Battlestar-Victorious-Volume-4-When-Fates-Collide
FFF: DEBUG: 2024-03-21 08:23:54,640: fetcher_flaresolverr_proxy.py(101): 
---------- REQ (GET) FlareSolverr_ProxyFetcher
https://www.fanfiction.net/s/8275357/1/Battlestar-Victorious-Volume-4-When-Fates-Collide
FFF: DEBUG: 2024-03-21 08:23:54,641: fetcher_flaresolverr_proxy.py(182): url:https://www.fanfiction.net/s/8275357/1/Battlestar-Victorious-Volume-4-When-Fates-Collide
FFF: DEBUG: 2024-03-21 08:23:54,641: fetcher_requests.py(114): 
---------- REQ (POST) RequestsFetcher
http://localhost:8191/v1
FFF: DEBUG: 2024-03-21 08:23:59,311: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2024-03-21 08:23:59,312: fetcher_flaresolverr_proxy.py(123): response code:200
FFF: DEBUG: 2024-03-21 08:23:59,312: decorators.py(112): fromcache:False
FFF: DEBUG: 2024-03-21 08:23:59,312: decorators.py(123): random sleep(3.33-10.00):4.22
FFF: DEBUG: 2024-03-21 08:24:03,638: calibre_plugins.fanficfare_plugin.fff_plugin(1460): update existing id:5234
bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor.
FFF: DEBUG: 2024-03-21 08:24:04,149: story.py(738): use_flaresolverr_proxy:true
FFF: DEBUG: 2024-03-21 08:24:04,151: configurable.py(1055): use_flaresolverr_proxy:true
FFF: WARNING: 2024-03-21 08:24:04,152: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false
FFF: WARNING: 2024-03-21 08:24:04,152: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images
FFF: WARNING: 2024-03-21 08:24:04,152: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver
FFF: DEBUG: 2024-03-21 08:24:04,152: fetcher_flaresolverr_proxy.py(42): using FlareSolverr_ProxyFetcher
FFF: DEBUG: 2024-03-21 08:24:04,152: configurable.py(1080): use_browser_cache:
FFF: DEBUG: 2024-03-21 08:24:04,152: configurable.py(1100): use_basic_cache:true
FFF: DEBUG: 2024-03-21 08:24:04,152: adapter_fanfictionnet.py(111): URL: https://www.fanfiction.net/s/11827512/1/Worldwar-Establishing-A-Balance
FFF: DEBUG: 2024-03-21 08:24:04,153: cache_basic.py(116): 
========== MISS (GET) BasicCache
https://www.fanfiction.net/s/11827512/1/Worldwar-Establishing-A-Balance
FFF: DEBUG: 2024-03-21 08:24:04,153: fetcher_flaresolverr_proxy.py(101): 
---------- REQ (GET) FlareSolverr_ProxyFetcher
https://www.fanfiction.net/s/11827512/1/Worldwar-Establishing-A-Balance
FFF: DEBUG: 2024-03-21 08:24:04,153: fetcher_flaresolverr_proxy.py(182): url:https://www.fanfiction.net/s/11827512/1/Worldwar-Establishing-A-Balance
FFF: DEBUG: 2024-03-21 08:24:04,153: fetcher_requests.py(114): 
---------- REQ (POST) RequestsFetcher
http://localhost:8191/v1
FFF: DEBUG: 2024-03-21 08:24:12,475: fetcher_requests.py(127): response code:200
FFF: DEBUG: 2024-03-21 08:24:12,476: fetcher_flaresolverr_proxy.py(123): response code:200
FFF: DEBUG: 2024-03-21 08:24:12,476: decorators.py(112): fromcache:False
FFF: DEBUG: 2024-03-21 08:24:12,476: decorators.py(123): random sleep(3.33-10.00):5.35
FFF: DEBUG: 2024-03-21 08:24:17,924: calibre_plugins.fanficfare_plugin.fff_plugin(1460): update existing id:2412
[4.40] splash screen hidden
[4.40] Started up in 4.40 seconds with 15335 books
kf.service.services: KApplicationTrader: mimeType "x-scheme-handler/file" not found

I also took a closer look at the default.ini and noticed the fg_sleep settings. Those seem quite interesting and I plan to play with them, what do they do exactly? Are there min and/or max values I have to set to avoid timeout and such?

[ownedbycats]

Enhancement request, if feasible: A prompt when the author or series URL changes in Calibre. Keep the original URL, or update to new one.

I downloaded a fic on AO3, and this populated the author URL (per the "set Calibre Author URL" setting). I later downloaded a fic from the same author on FanFiction.net, and this automatically changed the URL.

This would also help when there's multiple series sharing the same name.

[JimmXinu]

multi-reply

Originally Posted by Da Man

Here we go, worked again today. Slow down still set to 99seconds, still basically no pause between requests. Here's the debug file:

I also took a closer look at the default.ini and noticed the fg_sleep settings. Those seem quite interesting and I plan to play with them, what do they do exactly? Are there min and/or max values I have to set to avoid timeout and such?

Probably. ffnet is the only site that uses those settings and because Flaresolverr doesn't work for me, I didn't see it. Forgot about it, honestly.

Try setting tweak_fg_sleep:false. That should give you what you're looking for. Here's the code if you want to tweak them.

Those settings were added just to try and keep ffnet happy, which obviously didn't work. Perhaps I should remove them.

Originally Posted by ownedbycats

Enhancement request, if feasible: A prompt when the author or series URL changes in Calibre. Keep the original URL, or update to new one.

I downloaded a fic on AO3, and this populated the author URL (per the "set Calibre Author URL" setting). I later downloaded a fic from the same author on FanFiction.net, and this automatically changed the URL.

This would also help when there's multiple series sharing the same name.

As you've found, Calibre fundamentally does not accept 'different' series or authors with the same name. Same name == same author (or series) record.

I've been dealing with this for years by either ignoring it (click the story URL instead of the author URL), or changing the value, like:

[www.tthfanfic.org]
add_to_replace_metadata:
 author=>^dogbertcarroll$=>dogbertcarroll-tth

Unless this is a lot more common that I think it is, I really don't want to create and test a bunch of new GUI elements and interactions for this corner case.

[Da Man]

Originally Posted by JimmXinu

Probably. ffnet is the only site that uses those settings and because Flaresolverr doesn't work for me, I didn't see it. Forgot about it, honestly.

Try setting tweak_fg_sleep:false. That should give you what you're looking for. Here's the code if you want to tweak them.

Those settings were added just to try and keep ffnet happy, which obviously didn't work. Perhaps I should remove them.

I see, so those fg_sleep settings override the sleep_down. That should be mentioned in the ini. Glancing over the code this looks really interesting and enables more fine tuning. Very nice, I will definitely tweak those as they are to low now for my taste - and obviously that of FFN.
I guess this also works for fictionpress.com?

I'd suggest not removing them but including tweak_fg_sleep:false as default combined with a short explanation what they can do. With growing cloudflare use I fear we'll also have more use for them.

[ownedbycats]

Question: Is it possible to use replace_metadata to copy a portion of one metadata entry to another?

Probably easiest to link my use-case (warning for a few explicits), all consisting of "Story Title" by Author Name.

Rather than using a dozen storyUrl sections, I thought that I could regex a replace_metadata with seriesUrl conditional. I got the title extracted:

[archiveofourown.org]
add_to_replace_metadata:
 title=>^\"(.*)\" by (.*)$=>\1&&seriesUrl==https://archiveofourown.org/series/1618510

but I'm not sure how to copy title over to author, or if that's possible.

[JimmXinu]

The only way to 'copy' between metadata entries is include_in_X.

This works for me, but I only tested a couple of cases in a clean config.

Given I know you have a considerable amount of other metadata manipulation, I would not be surprised if this had problems with something else. I have a vague recollection of empty author list entries behaving weirdly with some esoteric configuration.

Personally, I would use this while downloading those stories then remove it or comment it out.

# make a copy of title
add_to_extra_valid_entries:,titlebyauthor
include_in_titlebyauthor:title.NOREPL

# add that copy to author
include_in_author:author,titlebyauthor

add_to_replace_metadata:
# clear titlebyauthor when NOT that series
 titlebyauthor=>^(.*)$=>&&seriesUrl!=https://archiveofourown.org/series/1618510
# rest only for that one series:
# Remove author entries that DON'T start with "
 author=>^([^"].*)$=>&&seriesUrl==https://archiveofourown.org/series/1618510
# Remove '"title" by ' from remaining author
 author=>^\"(.*)\" by (.*)$=>\2&&seriesUrl==https://archiveofourown.org/series/1618510
# Remove "s and author from title
 title=>^\"(.*)\" by (.*)$=>\1&&seriesUrl==https://archiveofourown.org/series/1618510

The author from the title will still have the real authorUrl. I consider that a feature.

[ownedbycats]

Yes, that did it! Thank you.

[ownedbycats]

Would it be much effort to add a 'paste as plain text' in the personal.ini editor? When I copy text from my browser for replace_metadatas, it also copies the formatting and occasionally makes it unreadable until I close and re-open.

[JimmXinu]

New Test Version Posted

2023-03-23
- Paste into ini edit as plain text only.


I've seen that before, but honestly didn't consider it enough of an issue to do anything about. And GUI changes always make me nervous.

OTOH, stackoverflow seems to think it's an easy fix.

Fair warning, I've only tested this on win, not on linux or mac. But I did try Cal2.85.1 and it worked.

[ownedbycats]

It's working in my few quick tests. I'll mention if I come across any weirdnesses.

[JimmXinu]

New Test Version Posted

2023-03-23
- adapter_deviantartcom: Bad username fails separately than bad pass. #1052

[JimmXinu]

New Test Version Posted

2023-03-24
- adapter_deviantartcom: Add a 6th different message to indicate 'mature content'. #1052

[madame_newbie]

Hello, recently whenever I download a fic that's suddenly deleted I get the following error window:

img link cause it won't embed:

https://imgur.com/JDWwPI9

It says an error has occurred while Fanficfare was updating calibre's metadata (link to fic here).

Is there a way for me to bypass this window entirely rather than having to check back whenever I'm downloading?

Thanks.

[JimmXinu]

Try the latest test version. There was a change addresses that.

Also, I recommend using update notifications rather than updating from your library. IMO, you should not be regularly attempting to update deleted stories. At least, not more than once for the same story.

FAQ: Why shouldn't I just select my entire library and use FanFicFare plugin to check for updates?