[GUI Plugin] FanFicFare - Page 627

Da Man · 03-20-2024, 09:34 AM

Quote:

Originally Posted by JimmXinu

It's working as intended for me. And that code hasn't been touched recently.

A number of [<site>] sections have a slow_down_sleep_time setting in defaults.ini.

If you put slow_down_sleep_time under [defaults] in personal.ini, those [<site>] sections in defaults.ini will take precedence over [defaults] in personal.ini.

Set it in personal.ini under the specific [<site>] section or [overrides] instead.

Alternatively, since the numbers you quote are larger than site's default, I would you have more than one slow_down_sleep_time setting, possibly in more than one copy of the same [<section>] that are conflicting.

If you run in debug mode, each fetch will explicitly state it's sleep time like:

Code:

FFF: DEBUG: 2024-03-19 17:19:15,288: decorators.py(123): random sleep(49.50-148.50):139.64

Well, no idea how long this is going on, I just noticed it yesterday when updating several stories. The setting is in one site section, there is only one in it and no twin of the section. It looks like this:

Code:

[www.fanfiction.net]
#use_nsapa_proxy:true
use_flaresolverr_proxy:true
use_cloudscraper:false
slow_down_sleep_time:99

Checked and double checked, still doesn't work. I placed the setting in [overrides] and there it does work - but for all sites. I want the heavy slow down for FFN, not everywhere. Any idea?

BTW, how do you set the plugin into debug mode?

JimmXinu · 03-20-2024, 10:53 AM

Quote:

Originally Posted by w4tchdoge

I've started fiddling with cover generation settings
...

Cover settings are complex and have a wiki page to help explain them.

If you haven't read it yet, please see if that answers any of your questions.

JimmXinu · 03-20-2024, 11:05 AM

Quote:

Originally Posted by Da Man

Well, no idea how long this is going on, I just noticed it yesterday when updating several stories. The setting is in one site section, there is only one in it and no twin of the section. It looks like this:

Code:

[www.fanfiction.net]
#use_nsapa_proxy:true
use_flaresolverr_proxy:true
use_cloudscraper:false
slow_down_sleep_time:99

Checked and double checked, still doesn't work. I placed the setting in [overrides] and there it does work - but for all sites. I want the heavy slow down for FFN, not everywhere. Any idea?

Nope. But [overrides] working still suggests settings shenanigans. I'd have to see the whole INI. There's a button for making a 'safe' copy without user/pass settings.

You could also try copying those setting to a new library as the only ini settings.

Quote:

Originally Posted by Da Man

BTW, how do you set the plugin into debug mode?

Calibre Preferences drop down -> Restart in Debug

Da Man · 03-20-2024, 11:29 AM

K, here is the safe copy, not much in there:

Spoiler:

Will try debug now.

Da Man · 03-20-2024, 12:08 PM

Tried in debug mode and checked the resulting txt file, there was no sleep whatsoever.

JimmXinu · 03-20-2024, 12:18 PM

Quote:

Originally Posted by Da Man

K, here is the safe copy, not much in there:

Will try debug now.

I don't see anything wrong here.

However, I'm unable to test it exactly as is because flaresolverr doesn't work on ffnet for me at all.

Quote:

Originally Posted by Da Man

Tried in debug mode and checked the resulting txt file, there was no sleep whatsoever.

I don't suppose I can see that?

Da Man · 03-20-2024, 12:39 PM

Quote:

Originally Posted by JimmXinu

I don't see anything wrong here.

However, I'm unable to test it exactly as is because flaresolverr doesn't work on ffnet for me at all.

I don't suppose I can see that?

And right now it stopped working for me as well. Probably too much requests due to lack of slow down. https://www2.mobileread.com/i/smiliesadd1/y_rofl.gif

Oh well, do you want the whole file? Just one try this time because of flaresolverr...

Spoiler:

JimmXinu · 03-20-2024, 01:05 PM

slow_down_sleep_time is applied after successes, not failures. There's no benefit waiting to report a failure most of the time.

And FFF doesn't retry when connecting through Flaresolverr--that's FS's job in that case. Log shows you trying twice?

So this output doesn't help with identifying your reported problem. Unless you were always getting failures.

Da Man · 03-20-2024, 01:41 PM

As of now I only get timeout errors using flaresolverr. Maybe it works again tomorrow or something, if so I try debug mode again and post the result.

Da Man · 03-20-2024, 01:56 PM

I thought what the hell, just copy the settings from FFN to AO3 and try that.

Despite the 99sec setting there's once again no pause between requests. Here's the file:

Spoiler:

Da Man · 03-20-2024, 02:16 PM

K, spoiler next time, will do.

JimmXinu · 03-20-2024, 02:48 PM

Quote:

Originally Posted by Da Man

I thought what the hell, just copy the settings from FFN to AO3 and try that.

Despite the 99sec setting there's once again no pause between requests. Here's the file:

Code:

...
FFF: DEBUG: 2024-03-20 18:49:02,665: decorators.py(123): random sleep(1.00-3.00):1.70
...

It's not as big as you've set, but it is there.

I was a stumped by this one for a minute. You used [www.archiveofourown.org] which is technically lower priority than defaults.ini's [archiveofourown.org] section.

I should add a warning about that to the wiki in [www.domain] Sections. It currently only mentions having both in personal.ini, not one each in defaults and personal.

Some sites require www in the domain, some sites forbid it, more will accept both. FFF tends to use whichever the site 'prefers' as the section name in defaults.ini. For example, any request to www.archiveofourown.org is immediately redirected to the archiveofourown.org equivalent.

This doesn't explain your original problem, however, as [www.fanfiction.net] is in defaults.ini and your personal.ini.

Quote:

Originally Posted by Da Man

K, spoiler next time, will do.

[code] tags are better--both scrolls at a reasonable size and preserves formatting. Can also wrap in [spoiler]

Da Man · 03-20-2024, 03:11 PM

Removed the www et voila, long slowdown with AO3. K, that was a weird one. What happens when there are two sections in personal.ini, one with one without www, otherwise identical?

Regarding FFN, I'll have to wait until flaresolverr works again, hope for the best and see what happens.

ownedbycats · 03-20-2024, 08:31 PM

On this ffnet story, it picked up the cover image despite it being identical the author image. Is this confirmed, or a gremlin?

JimmXinu · 03-20-2024, 08:50 PM

Quote:

Originally Posted by ownedbycats

On this ffnet story, it picked up the cover image despite it being identical the author image. Is this confirmed, or a temporary glitch?

I'm not seeing that happen.

In the download debug I see:

Code:

adapter_fanfictionnet.py(349): skip_author_cover: cover_url matches authimg_url: don't use

Manual inspection agrees, same image (modulo /150/ vs /180/ for size).

03-20-2024, 11:29 AM	#9394
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	K, here is the safe copy, not much in there: Spoiler: ## This is an example of what your personal configuration might look ## like. Uncomment options by removing the '#' in front of them. [defaults] ## [defaults] section applies to all formats and sites but may be ## overridden at several levels. Example: ## [defaults] ## titlepage_entries: category,genre, status ## [www.whofic.com] ## # overrides defaults. ## titlepage_entries: category,genre, status,dateUpdated,rating ## [epub] ## # overrides defaults & site section ## titlepage_entries: category,genre, status,datePublished,dateUpdated,dateCreated ## [www.whofic.com:epub] ## # overrides defaults, site section & format section ## titlepage_entries: category,genre, status,datePublished ## [overrides] ## # overrides all other sections ## titlepage_entries: category ## Some sites also require the user to confirm they are adult for ## adult content. Uncomment by removing '#' in front of is_adult. is_adult:true ## Don't like the numbers at the start of chapter titles on some ## sites? You can use strip_chapter_numbers to strip them off. Just ## want to make them all look the same? Strip them off, then add them ## back on with add_chapter_numbers. Don't like the way it strips ## numbers or adds them back? See chapter_title_strip_pattern and ## chapter_title_add_pattern. #strip_chapter_numbers:true #add_chapter_numbers:true ## Add this to genre if there's more than one category. #add_genre_when_multi_category: Crossover [epub] ## include images from img tags in the body and summary of stories. ## Images will be converted to jpg for size if possible. Images work ## in epub format only. To get mobi or other format with images, ## download as epub and use Calibre to convert. #include_images:true ## If not set, the summary will have all html stripped for safety. ## Both this and include_images must be true to get images in the ## summary. #keep_summary_html:true ## If set, the first image found will be made the cover image. If ## keep_summary_html is true, any images in summary will be before any ## in chapters. #make_firstimage_cover:true ## Resize images down to width, height, preserving aspect ratio. ## Nook size, with margin. #image_max_size: 768, 1024 ## Change image to grayscale, if graphics library allows, to save ## space. #grayscale_images: false ## Most common, I expect will be using this to save username/passwords ## for different sites. Here are a few examples. See defaults.ini ## for the full list. [www.twilighted.net] #username:XXXXXXXX #password:XXXXXXXX ## default is false #collect_series: true [ficwad.com] #username:XXXXXXXX #password:XXXXXXXX [www.adastrafanfic.com] ## Some sites do not require a login, but do require the user to ## confirm they are adult for adult content. #is_adult:true [www.twcslibrary.net] #username:XXXXXXXX #password:XXXXXXXX #is_adult:true ## default is false #collect_series: true [www.fimfiction.net] #is_adult:true #fail_on_password:XXXXXXXX [www.tthfanfic.org] #is_adult:true ## tth is a little unusual--it doesn't require user/pass, but the site ## keeps track of which chapters you've read and won't send another ## update until it thinks you're up to date. This way, on download, ## it thinks you're up to date. #username:XXXXXXXX #password:XXXXXXXX [www.fanfiction.net] #use_nsapa_proxy:true use_flaresolverr_proxy:true use_cloudscraper:false slow_down_sleep_time:99 [www.fictionpress.com] slow_down_sleep_time:99 #use_nsapa_proxy:true use_flaresolverr_proxy:true use_cloudscraper:false [www.archiveofourown.org] slow_down_sleep_time:25 always_login:false ## This section will override anything in the system defaults or other ## sections here. [overrides] ## default varies by site. Set true here to force all sites to ## collect series. #collect_series: true Will try debug now. Last edited by Da Man; 03-20-2024 at 02:19 PM.

03-20-2024, 01:56 PM	#9400
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	I thought what the hell, just copy the settings from FFN to AO3 and try that. Despite the 99sec setting there's once again no pause between requests. Here's the file: Spoiler: ApplicationPaletteChange event ignored Using calibre Qt style: True EpubMerge: DEBUG: 2024-03-20 18:47:35,379: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:/home/chr/.config/calibre/plugins/fanficfare_macmenuhack.txt FFF: DEBUG: 2024-03-20 18:47:35,383: calibre_plugins.fanficfare_plugin.fff_plugin(215): Plugin FanFicFare macmenuhack file_path:/home/chr/.config/calibre/plugins/fanficfare_macmenuhack.txt calibre Fehlerdiagnoseprotokoll calibre 7.7 embedded-python: True Linux-6.5.0-26-generic-x86_64-with-glibc2.38 Linux ('64bit', 'ELF') ('Linux', '6.5.0-26-generic', '#26-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 5 21:19:28 UTC 2024') Python 3.11.5 Interface language: de EXE path: /opt/calibre/bin/calibre-debug Successfully initialized third party plugins: EpubMerge (2, 17, 0) && FanFicFare (4, 32, 4) && Find Duplicates (1, 10, 9) && Reading List (1, 15, 2) calibre 7.7 embedded-python: True Linux-6.5.0-26-generic-x86_64-with-glibc2.38 Linux ('64bit', 'ELF') ('Linux', '6.5.0-26-generic', '#26-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 5 21:19:28 UTC 2024') Python 3.11.5 Interface language: de EXE path: /opt/calibre/bin/calibre-debug Successfully initialized third party plugins: EpubMerge (2, 17, 0) && FanFicFare (4, 32, 4) && Find Duplicates (1, 10, 9) && Reading List (1, 15, 2) QPA platform: xcb devicePixelRatio: 1.0 logicalDpi: 96.0 x 96.0 physicalDpi: 127.66492146596858 x 127.59069767441859 [0.00] Starting up... [0.00] Showing splash screen... [0.04] splash screen shown [0.04] Initializing db... [0.52] db initialized [0.52] Constructing main UI... [2.03] GUI main window shown [2.08] main UI initialized... [2.08] Hiding splash screen FFF: DEBUG: 2024-03-20 18:48:22,400: calibre_plugins.fanficfare_plugin.fff_plugin(1150) : FanFicFare v4.32.4 FFF: INFO: 2024-03-20 18:48:22,517: calibre_plugins.fanficfare_plugin.prefs(216): Using default settings FFF: DEBUG: 2024-03-20 18:48:22,526: story.py(738): use_flaresolverr_proxy:true FFF: DEBUG: 2024-03-20 18:48:22,530: configurable.py(1055): use_flaresolverr_proxy:true FFF: WARNING: 2024-03-20 18:48:22,530: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false FFF: WARNING: 2024-03-20 18:48:22,531: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images FFF: WARNING: 2024-03-20 18:48:22,531: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver FFF: DEBUG: 2024-03-20 18:48:22,531: fetcher_flaresolverr_proxy.py(42): using FlareSolverr_ProxyFetcher FFF: DEBUG: 2024-03-20 18:48:22,531: configurable.py(1080): use_browser_cache: FFF: DEBUG: 2024-03-20 18:48:22,531: configurable.py(1100): use_basic_cache:true FFF: INFO: 2024-03-20 18:48:22,531: base_otw_adapter.py(142): url: https://archiveofourown.org/works/16...iew_adult=true FFF: INFO: 2024-03-20 18:48:22,532: base_otw_adapter.py(143): metaurl: https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:22,532: cache_basic.py(116): ========== MISS (GET) BasicCache https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:22,532: fetcher_flaresolverr_proxy.py(101): ---------- REQ (GET) FlareSolverr_ProxyFetcher https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:22,532: fetcher_flaresolverr_proxy.py(182): url:https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:22,532: fetcher_requests.py(114): ---------- REQ (POST) RequestsFetcher http://localhost:8191/v1 FFF: DEBUG: 2024-03-20 18:48:27,501: fetcher_requests.py(127): response code:200 FFF: DEBUG: 2024-03-20 18:48:27,501: fetcher_flaresolverr_proxy.py(123): response code:200 FFF: DEBUG: 2024-03-20 18:48:27,501: decorators.py(112): fromcache:False FFF: DEBUG: 2024-03-20 18:48:27,502: decorators.py(123): random sleep(1.00-3.00):2.83 FFF: DEBUG: 2024-03-20 18:48:30,328: cache_basic.py(116): ========== MISS (GET) BasicCache https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:30,328: fetcher_flaresolverr_proxy.py(101): ---------- REQ (GET) FlareSolverr_ProxyFetcher https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:30,329: fetcher_flaresolverr_proxy.py(182): url:https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:30,329: fetcher_requests.py(114): ---------- REQ (POST) RequestsFetcher http://localhost:8191/v1 FFF: DEBUG: 2024-03-20 18:48:39,515: fetcher_requests.py(127): response code:200 FFF: DEBUG: 2024-03-20 18:48:39,516: fetcher_flaresolverr_proxy.py(123): response code:200 FFF: DEBUG: 2024-03-20 18:48:39,516: decorators.py(112): fromcache:False FFF: DEBUG: 2024-03-20 18:48:39,516: decorators.py(123): random sleep(1.00-3.00):2.55 FFF: DEBUG: 2024-03-20 18:48:42,159: base_otw_adapter.py(269): numChapters: (26) FFF: DEBUG: 2024-03-20 18:48:42,387: calibre_plugins.fanficfare_plugin.fff_plugin(1460) : update existing id:13258 bs4/builder/__init__.py:545: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. FFF: DEBUG: 2024-03-20 18:48:42,720: story.py(738): use_flaresolverr_proxy:true FFF: DEBUG: 2024-03-20 18:48:42,724: configurable.py(1055): use_flaresolverr_proxy:true FFF: WARNING: 2024-03-20 18:48:42,724: configurable.py(1059): FlareSolverr v2+ doesn't work with images: include_images automatically set false FFF: WARNING: 2024-03-20 18:48:42,724: configurable.py(1060): Set use_flaresolverr_proxy:withimages if your are using FlareSolver v1 and want images FFF: WARNING: 2024-03-20 18:48:42,724: configurable.py(1061): Set use_flaresolverr_proxy:directimages to download images directly while using FlareSolver FFF: DEBUG: 2024-03-20 18:48:42,724: fetcher_flaresolverr_proxy.py(42): using FlareSolverr_ProxyFetcher FFF: DEBUG: 2024-03-20 18:48:42,724: configurable.py(1080): use_browser_cache: FFF: DEBUG: 2024-03-20 18:48:42,724: configurable.py(1100): use_basic_cache:true FFF: INFO: 2024-03-20 18:48:42,725: base_otw_adapter.py(142): url: https://archiveofourown.org/works/16...iew_adult=true FFF: INFO: 2024-03-20 18:48:42,725: base_otw_adapter.py(143): metaurl: https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:42,725: cache_basic.py(116): ========== MISS (GET) BasicCache https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:42,725: fetcher_flaresolverr_proxy.py(101): ---------- REQ (GET) FlareSolverr_ProxyFetcher https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:42,725: fetcher_flaresolverr_proxy.py(182): url:https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:42,726: fetcher_requests.py(114): ---------- REQ (POST) RequestsFetcher http://localhost:8191/v1 FFF: DEBUG: 2024-03-20 18:48:51,545: fetcher_requests.py(127): response code:200 FFF: DEBUG: 2024-03-20 18:48:51,546: fetcher_flaresolverr_proxy.py(123): response code:200 FFF: DEBUG: 2024-03-20 18:48:51,546: decorators.py(112): fromcache:False FFF: DEBUG: 2024-03-20 18:48:51,546: decorators.py(123): random sleep(1.00-3.00):1.31 FFF: DEBUG: 2024-03-20 18:48:52,855: cache_basic.py(116): ========== MISS (GET) BasicCache https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:52,855: fetcher_flaresolverr_proxy.py(101): ---------- REQ (GET) FlareSolverr_ProxyFetcher https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:52,856: fetcher_flaresolverr_proxy.py(182): url:https://archiveofourown.org/works/16...iew_adult=true FFF: DEBUG: 2024-03-20 18:48:52,856: fetcher_requests.py(114): ---------- REQ (POST) RequestsFetcher http://localhost:8191/v1 FFF: DEBUG: 2024-03-20 18:49:02,664: fetcher_requests.py(127): response code:200 FFF: DEBUG: 2024-03-20 18:49:02,665: fetcher_flaresolverr_proxy.py(123): response code:200 FFF: DEBUG: 2024-03-20 18:49:02,665: decorators.py(112): fromcache:False FFF: DEBUG: 2024-03-20 18:49:02,665: decorators.py(123): random sleep(1.00-3.00):1.70 FFF: DEBUG: 2024-03-20 18:49:04,479: base_otw_adapter.py(269): numChapters: (44) FFF: DEBUG: 2024-03-20 18:49:04,570: calibre_plugins.fanficfare_plugin.fff_plugin(1460) : update existing id:13257 [4.32] splash screen hidden [4.32] Started up in 4.32 seconds with 15335 books kf.service.services: KApplicationTrader: mimeType "x-scheme-handler/file" not found Last edited by theducks; 03-20-2024 at 02:09 PM. Reason: SPOILER LOG files

03-20-2024, 08:31 PM	#9404
ownedbycats Custom User Title Posts: 9,048 Karma: 62040591 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	On this ffnet story, it picked up the cover image despite it being identical the author image. Is this confirmed, or a gremlin? Last edited by ownedbycats; 03-20-2024 at 08:44 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	494	05-18-2024 02:34 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 08:39 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 12:48 PM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

03-20-2024, 12:08 PM	#9395
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	Tried in debug mode and checked the resulting txt file, there was no sleep whatsoever.

03-20-2024, 01:05 PM	#9398
JimmXinu Plugin Developer Posts: 6,447 Karma: 4276871 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	slow_down_sleep_time is applied after successes, not failures. There's no benefit waiting to report a failure most of the time. And FFF doesn't retry when connecting through Flaresolverr--that's FS's job in that case. Log shows you trying twice? So this output doesn't help with identifying your reported problem. Unless you were always getting failures.

03-20-2024, 01:41 PM	#9399
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	As of now I only get timeout errors using flaresolverr. Maybe it works again tomorrow or something, if so I try debug mode again and post the result.

03-20-2024, 02:16 PM	#9401
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	K, spoiler next time, will do.

03-20-2024, 03:11 PM	#9403
Da Man Enthusiast Posts: 27 Karma: 10 Join Date: Jun 2018 Device: none	Removed the www et voila, long slowdown with AO3. K, that was a weird one. What happens when there are two sections in personal.ini, one with one without www, otherwise identical? Regarding FFN, I'll have to wait until flaresolverr works again, hope for the best and see what happens.

Advert

Advert