[GUI Plugin] FanFictionDownLoader

[Sefiriot]

Just updated to calibre 1.6.0, got this trying to update an existing story:

calibre, version 1.6.0
ERROR: Unhandled exception: <b>ValueError</b>:tuple.index(x): x not in tuple

calibre 1.6 [64bit]  isfrozen: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.dialogs", line 568, in do_loop
  File "calibre_plugins.fanfictiondownloader_plugin.dialogs", line 575, in do_when_finished
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1164, in update_books_finish
  File "site-packages\calibre\db\view.py", line 383, in set_marked_ids
  File "site-packages\calibre\gui2\__init__.py", line 421, in __call__
  File "site-packages\calibre\gui2\library\views.py", line 696, in marked_changed
  File "site-packages\calibre\db\view.py", line 193, in id_to_index
ValueError: tuple.index(x): x not in tuple

The ePub was updated with the new chapter, but I wonder what that error is about. Is this possibly to do with the changes to the Temp Marker feature?

[JimmXinu]

Originally Posted by Sefiriot

Just updated to calibre 1.6.0, got this trying to update an existing story:

calibre, version 1.6.0
ERROR: Unhandled exception: <b>ValueError</b>:tuple.index(x): x not in tuple
...

The ePub was updated with the new chapter, but I wonder what that error is about. Is this possibly to do with the changes to the Temp Marker feature?

Possibly. I'm not seeing this issue with the lastest FFDL test version. What is the story URL you saw it with?

[Sefiriot]

First saw the issue with http://www.fanfiction.net/s/7251431/1/. I later updated an anthology (AO3 series) that added a new story, single chapter with no problems.

I then updated a second anthology, also an AO3 series, but the update was to add a new chapter to one of the fics in the anthology, again with the same error message as above.

For the latest single story updated with that error, a yellow pin icon is shown in the normal records view (see attached screencap); not sure why it should.

[redscarlett]

Originally Posted by Sefiriot

Just updated to calibre 1.6.0, got this trying to update an existing story:

calibre, version 1.6.0
ERROR: Unhandled exception: <b>ValueError</b>:tuple.index(x): x not in tuple

calibre 1.6 [64bit]  isfrozen: True is64bit: True
Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.dialogs", line 568, in do_loop
  File "calibre_plugins.fanfictiondownloader_plugin.dialogs", line 575, in do_when_finished
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1164, in update_books_finish
  File "site-packages\calibre\db\view.py", line 383, in set_marked_ids
  File "site-packages\calibre\gui2\__init__.py", line 421, in __call__
  File "site-packages\calibre\gui2\library\views.py", line 696, in marked_changed
  File "site-packages\calibre\db\view.py", line 193, in id_to_index
ValueError: tuple.index(x): x not in tuple

The ePub was updated with the new chapter, but I wonder what that error is about. Is this possibly to do with the changes to the Temp Marker feature?

I've had the same problem, and it occurs with Generate Cover too.

[JimmXinu]

Originally Posted by Sefiriot

First saw the issue with http://www.fanfiction.net/s/7251431/1/. I later updated an anthology (AO3 series) that added a new story, single chapter with no problems.

I then updated a second anthology, also an AO3 series, but the update was to add a new chapter to one of the fics in the anthology, again with the same error message as above.

For the latest single story updated with that error, a yellow pin icon is shown in the normal records view (see attached screencap); not sure why it should.

??? Anthologies don't even call the piece of code your first error report showed.

Please try the latest test version. If nothing else, unchecking the 'Marked added/updated books' option in FFDL config will bypass that code.

The new 'Mark Books' feature in the main calibre program makes marked books visible that way. calibre's had the marking ability for a long time, but it wasn't visible before. That's why I'm adding an option to not mark added/updated books.

[JimmXinu]

Originally Posted by redscarlett

I've had the same problem, and it occurs with Generate Cover too.

Can you clarify that statement?

The Generate Cover PI doesn't mark books, so it won't generate that same error when used alone.

If you mean you have GC turned on in FFDL to run on add/update, calling GC isn't really relevant to the problem this time.

[redscarlett]

Originally Posted by JimmXinu

Can you clarify that statement?

The Generate Cover PI doesn't mark books, so it won't generate that same error when used alone.

If you mean you have GC turned on in FFDL to run on add/update, calling GC isn't really relevant to the problem this time.

I meant a similar error arising - as in related to tuple, but I think I've sorted it now thanks, though the FFDL one hasn't been fixed yet.

[Sefiriot]

Originally Posted by JimmXinu

??? Anthologies don't even call the piece of code your first error report showed.

Please try the latest test version. If nothing else, unchecking the 'Marked added/updated books' option in FFDL config will bypass that code.

The new 'Mark Books' feature in the main calibre program makes marked books visible that way. calibre's had the marking ability for a long time, but it wasn't visible before. That's why I'm adding an option to not mark added/updated books.

I've never activated the 'marked added/updated books' option in the configs, actually; there was no point to doing so when I sort my library by latest modified. Even so the yellow pin marker seems to be turning up for the last single book I add/update, doesn't happen with anthologies.

And you're right, anthologies updated this morning don't trigger that tuple error even though I could've sworn one of the anthologies I updated last night did trigger that.

Meanwhile, adding these stories triggered the tuple error message, after several updates for other stories that went off without a hitch. http://archiveofourown.org/works/1001954, http://archiveofourown.org/works/1002277 and http://www.fanfiction.net/s/8576811/1/

Attached calibre debug log, but don't think I see anything unusual in it. :S I'll update to the test version for the later afternoon/evening spate of updates and see what happens.

ETA: Updated to the test version of FFDL, downloaded a new fic, the tuple error still occurs. Didn't have calibre-debug on however, for which I'm sorry. I still can't seem to see any logic to which stories trigger the error and which don't.

[JimmXinu]

Originally Posted by Sefiriot

I've never activated the 'marked added/updated books' option in the configs, actually; there was no point to doing so when I sort my library by latest modified. Even so the yellow pin marker seems to be turning up for the last single book I add/update, doesn't happen with anthologies.

The yellow pin marker is a new calibre feature indicating marked books. Marking the added/updated books didn't make any difference to users who didn't care before--that's why it wasn't even an option not to do it before.

You never turned it on because it wasn't there before--it was always on and still defaults to on if you don't turn it off in the text version posted. It's a checkbox on the Basic tab. I'll default it to off when I release the next release.

This is a calibre bug that is already fixed. But because the next calibre release isn't until Friday, turning off "Mark added/updated books when finished" is the only fix right now.

[JimmXinu]

Version 1.7.47 - 13 Oct 2013

• New site: fictionpad.com
• Debug/information output reduced and changed to logging.
• New Basic option "Mark added/updated books when finished?"

Calibre's new 'Mark Book' feature is an expansion of the existing system for marking books. FFDL has for some time marked added/updated books for 'Show added/updated books' feature and the minority who knew how to search for them.

Now, however, marked books show up visibly (yellow pin to the left) and the current version of calibre has a bug that sometimes crops up when marking the books. It's already fixed. But because the next calibre release isn't until Friday, turning off "Mark added/updated books when finished" is the only fix right now.

[Sefiriot]

Attempting to update a fic from FFN, hit this error with latest release of FFDL:

Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.jobs", line 212, in do_download_for_worker
  File "calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.writers.base_writer", line 227, in writeStory
  File "calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.adapters.base_adapter", line 211, in getStory
  File "calibre_plugins.fanfictiondownloader_plugin.fanficdownloader.adapters.adapter_fanfictionnet", line 285, in getChapterText
FailedToDownload: Error downloading Chapter: http://www.fanfiction.net/s/8897431/22/!  Missing required element!

FFDL:INFO:calibre_plugins.fanfictiondownloader_plugin.jobs(98):Successfully downloaded:
FFDL:INFO:calibre_plugins.fanfictiondownloader_plugin.jobs(102):
Unsuccessful:
FFDL:INFO:calibre_plugins.fanfictiondownloader_plugin.jobs(105):Child of the Storm http://www.fanfiction.net/s/8897431/1/

[Ellachanted]

I'm suddenly having issues with FFnet. It was working perfectly this morning. Now I am getting error messages such as this:

Status
Title
Author
Comment
URL
Error
Femme Fatale
bexie25
Error downloading Chapter: http://www.fanfiction.net/s/8002577/1/! Missing required element!
http://www.fanfiction.net/s/8002577/1/

It doesn't really make sense unless they are doing something to the database. I know Flagfic is having issues with newer stories since yesterday. I did update to the new version. Can I go back to the previous version somehow?

Thanks

[BaconButty]

FFN fics are showing are error message when being updated.
Could this be part of the new copy-protection to help prevent plagiarism?

Error - How to Forget - Shadow Masen
Error downloading Chapter: http://www.fanfiction.net/s/9708631/8/! Missing required element!
http://www.fanfiction.net/s/9708631/1/
Bad

Error
When Day Ends and Night Begins - PuppetStrings19
Error downloading Chapter: http://www.fanfiction.net/s/9469398/40/! Missing required element!
http://www.fanfiction.net/s/9469398/1/
Bad

Error - Beaches and Boardwalks - jaxon22
Error downloading Chapter: http://www.fanfiction.net/s/9692076/14/! Missing required element!
http://www.fanfiction.net/s/9692076/1/
Bad

Error - The Perfect Partner Program - MarieCarro
Error downloading Chapter: http://www.fanfiction.net/s/9201608/11/! Missing required element!
http://www.fanfiction.net/s/9201608/1/
Bad

Error - Friend of the Family - SweetUmami
Error downloading Chapter: http://www.fanfiction.net/s/9374916/19/! Missing required element!
http://www.fanfiction.net/s/9374916/1/
Bad


I have tried downloading before and after the FFDL update, without any difference.

Thank you.

[JimmXinu]

ffnet has tweaked their HTML again in a way that breaks FFDL. Going back a version of FFDL won't help.

I hope to have a fixed version shortly.

[JimmXinu]

Version 1.7.48 - 13 Oct 2013

- Fix for an HTML change on fanfiction.net

(You might need to get it manually from the first post for immediate effect. There's now an hourly cron that collects the plugin zip files for calibre's built-in plugin installer.

From 1.7.47:

• New site: fictionpad.com
• Debug/information output reduced and changed to logging.
• New Basic option "Mark added/updated books when finished?"

Calibre's new 'Mark Book' feature is an expansion of the existing system for marking books. FFDL has for some time marked added/updated books for 'Show added/updated books' feature and the minority who knew how to search for them.

Now, however, marked books show up visibly (yellow pin to the left) and the current version of calibre has a bug that sometimes crops up when marking the books. It's already fixed. But because the next calibre release isn't until Friday, turning off "Mark added/updated books when finished" is the only fix right now.