[GUI Plugin] FanFictionDownLoader - Page 63

JimmXinu · 10-16-2012, 12:54 PM

@Dylan Tomorrow - Wow, you've put a lot of effort into this.

Do you mind if I link to this post from the FFDL FAQs?

I assume since you went with KindleGen that calibre conversion from epub to mobi didn't solve the issue.

Dylan Tomorrow · 10-16-2012, 04:01 PM

@JimmXinu: Thanks! I was just getting more and more annoyed that certain fanfic ebooks only indexed properly in their txt form!

Of course I don't mind, I would be honoured to be linked from there

. [Also, I did not know this FAQ exists. *cough* Because I had been wondering whether the missing TOC in MOBI was a known issue and turns out it was. ^^]

Yes, the calibre conversion from EPUB to MOBI, while fixing the missing TOC, did not work to fix the indexing, nor did MOBI to MOBI (In hindsight I have not tried EPUB to KF8 nor MOBI to KF8 conversion though (option became only available in recent update IIRC), so it would be kinda funny if that works

). I think I read somewhere that the calibre devs care more about the files working then them conforming to XHTML specs and validating against Epubcheck, which makes alot of sense.

It could be that the indexing engine on the KT is less forgiving towards tag soup than with earlier kindles? I have no idea as I don't have another Kindle device to test the files on ...

On an somewhat unrelated note, after trying out various similar services, none of which really worked, I found a workable tool for livejournal fics: Readability.com. You just have to append ?format=light (intended for mobile use) to the end of livejournal urls and then give the URL to Readability via one of their many apps/extensions or directly on their website. You can either send an MOBI to your Kindle or download an EPUB!

Of course this leaves you with separate ebook files for each chapter which you will have to merge manually if you care enough to go to that effort (I didn't, fic was only 3 chapters

) and there is some stuff at the end that Readabiltiy does not filter out, but it just works wonderful otherwise. According to Readability, the Kindle uses their code for the Article Mode of its browser. The results are indeed very similar.

djrm · 10-16-2012, 04:52 PM

Hi all. I have a question I hope someone can help me with. When I download from fanfiction.net to html, the name of the html file inside the zip file is a bit weird. e.g. new-42-xegfph.html (downloaded from http://www.fanfiction.net/s/6865661/1/). Is there any way to make FFDL use the title and author as the html name?

Thanks.
DJRM

P.S. Love this plugin!

JimmXinu · 10-16-2012, 06:06 PM

Quote:

Originally Posted by djrm

Is there any way to make FFDL use the title and author as the html name?

That's not an unreasonable request.

However, there are limitations due to filesystem naming limitations. I can make it better, but I can't make it perfect.

Try this version and see what you think.

JimmXinu · 10-16-2012, 06:47 PM

Quote:

Originally Posted by Dylan Tomorrow

Of course I don't mind, I would be honoured to be linked from there

. [Also, I did not know this FAQ exists. *cough* Because I had been wondering whether the missing TOC in MOBI was a known issue and turns out it was. ^^]

I've added an FAQ about this to the project wiki, thanks.

And I've added a link to the FAQ wiki to the first post of this thread.

Quote:

Originally Posted by Dylan Tomorrow

Yes, the calibre conversion from EPUB to MOBI, while fixing the missing TOC, did not work to fix the indexing, nor did MOBI to MOBI (In hindsight I have not tried EPUB to KF8 nor MOBI to KF8 conversion though (option became only available in recent update IIRC), so it would be kinda funny if that works

). I think I read somewhere that the calibre devs care more about the files working then them conforming to XHTML specs and validating against Epubcheck, which makes alot of sense.

Yeah, I use epubcheck once in a while, but I'm not too zealous about it because it reports problems that nothing else seems to care about while accepting some things that fail on my devices. (Usually because of non-standard limitations on the devices, but still...)

Quote:

Originally Posted by Dylan Tomorrow

It could be that the indexing engine on the KT is less forgiving towards tag soup than with earlier kindles? I have no idea as I don't have another Kindle device to test the files on ...

I have a Kindle 3g, but frankly I gave up on using search on it long ago when my library hit ~1000 ebooks. It's entirely possible it has the same problem. Even when it worked, I didn't found it all that much of a help, since I couldn't restrict the search to title / author.

Quote:

Originally Posted by Dylan Tomorrow

On an somewhat unrelated note, after trying out various similar services, none of which really worked, I found a workable tool for livejournal fics: Readability.com. You just have to append ?format=light (intended for mobile use) to the end of livejournal urls and then give the URL to Readability via one of their many apps/extensions or directly on their website. You can either send an MOBI to your Kindle or download an EPUB!

I've added a note and a link here to the LiveJournal note on our Supported Sites page, thanks.

Quote:

Originally Posted by Dylan Tomorrow

Of course this leaves you with separate ebook files for each chapter which you will have to merge manually if you care enough to go to that effort (I didn't, fic was only 3 chapters

) and there is some stuff at the end that Readabiltiy does not filter out, but it just works wonderful otherwise. According to Readability, the Kindle uses their code for the Article Mode of its browser. The results are indeed very similar.

If I may humbly point you in the direct of EpubMerge...

djrm · 10-16-2012, 11:20 PM

Quote:

Originally Posted by JimmXinu

That's not an unreasonable request.

However, there are limitations due to filesystem naming limitations. I can make it better, but I can't make it perfect.

Try this version and see what you think.

Wow, thanks! I will try it out as soon as possible tomorrow after work and report back.

stormharper · 10-17-2012, 12:29 AM

I'm having a problem getting updates onto my Kindle & am hoping someone here can tell me what I'm doing wrong. When I update a story using FFDL & then want to put the updated story on my Kindle - it reloads an older file that does not have the new chapter. I've tried it both ways - with the story not on the Kindle & once with it on the Kindle. Both times loaded an older file. However, if I check the file in Calibre it shows the new chapters. On the add to Kindle list I've got it set to add/overwrite.

If some one knows what I'm doing wrong I would appreciate the help.

JimmXinu · 10-17-2012, 12:37 AM

Quote:

Originally Posted by stormharper

I'm having a problem getting updates onto my Kindle & am hoping someone here can tell me what I'm doing wrong. When I update a story using FFDL & then want to put the updated story on my Kindle - it reloads an older file that does not have the new chapter. I've tried it both ways - with the story not on the Kindle & once with it on the Kindle. Both times loaded an older file. However, if I check the file in Calibre it shows the new chapters. On the add to Kindle list I've got it set to add/overwrite.

If some one knows what I'm doing wrong I would appreciate the help.

I suspect this is more likely an issue with your FFDL config than it is with Reading List.

Do you have the 'Delete other existing formats?' option in FFDL turned on? I suspect what's happening is that you're loading an out of date mobi that needs to be converted. Assuming you're downloading epub and converting to mobi, that is.

Dylan Tomorrow · 10-17-2012, 06:49 AM

Quote:

Originally Posted by JimmXinu

I've added an FAQ about this to the project wiki, thanks.

And I've added a link to the FAQ wiki to the first post of this thread.

Quote:

Originally Posted by JimmXinu

I've added a note and a link here to the LiveJournal note on our Supported Sites page, thanks.

Awesome!

Quote:

Originally Posted by JimmXinu

Yeah, I use epubcheck once in a while, but I'm not too zealous about it because it reports problems that nothing else seems to care about while accepting some things that fail on my devices. (Usually because of non-standard limitations on the devices, but still...)

Yeah, it feels like the web 10 years ago. Being technically standards compliant does not mean your ebook actually working on all devices

.

Quote:

Originally Posted by JimmXinu

I have a Kindle 3g, but frankly I gave up on using search on it long ago when my library hit ~1000 ebooks. It's entirely possible it has the same problem. Even when it worked, I didn't found it all that much of a help, since I couldn't restrict the search to title / author.

I'm currently at about 500 ebooks, ~250 of those are samples (I know, I need to clean out the old ones some time

) and ~100 are fanfics, so right now search works pretty well for me

.

Quote:

Originally Posted by JimmXinu

If I may humbly point you in the direct of EpubMerge...

I know that tool, I just was too lazy to use if for a three chapter fic

. I've used it once for a series of 19 or so shorts on AO3 which in reality were more like chapters of one big story.

JimmXinu · 10-17-2012, 01:08 PM

Version 1.6.14 - 17 Oct 2012

This one is mostly internal changes and minor fixes rather than new features for plugin users.

Changes to logging to make more logging visible in PI and the same across web/CLI/PI.
Don't number chapter when only one chapter even if add_chapter_numbers is on.
Fix a bug where cover could still be partially set to an image in the cover_exclusion_regexp list.
Add include_images support for HTML format.
Change HTML format output to include title/author in file name that appears inside zip file.
Preserve id and class attributes on the HTML tags. (This is for users who want more control from output_css. I don't *think* it will hurt anything for other users...)
Change grangerenchanted.com to use more relaxed HTML parser due to problems with some stories.

djrm · 10-17-2012, 09:14 PM

Quote:

Originally Posted by JimmXinu

That's not an unreasonable request.

However, there are limitations due to filesystem naming limitations. I can make it better, but I can't make it perfect.

Try this version and see what you think.

Looks good! Title and author now in the name: Random Glimpses-0atis-nsdjaj.html . Is the nsdjaj due to the filesystem naming limitation you mentioned?

DJRM

JimmXinu · 10-17-2012, 09:33 PM

Quote:

Originally Posted by djrm

Looks good! Title and author now in the name: Random Glimpses-0atis-nsdjaj.html . Is the nsdjaj due to the filesystem naming limitation you mentioned?

DJRM

Sort of. It's generated by the 'temporary file' utility in calibre that I use to guarantee unique file names. If it wasn't that, it would be something else.

It's really weird, IMO, how calibre treats HTML books. While it renames the files for all the other times, it keeps whatever the original file name was for HTML. Inside the zip file, that is, that it forces them to be in--but doesn't unzip again for viewing. It does follow links and images--but then it flattens the structure and changes the HTML to match.

And you're welcome.

Jim

ilovejedd · 10-18-2012, 11:16 PM

Quote:

Originally Posted by JimmXinu

It's really weird, IMO, how calibre treats HTML books. While it renames the files for all the other times, it keeps whatever the original file name was for HTML. Inside the zip file, that is, that it forces them to be in--but doesn't unzip again for viewing. It does follow links and images--but then it flattens the structure and changes the HTML to match.

If I remember correctly, Calibre converts all non-epub files to epub for viewing. Perhaps the behavior you're seeing is because of that?

JimmXinu · 10-18-2012, 11:30 PM

Quote:

Originally Posted by ilovejedd

If I remember correctly, Calibre converts all non-epub files to epub for viewing. Perhaps the behavior you're seeing is because of that?

I doubt it, wouldn't there be a very noticeable delay when viewing mobi, etc if that were true? Plus I found an article from a year ago claiming ebook-viewer could handle all the same types calibre can convert.

That did start me looking around, though. Under Preferences, Behavior, "Use internal view for:", you can check ZIP and then it does use it's own viewer. Much better.

davidfor · 10-18-2012, 11:37 PM

Quote:

Originally Posted by JimmXinu

I doubt it, wouldn't there be a very noticeable delay when viewing mobi, etc if that were true? Plus I found an article from a year ago claiming ebook-viewer could handle all the same types calibre can convert.

It does, but it does it differently to the conversion process. Don't ask me how it is different, but it has been discussed on the forum somewhere.

10-16-2012, 04:01 PM	#932
Dylan Tomorrow Connoisseur Posts: 62 Karma: 640495 Join Date: Jul 2012 Location: Germany Device: Kindle Touch, Android smartphone w/ FBReader	@JimmXinu: Thanks! I was just getting more and more annoyed that certain fanfic ebooks only indexed properly in their txt form! Of course I don't mind, I would be honoured to be linked from there . [Also, I did not know this FAQ exists. cough Because I had been wondering whether the missing TOC in MOBI was a known issue and turns out it was. ^^] Yes, the calibre conversion from EPUB to MOBI, while fixing the missing TOC, did not work to fix the indexing, nor did MOBI to MOBI (In hindsight I have not tried EPUB to KF8 nor MOBI to KF8 conversion though (option became only available in recent update IIRC), so it would be kinda funny if that works ). I think I read somewhere that the calibre devs care more about the files working then them conforming to XHTML specs and validating against Epubcheck, which makes alot of sense. It could be that the indexing engine on the KT is less forgiving towards tag soup than with earlier kindles? I have no idea as I don't have another Kindle device to test the files on ... On an somewhat unrelated note, after trying out various similar services, none of which really worked, I found a workable tool for livejournal fics: Readability.com. You just have to append ?format=light (intended for mobile use) to the end of livejournal urls and then give the URL to Readability via one of their many apps/extensions or directly on their website. You can either send an MOBI to your Kindle or download an EPUB! Of course this leaves you with separate ebook files for each chapter which you will have to merge manually if you care enough to go to that effort (I didn't, fic was only 3 chapters ) and there is some stuff at the end that Readabiltiy does not filter out, but it just works wonderful otherwise. According to Readability, the Kindle uses their code for the Article Mode of its browser. The results are indeed very similar. Last edited by Dylan Tomorrow; 10-16-2012 at 04:28 PM. Reason: +Aside on Livejournal fics via Readability

10-17-2012, 12:29 AM	#937
stormharper Junior Member Posts: 1 Karma: 10 Join Date: Oct 2012 Device: kindle touch	Overwriting Issue I'm having a problem getting updates onto my Kindle & am hoping someone here can tell me what I'm doing wrong. When I update a story using FFDL & then want to put the updated story on my Kindle - it reloads an older file that does not have the new chapter. I've tried it both ways - with the story not on the Kindle & once with it on the Kindle. Both times loaded an older file. However, if I check the file in Calibre it shows the new chapters. On the add to Kindle list I've got it set to add/overwrite. If some one knows what I'm doing wrong I would appreciate the help.

10-17-2012, 01:08 PM	#940
JimmXinu Plugin Developer Posts: 6,590 Karma: 4600349 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	Version 1.6.14 - 17 Oct 2012 This one is mostly internal changes and minor fixes rather than new features for plugin users. Changes to logging to make more logging visible in PI and the same across web/CLI/PI. Don't number chapter when only one chapter even if add_chapter_numbers is on. Fix a bug where cover could still be partially set to an image in the cover_exclusion_regexp list. Add include_images support for HTML format. Change HTML format output to include title/author in file name that appears inside zip file. Preserve id and class attributes on the HTML tags. (This is for users who want more control from output_css. I don't think it will hurt anything for other users...) Change grangerenchanted.com to use more relaxed HTML parser due to problems with some stories.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Count Pages	kiwidude	Plugins	1780	11-05-2024 10:30 AM
[GUI Plugin] Find Duplicates	kiwidude	Plugins	1111	10-07-2024 01:13 AM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 09:39 AM
[GUI Plugin] Resize Cover	kiwidude	Plugins	95	03-17-2024 12:55 AM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 01:27 PM

10-16-2012, 12:54 PM	#931
JimmXinu Plugin Developer Posts: 6,590 Karma: 4600349 Join Date: Dec 2011 Location: Midwest USA Device: Kindle Paperwhite(10th)	@Dylan Tomorrow - Wow, you've put a lot of effort into this. Do you mind if I link to this post from the FFDL FAQs? I assume since you went with KindleGen that calibre conversion from epub to mobi didn't solve the issue.

10-16-2012, 04:52 PM	#933
djrm Enthusiast Posts: 33 Karma: 10 Join Date: Aug 2010 Location: NJ, USA Device: iPad mini 4, iPhone 6s	Hi all. I have a question I hope someone can help me with. When I download from fanfiction.net to html, the name of the html file inside the zip file is a bit weird. e.g. new-42-xegfph.html (downloaded from http://www.fanfiction.net/s/6865661/1/). Is there any way to make FFDL use the title and author as the html name? Thanks. DJRM P.S. Love this plugin!

Advert

Advert