FLAG (Fanfiction.net Lightweight Automated Grabber)

[emmadiva]

Originally Posted by ilovejedd

Basically, you'll be running software for running websites on your home PC. Don't worry, it's not really all that difficult, you just need to follow instructions.

WOW - It's really not that difficult to do? LOL - I guess the word "server" sounds a lot bigger and scarier than I thought. I'm so relieved you're writing instructions. If it's outlined for me then I usually don't have much of a problem. However, when it isn't... You don't even want to know the ways I've butchered computers over the years, attempting to do things I had no clue about. Thanks for taking the time to write the guide up for everyone.

Originally Posted by ilovejedd

Actually, I already had it fixed Sunday night. I was planning on reading some fanfics before I went to sleep and bam, it wasn't working. I had to fix it. I just haven't had the time to upload the fixed script to my webhost.

HA! I don't know what I'm more happier about -- That you've fixed the script already or that I'm not the only one figuring out how to get their fanfic off the net in the middle of the night. I truly am, shamelessly addicted to them.

Originally Posted by ilovejedd

This isn't actually my project. It's erayd's. I'm just helping out a bit. Besides, the social life will be gone soon. My best friend from Colorado is visiting since she'll be gone to South Korea for a year or so. Her flight leaves next week so we're making the most of the time we have.

Well huge thanks to the both of you then. Erayd is amazing for creating the program and you for all your help. It's members like you that make forums like this a great place.

South Korea?! That's incredible. You have to go visit her. I'd kill to know someone there to go visit. Enjoy the remaining time with your best friend. Mine lives across the country so we webcam/skype all the time. I highly recommend it.

[emmadiva]

Hi all! Hope everyone had a fun weekend.

I absolutely LOVE the FLAG Webservice. It makes downloading my fics sooooo much easier. However, I've been desperately trying to figure out how to enter the ff.net recipe into calibre and for the life of me can't. LOL - I'm not in the least bit computer savvy but I can follow clear directions. Can anyone point me to some detailed instructions? I've tried Googling and haven't had much success.

I'm running Calibre on a Mac and would prefer to run FLAG in that operating system. However, I do have access to virtual PC software and will install it if necessary.

I apologize for being such a newbie to all of this. I truly love the program and would appreciate the help if anyone has the patience and time. If not, no worries. I'll just continue with the FLAG Webservice.

Quick question -- Is anyone else having problems connecting to webservice this morning? I keep getting the following message:

'Safari can’t open the page “http://flag.erayd.net/getstory.php” because it can’t find the server “flag.erayd.net”.'

[araevin]

I second that motion mac needs more representation!

[ilovejedd]

The code posted in this thread are not Calibre-compatible recipes. They are PHP scripts for creating/downloading printable HTML or ebook versions of fanfics.

You can find a Calibre recipe for FanFiction.Net here. Read the notes for recipe-specific instructions/tips. Refer to the Calibre online manual for instructions on how to add custom recipes. One caveat, though, is the recipe downloads a whole bunch of fanfics at the same time and saves them as one "omnibus"/collection file. Unfortunately, the current version of Calibre makes it difficult to get it to download files one at a time. Kovid has plans of creating a mechanism to download individual files in the future, but how soon he'll be able to implement it is unknown.

For getting individual HTML files which you can then convert to ebook format using Calibre, you can use

http://utterlyinsane.org/fflag/print.php?storyid=1234567

in case erayd's website is ever down. Obviously you need to substitute the actual story id for 1234567.

I know I promised a guide and clean/updated scripts and I'm really sorry I still haven't gotten to it. I just got hooked on watching Heroes and I'm currently distracted by reading Sylar/Elle fanfiction (yes, I'm twisted). I'm running out of fics to read so maybe I'll finally be able to get to working on the scripts this weekend.

As for Mac... I tried it (Hackintosh, OSX 10.5.5) and I found Linux (Kubuntu) easier to use. Sorry, won't be able to help in that area, aside from the Calibre recipe which should be universal.

[emmadiva]

Originally Posted by ilovejedd

The code posted in this thread are not Calibre-compatible recipes. They are PHP scripts for creating/downloading printable HTML or ebook versions of fanfics.

No wonder it wasn't working for me. Only I would do something so silly.

You can find a Calibre recipe for FanFiction.Net here. Read the notes for recipe-specific instructions/tips. Refer to the Calibre online manual for instructions on how to add custom recipes. One caveat, though, is the recipe downloads a whole bunch of fanfics at the same time and saves them as one "omnibus"/collection file. Unfortunately, the current version of Calibre makes it difficult to get it to download files one at a time.

I finally got it up and running on Calibre. It was actually pretty simple once I had the actual recipe. I must admit as fantastic as it is to get a huge file of goodies -- It's a HUGE file. A bit too much, even weened down to the most basic of my preferences in the RSS. I also don't like that you can't separate the fics to keep what you like and toss the others.

I have to admit that I much prefer the html/PHP version of Flag. It's much cleaner and more manageable. Thanks for giving the utterlyinsane link for individual HTML download when the other down. It's greatly appreciated.

I really do love this program and it's possibilities. I definitely need to figure out how to set this puppy up.

I know I promised a guide and clean/updated scripts and I'm really sorry I still haven't gotten to it. I just got hooked on watching Heroes and I'm currently distracted by reading Sylar/Elle fanfiction (yes, I'm twisted). I'm running out of fics to read so maybe I'll finally be able to get to working on the scripts this weekend.

I hope you enjoyed the time with your friend. Please don't apologize. You're doing a tremendous favor by doing it so definitely write it at your own pace. I have to figure out how to set up a server on my computer before I do anything. LOL - After I figure out what exactly one is and how to do it.

Is there any specific type of server? I'd only be using it for this specific task so if there's a basic type to set up, just give me the heads up and I'll start googling.

Have a fun night!!

[ilovejedd]

Originally Posted by emmadiva

Is there any specific type of server? I'd only be using it for this specific task so if there's a basic type to set up, just give me the heads up and I'll start googling.

I actually have extremely vague instructions on the custom recipe thread

Originally Posted by ilovejedd

I have a working recipe but it involves running an HTTP server w/PHP on your PC. How comfortable are you with that?

<snip>

Attached are the PHP scripts to create a printable version of a story. Easiest way to get it working would be to install XAMPP. You need to enable the php curl and php tidy extensions. In XAMPP for Windows, I just had to uncomment the following lines from \xampp\apache\bin\php.ini:

extension=php_curl.dll
extension=php_tidy.dll

Alas, the scripts attached to the original message are the ones FanFiction.Net broke in their recent update and the code for updated/fixed scripts are all over the place. Well, they're all over this thread at any rate. That's one of the reasons I need to create a clean updated version. So erayd can consolidate the sources and get rid of the script mess we have now.

[Onion Knight]

I love fan fiction so this has been pretty much a god send for me however the web service seems to have stopped working. I'm getting a 404 for every story id I try.

[ilovejedd]

Temporary measure, try using:

http://www.utterlyinsane.org/fflag/print.php?storyid=1234567

Obviously, you need to replace 1234567 with the actual story id. It only gives you HTML but you can easily convert it using Calibre or similar.

[sherryk_us]

Thanks for this and for stating the "obvious" about the story ID. I've learned to do some things on the computer, but more by memorizing steps than really understanding it. Many easy instructions are beyond me!

Sherry

[sherryk_us]

Wow, I tried this on a really long story with many parts and it worked. One file instead of many. Thanks! Now I can just put it in Sony format using Calibre--no TOC formatting required. Cool.

Sherry

[Mnementh]

Gah, looks like FFnet might have changed things again, numerous storyid's I try now are only pulling down a few chapters with some/most chapters just pulling down the title of the chapter but no body.

It's weird because some of the chapters are ok.

Anybody have any ideas?

[ilovejedd]

Originally Posted by Mnementh

Gah, looks like FFnet might have changed things again, numerous storyid's I try now are only pulling down a few chapters with some/most chapters just pulling down the title of the chapter but no body.

It's weird because some of the chapters are ok.

Anybody have any ideas?

Can you give me the story ids of the non-working titles? I'm not aware of FFNet having changed anything that would break FLAG compatibility. At a guess, it might have something to the with the default preg settings on PHP. Can't recall what it's called right now.

[Mnementh]

Here's two quick ones, the first is a huge fic (nearly 900,000 words) the second's a fair bit smaller...

2697521 this one misses chapter 5 to 8 iirc and then 11 onwards minus the odd chapter which it does pull down.

4605681 this one is only missing a few chapters 7, 8 & 9 but pulls down the last chapter.

The weird thing is it's not happening on all fics, as far as I can see it's random but as I tend to only read HP or PERN from FF I might have missed a pattern if there is one.

[ilovejedd]

Originally Posted by ilovejedd

Doesn't fetch story id: 4880173. Meta information is detected correctly but the chapter content is blank.

Originally Posted by ilovejedd

Figured it out. Lol, I didn't sleep until it was fixed. The fanfic was fairly long and the pcre.backtrack_limit was too restrictive (100,000). Changed the value to 1,000,000 and it works now.

Found the post. I think this might be the culprit. You'll have to restart Apache for the new settings to take effect.

[Mnementh]

You know just as you were posting that I re-read my post and saw the length and thought I'd read something about possible story length issues I was about to go re-reading the whole thread lol.

I'll try that and see if it works thanks.