09-21-2005, 08:33 PM | #1 |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Track Yahoo Finance message boards via RSS
I just wrote YahooMsg2RSS, a new tool which allows you to track Yahoo Finance message boards via RSS feeds. You can use it for instance to track everything that is going on in the Palm message board.
Your suggestions are always welcomed. |
09-22-2005, 09:58 AM | #2 |
Nameless Being
|
very nice! i was wondering when someone would do this (like yahoo).
would something like this be adaptable to, say, treocentral boards, or others very generically? thanks. |
Advert | |
|
09-22-2005, 10:01 AM | #3 | |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Quote:
|
|
09-22-2005, 10:25 AM | #4 |
Nameless Being
|
yeah, i realized that, then thought it could work like a plugin to a reader and each board could be user configurable, since all boards have certain things in common despite different layout. anyway, just smells like a useful bigger thing to me!
i'm checking palm now, seems updating is sporadic. do you have the bandwidth to run this through mobileread (assuming that's how you do it)? i use greatnews (and thunderbird and feedreader depending) and the validate feed tool came back with an error (for palm) but a few recent posts updated though not the latest. |
09-22-2005, 10:49 AM | #5 | ||||
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Quote:
Quote:
Quote:
Quote:
|
||||
Advert | |
|
09-22-2005, 11:26 AM | #6 |
Nameless Being
|
you and jack should get together on this plugin idea
so, you mean regardless of the interval i have set in greatnews or manual force update, the tool pops max once an hour and then only gets last 40? i mean, i can see why, but.. |
09-22-2005, 11:38 AM | #7 | |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Quote:
The problem is that there are some smart people out there who update their feeds every 10 seconds (!) and that would definitely be a killer for a project like this one (not to mention Y! probably banning our server). |
|
09-22-2005, 12:39 PM | #8 |
Nameless Being
|
how about once every half hour?
much of yahoo is entertainment only, but some boards are actually amazingly useful and knowledgable.. funny that you're in switz; i'm in rome these days. btw, you've posted good stuff over on GN. best. |
09-22-2005, 01:33 PM | #9 |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Lemme explore the logs in the next few days and depending on traffic I'll reduce it to every 30 mins.
|
09-26-2005, 07:30 AM | #10 |
Nameless Being
|
alexander,
i've been using your tool for a few days now, and it's changed forever how i read the Y! message boards. really terrific. in GN i can sort by author and get rid of known bad ones/keep known good ones, and save useful stuff. for multiples faster than before. i don't know your plans for this, but hope you turn it into a generic tool. someday phpBB and the others might have this sort of feature built in, but.. it'll be a one by one progression etc. msg boards are a different beast, but i wish more sites did what PubMed does, ie allow sending a user customized keyword search as an RSS feed. some feedback. i know it's only a yahoo message board, and you don't need two-phased commit or checksum architecture to guarantee message integrity. but, you sort of have to guarantee no message gaps. palm eg. is in a very interesting period, and heavy msg volume friday led to hourly gaps. it's the ancient problem of tradeoff between getting, in realtime, both very high volume and very low volume messages. conversely, on one very lightly used board, i got duplicates after i deleted posts i didn't want. but not all, just a few reappeared. i will watch this carefully again and report a better description. thanks! |
09-26-2005, 11:30 AM | #11 |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
alta, I am happy you like the tool!
Regarding 'generic': It's often up to the webmaster whether to offer RSS feeds for his forum. For instance, Mobileread has RSS feeds for every forum section. Returning RSS feeds based on customized keyword searches sounds useful, but causes some serious scaling issues. As long as you restrict the number of feeds, you can easily cache them for use by multiple people. But if you offer keyword searches, it's possible that your server has to great hundreds if not thousands of individual RSS feeds, definitely putting a lot of stress on any server (bandwidth and CPU). I am still not sure about refreshing the feeds more than every one hour. So far word hasn't spread much of this tool - but if more people start using it, one hour might be the best frame to limit stress on the servers. Regarding duplicate posts - that issue is most likely related to your feed client (GN) which doesn't keep track of what items you've deleted before and once you refetch the feed, the previously deleted items come back. update: I've changed the update frequency to every 30 minutes. Let's see how it goes... Best Alex |
09-27-2005, 10:34 AM | #12 |
Nameless Being
|
yes, the best way is for all boards to offer a feed. but they don't. surprising about yahoo, they have almost everything else.
as an fyi, check out http://www.ncbi.nlm.nih.gov/entrez/q...arch&DB=pubmed where a p53 gene search is returned. Send to dropdown, RSS feed. i'm guessing they have a lot of resources, every reasearcher in the world can do a custom search(es) on a very large database.. yes, it's GN reposting deletes, i recall the bug mentioned now. i guess the refresh rate is of secondary importance to message gaps. thanks. |
09-27-2005, 10:41 AM | #13 |
Nameless Being
|
forgot to mention, feed for EXAS doesn't validate..
thanks. |
09-27-2005, 11:42 AM | #14 | ||||
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
Quote:
Quote:
Quote:
Quote:
|
||||
09-27-2005, 11:57 AM | #15 |
Fully Converged
Posts: 18,171
Karma: 14021202
Join Date: Oct 2002
Location: Switzerland
Device: Too many to count here.
|
OK, I added some extra sanity checks in 1.01 to remove illegal content from broken boards such as EXAS.
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Hello from WA, love the boards! | Ayrahvon | Introduce Yourself | 11 | 02-16-2010 06:06 PM |
School boards, Religion and Politics... | kennyc | Lounge | 7 | 02-15-2010 04:11 PM |
30 Free Ebooks on Personal Finance | Nate the great | Deals and Resources (No Self-Promotion or Affiliate Links) | 0 | 01-28-2008 02:45 PM |
Yahoo! Finance News through RSS feeds | TadW | Lounge | 0 | 01-13-2005 09:06 AM |