Sunrise 0.3 released

[Laurens]

This release adds Atom support, improves scripting and caching and contains a slew of changes and bug fixes. Thanks everyone for the feedback.

Quick Tour (lots of screenshots)

Change Log

Download from PalmGear

[hacker]

I don't see an option in here to increase the amount of available RAM used by the JVM in Windows, as you mentioned you were going to add. This is easy to do in Linux and FreeBSD, but not so easy in Windows.

Using java.exe -Xmx512M %* doesn't work here in a wrapper, obviously.

As such, it can't convert about 90% of the documents I regularly convert for users and commercial clients with the Perl and Python Plucker tools. Since Sunrise itself is an executable stub, I can't just run the jar directly, without decompilation of the .jar files to fix the manifests.

Any ETA on that feature being added?

I'm just using Sunrise as a datapoint in my feasibility analysis right now, and it still has quite a bit of catching up to do.

[Laurens]

The max. amount of RAM was already increased from 128Mb to 256Mb and the image handling was optimized so that images are cached to disk rather than kept in memory. This is basically all I'm going to do about the memory usage. There's no point in spending any more effort, my plate is full already. (I have food spilling all over the floor, so to speak.) Even so, Sunrise still converts typical documents just fine.

Both Sunrise and JPluck are simply not suitable for converting super-size documents because of the wasteful use of memory in the core JPluck library, which I talked about in the other thread. (For the sake of comparison, the JPluck EXE launcher only allocates a maximum of 128Mb.)

As for a solution: you can launch Sunrise from the command-line and thus pass the maximum amount of memory to use:

java -Xmx1024m -cp .;build\sunrise.jar;lib\commons-logging.jar;lib\hsqldbmain.jar;lib\jetty.jar;lib\n ekohtml.jar;lib\servlet.jar;lib\smalljs.jar;lib\sp inner.jar;lib\swt.jar;lib\xercesMinimal.jar com.distantchord.sunrise.Desktop

where -Xmx####m is max. amount of megabytes that the VM is allowed to allocate. (Naturally, everything should go on one line.)

Starting multiple instances this way will fail. (The EXE launcher creates a mutex to prevent this.) If you're on Windows XP you'll lose the "rounded" look-and-feel as the EXE contains a manifest for enabling it.

I just looked on plkr.org and your Perl spider isn't even available, at least not from there. The only thing Perl thing I could find were some 4-year-old files in CVS. Seriously, I think it's about time you made it public. If it's really so great, it deserves a release.

Anyway, I'm not interested in a competition in who has the best Plucker tool. Sunrise simply is what it is, take it or leave it.

[hacker]

The only thing Perl thing I could find were some 4-year-old files in CVS.

Right, that stuff is ancient, old, crufty, and probably nowhere near a working solution. Thats why it doesn't appear in most releases. It was also not entirely authored by me. Pat and I worked on that for a day or two, and decided to just drop that approach altogether.

Also, I'm not releasing my Perl spider at this point, because there is no compelling reason to, and it isn't ready for public consumption. I'm constantly tweaking and adding functions, in my diminishing spare time (having a daughter, a new house, and an influx of freelance work can do that to you).

What's more-interesting though, is that JPluck has no problems converting the sample documents I have here (2.1.6b, at this point), and it takes about 1/20th as long to convert them, as Sunrise. Sunrise dies right around 80%, after taking about 15 minutes to try to convert a single site.

JPluck, by comparison, takes about 1-2 minutes. I don't care if both tool converts them to pdf, or text, or separate image files.. the point I'm trying to elucidate here, is that JPluck still beats Sunrise in speed and function, and both are from the same author.

Sunrise is definately significantly slower on the same hardware, converting the same (validated HTML) sites than JPluck does. With that information in hand, what would the difference be? Why such a dramatic difference?

At one point, you stated (either here, or on one of your news articles; I don't seem to recall, and it seems to have disappeared at this point), that Sunrise included NONE of the JPluck code, and now you seem to imply that it does, but that parts of it were rewritten to work better. Which is it?

[Laurens]

Would you care to point out which site you're talking about here? There's no point in debating if you make such vague assertions. I know for a fact that Sunrise converts typical web content (i.e. BBC News, Onion, etc.) way faster than JPluck by virtue of it being able to convert multiple documents simultaneously. That's what users have been telling me as well. Apparently, your experience is different, but without any concrete pointers to the content that you're dealing with I cannot dispute that.

As for Sunrise containing JPluck code. It only uses the library for generating Plucker content, the bulk of which I wrote two years ago and used throughout JPluck 0.x, BlogPluck and JPluck 2.x. This is only a tiny fraction of the whole. Everything else - the UI, spidering, threading, SDL handling, HTML/RSS/Atom parsing - is totally new. (When I remove the Plucker generation library it shares none of the code.)

[hacker]

Would you care to point out which site you're talking about here? There's no point in debating if you make such vague assertions.

So "vague", that it was clearly posted more than a week ago, in a thread which you directly responded to?

You know what, don't even bother replying to this thread. Every time someone questions your software in any way, even with 100% valid technical merit, in a clear and concise way, you get defensive, and start ranting. Its not even worth having a discussion with you about it.

You did it on the mailing lists, you do it here, and myself and others are rapidly tiring of it. Nobody is attacking you, so drop the rhetoric please.

I'm done, this is a waste of my time, and the time of my clients who are interested in the alternatives out there.

[Bob Russell]

Hey, hey, hey guys!
You are BOTH such a great asset to us both at MobileRead and in the mobile device reading community!!!

Lauren, Sunrise looks like a fantastic tool to me, and I am excited to give it a go when I upgrade my PDA. I doubt any tool is going to cover every base perfectly, but it sure looks like it does a lot of things very well and I'm anxious to use it.

And, Hacker, I don't know details, but it sounds like you've contributed a lot with the Plucker team and are now preparing more for us in the PDA world to be revealed later. I'm also looking forward to seeing your contributions as I expect they will be significant. Your 9/11 commission iSilo report, for example, is great!!

You both have not just a lot of knowledge, but a passion for your work. I want to see that grow and hope that you will continue to enjoy the work you are doing for the sake of all of us. Certainly there is a temptation to aggressively challenge the strengths and weaknesses of your work as well as others. But, please, let's take extra care to do it in a way that stays positive and supportive. I know when I am giving as much as you guys are certainly giving of yourselves to do this kind of quality work, criticism or a lack of appreciation can sap me emotionally even if it doesn't stop me. I want to see you guys both stay happy (and productive! )

Bottom line, we really need you both! I can't emphasize that enough, and I hope you'll consider it in any moments of frustration. Beyond the two of you and various technical disputes, there's a whole lot of people here (like me!) that depend on the work of experts like you that generously provide wonderful tools and we are eagerly looking forward to all that you guys contribute. You're both part of what's really great about the PDA world and it's much of what draws me to it.

So pleeeeeeeze... be patient with each other and with us. And understand how much you are both appreciated, not just by me, but by many!!! We may not ever properly and completely appreciate all the long hours and expertise you provide, and sometimes we may even accidently mis-assign credit that is due (sorry hacker!!), but we do send much much thanks your way.

And not just for your "products", but it's also really nice to have you both sharing here in the forums. If either of you were to withdraw even partly, you'd be sorely missed. I may have embarrassed the other editors by my plea here, but I'm sure even if they don't like my style they would agree with me that you are both needed, wanted and appreciated and that it wouldn't be the same without you.

Take care,
and Happy Labor Day!!!
Bob R

[mercedes]

Sunrise works fine for me. I am sure that the majority of end users will use the tool to covert rss feeds and palm-friendly web sites not for converting manuals or big documents. I have both Sunrise and Jpluck installed but I will use Sunrise exclusively as soon as we have SDLs for Reuters, CNET, The Register, and hopefully NYT text edition. Laurens is a very talented developer and I look forward to his new reader.

[Laurens]

Originally Posted by hacker

So "vague", that it was clearly posted more than a week ago, in a thread which you directly responded to?

You know what, don't even bother replying to this thread. Every time someone questions your software in any way, even with 100% valid technical merit, in a clear and concise way, you get defensive, and start ranting. Its not even worth having a discussion with you about it.

You did it on the mailing lists, you do it here, and myself and others are rapidly tiring of it. Nobody is attacking you, so drop the rhetoric please.

I'm done, this is a waste of my time, and the time of my clients who are interested in the alternatives out there.

David, please understand I'm just calling for a fair and honest argument. Your own opinions are strongly-worded and I respond in kind. Maybe we should both chill out for a while. The FreeBSD manual you are talking about works fine here. (It already did with Sunrise 0.2.1.) Have you actually tried converting it with the latest release?

I take the time to address your concerns, but all I get from you is resentment. You may question my work on its technical merits, absolutely, but leave the character assassination out, please. I can honestly say that I'm sad to hear this coming from you. I've put in hundreds of hours in developing JPluck and Sunrise and many users are thankful for that. Indirectly, I've furthered the Plucker cause as well, so we are both on the same page here.

[Laurens]

Originally Posted by mercedes

I have both Sunrise and Jpluck installed but I will use Sunrise exclusively as soon as we have SDLs for Reuters, CNET, The Register, and hopefully NYT text edition.

You get good service today! I just posted a ZIP with example SDLs. It contains SDLs for all the sites you mention.

[mercedes]

Excellent! Thanks Laurens. They work fine, no problems at all.

[Jeff Kirvin]

Just downloaded 0.3, and all the Help screens try to open a port on 127.0.0.1 (localhost). Kinda need this to work so I can get the link to "Send to Sunrise" for Firefox...

[Laurens]

Originally Posted by Jeff Kirvin

Just downloaded 0.3, and all the Help screens try to open a port on 127.0.0.1 (localhost). Kinda need this to work so I can get the link to "Send to Sunrise" for Firefox...

Sunrise creates a web server on 127.0.0.1:2575. Are you using a firewall that won't allow this? Or running another server that uses this port? The server needs to be enabled for "Send to Sunrise" to work.

[taylor]

Just out of curiousity, I tried the freeBSD manual with Sunrise 3.0 and had no problems generating a 5MB plucker document (this included the indexed page as well as the whole thing on one HTML page, so it really was two copies of the manual). On my machine it took Sunrise a bit under 2 minutes to complete.

Laurens,

I've been using Sunrise with Java 1.5.0 RC and it works great for me. I do have a few questions/comments. What is your preferred method for dealing with these? Posts here, somewhere else, or email?

[Laurens]

Originally Posted by taylor

I do have a few questions/comments. What is your preferred method for dealing with these? Posts here, somewhere else, or email?

If you think the discussion would be interesting to other people, just ask in this thread. Otherwise feel free to e-mail me directly. You can find my address in the Sunrise readme or you can launch your e-mail client by selecting "Help -> E-Mail Author" from within Sunrise.

(I have also enabled "Receive Email from Other Members" in my MobileRead user profile. I'm not sure if this works, though. If I click on it myself I get the message: "Sorry! That user has specified that they do not wish to receive emails.")