PRS-500 RTF Tools - a simple tool for batch edtiting your rtf ebooks

[boradicus]

After getting my Sony Reader I determined to find some free ebooks that I could read on it. I already knew about Project Gutenberg, but unfortunately, many of the free texts there are not available in a format that is readily accessible to the Sony Reader.

After some searching I was able to locate in addition to some RTF conversion tools such as the free Amber Converters, a couple of websites that already provided ebooks in RTF format. I didn't happen to find much in the way of LRF format at all, however, I did notice a marked decrease in the RTF file size for an ebook in comparison to the LRF files for the same text.

The two websites I found were www.manybooks.net (my favorite for ease of use and nice presentation, as well as a great number of texts), and www.munseys.com. I also discovered a conversion utility called HTMLtoRTF Converter (the free version) that makes it quite easy to obtain texts that are only available in HTML format or displayed on websites (just right click the website use the 'view source' option to save the html of the page to a text file and then use the converter to convert the text file).

Because of the more universally acceptable format of the RTF standard, it's immediate usability by the Sony Reader, its leaness of file size, and its simplicity, I decided that this would be the best format for me to use for all freely obtainable ebooks for my Sony Reader.

This was half of the battle in creating a consistent library of ebooks that could be managed efficiently. Consequently, it is my recommendation to the Sony Reader community to adopt the RTF format for freely obtainable ebooks and etexts.

The next stage in realizing a consistent library of ebooks was the ability to provide Author/Title information. Here enters the RTF Tools project. This tool, so far, is designed to select an rtf file in a directory, and update all of the Author information in that file as well as the remaining files in that directory with a single click of a button. The current release provided this ability, but is not without bugs. Eventually, it would be nice to clean up any bugs in the project, and add a separate form with which to edit the Title information in separate files. I envision a button to switch between the Author editing window and the Title editing window.

This little tool should help reduce the sheer amount of keystrokes and savvy needed to edit the Author information in the RTF files. The Author information is set up to automatically display in the Sony Reader Library as <LastName, FirstName>.

I have posted the project, including its source (written in C#), and the wndows binary here: http://www.codeplex.com/rtftools/Wik...spx?title=Home

.NET will need to be installed in order to run the program.

Have fun ! And please feel free to modify and continue the original code as I am not certain how much time I will be able to devote to its continued development. Good luck and I hope this will be a useful tool for some of you if not all of you

-Mark

[HarryT]

"leaness of file size"? RTF? Are you joking?

RTF files are enormous compared with virtually any other format, especially if they have pictures in them.

Eg, to take an example from my eBook library completely at random: David Weber's "On Basilisk Station":

LRF file: 608kb
MobiPocket file: 676kb
RTF file: 1358kb

The RTF version is double the size of the others. For files with a lot of pictures in them, RTF files can be 10x the size of other formats - 20x the size, even.

RTF is a good choice for a long-term storage format, but please convert it to a proper eBook format for reading. The Sony's support for RTF is very poor - eg, it doesn't support pictures or a table of contents.

Using a tool like "Book Designer", it's the work of a few minutes to convert RTF to LRF and that way be able to view the pictures on the Sony, have a proper table of contents, etc.

[boradicus]

Harry,

I'm not sure what converter you have used for your RTF ebook files, but I have not found a case yet in which my RTF files are not smaller in size than LRF. I suppose it could be the fault of the LRF converter I had been using, but I suspect the difference in file size had more to do with DRM than anything else.

I suppose it is also possible that you have a poor RTF converter, which has yielded overly large file sizes through excessive tagging. In any case, it certainly was not my intention to upset you about your preferred ebook format. If you have a strong emotional attachment to the LRF format, then please continue using it. I however, have found it quite rational to use RTF for my own purposes - which are namely accessiblity of conversion, and ebook format availability. In fact, it is MUCH easier to convert an RTF formatted ebook into and LRF formatted ebook than to find the LRF formatted book in question for free on the web.

As far as tables of contents are concerned, while being able to jump around through a book's chapters as you would an HTML document - which may be useful to some readers, I find that if I desire such functionality, that my laptop serves beautifully. But as most readers of paper-based books will probably concur, a table of contents is used rarely if at all in most cases, except perhaps where techincal documents and reference material is concerned. However for the casual reader, a linked table of contents isn't very useful, and in fact can be rather annoying, detracting from the reader's experience.

As for pictures, I don't find them to be of great use on the Sony Reader. Again, for the casual reader, except perhaps for where periodicals and coloring books are concerned, pictures, other than perhaps the familiar cover art of a book, are not of great consequence. Neither is the Sony Ebook designed (at least the 500 model) to be a picture intensive device: the dispaly is in black and white LCD, which in my opinion is designed more for the purpose of providing PDF functionality than anything else. Sony probably made a good choice in choosing to support the PDF format for its universal accessibility as an ebook format. However, reading PDF formatted texts on the Sony Reader is rather impractical in most cases because PDF documents have only two resize options, and the text is often in raster format as opposed to vector image format, making resizing less clear and visible to the eye. Even if the text is stored in vector image format, there are much more efficient, smaller file sized formats that can do the same, resulting in a greater number of books one can load on the Sony Reader.

Part of an ebook reader's appeal is its similarity to an actual book. The fact that the LCD screen is not particularly bright or invasive to the eye for protracted periods of reading, the small size and portability, and even the cover flaps of the Sony Reader all contribute to this experience and help bridge the experiential gap between an electronically rendered text and a paper and ink one. The elegance of the Sony Reader is that it can hold a large number of texts, run for days on its battery, and be easily carried to work, to a coffee shop, to a doctor's office, or used on an airplane or bus, living room couch, or even be used at home in bed. It is a comfortable design, paired with the accessibility of easily obtainable public domain or free texts as well as newly published purchasable texts. It bridges the gap between the natural comfort zone of the casual reader and the accessiblity of the information age. I really think that the emergence of readers like Sony's is pivotal in transforming the world of modern readership, and whether one desires to use their proprietary LRF format, or to use RTF or any other format is just another example of the great amount of flexibility the Sony Reader offers.

Respectfully,

Mark

[AnemicOak]

Originally Posted by boradicus

Harry,

I'm not sure what converter you have used for your RTF ebook files, but I have not found a case yet in which my RTF files are not smaller in size than LRF. I suppose it could be the fault of the LRF converter I had been using, but I suspect the difference in file size had more to do with DRM than anything else.

In almost every case I've found RTF's to be much larger than any LRF I've created (not to mention larger than pretty much any other ebook format). Neither LRF or RTF have DRM (that would be LRX) so that would have nothing to do with file size (not that DRM adds much to file size from what I've seen).

[Dr. Drib]

Hi boradicus, and welcome to MobileRead.

RTF files sizes:
I'll have to disagree with you on the size of RTF files. They're huge. Harry and I (among others), routinely convert (read: lovingly assemble) LRF files for the Sony upload section.

Photos that enhance one's reading experience:
Additionally, the absence of photos is a downside to having RTF files on the Sony Reader. Check out Harry's work on Charles Dickens: In my opinion, these are commercial-grade assembled books, with beautiful illustrations and presented in a format that puts Sony's classics to shame.

When I make available novels from old pulps or old paperbacks, I often like to add old cover photos. I find they add to the reading and/or presentation of what's going to be read.

Links to chapter headings:
Having links - on the Reader and not on the computer - is a very utilitarian advantage over RTF. Frankly, Sony's support for RTF is rather poor.

No one's mad:
We're all passionate, and I think that's where most of us are coming from.

Welcome:
Let me again welcome you to this Forum and to MobileRead. Your comments are thoughtful and are appreciated.

Respectfully,
Don

[boradicus]

Originally Posted by AnemicOak

In almost every case I've found RTF's to be much larger than any LRF I've created (not to mention larger than pretty much any other ebook format). Neither LRF or RTF have DRM (that would be LRX) so that would have nothing to do with file size (not that DRM adds much to file size from what I've seen).

Brian, I stand corrected ! Thanks for your reply, and I hope that this tool will yet be found useful .

-Mark

[JSWolf]

If I have an RTF file I need or want to edit, I do use Microsoft Word 2003. When I download eBooks from say project Gutenberg, i edit them in MS Word, save to RTF if originally text or back to HTML if HTML. Then I load into Book Designer. to finsih the process and create the LRF. I have used html2lrf for some eBooks from PG. But if they start off as text, it will be Book Designer.

[NatCh]

I've found that RTFs are much bigger than LRFs tend to be as well. That being said, running them through WordPad always reduced their size drastically, but one of the things that WordPad strips off is the Author/Title metadata. That's one of the things that moved me into LRFs, actually.

Even so, I appreciate your making a tool like this, boradicus. An excellent "entry" to MobileRead!

And welcome, by the way.

[kovidgoyal]

The reason RTF tends to be bigger is because it encodes everything including picture data in ASCII, which is very inefficient. In contrast most e-book formats, LRF included typically use some form of compression on their data internally.

[HarryT]

It is specifically having pictures in an RTF file which makes it large. Each pixel in the image ends up as several ASCII characters in the RTF. Adding a 100kb JPG image to an RTF file commonly makes the RTF file 2-3MB larger. Some of the Dickens books I've created have 70 or so pictures - imagine what that would do to the RTF .

No, I'm not particularly "passionate" about LRF files - I no longer even own a Sony Reader (although I used to) - but I do genuinely believe that, for the Sony Reader, LRF is by far the best choice for "reading" format, and we are lucky enough to have a number of excellent tools for creating LRF files - "BookDesigner" and "libprs500" to name but two, both of which have been created by members of this board.

[boradicus]

Thank you all for the wonderful recommendations ! - from what I have heard I think that we can benefit from the best of both worlds - especially if I can get the latest rev finished for my RTF tools program. Although I have not done any DOS batch file programming in a while, it seems to be that there might be a way to create a batch file routine for calling Wordpad to open and save RTF files, thereby slimming them down prior to LRF conversion. Here is what I am currently working on, and here is my idea about how this could be used in conjunction with an LRF conversion tool -

Currently, I am working on a variation of the initial RTF Tools program I wrote that uses a directory structure to obtain and write Author and Title information readable by the Sony Reader. Basically, my intention is to read through all the folders in a directory (one level deep) and obtain the Author's name from the Folder, and the book's Title from the filename, inscribing via a one-shot batch routine both Author and Title information into the RTF files. Originally, I was planning on using this as a sole means to an end - the use of RTF files in the Sony Reader, however thanks to members of the community here is my revised idea involving the LRF format:

1) because ebooks are easily and freely obtainable in RTF format - which is also easily manipulated - acquire ebooks in RTF format or convert titles not already in LRF or RTF to RTF. There are a variety of tools with which to do this, including the free Amber Converter, and a program called HTMLtoRTF Converter (also free). Ebooks already in RTF format can be found on www.manybooks.net (free) and www.munseys.com (free).

2) Save the RTF formatted ebooks to a directory structure comprised of the file name as the Title of the book, and all books for a particular Author (or primary author in the case of Multiple authors saved in a folder name comprised of the Last name and First name of the Author, separated by a comma - this limitation only because I believe there is currently only one Author allowed for the purposes of the Sony Reader).

3) Run a DOS based batch program to call Wordpad to open and save the various RTF formatted ebooks, reducing thereby their file sizes and any unecessary formatting.

4) Because Author and Title information are purportedly destroyed by Wordpad, and because not all ebooks will have the Author and Title information already available, run my RTF Tools batch program to add the Author and Title information back into the RTF ebook files.

5) Run the appropriate RTF to LRF conversion program

6) Voila ! YOU have just obtained and formatted a hopefully large library of freely obtainable ebooks for your Sony Reader !

THanks - and please let me know what you think of my ideas !

Mark

[kovidgoyal]

take a look at calibre, i wrote it to solve exactly these problems, but in a format neutral way.