Yet Another Gutenberg Book/HTML converter

[AprilHare]

OK: I downloaded FangornUK's script and tried it. Other then it complaining it couldn't find perl (the first line in the script appeared to have ^M next to it; fixed by editing with gedit) I get the following error:

Can't locate Archive/Zip.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at ./gutlrf.pl line 17.
BEGIN failed--compilation aborted at ./gutlrf.pl line 17.

[tompe]

You have to install Zip.pm in some way. Either using a package in your distribution or using CPAN (e.g. "perl -MCPAN -e shell").

[pilotbob]

Originally Posted by AprilHare

OK: I downloaded FangornUK's script and tried it. Other then it complaining it couldn't find perl (the first line in the script appeared to have ^M next to it; fixed by editing with gedit) I get the following error:

Yes, alot of the .py scripts you will find out there have CRLF line endings which Mac/Linux doesn't like. You should convert to LF only.

BOb

[AprilHare]

Very good, finally got the script going:
Of course, it didn't have to be all fair sailing and here is the output from my first conversion:

:~/Desktop/untitled folder/CleanMe!!!/gutlrf$ ./gutlrf.pl http://www.gutenberg.org/files/17297/17297-h.zip
... 0KBytes

Extracting files...

Book Title: British Highways And Byways From A Motor Car
Author : Thomas D Murphy

Cleaning HTML...
Wrote cleaned HTML "/tmp/17297-h/new.htm"
Converting to BBeB...
Processing u'new.htm'
Parsing HTML...
Converting to BBeB...
An error occurred while processing a table: AttributeError("'module' object has no attribute 'tt0011m_'",). Ignoring table markup.
An error occurred while processing a table: AttributeError("'module' object has no attribute 'tt0011m_'",). Ignoring table markup.
Rationalizing font sizes...
Output written to /tmp/17297-h/British Highways And Byways From A Motor Car.lrf
Segmentation fault
Died at ./gutlrf.pl line 261.

[FangornUK]

No idea, that book conversion works fine for me (coincidentally one of my favourites).
gutlrf.pl 17297
Your error is coming from Calibre, make sure you've installed the latest version and have all the dependencies Calibre requires (as I assume you're on Unix).

P.S. I've updated the gutlrf.pl to be in Unix/OSX text type so it doesn't have the ^M problems, will also work fine on Windows.

[kovidgoyal]

the table error is fixed in 125

[AprilHare]

Kovid, I had 125 installed. I'll upgrade to 126 and the latest gutlrf and see what happens.
I'm using Ubuntu 8.10 Intrepid Ibex AMD64.

[AprilHare]

OK; the new script behaved differently, with 126 and the latest gutlrf.pl:
The contents links all point to page one according to LRF viewer.
Output:

:~/Desktop/untitled folder/CleanMe!!!/gutlrf$ ./gutlrf.pl http://www.gutenberg.org/files/17297/17297-h.zip
Extracting files...

Book Title: British Highways And Byways From A Motor Car
Author : Thomas D Murphy

Cleaning HTML...
Wrote cleaned HTML "/tmp/17297-h/new.htm"
Converting to LRF BBeB...
Processing u'new.htm'
Parsing HTML...
Converting to BBeB...
Rationalizing font sizes...
Output written to /tmp/17297-h/British Highways And Byways From A Motor Car.lrf
Segmentation fault
Died at ./gutlrf.pl line 261.

[AprilHare]

I opened new.htm with Amaya 9.5.5 and it gave me the following:

*** Errors/warnings in /tmp/17297-h/new.htm
line 6, char 0: Invalid or unsupported XHTML element <META>
line 7, char 0: Invalid or unsupported XHTML element <META>
line 82, char 4: mismatched tag

Line 82 refers to a errant </head>

[FangornUK]

Updated:

• Now process etext by paragraph "<p>" to catch split lines more documents appear to be using.
• Improved detection of HTML markup

Download it from the link on the first post of this topic.

[Alpharesearch]

Hello FangornUK,

first thank you for your Perl script... I added two lines and now the script also creates ePUB files:

$lrfbook=File::Spec->catfile($outdir, "$BOOKTITLE.lrf");
$epubbook=File::Spec->catfile($outdir, "$BOOKTITLE.epub");
#Everything setup so create the actual LRF BBeB book
system ("html2lrf \"$outfile\" -t \"$BOOKTITLE\" -a \"$AUTHOR\" --category=\"Project Gutenberg\" -o \"$lrfbook\" --disable-chapter-detection --left-margin=25 --right-margin=25 --link-exclude=http:") == 0 || die $!;
system ("html2epub \"$outfile\" -t \"$BOOKTITLE\" -a \"$AUTHOR\" --publisher=\"Project Gutenberg\" -o \"$epubbook\" --profile=PRS505") == 0 || die $!;

What license does your script uses GNU GPL? I have not problem to put my change under GPL... I'm asking because I found a bug in html2epub and I would like to attach your script for reproduction to the bug tracer...

With the Calibre viewer look at PG book 201, the ASCII art looks good in ePUB but not so good in LRF... but there is a bug in the second half of the ePUB book... just scroll down.

the original html form PG

[ASCII approximation follows]
</H3>

<BR>

<PRE>
                           My view of Lineland

                                ---------
                                |       |
                                | Myself|
                                |       |
                        My eye  o--------


   Women  A boy       Men        The KING        Men       A boy  Women
  + + + +   -   --- -- -- -- --  (>----<)  -- -- -- -- ---   -   + + + +
                                  ^    ^
                                The KING'S eyes
                                much larger than the reality
                                shewing that HIS MAJESTY
                                could see nothing but a point.
</PRE>

<BR>

<P>
"I am no Woman,"

in ePUB some Text and a </PRE> is missing...

[ASCII approximation follows] 
</h3> 
 
<p style="margin: 0pt; border:0pt; height:0pt">Â*</p><pre> 
                           My view of Lineland 
 
                                --------- 
                                |       | 
                                | Myself| 
                                |       | 
                        My eye  o-------- 
 
 
   Women  A boy       Men        The KING        Men       A boy  Women 
  + + + +   -   --- -- -- -- --  (&gt;---- 
 
<br/><p> 
"I am no Woman," replied the small Line.

Thanks,
Markus

[FangornUK]

There's no license. I can easily add the ePub output as an option. I have a PRS500 so it doesn't support ePub and I assumed LRF and ePub output would be the same so I would have sticked with the native LRF.

[Alpharesearch]

Originally Posted by FangornUK

There's no license. I can easily add the ePub output as an option. I have a PRS500 so it doesn't support ePub and I assumed LRF and ePub output would be the same so I would have sticked with the native LRF.

ePUB seams to preserves the ASCII art (for example the PG 201 book). To me it also feels like it loads faster than the LRF on the PRS505. It would be nice if you could add a switch to your script for ePUB, thanks.

Markus

[FangornUK]

gutlrf strips out the <pre> sections as I've only seen the Gutenberg disclaimer in that section before. That needs a bit of work.

ePub is still not quite mature on the PRS-505 (no right justification) so I personally would stick with LRF for now. ePub looks to be the future though and has better formatting than LRF, nice inline image support etc.

I also do not know what ePub looks like on the PRS-505, it looks great on the Calibre Viewer, but that's not exactly the same as the PRS-505.

[FangornUK]

Updated:

• Now can also output ePub files as well as LRF.

Finally I've upgraded my PRS-500 to a PRS-505 so now I can read ePub files (also the battery was dying). I recommend converting Gutenberg HTML files in ePub format as it supports inline image display - much better than LRF files.


Download it from the link on the first post of this topic.