soPdf - Better than Yet another PDF to LRF converter

[theguru]

I really liked the pdflrf tool from the "Yet another PDF to LRF converter" thread, but it has been taken down by the moderator for violation of GPL and has been down for quite some time because it seems like the author is not interested in providing the source for his tool. But there are some issues with the pdflrf tool.

1. pdflrf renderes the pdf into image and then creates the lrf file.
   This makes the 4mb pdf file grow into more than 40mb file.
2. No text information is preserved because of the image conversion
3. Very slow
4. No source for the tool <-- biggest disadvantage

So I decided to write a tool for myself. soPdf is a pdf formatter for sony reader. It is based on sumatrapdf's version of mupdf and fitz.

The advantages of soPdf over pdflrf

1. Pdf to Pdf conversion
2. Text and other contents of pdf are preserved
3. Size of the output file is very close to size of input file
   and in some cases smaller than input file.
4. Super fast conversion compared to pdflrf.
5. Source available to make further changes !!!!!! <-- biggest advantage

The disadvantages over pdflrf

1. Cannot yet convert the comic book. It can still split the image pdfs into two.
2. soPdf is in alpha stage. (ver 0.1). There may be lots of bugs to be found yet. At least all of the mupdf bugs.
3. ???

soPdf command line options

about: soPdf
   author: Navin Pai, soPdf ver 0.1 alpha
usage:
   soPdf -i file_name [options]
   -i file_name   input file name
   -p password  password for input file
   -o file_name  output file name
   -w               turn off white space cropping
                        default is on
   -m nn           mode of operation
                       0 = fit 2xWidth *
                       1 = fit 2xHeight
                       2 = fit Width
                       3 = fit Height
                       4 = smart fit Width (not yet implemented)
                       5 = smart fit Height (not yet implemented)
   -v nn          overlap percentage
                       nn = 2 percent overlap *
   -t title         set the file title
   -a author     set the file author
   -b publisher  set the publisher
   -c category  set the category
   -s subject    set the subject
   -e               proceed with errors
   -r               reverse landscape

   * = default values

The conversion algorithm is as follows

1. If user specified Fit2xWidth or Fit2xHeight then simply make two copies of pdf page from source into destination pdf file.
2. Render the page and get the actual boundary box that encompasses all of the content in the page. This step removes all the white space border of the page.
3. If page cannot be rendered by mupdf and error option is specified then split the page w/o rendering by setting the MediaBox of the page.
4. Try to split the file first by iterating all the elements that can fit in half a page and if that does not work then split the file half way with 2% overlap (this can be changed).
5. If FitWidth or Fit2xWidth is specified then rotate the page by -90 deg.

Source code for soPdf is available from google code.
http://sopdf.googlecode.com

To compile the source code you will need Visual Studio 8.0 (Even free edition will work). Visual studio is not required if you just want to run the soPdf tool. If you are having issues running the binary then make sure you have VC runtime library. You can download the VC runtime library from Microsoft website.

Coming soon

• Output to image pdf - for complex pdf that renders slowly on the reader devices.

Update 0.1 Rev 12

• Added reverse landscape mode. Ever wished that you could hold your reader the other way around in landscape mode and scroll thru the pages using your right thumb. Use reverse landscape mode and start reading from last page onwards.

Update 0.1 Rev 10

• Proceed with error option. With this option, soPdf can now process any pdf file, even the ones mupdf cannot handle. If mupdf cannot load the contents then it simply splits the page into two w/o any processing. The disadvantage is that the white space border in this case is not removed but you can still get a pdf output file.
• Set subject of the pdf file option
• Fixed stack over flow when processing complex pdf files
• Better clipping algorithm

Update 0.1 Rev 7

• Work around a mupdf bug where it is not able to allocate oid and gid numbers. This prevented some of the files from being split properly.

[godel10]

Thanks for your effort.

I am not an user of Windows, so I wonder if anyone could upload an example of an input file and an output file.

[ProDigit]

I'd suggest you to try recoding the prs-505's manual again,it seems kind of buggy!

[ddavtian]

Does this mean I need VC runtime (have no idea what it means)?


Error: .\mupdf\pdf_xref.c(459) : pdf_loadindirect() - cannot load indirect objec
t 1586
Error: .\mupdf\pdf_xref.c(442) : pdf_loadobject() - cannot load object 1586 into
cache
Error: .\mupdf\pdf_xref.c(416) : pdf_cacheobject() - found object 1636 0 instead
of 1586 0

[theguru]

Originally Posted by ProDigit

I'd suggest you to try recoding the prs-505's manual again,it seems kind of buggy!

This bug has been fixed.

[theguru]

Originally Posted by ddavtian

Does this mean I need VC runtime (have no idea what it means)?


Error: .\mupdf\pdf_xref.c(459) : pdf_loadindirect() - cannot load indirect objec
t 1586
Error: .\mupdf\pdf_xref.c(442) : pdf_loadobject() - cannot load object 1586 into
cache
Error: .\mupdf\pdf_xref.c(416) : pdf_cacheobject() - found object 1636 0 instead
of 1586 0

It means that there is error in your pdf file. Check if the pdf file can be loaded by sumatrapdf viewer. If the file cannot be handled by sumatrapdf viewer then soPdf cannot handle the file as well.

[=X=]

Quite an excellent app. This tool provides the feature I have been sorely looking for. I have some scripts that do remove the margins but none provided this level of success. I have a feeling this tool will become my new favorite PDf tool.

This tool does struggle with the more complicated PDF but for those there are PDFLRF/PDFRead/PaperCrop

Thanks.


One recommendation is since the tool is written in CPP there is no reason to tie it to one platform. There is a surprising large number of users on this board that use Linux/Mac OSX.


Thank you,
=X=

[theguru]

I am working on fixing the bugs for the complicated pdf's. And yes it can be easily ported to any platform. There is no platform specific stuff in the code and since the source is available, anyone who is interested in creating a port for Linux/Mac is welcome to do so.

[ProDigit]

So far I've only managed to get PDF to PDF working here.
So how do I convert it to LRF,or do you suggest keeping those documents in PDF?

(BTW thank you for the program,I've only had a brief look at it)

[ProDigit]

So far I've only managed to get PDF to PDF working here.
So how do I convert it to LRF,or do you suggest keeping those documents in PDF?

(BTW thank you for the program,I've only had a brief look at it)

[theguru]

Originally Posted by ProDigit

So far I've only managed to get PDF to PDF working here.
So how do I convert it to LRF,or do you suggest keeping those documents in PDF?

(BTW thank you for the program,I've only had a brief look at it)

That was the original plan. I wanted to keep the files in pdf format. These reformatted pdf files can be read easily on the reader.

[=X=]

Okay I found my first bug. It seems bookmarks on PDF are getting removed from the PDF.

These are bookmarks in the PDF used by the SONY reader for the table of contents.

=X=

[DDHarriman]

Not bad at all.

Big files resulting from scanning still crash my Cybook or are so slow that hey are useless, but text ones (with images or not) are quite good!

One more info: does not convert PDF files with monochrome images JBIG2 compressed, CCITT Group 4 compressed, no problem.

Bets regards,

[theguru]

Originally Posted by =X=

Okay I found my first bug. It seems bookmarks on PDF are getting removed from the PDF.

These are bookmarks in the PDF used by the SONY reader for the table of contents.

=X=

I am aware of the issue. I do not quite understand how the bookmarks work in PDF file. I will update the tool once I do or if anyone here does understand how bookmarks work then they are free to update the source.

[DDHarriman]

One question: any plans on evolving into a gui with options to choose from?