[Conversion Input] Microsoft Doc Input Plugin

[BetterRed]

@davidfor - that's fixed the double up in prefs, but couple of things before you push it

• Update version number, I'm still seeing 1.0.0
• After I installed into 2.0 I got a 'python' error, but I didn't copy it I thought oh well, let's try doing a restart. Goddam, that fixed it. Some PI's offer a restart after an update, if its easy you might want to do that with this puppy.
• Maybe wait for rebl to give it a spin around the block.

I think I'm done with this PI, given I don't even wanna use the darn thing

BR

[igi]

I just it made because i have same doc files. ANd when i create this plugin i little bit learn this python so appologize not standard way how i implemented.


Thanks igi.

[rebl]

Hi all and thank you again for all you guys do.
BetterRed, In post #5 I was writing I've also tried editing the doc_input.py file (I'm not sure if it's the same) but somehow it hasn't worked:

I found the doc_input.py file in folder Calibre Settings\conversion
and manually edited path to
u'wordconv_exe_path' : u'C:\\Program Files (x86)\\Microsoft
This is quite bizare! Even after I've corrected the path by editing the file (and checked in GUI, it's there) the plugin still looks for Office12, like it's hardcoded.

I haven't noticed the duplicate entry in input preferences and now it's too late to see that, since i've upgraded to 1.0.1.

I've tested the plugin again and I noticed the following:
1. In Preferences -> Input -> DOC it is possible to change the default path and appears to be saved.
I had to change it to:
C:/Program Files (x86)/Microsoft Office/Office12/Wordconv.exe
2. When trying to do a bulk conversion from 3 DOC-only books, I get the same error as listed in post #23
3. Then I've changed the path to
C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe
because I do have both intances of wordconv.exe (files seem to be identical)
but the error persisted:

Convert book 3 of 3 (Monte Cassino)
Resolved conversion options
calibre version: 2.1.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': u'original',
 'chapter': u"//*[((name()='h1' or name()='h2') and re:test(., '\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', 'i')) or @class = 'chapter']",
 'chapter_mark': u'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'docx_no_cover': False,
 'dont_compress': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': u'',
 'fix_indents': True,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000000004A9EA20>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'mobi_toc_at_start': False,
 'no_chapters_in_toc': False,
 'no_inline_navbars': False,
 'no_inline_toc': False,
 'output_profile': <calibre.customize.profiles.KindleOutput object at 0x0000000004A9EFD0>,
 'page_breaks_before': '/',
 'prefer_author_sort': False,
 'prefer_metadata_cover': False,
 'pretty_print': False,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': u'C:\\Users\\xxxx\\AppData\\Local\\Temp\\calibre_9kuik7\\vetaot.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': u'',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'share_not_sync': False,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2,
 'wordconv_exe_path': u'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe'}
InputFormatPlugin: DOC Input running
on C:\Users\xxxx\AppData\Local\Temp\calibre_9kuik7\kzqevx.doc
Convert doc C:\Users\xxxx\AppData\Local\Temp\calibre_9kuik7\kzqevx.doc to docx via c:\Program Files\Microsoft Office\Office12\Wordconv.exe
Temp directory C:\Users\xxxx\AppData\Local\Temp\calibre_9kuik7\xckythdoc_input temp output fileC:\Users\xxxx\AppData\Local\Temp\calibre_9kuik7\xckythdoc_input\kzqevx.doc.docx
Python function terminated unexpectedly
  Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe (Error Code: 1)
Traceback (most recent call last):
  File "site.py", line 132, in main
  File "site.py", line 109, in run_entry_point
  File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "site-packages\calibre\ebooks\conversion\plumber.py", line 1041, in run
  File "site-packages\calibre\customize\conversion.py", line 241, in __call__
  File "calibre_plugins.doc_input.__init__", line 51, in convert
ValueError: Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe

My calibre installation (x64 2.1) is pretty much default settings with a few minor tweaks related to date format and custom columns.
It seems to me that even if the Preferences -> Input -> Doc saves the correct path or even if editing manually the py file, somehow the plugin's code contains the wrong (32-bit only) path, used in the code as a constant and not using the externalized variable.

This can also be noticed if starting a single book conversion. Even after the path in preferences has been edited, the path in the single book conversion is still the old one.
In the attached picture you can see the discrepancy between the path saved in preferences and the path the plugin is using for a single book conversion (supposedly, the same is used in bulk conversion but you can't edit it the latter case).

[igi]

Just probably hard core change path in file "c:\Users\igi\AppData\Roaming\calibre\plugins\ DOC Input.zip\__init__.py"

Its looks like calibre don;t load setting for plugin. I need look into current calibre source code and debug why configuration is not loaded till that just hardoce change path in __init__.py

replace options with

options = {
        OptionRecommendation(name='wordconv_exe_path', recommended_value='C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe', 
            help=_('Path where is Wordconv.exe. Usually it is in "c:\Program Files\Microsoft Office\Office12\Wordconv.exe". If you don\'t have it you could download see http://www.microsoft.com/en-us/downl...ails.aspx?id=3 ')),
        OptionRecommendation(name='docx_no_cover', recommended_value=False,
            help=_('Normally, if a large image is present at the start of the document that looks like a cover, '
                   'it will be removed from the document and used as the cover for created ebook. This option '
                   'turns off that behavior.')),

    }

[rebl]

@igi
I've edited the file inside the zip to:

 options = {
        OptionRecommendation(name='wordconv_exe_path', recommended_value='c:\Program Files (x86)\Microsoft Office\Office12\Wordconv.exe', 
            help=_('test tooltip ')),
        OptionRecommendation(name='docx_no_cover', recommended_value=False,
            help=_('Normally, if a large image is present at the start of the document that looks like a cover, '
                   'it will be removed from the document and used as the cover for created ebook. This option '
                   'turns off that behavior.')),

    }

I know, I've also edited to tooltip text, just to see it working (and it works).
But I still had the same error:

Python function terminated unexpectedly
  Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe (Error Code: 1)
Traceback (most recent call last):
  File "site.py", line 132, in main
  File "site.py", line 109, in run_entry_point
  File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "site-packages\calibre\ebooks\conversion\plumber.py", line 1041, in run
  File "site-packages\calibre\customize\conversion.py", line 241, in __call__
  File "calibre_plugins.doc_input.__init__", line 51, in convert
ValueError: Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe

What I've noticed is that __init__.py uses the slashback separator '\' in the path while the doc_input.py uses '/'. Also, one was using capital C for drive letter, while the other lowercase c. Not sure if this is relevant.

Then I've tried once more this time with Office14. I've edited
__init__.py
to contain:

 OptionRecommendation(name='wordconv_exe_path', recommended_value='C:\Program Files (x86)\Microsoft Office\Office14\Wordconv.exe',

Then doc_input.py to contain:

u'wordconv_exe_path' : u'C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe'

Now the Preferences -> Input dialogue shows:
C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe

Converting one single book from doc works (and the path shown in the single book convert dialogue is correct so I don't need to change it).
Trying to bulk convert two books raises the old path error.
Trying to convert only the second book presents me with a surprise. The path in the single conversion dialogue is (again !?):
c:\Program Files\Microsoft Office\Office12\Wordconv.exe
With lowercase c:
Yet, the tooltip displays my text (tooltip test).
I don't understand where this path comes from. I've edited both files and used the uppercase C:\ (if that matters), x86 and Office14.

If i manually edit doc_input.py and capitalize OFFICE14 (C:/Program Files (x86)/Microsoft Office/OFFICE14/Wordconv.exe) it appears this way in Preferences-> Input -> Doc input.
If i manually edit __init__.py and capitalize the path with different case (e.g. MiCrOsOfT), it does not appear anywhere in calibre.
Also please note that your example uses "/" in __init__.py while initially it had "\".
Then I've tried in __init__.py both with / and \ but it doesn't seem to matter. For some books the path in the single convert dialogue appears:
C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe
For others appears:
c:\Program Files\Microsoft Office\Office12\Wordconv.exe
(Note the lowercase c
I've restarted calibre after every change with no effect.
When doing a bulk convert, the plugin will use the paths listed above, so for first book will work, second will raise the same error:

Python function terminated unexpectedly
Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1041, in run
File "site-packages\calibre\customize\conversion.py", line 241, in __call__
File "calibre_plugins.doc_input.__init__", line 51, in convert
ValueError: Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe

[rebl]

I've performed a clean install of both Calibre 64 and portable calibre.
I've deleted all the files in roming/calibre and the install directory.
All the below is valid both for Calibre64 and Calibre Portable:

After more testing I've noticed that most of the books have the wrong path in the single book convert dialogue (except for the first one and I think I've manually changed the path for that one) and pressing the reset to defaults button in the lower part of the dialogue will reset from the wrong to the correct path (that is stored in the Preferences).
That might seem to imply that a different input plugin settings set is saved for each of the books and somehow each of them was using this set on its own, ignoring the Preferences, and using a path different than the one saved in preferences...

[BetterRed]

Originally Posted by rebl

I've performed a clean install of both Calibre 64 and portable calibre.
I've deleted all the files in roming/calibre and the install directory.
All the below is valid both for Calibre64 and Calibre Portable:

After more testing I've noticed that most of the books have the wrong path in the single book convert dialogue (except for the first one and I think I've manually changed the path for that one) and pressing the reset to defaults button in the lower part of the dialogue will reset from the wrong to the correct path (that is stored in the Preferences).
That might seem to imply that a different input plugin settings set is saved for each of the books and somehow each of them was using this set on its own, ignoring the Preferences, and using a path different than the one saved in preferences...

@rebl - in bulk metadata edit there is a check box to reset the individual conversion settings back to the defaults - suggest you select all your books at use that setting.

If you click on any of the settings icons (Look & Feel, Heuristics, Doc Input etc) in the Book Conversion dialogue you will get book specific conversion settings, you'll notice that there's a Reset Defaults button in bottom left corner of that dialogue.

Sorry, I missed your reference to the doc_input.py in conversions.

@igi - I'll let davidfor respond to your posts, as he was the one who did the code changes.

BR

[davidfor]

As far as I can see, it is all working correctly.

If I remove the doc_input.py file in the conversions directory of my calibre settings, the defaults from the plugin are used for the preferences and any new conversion. This file is where the preferences are stored for this plugin.

If I select a doc that has not been converted, it uses whatever I have set in the preferences. I can change this to another location without any problems.

The next time I convert the book, it uses the setting from the last time I converted the book. Pressing the "Restore defaults" button at the bottom of the screen restores whatever I have in the preferences, or the defaults from the plugin if I hadn't changed them.

A bulk conversion has an option at the top about using the stored settings for each book. The settings are store with the book for the next conversion.

As to the format of the path, exactly what form is used for the separator depends on how it is used. Either "\\" or "/" is safer. Depending on what is done with the string, a single slash can be interpreted as an escape character. Hence "\\" will actually be a single slash and "\t" a tab character. But, it appears that everything is handling the different versions of the path OK.

Unless I have missed something, the errors are all related to the existence of wordconv.exe and the differences between the preferences and saved settings for a book. As I mentioned before, I have not actually installed this and just have a few copies of empty files called "wordconv.exe". The conversion doesn't happen, but the error is because the program won't run, not that it can't be found. I suppose I should install it. But, when I searched, neither Google or MS wanted to tell me where it was. I didn't try that hard, but if someone has a link, I'll try it.

If I have missed something in the reports, point it out and I'll have another look.

[BetterRed]

@davidfor - I have a suspicion that wordconv needs other components that are in the Office folder hierarchy and/or the Common Modules folder hierarchy.

IMO the PI should not have a hard coded default path for wordconv.exe, just the suggestion where it might be - if it hasn't been set then let the PI fail with a suitable error message - who knows where MS might put it in Office 2015.

BR

[rebl]

Originally Posted by davidfor

The next time I convert the book, it uses the setting from the last time I converted the book. Pressing the "Restore defaults" button at the bottom of the screen restores whatever I have in the preferences, or the defaults from the plugin if I hadn't changed them.

A bulk conversion has an option at the top about using the stored settings for each book. The settings are store with the book for the next conversion.

I think that was it, davidfor !
Some of the books had custom conversion options and somehow they were saved based on the old default setting that was wrong.
I noticed in the bulk converison dialogue, in the upper left corner a checkbox, "Used saved conversion settings for individual books". I've just started a 723 books conversion from doc-only to AZW3 and will post back, but I'm 99% sure that unchecking that checbox will solve the problem, finally!
Thank you all.

L.E.
So far out of 20 books about 4 gave an error, but not the infamous can't find wordconv error. So unchecking the checkbox actually mght have done the trick regarding the stored path.
The current error seems to be related to the conversion itself (maybe even a corrupt file?)

Python function terminated unexpectedly
  Command '[u'C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe', '-oice', '-nme', u'C:\\Users\\Paul\\AppData\\Local\\Temp\\calibre_koo9rt\\sex7sy.doc', u'C:\\Users\\Paul\\AppData\\Local\\Temp\\calibre_koo9rt\\lkjg5mdoc_input\\sex7sy.doc.docx']' returned non-zero exit status -14 (Error Code: 1)
Traceback (most recent call last):
  File "site.py", line 132, in main
  File "site.py", line 109, in run_entry_point
  File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
  File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "site-packages\calibre\ebooks\conversion\plumber.py", line 1041, in run
  File "site-packages\calibre\customize\conversion.py", line 241, in __call__
  File "calibre_plugins.doc_input.__init__", line 53, in convert
  File "subprocess.py", line 540, in check_call
subprocess.CalledProcessError: Command '[u'C:/Program Files (x86)/Microsoft Office/Office14/Wordconv.exe', '-oice', '-nme', u'C:\\Users\\Paul\\AppData\\Local\\Temp\\calibre_koo9rt\\sex7sy.doc', u'C:\\Users\\Paul\\AppData\\Local\\Temp\\calibre_koo9rt\\lkjg5mdoc_input\\sex7sy.doc.docx']' returned non-zero exit status -14

Using a custom column, "Formats" I'll be able to easily identify the books which did not convert (they only have the DOC format) and check them.


Off topic:
And when you think all my effort was only to be able to count words using Count Pages plugin... and yet its results seem so inaccurate (same book converted to 3 different formats in calibre, split to three different ids and counted words ... and the results differe by large, even 5-10% in word numbers... who knows, is it counting empty paragraphs...)

[davidfor]

Originally Posted by rebl

I think that was it, davidfor !
Some of the books had custom conversion options and somehow they were saved based on the old default setting that was wrong.
I noticed in the bulk converison dialogue, in the upper left corner a checkbox, "Used saved conversion settings for individual books". I've just started a 723 books conversion from doc-only to AZW3 and will post back, but I'm 99% sure that unchecking that checbox will solve the problem, finally!
Thank you all.

Good to hear we have solved it.

L.E.
So far out of 20 books about 4 gave an error, but not the infamous can't find wordconv error. So unchecking the checkbox actually mght have done the trick regarding the stored path.
The current error seems to be related to the conversion itself (maybe even a corrupt file?)

I'd suggest running wordconv against them manually to see what happened. Hopefully that will show errors if it is the file.

Using a custom column, "Formats" I'll be able to easily identify the books which did not convert (they only have the DOC format) and check them.


Off topic:
And when you think all my effort was only to be able to count words using Count Pages plugin... and yet its results seem so inaccurate (same book converted to 3 different formats in calibre, split to three different ids and counted words ... and the results differe by large, even 5-10% in word numbers... who knows, is it counting empty paragraphs...)

I haven't tried that. Hopefully there isn't any text being lost. It's probably some artefact of the conversions adding tags or something that are getting counted as words.

[HarryT]

Moderator Notice
The moderating team has deleted a number of posts from this thread because they were providing a member with assistance with unauthorised copyrighted material.

Please note that just because you "got books from a friend", this does not make it OK. Your friend needs to have the permission of the copyright holder to give you the books.

[Janeygee]

Harry,
I understand why you have done this, however I still would like to resolve the issue of the plugin not working.
At this moment in time, as I had said in a leter post, the conversion problem using other utilities had been resolved.
Am I now stopped from resolving issues with a plugin ?

[HarryT]

Yes, you are, because it is for purposes of piracy, and we will not permit this site to be used for that purpose. Please drop the subject now.

[JGB]

Not sure what is causing this, I installed the doc converter plugin and get this error when I try to convert a doc file.

calibre, version 2.45.0

'wordconv_exe_path': 'c:\\Program Files\\Microsoft Office\\Office12\\Wordconv.exe'}
InputFormatPlugin: DOC Input running
on C:\Users\user\AppData\Local\Temp\calibre_mq2dr3\kj y6pz.doc
Convert doc C:\Users\user\AppData\Local\Temp\calibre_mq2dr3\kj y6pz.doc to docx via c:\Program Files\Microsoft Office\Office12\Wordconv.exe
Temp directory C:\Users\user\AppData\Local\Temp\calibre_mq2dr3\zn kxvldoc_input temp output fileC:\Users\user\AppData\Local\Temp\calibre_mq2dr 3\znkxvldoc_input\kjy6pz.doc.docx
Python function terminated unexpectedly
Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 193, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1051, in run
File "site-packages\calibre\customize\conversion.py", line 241, in __call__
File "calibre_plugins.doc_input.__init__", line 51, in convert
ValueError: Not found c:\Program Files\Microsoft Office\Office12\Wordconv.exe