Bookit plugin for Firefox

[acidzebra]

Beowulf, that is an awesome extension!

Regarding HTML cleanup, I've always used the Aardvark extension to clean up HTML before I print, it might be worth looking at if your considering integrating some kind of cleanup mechanism - why invent the wheel twice?

[beowulf573]

Originally Posted by kovidgoyal

How are you launching web2lrf on windows? Using a windows API call?

Yep, I'm using the unicode version of CreateProcess. Passing Unicode titles work, it's the file names that are an issue. I ran dumpbin against web2lrf.exe and it's only importing the ASCII version of CreateFile (CreateFileA) so it's incapable of creating files with Unicode file names.

Apparently this is a long standing issue with python that I don't think is fully resolved yet. From what I've read googling about it's sticky problem depending upon what version of python you're using and how you instantiate the file object. However, I'm far from a python expert and am most likely wrong.

If you want I can create a set of testcases and sample code and create a ticket on your site.

[beowulf573]

Originally Posted by acidzebra

Beowulf, that is an awesome extension!

Regarding HTML cleanup, I've always used the Aardvark extension to clean up HTML before I print, it might be worth looking at if your considering integrating some kind of cleanup mechanism - why invent the wheel twice?

Cool, I'll take a look at it.

[kovidgoyal]

Originally Posted by beowulf573

Yep, I'm using the unicode version of CreateProcess. Passing Unicode titles work, it's the file names that are an issue. I ran dumpbin against web2lrf.exe and it's only importing the ASCII version of CreateFile (CreateFileA) so it's incapable of creating files with Unicode file names.

Apparently this is a long standing issue with python that I don't think is fully resolved yet. From what I've read googling about it's sticky problem depending upon what version of python you're using and how you instantiate the file object. However, I'm far from a python expert and am most likely wrong.

If you want I can create a set of testcases and sample code and create a ticket on your site.

Do that, I'll look into it. calibre already has an extension module to make interfacing with windows a little less bug prone. Worst case, I could just add a wrapper around CreateFileW and replace open with that.

[beowulf573]

Originally Posted by kovidgoyal

Do that, I'll look into it. calibre already has an extension module to make interfacing with windows a little less bug prone. Worst case, I could just add a wrapper around CreateFileW and replace open with that.

Will do. I'll put something together tonight.

[igorsk]

Here's a little wrapper I had used to handle unicode commandline arguments.

import sys, os, ctypes, pprint
class Argvu:
    def __init__(self,CommandLine=None):
        if CommandLine==None:
            CommandLine=ctypes.windll.kernel32.GetCommandLineW()
        argv_count=ctypes.c_int()
        cmd_string=ctypes.c_wchar_p(CommandLine)
        array_memory_address=ctypes.windll.shell32.CommandLineToArgvW(cmd_string,ctypes.byref(argv_count))
        match_array_type=ctypes.c_wchar_p*argv_count.value
        self.argv=[arg for arg in match_array_type.from_address(array_memory_address)]

        #running from a script:
        #sys.argv[0] == script.py
        #our argv[0] == python.exe
        #running from an exe:
        #sys.argv[0] == prog.exe
        #our argv[0] == prog.exe
        #if we have the first case, we need to remove the interpreter from argv

        if not sys.argv[0].endswith(".exe") and self.argv[0].endswith(".exe"):
            self.argv=self.argv[1:]

        ctypes.windll.kernel32.LocalFree(array_memory_address)
    def __str__(self):
        return pprint.pformat(self.result)

argv = Argvu().argv

if __name__=='__main__':
    c2a=Argvu()
    print c2a
    print c2a.argv

[igorsk]

Originally Posted by beowulf573

Yep, I'm using the unicode version of CreateProcess. Passing Unicode titles work, it's the file names that are an issue. I ran dumpbin against web2lrf.exe and it's only importing the ASCII version of CreateFile (CreateFileA) so it's incapable of creating files with Unicode file names.

Try dumping python25.dll instead

[kovidgoyal]

Yeah calibre uses something very similar, except its implemented in C. But if I understand beowulf's problem, it's that calibre is not creating the correct output file name.

Originally Posted by igorsk

Here's a little wrapper I had used to handle unicode commandline arguments.

import sys, os, ctypes, pprint
class Argvu:
    def __init__(self,CommandLine=None):
        if CommandLine==None:
            CommandLine=ctypes.windll.kernel32.GetCommandLineW()
        argv_count=ctypes.c_int()
        cmd_string=ctypes.c_wchar_p(CommandLine)
        array_memory_address=ctypes.windll.shell32.CommandLineToArgvW(cmd_string,ctypes.byref(argv_count))
        match_array_type=ctypes.c_wchar_p*argv_count.value
        self.argv=[arg for arg in match_array_type.from_address(array_memory_address)]

        #running from a script:
        #sys.argv[0] == script.py
        #our argv[0] == python.exe
        #running from an exe:
        #sys.argv[0] == prog.exe
        #our argv[0] == prog.exe
        #if we have the first case, we need to remove the interpreter from argv

        if not sys.argv[0].endswith(".exe") and self.argv[0].endswith(".exe"):
            self.argv=self.argv[1:]

        ctypes.windll.kernel32.LocalFree(array_memory_address)
    def __str__(self):
        return pprint.pformat(self.result)

argv = Argvu().argv

if __name__=='__main__':
    c2a=Argvu()
    print c2a
    print c2a.argv

[beowulf573]

You're right, I see python25.dll importing all of those functions.

So, you're code did the trick. I wrote a simple python script that created a file and wrote a utf-8 string to it. If I passed a unicode string in as the filename, it failed in the same we I was seeing. However, if i used your code to obtain the argv array, it works like a champ.

I'll write a ticket up for Kovid with details.

[beowulf573]

Sorry Kovid, I was writing my reply as you posted yours.

I think the problem is either the command line arguments are not being converted to python unicode strings from the get go, or, somewhere along the line the string variable containing the output filename is being converted to ascii or some other encoding.

The open method must be a true python unicode string not a utf-8 encoded string.

[kovidgoyal]

I'll look at the ticket when I get time. But calibre processes unicode arguments correctly on windows (it also uses GetCommandLineW) so the bug is probably elsewhere.

[jotheman]

Originally Posted by beowulf573

I just tested with an ascii title and it worked, are you trying a page with a unicode title? There's a problem passing a unicode output filename to web2lrf or html2lrf so I replace any non-ascii title. I don't like it but it appears to be a python issue on certain os versions and python versions. I'm going to continue to look at it but right now I don't have a good solution.

There's an edit field on the advanced tab that will let you override the filename.

I'm glad you found the editor menu item, I'll add a new menu item to the Tools menu. I need to sit down and make the gui more consistent. For 0.5.0 I'll do a complete overhaul.

Beowulf,

I think the editor feature is a great idea and it already does a great job.

I wasn't talking about the file names, though, I meant document title in the metadata. It is always empty now.

I also found that I cant un-set bold, italic etc. attributes added with the editor.

Cheers,


jo.

[beowulf573]

Originally Posted by kovidgoyal

I'll look at the ticket when I get time. But calibre processes unicode arguments correctly on windows (it also uses GetCommandLineW) so the bug is probably elsewhere.

You're right, since unicode titles are processed correctly it's probably not the argument code.

I've found an easier way to reproduce it, cut and paste the following into a console window:

"c:\program files\calibre\web2lrf.exe" -u http://localhost -o abc_ΠΣΩ.lrf default

I've updated the ticket but in case you read this first. As time permits I'll trace through the calibre code and see if anything jumps out.

[beowulf573]

A quick lunchtime release.

Notes for 0.4.3

1. Ordered and Unordered lists in the html editor
2. A Tools popup menu for folks who don't use the status bar icon
3. Quick fix for windows conversion, fixes a bug no one should have seen.

[beowulf573]

Originally Posted by jotheman

Beowulf,

I wasn't talking about the file names, though, I meant document title in the metadata. It is always empty now.

I also found that I cant un-set bold, italic etc. attributes added with the editor.

I can't reproduce either of these, can you make sure to update to the latest version and try again? Also, make sure you didn't set a fixed title in the options dialog.

If it's still broken send me platform, firefox version details, and a sample url you see this on.

thanks.