Obelisk -- legal distribution of format-shifted copyrighted works

[llasram]

I love curly quotation marks. They're so round and inviting. I also love free e-books, and so have been delighted by Tor's current free–e-book–each–week program. Perhaps by Tor my loves may be joined? But alas not – the HTML versions Tor provides have ASCII quotation marks, and when I asked if this could be rectified was told “I'm afraid the quotation-mark conversion has to stay.”

So for Robert Charles Wilson’s Spin I rolled up my crazy-sleeves, pulled out by regexps, and fixed them myself. Every last one. And modified the CSS and some of the markup to much more more closely resemble the formatting in the PDF version. Then wrapped it up as a valid .epub book. Then converted/tweaked to produce a great-looking Sony Reader BBeB book.

And they’re all for only me! Nope, can’t give them to you. The power of copyright compels me! I can add those curly quotes myself because I have the source HTML to start with. If I start handing people my curly-quoted version I have no means to stop it from falling into new hands which didn’t already have the straight-from-Tor edition.

Or do I?

I could provide you with a grid of just the byte offsets of the various curly quotes. Some extreme variant of diff/patch in which nothing of the original copyrighted text persists. It would contain just my curly quotes, owned by me under copyright law and free to give you as I wish. You provide the straight-from-Tor e-book, mix in my curly quotes and poof! – you have a be-curled edition of Spin. But this doesn’t work for format-shifting over compression, encoding changes, etc., where “put a curly quote here” ceases to makes sense.

Unless we distill the idea down to the lowest level – what is XOR but the difference between two bits?

Let’s try an experiment, which I’m calling Obelisk[1]. Download the following files:

obelisk.py
Mohm5pei#WilsonSpin_HTML.zip#Spin.epub.obelisk
AhZe5shu#WilsonSpin_HTML.zip#Spin.lrf.obelisk

Then get your copy of WilsonSpin_HTML.zip handy, pop open your favorite shell, and run:

python obelisk.py Mohm5pei WilsonSpin_HTML.zip Mohm5pei#WilsonSpin_HTML.zip#Spin.epub.obelisk Spin.epub
python obelisk.py AhZe5shu WilsonSpin_HTML.zip AhZe5shu#WilsonSpin_HTML.zip#Spin.lrf.obelisk Spin.lrf

The results should be curly-quoted .epub and BBeB versions of Spin, seamlessly merging Tor’s bits with mine into unified wholes.

Let me know what you think.

[1] Obelisk is similar to and inspired by a “project” called Monolith, although with rather different goals.

[kovidgoyal]

Assuming the source file has an even number of quotes, shouldn't replacing them with curly quotes be as simple as

intag = False
inquote = False
for i, chr in enumerate(data):
  if chr == '<':
    intag = True
  elif chr == '>'
    intag = False
  elif not intag and chr == '"':
    if inquote:
      data[i] = right curly quote
      inquote = False
    else:
      data[i] = left curly quote
      inquote = True

Or is there something about curly quotes I'm missing?

[llasram]

Originally Posted by kovidgoyal

Assuming the source file has an even number of quotes, shouldn't replacing them with curly quotes be as simple as

It’s mostly mechanizable, but not quite that simply. For example:

“This quotation-marked bit goes on for more than one paragraph. It doesn’t end with a double quote.

“And here I have some ‘examples’ of single quotes. I’ve got several of ’em. The examples’ quotation marks point in all kinds of directions.

“And here ends the quote.”

So pretty much the rules are:

<ws>" == “
"<ws> == ”
\w'\w == ’
'<ws> == ’
<ws>' == ‘

Where <ws> is whitespace plus ( ) [ ] - – —.

But then have to manually check all the instances of “<ws>‘” and probaly start by looking for any quotations marks with white space on both sides (usually found when doing "something like 'this' ").

So anyway. Mostly mechanizable, but still some manual labor to get it perfect. And can’t automate improving the CSS. :-)

[kovidgoyal]

Ah I see, well lets see if Tor starts beating on your door in the middle of the night.

[JSWolf]

Um... it won't work because it never came zipped. And how do we know the filename to use in the ZIP file or even if we have the exact same contents?

[llasram]

Originally Posted by JSWolf

Um... it won't work because it never came zipped. And how do we know the filename to use in the ZIP file or even if we have the exact same contents?

The e-mails actually contain links to two separate HTML versions. One is the HTML content served directly, the other is a ZIP archive which contains the images used in the book, a (broken) OPF file, etc.

[JSWolf]

Originally Posted by llasram

The e-mails actually contain links to two separate HTML versions. One is the HTML content served directly, the other is a ZIP archive which contains the images used in the book, a (broken) OPF file, etc.

Yes, you are correct. My apologies. I'll give your script another go and see how it works out.

[JSWolf]

How do I use your script to generate a diff file for other content? I'd love to do one for Mistborn based on the PDF to make the LRF from it.

[JSWolf]

I've taken the EPUB edition and built an LRF to my specification. Looks nice. Now all I need to do is build a proper ToC and I'll be all set.

[llasram]

Originally Posted by JSWolf

How do I use your script to generate a diff file for other content? I'd love to do one for Mistborn based on the PDF to make the LRF from it.

It's symmetric, so:

python obelisk.py SALT KEYFILE INFILE OUTFILE

For both decryption and encryption. The SALT parameter is some string of your choosing but should not be reused for a particular KEYFILE. For example:

python obelisk.py sai3sahS 9780765350381.zip Mistborn.lrf sai3sahS#9780765350381.zip#Mistborn.lrf.obelisk

HOWEVER – I am not a lawyer. This certainly seems reasonable given that one needs the original file to reconstitute the derived file, but I don’t really know if Tor and/or your nation’s legal system will see it that way. This is an experiment – use at your own risk.

[JSWolf]

I got it to work. Thank you. This will make it a lot easier now to post conversions without having to post the converted file.

[NatCh]

Heh, we may have to have another category in the Book Uploads area.

[JSWolf]

Originally Posted by NatCh

Heh, we may have to have another category in the Book Uploads area.

That is a good idea.

[llasram]

Originally Posted by kovidgoyal

Ah I see, well lets see if Tor starts beating on your door in the middle of the night.

I decided to – you know – just go ahead and ask them. One Laurence Hewitt first says no, but I ask:

Just to clarify -- you are asking me not to distribute the derivative work even in the form which requires the recipient of the derived work already have a copy of the original?

Which to clarify one step further, what I'm actually wishing to distribute is instructions for transforming the original version of the e-book into my modified version. So not directly distributing either the original or the derived version, but the means to transform the original into the derived version.

And he responds:

Sorry, I utterly misunderstood your message. I've gotten into the bad habit of merely skimming my e-mail rather than actually reading it. Yes, of course you can distribute.

So Tor at least is fine with this. Hooray! :-)

[JSWolf]

Nice! I don't see any issue. Because as you said without the source the diff file is useless. What if you've downloaded the PDF of Mistborn then you can get my LRF conversion over at https://www.mobileread.com/forums/sho...356#post156356