Hacking EBW-1150 to browse the Internet

[derrell]

Have you gotten your perl script to work in Linux? It uses Win32::OLE doesn't it? The last time I tried running it in native Linux there was no way for it to make use of the commands to eBook Publisher to create an imp file. The mobi scripts from tommy work but they all only use perl modules.

I've installed perl 5.8.8 in wine and have been trying to get the Mobi2Imp gui and scripts to work that way but haven't spent much time on it yet. My answer so far is to run the script in an install of Windows XP using VirtualBox.

[nrapallo]

Originally Posted by derrell

Have you gotten your perl script to work in Linux? It uses Win32::OLE doesn't it? The last time I tried running it in native Linux there was no way for it to make use of the commands to eBook Publisher to create an imp file. The mobi scripts from tommy work but they all only use perl modules.

I've installed perl 5.8.8 in wine and have been trying to get the Mobi2Imp gui and scripts to work that way but haven't spent much time on it yet. My answer so far is to run the script in an install of Windows XP using VirtualBox.

Oops, your are absolutely right. There is no .imp creation in linux; eBook Publisher is required to generate the .imp from the .html and images extracted. Now only if eBook Publisher worked in linux...

Mobi2IMP is written in perl and then compiled for windows users, along with that GUI. It uses the same interface that was created for PDFRead. PDFRead has the same issues working in linux; can create the "parts", but can't assemble them into a .imp.

The original creator of PDFRead, ashkulz, recommended taking those "parts" to the "windows"-side to complete the .imp creation. His solution was to just get eBook Publisher to finish the job in windows; your solution is to use Mobi2IMP in windows to begin with.

Which is easier? better?

Can linux use/simulate the eBook Publisher .dll / COM and OLE calls? The SBPubX.doc documentation is quite extensive but I'm not sure how you could get that working in linux. But then again, I'm "The REB1200 Guy", not "The Linux Guy"...

[nrapallo]

Originally Posted by Ret

I think you don't really need my code but if you want it you can get it from here

OK, I must admit, you've managed to stump me.

I'm using WinXP with java available at the command prompt. When I issue "java -version", I get:

java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

How do you start this proxy server written in java?

Anyone know what to do when using a windows (brain-dead) machine to execute this? I know derrell stays away from the "windows"-side (Star Wars analogy?)

[derrell]

Actually what I've been doing with the conversions from lit and mobi is to run the tool that will convert whichever file type to the oeb style directory and using eBook Publisher run through wine to create the imp file. What I like about your scripts is that they turn that into a one step process.

Even when I get the whole wine setup working thats still going to be a huge round about kludge. I've searched around looking for something that will create an imp file without using eBook Publisher and haven't found anything.

Seeing as how I need my Virtualbox XP to download lit files with IE that I've purchased I guess its not that big of a deal to start it and run the conversion scripts there.

Nick
I'm not one of those militant Linux guys I won't boo and hiss it just because Bill Gates had something to do with. I freely admit the man is one hell of a business man, locked nearly the entire industry into a voluntary monopoly. I just choose not to use when there is a choice. Sometimes there just isn't one though.

[nrapallo]

Derrell:

I am only using WinXP because it satisfies most of my needs.

On the programming side I wouldn't touch MS products with a ten-foot pole!

I have used mostly Unix-style programming since university and that's what I prefer to work with. C and C++ are my main choices to experiment in.

Prior to using windows I used the AmigaOS and loved it. I really felt comfortable there, just that with the demise of the hardware, the choice was windows or MAC. I chose windows and that's why I'm in this quandry!

Oh well, my most recent computer at least dual boots with WinXP and Ubuntu 8.04 so there is light at the end of the tunnel...

Oh, BTW, be sure to check ashkulz's recent post impmake: automated creation of IMP books via the command line for an all linux solution to imp creation.

[Ret]

OK... I've managed to let the reader surf the web AND browse books from Linreb's bookshelf!!
I've installed Privoxy which runs on port 8118. So I configured the reader to use proxy listening in 127.0.0.1:8118.
In Privoxy's main configuration file I've added ONE line "forward .ebooksystem.net 127.0.0.1:9090" (without quotes).
So, to surf the web, you only need to put an index.html file on Linrebs store directory pointing somewhere on the internet. In the reader you go to BOOKSTORE and click the icon and Linred should respond with that index page you created, after that, click on the link and VOILA!!
Is it clear?

[nrapallo]

Ret:

Great news! The home page to privoxy is http://www.privoxy.org/ for those so inclined to try this out!

Now onto the many Linreb mods planned.... boohhahahahahaha...

[nrapallo]

Originally Posted by Ret

OK... I've managed to let the reader surf the web AND browse books from Linreb's bookshelf!!

I've installed Privoxy which runs on port 8118. So I configured the reader to use proxy listening in 127.0.0.1:8118. In Privoxy's main configuration file I've added ONE line "forward .ebooksystem.net 127.0.0.1:9090" (without quotes).

So, to surf the web, you only need to put an index.html file on Linrebs store directory pointing somewhere on the internet. In the reader you go to BOOKSTORE and click the icon and Linred should respond with that index page you created, after that, click on the link and VOILA!!
Is it clear?

This works fantastic using the EBW1150 to browse the internet.

I surfed to the MR wiki pages and everything downloaded fine (except .imp links! ), just that none of the hyperlinks are underlined (current limitation) so choosing what to click becomes cumbersome.

Alas, my REB1200 did not have any success using this approach to 'surf the internet' since it is connected to a router with a local IP address (192.168.0.X) and not connected to the computer running Linreb (127.0.0.1). The Linreb part works okay but the surfing HTTP requests are not passed through somehow.

I tried different configurations, but still no go! I'll keep trying to get the REB1200 fully working !

Ret, is it possible to use your local (WinXP box) IP address instead of 127.0.0.1? My situation, in essence, is one where the hardware reader is at one local IP address and needs to go through a computer running Linreb at another IP address which inturn needs to proxy properly with privoxy. My brain hurts now just thinking about this...

Any insights? suggestions?

EDIT: I removed the HTTP proxy setting on the REB1200 that is used to communicate with Linreb, but already had its content directory page loaded in and managed to get the REB1200 to surf the internet. So it is possible; just takes some more thinking...

Proof of concept is the surfing to the ETI home page with the REB1200. I just grabbed the closest capturing device to record this. Sorry, it is very low quality and low light!

EDIT: I just took some more pictures of surfing the internet (better lighting and quality). Have a look at my REB1200 Album and EBW1150 Album!

(attached thumbnails of movie-low quality)

[Ret]

I guess REB1200 uses other servers to connect for bookshelves so if I'm not wrong you should add to privoxy settings the line "forward .softbook.net 127.0.0.1:9090".

Are you following me on why we are using privoxy, adding those lines, etc?

So, basically you need to do this:
1- Run Linreb (usually uses port 9090)
2- Run privoxy (usually uses port 8118)
3- add the lines "forward .ebooksystem.net LINREB_IP:LINREB_PORT" and "forward .softbook.net LINREB_IP:LINREB_PORT"
(where LINREB_IP is the IP address where Linreb is runnning and LINREBPORT is the port in which it listens)
4- Set the reader to use HTTP proxy pointing to the IP address and port of privoxy.

In the case that privoxy, Linreb and the reader are connected to the same PC just use 127.0.0.1 (I don't know why it didn't work when I use my PC's local IP 192.168.#.1)

[nrapallo]

Originally Posted by Ret

I guess REB1200 uses other servers to connect for bookshelves so if I'm not wrong you should add to privoxy settings the line "forward .softbook.net 127.0.0.1:9090".

No, it uses the same servers as the EBW1150 (I migrated it over already)

Are you following me on why we are using privoxy, adding those lines, etc?

You got me on WHY? other than the obvious to handle non *.ebooksystem.net HTTP requests. Now the task begins getting Linreb to do all of this! Is their a reason, in the future, to keep Privoxy "in the loop"?

So, basically you need to do this:
1- Run Linreb (usually uses port 9090)
2- Run privoxy (usually uses port 8118)
3- add the lines "forward .ebooksystem.net LINREB_IP:LINREB_PORT" and "forward .softbook.net LINREB_IP:LINREB_PORT"
(where LINREB_IP is the IP address where Linreb is runnning and LINREBPORT is the port in which it listens)
4- Set the reader to use HTTP proxy pointing to the IP address and port of privoxy.

In the case that privoxy, Linreb and the reader are connected to the same PC just use 127.0.0.1 (I don't know why it didn't work when I use my PC's local IP 192.168.#.1)

That's my problem! (in bold above); my reader is at IP_address_1 and Linreb and Privoxy are at IP_address_2. I will try your suggestions and experiment more, but the BATTLE has been won. The readers can access and surf the 'net!

Job well done, Ret! I'd give you more karma, but the system here is not allowing me to do so soon after already giving you karma!

p.s. is there a capture log that can be saved from Privoxy and posted here to see which HTTP requests don't make it to Linreb? and then post Linreb's capture file to refer to it? This should start us off in adapting the Linreb code to hopefully cut out Privoxy from the equation.

Has anyone tried this in linux yet? how about MacOS! The possibilities could be endless....

[Ret]

That's my problem! (in bold above); my reader is at IP_address_1 and Linreb and Privoxy are at IP_address_2. I will try your suggestions and experiment more

I don't see why that would be a problem. That should work. When things are run locally, maybe the packets don't get routed correctly to privoxy if you don't use the local loop 127.0.0.1.

Job well done, Ret! I'd give you more karma, but the system here is not allowing me to do so soon after already giving you karma!

I just realized about those karma points!. Didn't know they existed! What are they for?

Sure, you can code some proxy features into Linreb and get rid of privoxy. You'll have to work on linreb.c. There's an if..then that handles GET requests. Take a look here:

// if it is a request for bookshelf.ebooksystem.net,
						// then serve our page

						if (strstr(orl,BOOKLIST_PREFIX))
						{
								handle_booklist_request(rs,orl,orl+
												strlen(BOOKLIST_PREFIX));
.
.
.
.
.

else
				{
					// just print the first line, probably not a
					// GET request
					char line[1024];
					char *a, *b;

					for (b=buf,a=line; *b ; /* no increment */)
					{
							if (*b == '\r')
									break;
							*a++ = *b++;
					}
					*a='\0';
					if (verbose)
					{
							printf("%s\n",line);
					}


					not_found(rs);
				}

[Nate the great]

Originally Posted by nrapallo

I surfed to the MR wiki pages and everything downloaded fine (except .imp links! ), just that none of the hyperlinks are underlined (current limitation) so choosing what to click becomes cumbersome.

That's because the ebook links do not end in IMP, they end in PHP. The Ebw-1150 is confused by the suffix. The Kindle has the same problem, BTW.

[nrapallo]

Originally Posted by Ret

I don't see why that would be a problem. That should work. When things are run locally, maybe the packets don't get routed correctly to privoxy if you don't use the local loop 127.0.0.1.

It's strange! If the REB1200 page with the internet links is displayed and I turn off the HTTP proxy setting, then I can get the links to not go through Linreb and pass onto (without Privoxy?) the internet. I have to get this nailed down, but for now I'm happy it works!

I just realized about those karma points!. Didn't know they existed! What are they for?

I think you collect as many as you can and then redeem them later in life (or after-life), I think NOT!

They are a reward system with no apparant purpose other than to electronically "pat someone on the back" so to speak.

Sure, you can code some proxy features into Linreb and get rid of privoxy. You'll have to work on linreb.c. There's an if..then that handles GET requests. Take a look here:

Yep, that "not_found(rs)" message is going to get an overhaul!!!!

BTW, I just took some more pictures of the REB1200 surfing the internet (better lighting and quality). Have a look at my REB1200 Album set up here!

Next up, an Album for the EBW1150 browsing the internet. Have a look at my EBW1150 Album set up here!

[nrapallo]

Originally Posted by Nate the great

That's because the ebook links do not end in IMP, they end in PHP. The Ebw-1150 is confused by the suffix. The Kindle has the same problem, BTW.

Good to know this info. It will save a lot of debugging later when we try to code it and find out there is no .imp at the end of that link... WTF? Aha a .php ending!

Thanks Nate!

[nrapallo]

See continuation of this thread, in: HOW-TO: Surf the Internet with the EBW1150 or REB1200

Major thanks go out to Ret for originally having this idea and finally getting it to work AGAIN as well as to the original author of Linreb (Nandagopal Kirubanandan) wherever and whoever you are!!

Most with just a EBW1150 should follow the first and third HOW-TO at the above link, whereas those with multiple ETI devices (like me) should follow the second and third HOW-TO (follow?)!

EDIT: To easily surf the internet WITHOUT using the (limited) Linreb or Privoxy, please use Impserve and be sure to download all of the Impserve plugins to allow the downloading of content directly from the internet to your ebook reader!!!