Wikipedia / Porting calibre to android

donttrustben · 01-29-2010, 08:41 PM

Hi,

Firstly, thanks to the calibre peeps for creating such functional and easy to use software and releasing it as free software.

So, a blue sky question: Is it possible to read wikipedia on an ebook reader? If not, what would need to be done to get there?

My thoughts on the second question to date:

1. (English) wikipedia is big (5.4GB bz2-compressed currently). ePub is natively compressed format and seems the obvious format to convert to, but I wonder how much bigger a zip compressed file will be than a bz2 one (also need to take into consideration the differing XML). Perhaps size also matters because if you store all the articles in one big file then that file is a lot smaller than thousands of smaller files.

What would be cool is if you could mount a bz2 compressed folder of ePub files directly using archivemount/FUSE on the ereader (or computer for that matter). For my case (I'm likely buying a nook), I wonder if anyone has got fuse working on android?

2. Are there any efforts to port the reading part of calibre to android? I think this would be awesome, because when the tech gets better companies will stop supporting their old products. I'm thinking of rockbox being better than standard ipod software as an example of this. Given the open nature of android, it might be possible to get commercial interests working towards that as well.

3. If you have internet access, then you could start a calibre content server and just access each article as an ebook over the net. Not an ideal solution I know.

4. Storage problems aside, if you store each ebook as a file, then it might become unwieldy to find the article you are looking for because there is so many articles, assuming 1 article = 1 ebook.

Fire away.. I apologize if I've said uninformed or stupid things - I'm supposed to be working, and these thoughts are embryonic.
ben

kovidgoyal · 01-29-2010, 09:04 PM

Since most android devices apart from a few crippled e-book readers liek the nook all have internet access anyway, this would really only be useful ion the limited situations when you're without internet access.

donttrustben · 01-29-2010, 09:44 PM

So, to partially answer my questions about porting and compression, FBReaderJ apparently works on Android (though not sure specifically about nook), and can read from gzip (but not bz2, though the original FBReader can).

donttrustben · 01-29-2010, 09:50 PM

Thanks for the quick reply David.

I think it is a little more complicated than that, especially for non-US citizens. Also, reading wikipedia online is different than reading it as an ebook.

kovidgoyal · 01-29-2010, 09:52 PM

There are a couple of projects to make wikipedia available offline on ebook readers, dont recall them right now, but a little googleing should do the trick

donttrustben · 01-29-2010, 11:10 PM

Hmm, yes there are some interesting things out there. I still haven't found a pain-free method of getting the wikipedia xml to an ebook format, though it presumably can be done. One method might be to convert the wikipedia XML to HTML and then to epub, perhaps filtering out some stuff like sidebars along the way. Then gzip the whole thing and use FBReaderJ to read the one big gzip file on android. It seems a little long winded to me, but otherwise I don't see how delving into the details of wikipedia xml or epub can be avoided, which might get very complex.

See also a similar discussion where they had storage problems that don't appear to have been solved: https://www.mobileread.com/forums/showthread.php?t=31041

jadavbheda · 07-28-2010, 06:27 AM

Can't we try to port calibre on android?

That way it will convert almost anything to epub and then FBReader will just display epub.

Please help me for "how to compile conversion module only?"

-- Thanks

itimpi · 07-28-2010, 10:01 AM

I do not think that you can easily strip out just the parts of Calibre that you want!

Calibre has a very large number of dependencies, and you would need to get all of those onto Android first. You would then need Python itself (is that available as standard on android), and then once that is all in place you could start trying to get Calibre itself working.

Even if all the above steps were successful, it is doubtful that the device would have the resources to run Calibre (I assume that you are talking about a mobile device here?).

However despite the pessimistic tone of the above, I am sure that if anyone succeeded the results would be welcomed

01-29-2010, 08:41 PM	#1
donttrustben Junior Member Posts: 6 Karma: 10 Join Date: Jan 2010 Device: nook	Wikipedia / Porting calibre to android Hi, Firstly, thanks to the calibre peeps for creating such functional and easy to use software and releasing it as free software. So, a blue sky question: Is it possible to read wikipedia on an ebook reader? If not, what would need to be done to get there? My thoughts on the second question to date: 1. (English) wikipedia is big (5.4GB bz2-compressed currently). ePub is natively compressed format and seems the obvious format to convert to, but I wonder how much bigger a zip compressed file will be than a bz2 one (also need to take into consideration the differing XML). Perhaps size also matters because if you store all the articles in one big file then that file is a lot smaller than thousands of smaller files. What would be cool is if you could mount a bz2 compressed folder of ePub files directly using archivemount/FUSE on the ereader (or computer for that matter). For my case (I'm likely buying a nook), I wonder if anyone has got fuse working on android? 2. Are there any efforts to port the reading part of calibre to android? I think this would be awesome, because when the tech gets better companies will stop supporting their old products. I'm thinking of rockbox being better than standard ipod software as an example of this. Given the open nature of android, it might be possible to get commercial interests working towards that as well. 3. If you have internet access, then you could start a calibre content server and just access each article as an ebook over the net. Not an ideal solution I know. 4. Storage problems aside, if you store each ebook as a file, then it might become unwieldy to find the article you are looking for because there is so many articles, assuming 1 article = 1 ebook. Fire away.. I apologize if I've said uninformed or stupid things - I'm supposed to be working, and these thoughts are embryonic. ben

07-28-2010, 06:27 AM	#7
jadavbheda Junior Member Posts: 1 Karma: 10 Join Date: Jul 2010 Device: none	porting calibre to android Can't we try to port calibre on android? That way it will convert almost anything to epub and then FBReader will just display epub. Please help me for "how to compile conversion module only?" -- Thanks

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre Android App	Sentireader	Devices	11	06-19-2011 01:11 AM
Calibre doesn't see Android phone	notaguru	Calibre	5	08-09-2010 08:06 PM
Wikipedia on Calibre	franc	Calibre	7	05-28-2010 02:15 AM
Calibre and Android Device	gambarini	Calibre	9	04-09-2010 04:17 AM
Reference Wikipedia: SOS Children 2006 Wikipedia CD	hn_88	BBeB/LRF Books	0	01-29-2008 12:23 PM

01-29-2010, 09:04 PM	#2
kovidgoyal creator of calibre Posts: 44,087 Karma: 22669822 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Since most android devices apart from a few crippled e-book readers liek the nook all have internet access anyway, this would really only be useful ion the limited situations when you're without internet access.

01-29-2010, 09:44 PM	#3
donttrustben Junior Member Posts: 6 Karma: 10 Join Date: Jan 2010 Device: nook	So, to partially answer my questions about porting and compression, FBReaderJ apparently works on Android (though not sure specifically about nook), and can read from gzip (but not bz2, though the original FBReader can).

01-29-2010, 09:50 PM	#4
donttrustben Junior Member Posts: 6 Karma: 10 Join Date: Jan 2010 Device: nook	Thanks for the quick reply David. I think it is a little more complicated than that, especially for non-US citizens. Also, reading wikipedia online is different than reading it as an ebook.

01-29-2010, 09:52 PM	#5
kovidgoyal creator of calibre Posts: 44,087 Karma: 22669822 Join Date: Oct 2006 Location: Mumbai, India Device: Various	There are a couple of projects to make wikipedia available offline on ebook readers, dont recall them right now, but a little googleing should do the trick

01-29-2010, 11:10 PM	#6
donttrustben Junior Member Posts: 6 Karma: 10 Join Date: Jan 2010 Device: nook	Hmm, yes there are some interesting things out there. I still haven't found a pain-free method of getting the wikipedia xml to an ebook format, though it presumably can be done. One method might be to convert the wikipedia XML to HTML and then to epub, perhaps filtering out some stuff like sidebars along the way. Then gzip the whole thing and use FBReaderJ to read the one big gzip file on android. It seems a little long winded to me, but otherwise I don't see how delving into the details of wikipedia xml or epub can be avoided, which might get very complex. See also a similar discussion where they had storage problems that don't appear to have been solved: https://www.mobileread.com/forums/showthread.php?t=31041

07-28-2010, 10:01 AM	#8
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	I do not think that you can easily strip out just the parts of Calibre that you want! Calibre has a very large number of dependencies, and you would need to get all of those onto Android first. You would then need Python itself (is that available as standard on android), and then once that is all in place you could start trying to get Calibre itself working. Even if all the above steps were successful, it is doubtful that the device would have the resources to run Calibre (I assume that you are talking about a mobile device here?). However despite the pessimistic tone of the above, I am sure that if anyone succeeded the results would be welcomed

Advert

Advert