T?ol to create Structured catalogs?

itimpi · 11-19-2008, 11:47 AM

I am wondering if there are any tools around that can create a structured set of catalogs from calibre? I want to run the results on my existing web server (i.e. not use the calibre built-in one). The current calibredb command creates a flat structure constiting only of titles (or at least I think that is the only option) which is rather unwieldy as the number of books goes up.

I want to create a structure along the lines of:

Top Level

List of Titles

.. lots of books

Authors

Authors-A

... books for A authors

Authors-B

... books for B authors

etc

Series

SeriesNames

Books in Series

and so on. The idea being that each level is probably a separate catolog file, with the bottom levels finally pointing at real books.

If there is a tool that can already do this sort of thing it would be great, but if not I was thinking of writing one that would process the calibre database and produce the relevant atom style catalogs and book entries.

Any feedback/guidance/ideas would be great

kovidgoyal · 11-19-2008, 11:54 AM

What do you mean by a structured set of catalogs?

itimpi · 11-19-2008, 12:02 PM

I posted the original message too soon by accident . I have now completed editing it so hopefully it now makes more sense?

kovidgoyal · 11-19-2008, 12:07 PM

calibredb produces all the data in the database in XML format. See the --output-format option. It should be easy enough to process the generated XML and convert it into the catalogs you want in an external tool.

On the other hand you can patch the content server (the file calibre/library/server.py) to generate hierarchical atom feeds for stanza (which I suspect is your use case). If you do decide to patch server.py, send me the patch and I'll consider it for inclusion.

itimpi · 11-19-2008, 12:16 PM

Python is not one of the programming languages that I am familiar with. I was thinking of first prototyping it in perl to check my ideas. I know a wide variety of programming languages, but python is not one of them. I just wanted to check initially that I was not duplicating something already done elsewhere.

Assuming my ideas work out I will look at the server.py code and see if I can work out how to put things into Python syntax. While I am at it I will see if I can generalize the structure a bit (perhaps in an external XML file) to make it easier for others to have variants of the structure I want.

I thought for first attempt it would be easier to access the metadata.db file directly as I can then use SQL statements to do the hard work of selecting and sorting entries. Should be easier than processing the current XML output by calibred? Or do you think it is worth jumping straight in and trying to modify server.py?

kovidgoyal · 11-19-2008, 12:30 PM

I'd recommend against direct access to metadata.db. Its structure is not stable and generally speaking you dont want to have concurrent access to the db.

server.py already contains code to search and sort by various fields (it used for the HTMl content server), so you can just leverage that existing code if you modify server.py (have a look at the function library which generates the XML catalog of books for the content server to see sort and search in action). You'd have to modify the function stanza to handle multiple catalogs.

If you want to develop in perl, I'd suggest parsing the XML output of calibredb. It's very simple, so it shouldn't be a problem.

11-19-2008, 11:47 AM	#1
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Tool to create Structured catalogs? I am wondering if there are any tools around that can create a structured set of catalogs from calibre? I want to run the results on my existing web server (i.e. not use the calibre built-in one). The current calibredb command creates a flat structure constiting only of titles (or at least I think that is the only option) which is rather unwieldy as the number of books goes up. I want to create a structure along the lines of: Top Level List of Titles .. lots of books Authors Authors-A ... books for A authors Authors-B ... books for B authors etc Series SeriesNames Books in Series and so on. The idea being that each level is probably a separate catolog file, with the bottom levels finally pointing at real books. If there is a tool that can already do this sort of thing it would be great, but if not I was thinking of writing one that would process the calibre database and produce the relevant atom style catalogs and book entries. Any feedback/guidance/ideas would be great Last edited by itimpi; 11-19-2008 at 12:01 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[The Older version] Calibre2opds V2	dpierron	Related Tools	537	04-06-2012 10:43 AM
Creating Catalogs (Archived)	GRiker	Library Management	419	01-23-2011 04:16 PM
Classic Broken search function in Trook / lexcycle catalogs	Astro	Nook Developer's Corner	0	06-29-2010 09:46 PM
Call for comments: OPDS Catalogs 0.9 draft (open catalog standard ebooks)	zelda_pinwheel	News	4	05-14-2010 02:02 PM
Sony 300 or 505? What about catalogs?	coredump	Which one should I buy?	2	11-14-2009 12:36 PM

11-19-2008, 11:54 AM	#2
kovidgoyal creator of calibre Posts: 44,718 Karma: 24967300 Join Date: Oct 2006 Location: Mumbai, India Device: Various	What do you mean by a structured set of catalogs?

11-19-2008, 12:02 PM	#3
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	I posted the original message too soon by accident . I have now completed editing it so hopefully it now makes more sense?

11-19-2008, 12:07 PM	#4
kovidgoyal creator of calibre Posts: 44,718 Karma: 24967300 Join Date: Oct 2006 Location: Mumbai, India Device: Various	calibredb produces all the data in the database in XML format. See the --output-format option. It should be easy enough to process the generated XML and convert it into the catalogs you want in an external tool. On the other hand you can patch the content server (the file calibre/library/server.py) to generate hierarchical atom feeds for stanza (which I suspect is your use case). If you do decide to patch server.py, send me the patch and I'll consider it for inclusion.

11-19-2008, 12:16 PM	#5
itimpi Wizard Posts: 4,553 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Python is not one of the programming languages that I am familiar with. I was thinking of first prototyping it in perl to check my ideas. I know a wide variety of programming languages, but python is not one of them. I just wanted to check initially that I was not duplicating something already done elsewhere. Assuming my ideas work out I will look at the server.py code and see if I can work out how to put things into Python syntax. While I am at it I will see if I can generalize the structure a bit (perhaps in an external XML file) to make it easier for others to have variants of the structure I want. I thought for first attempt it would be easier to access the metadata.db file directly as I can then use SQL statements to do the hard work of selecting and sorting entries. Should be easier than processing the current XML output by calibred? Or do you think it is worth jumping straight in and trying to modify server.py?

11-19-2008, 12:30 PM	#6
kovidgoyal creator of calibre Posts: 44,718 Karma: 24967300 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I'd recommend against direct access to metadata.db. Its structure is not stable and generally speaking you dont want to have concurrent access to the db. server.py already contains code to search and sort by various fields (it used for the HTMl content server), so you can just leverage that existing code if you modify server.py (have a look at the function library which generates the XML catalog of books for the content server to see sort and search in action). You'd have to modify the function stanza to handle multiple catalogs. If you want to develop in perl, I'd suggest parsing the XML output of calibredb. It's very simple, so it shouldn't be a problem.

Advert

Advert