<Command Line> Add multiple books in multiple formats

[himitsu]

I have a fairly extensive ebook Library. Some of my books are in multiple formats. I'd like to be able to from the commandline add books to the calibre database like. "AddBook2DB TomSawyer.lrf TomSawyer.zip TomSayer.mobi"
Is there a good way to do this?

-Joe

[kovidgoyal]

calibredb --help

[_Em]

Sorry to revive an old thread, but I'd like to keep the context of what's already been said.

According to calibredb --help, you can either add duplicates (and have to clean it up using merge from the gui later), or add by unique title, skipping files of the same title in a new format. There doesn't appear to be a way to add new formats to an existing book from the command line (no command line merge function either). Am I missing something, or has someone hacked up a script that will accomplish this?

Do I have to rip apart a few plugins to create a command line script to do this? I have found adding by command line to be orders of magnitude faster than using the GUI (either via interface or by dragging books over the window -- my preferred method).

What I'd really like to do is write a script that traverses a directory structure, verifies that my to-be-imported books have the correct naming format to be imported correctly, and then imports all of those books, correctly merging books with the same author and title. After this, the script would then trigger mekk's ISBN detection script, and after that would trigger the gui-style download metadata script.

Right now, this does not seem possible without scripting the GUI itself (which seems a bit backward, as the GUI is generated by scripts in the first place).

Thoughts? Ideas?

[kovidgoyal]

calibredb add_format -h

[Starson17]

Originally Posted by _Em

Sorry to revive an old thread, but I'd like to keep the context of what's already been said.

According to calibredb --help, you can either add duplicates (and have to clean it up using merge from the gui later), or add by unique title, skipping files of the same title in a new format. There doesn't appear to be a way to add new formats to an existing book from the command line (no command line merge function either). Am I missing something, or has someone hacked up a script that will accomplish this?

Do I have to rip apart a few plugins to create a command line script to do this? I have found adding by command line to be orders of magnitude faster than using the GUI (either via interface or by dragging books over the window -- my preferred method).

What I'd really like to do is write a script that traverses a directory structure, verifies that my to-be-imported books have the correct naming format to be imported correctly, and then imports all of those books, correctly merging books with the same author and title. After this, the script would then trigger mekk's ISBN detection script, and after that would trigger the gui-style download metadata script.

Right now, this does not seem possible without scripting the GUI itself (which seems a bit backward, as the GUI is generated by scripts in the first place).

Thoughts? Ideas?

I did the code for Merge and for AutoSort/AutoMerge (put new formats into existing records where author/title fuzzy match). On the issue of command line merge, how do you plan to identify the merge records? The GUI merge takes the selection made by the user for source and destination records, finds the book ids to access the records in the library and goes from there. If you have the book ids, you can use the code from the command line (calibre.library.database2 import LibraryDatabase2) with calibre-debug -c or -e. Give me the record ids and I can give you the command line code.

As to "verifies that my to-be-imported books have the correct naming format to be imported correctly," if you can code this, it would be useful for the GUI, too. I'm not sure how you tell a title, from a series, from an author.

[_Em]

Kovidgoyal, add_format won't work unless I already have an ID. I suppose I could re-write everything from scratch and do a loop (lookup to see if book already exists, grab ID if it does and add_format, otherwise just run add), but since all the code is already written and used by the GUI, that seems a bit too much effort.

Thanks, Starson17; that points me exactly where I want to go. I was actually hoping you'd respond to this

I plan to identify the merge records by parsing the following tree:

Check for Title
If exists,
  Check for author
  If exists,
    Check for format
    If exists, end
    else
      get book ID
      merge books
else
  add book

I was also thinking of pre-processing with mekk's ISBN code, so the first step could be to check for ISBN and merge if it's found.


As for the "correct naming format", all this test would do is verify that the RE I use properly captures all the expected fields. If everything ends up in the Title field, I know the RE failed and the book file name needs a look before that book gets imported. I could also apply normalization (eg. JRR Tolkien, J.R.R. Tolkien, J. R. R. Tolkien, and Bob Smith, Bob R. Smith, Bob Smith, Jr.) and ensure things like Jr, Esq, Sir, Mr, etc. get handled appropriately by author_sort.

Next time I have a moment, I'll try to write some of these generically in Python so I can use them on the command line but also have available for someone to port to the GUI who is more heavily involved (I don't have much spare time, which is why I'm trying to avoid duplicating effort).

[_Em]

Originally Posted by Starson17

On the issue of command line merge, how do you plan to identify the merge records? The GUI merge takes the selection made by the user for source and destination records, finds the book ids to access the records in the library and goes from there. If you have the book ids, you can use the code from the command line (calibre.library.database2 import LibraryDatabase2) with calibre-debug -c or -e. Give me the record ids and I can give you the command line code.

I just realized I didn't completely answer this. I plan to identify the merge records as the result of an in-db author/title fuzzy match, similar to what you've done in the gui. For a more standard tool, I was thinking something along the lines of calibredb merge targetid currid1 currid2 etc. This could then be passed the ID sets from a parsed calibredb list -s, using the logic from my previous post.

[Starson17]

Originally Posted by _Em

I plan to identify the merge records by parsing the following tree:

Check for Title
If exists,
  Check for author
  If exists,
    Check for format
    If exists, end
    else
      get book ID
      merge books
else
  add book

The code find_identical_books at line 561 of database2.py basically does this, but in reverse. It starts with the author, instead of title, then does a fuzzy match on the book titles by that author (removes spaces, punctuation, etc.) to find "identical" books.

That code is used in add.py (line 159) to do the autosort/automerge work.

I was a bit confused by your comment that there was "no command line merge function" as I couldn't see many situations where you'd know the books to be merged outside of the GUI (i.e. how you would know the ID, other than by writing code to search for books to be merged.) I can see where you'd like the autosort/automerge of new formats into existing records, however.

[_Em]

Originally Posted by Starson17

I was a bit confused by your comment that there was "no command line merge function" as I couldn't see many situations where you'd know the books to be merged outside of the GUI (i.e. how you would know the ID, other than by writing code to search for books to be merged.) I can see where you'd like the autosort/automerge of new formats into existing records, however.

Two methods: calibre_report_duplicates by mekk, and calibredb list --sort-by (series|author|title|author_sort|size)

both return the ID, which then gets piped into the merge. You can also parse it so that you select your base ID as the top result from hits that contains ISBN, then publisher, and then book type in order of preference.