[GUI Plugin] Find Duplicates

[Geremia]

Originally Posted by komali_2

[*] In that menu, click "Merge All Groups." Remember, this option is only available if you are using the "Find Duplicates" plugin with my changes (available at github link above, or maybe when you're reading this, merged into the main plugin).

I wish your commit was merged into master. An auto-merging feature is desperately needed.

[kiwidude]

Originally Posted by Geremia

I wish your commit was merged into master. An auto-merging feature is desperately needed.

That will never be in my interest to do so. I think it is the single most dangerous thing you could do to corrupt your library (I have other words for it but they may offend).

If you have yourself in a situation where you are creating so many duplicates that you feel you need to automate resolving them then go back and take a good hard look at how you got into that mess in the first place and change your workflows.

Automated merging has a 99% chance of making the problem even worse for you, not better. There are way too many reasons how this goes wrong - remember duplicate groups in many situations are not "guaranteed" duplicates, they are suggestions. There are no guarantees unless you have been meticulous about each book added to your library whether the book actually is what the metadata indicates it is - and if you were a meticulous person then you would not even be in the situation of thousands of duplicates. Let alone when you start doing any sort of non-exact searches with all the false positives it will find.

The plugin can make no determinations as to which of the items in the group it should merge into. One may have "better" metadata. One might have a better format conversion of the book.

So automated merging is the equivalent of "close your eyes and hope for the best". That is not a behavior I want to encourage by adding it as a feature to the plugin, because inevitably people will come here after it goes wrong for them and complain about why the plugin could not figure it all out for them to prevent it.

[ownedbycats]

Originally Posted by kiwidude

That will never be in my interest to do so. I think it is the single most dangerous thing you could do to corrupt your library (I have other words for it but they may offend).

corruption as in database corruption, or "severely eff'd up data" corruption?

[kiwidude]

Originally Posted by ownedbycats

corruption as in database corruption, or "severely eff'd up data" corruption?

Data. Lost books, lost metadata, metadata pointing to the wrong books.

[random733]

Is there any working version with the auto merging feature? komali2s version (1.8.5) here unfortunately does not work with the latest Calibre version (linux and windows) - keeps requiring a reboot without the plugin being fully installed.

I understand the concerns about the dangers to the integrity of the library. But I think you can manage the risk with a backup. And in the end, everyone is responsible for actually using this function (which is also explicitly declared as dangerous).

[theducks]

Originally Posted by random733

Is there any working version with the auto merging feature? komali2s version (1.8.5) here unfortunately does not work with the latest Calibre version (linux and windows) - keeps requiring a reboot without the plugin being fully installed.

I understand the concerns about the dangers to the integrity of the library. But I think you can manage the risk with a backup. And in the end, everyone is responsible for actually using this function (which is also explicitly declared as dangerous).

How much time does doing this manually from the results take?
Even previewing versions of a 'dup', only takes a couple of minutes to resolve (merge, dump one or let stand).

If you are having to process very many results after your initial culling, I would say you need to rethink your process.

[DNSB]

I find it rather interesting to wonder about how people are obtaining their ebooks that would cause so many duplicates. My thoughts tend to meander in the direction of downloading multiple collections of ebooks which suggests copyright violation in bulk.

[ownedbycats]

Originally Posted by DNSB

I find it rather interesting to wonder about how people are obtaining their ebooks that would cause so many duplicates. My thoughts tend to meander in the direction of downloading multiple collections of ebooks which suggests copyright violation in bulk.

Before I started using Calibre, I had slightly sloppy file management practices and ended up saving eBooks in several different folders. If I were to import those folders to Calibre I'd end up with quite a few duplicates.

But seeing as I have a brain injury and most people don't, the "collections of pirated eBooks" feels more likely.

[DNSB]

Originally Posted by ownedbycats

Before I started using Calibre, I had slightly sloppy file management practices and ended up saving eBooks in several different folders. If I were to import those folders to Calibre I'd end up with quite a few duplicates.

But seeing as I have a brain injury and most people don't, the "collections of pirated eBooks" feels more likely.

I had a similar experience with duplicates when I first started using calibre and was adding my collection of ebooks in a directory structure to calibre. However, since I only had one calibre library, when I imported the contents of a directory, calibre whined about my adding duplicates based on the book title.

[theducks]

And we probably only built (added books) the Library ONCE.
I move a nested folders worth at a time, then Archived that folder so it wold not be done again.
Repeated, fixed
until there was only THE Calibre library

[Rellwood]

I hate to ask if this is redundant, but I finally updated from a much older version to find that the advanced features is not included. Are we not doing this anymore? I am trying to match an identifier that isn't showing matched in one library but is showing them in another.

I am trying to make sure that all my ebooks that I have sent to Amazon after editing are matching up with what I have in my library. Amazon has a special id for those docs and I have called it "stk mobi-asin" I have a special library with those books and I can't seem to get the plugin to match with it. This is a new problem.

[ownedbycats]

https://www.mobileread.com/forums/sh...3&postcount=30

[JSWolf]

Originally Posted by DNSB

I find it rather interesting to wonder about how people are obtaining their ebooks that would cause so many duplicates. My thoughts tend to meander in the direction of downloading multiple collections of ebooks which suggests copyright violation in bulk.

One way I sometimes get duplicates is when I download the latest version of an eBook and the name in the eBook is changed so I don't get flagged about a possible duplicate.

[DNSB]

Originally Posted by JSWolf

One way I sometimes get duplicates is when I download the latest version of an eBook and the name in the eBook is changed so I don't get flagged about a possible duplicate.

I've had the same type of situation but of them when the new version has extraneous junk in the downloaded book title (does anyone really think that adding : A Novel to the book title is useful?) but it doesn't happen en masse to the point where I would want to accept the potential issues of a automated mass merge. Very few people would like to have their ebook library omeletted.

How are you finding the duplicates if the title has changed? The ones I've found have been when I've edited the metadata and run Find Duplicates on both my intake and main libraries afterwards.

"On ne saurait faire d'omelette sans casser des œufs"—François de Charette

[JSWolf]

Originally Posted by DNSB

I've had the same type of situation but of them when the new version has extraneous junk in the downloaded book title (does anyone really think that adding : A Novel to the book title is useful?) but it doesn't happen en masse to the point where I would want to accept the potential issues of a automated mass merge. Very few people would like to have their ebook library omeletted.

How are you finding the duplicates if the title has changed? The ones I've found have been when I've edited the metadata and run Find Duplicates on both my intake and main libraries afterwards.

"On ne saurait faire d'omelette sans casser des œufs"—François de Charette

Sorting by author when I know I've downloaded it before works. If not, sometimes finding by similar titles can work if the change is not too different.