04-14-2011, 03:40 PM | #46 |
Wizard
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
|
|
04-15-2011, 03:46 AM | #47 | ||||||
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
Quote:
Quote:
To keep the icon correct, call Code:
gui.set_highlight_only_button_icon() Quote:
Perhaps an 'I am done' menu item is sufficient? It would clear the search, clear (restore?) the search restriction, and restore the marking flag. Quote:
It could also be useful to be able to show exemptions as you describe, in order to give me the power of the GUI to see what is going on. Then, if I see a book that shouldn't be there, I would right-click on it to get the exemptions editor. That book would be the root, and I remove the offending books. Does this work for you? |
||||||
Advert | |
|
04-15-2011, 03:49 AM | #48 | ||
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
|
||
04-15-2011, 04:09 AM | #49 | |
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
There be some dragons in using sort history or the proposed method. What that tells you is that the last time the view was sorted, it used columns X. It does not tell you that the view is currently in that order. For example, added books will go on top, which is probably not where they should be. I am not sure you should assume that the last sort is still good. What I will look at is whether we can sort the filtered set (after search) instead of the entire cache. First blush look in the GUI says it will work there, because sorts are reapplied after a search. I don't know how the content server would treat this. Perhaps an option to db.sort would be best, so that the code that is prepared for this behavior can ask for it. |
|
04-15-2011, 04:30 AM | #50 |
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
OK, I have changes that sort the sublist instead of the entire cache. Before I submit them (they are very small, so Kovid might be able to look at them), a question:
Without this change, searches operated on the sorted cache. library.caches.search generates a list of IDs, then order that list in the same way as the cache order. Because of this, it is not necessary to sort again after a search. However, if we are sorting the sublist (the search results), then the sort must be redone after every search. Given that this matters only on a huge library, it isn't clear to me which is better. If searches tend to find large segments of the library, doing the sort again will be slower overall than what it is today. However, if the searches tend to find much smaller segments of the library, sorting the segment is a significant win. I can sort 400 records 100 times before I pay the cost of sorting 40,000 once. Opinions? |
Advert | |
|
04-15-2011, 04:50 AM | #51 | |
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Quote:
From a search perspective I would have thought optimising for the 99% case of a far smaller search subset and having that displayed faster would be a nice win. I guess the question is "how much" slower is it for very large searches? |
|
04-15-2011, 04:53 AM | #52 | |
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Edit: The worst case slowdown is the time it currently takes to do a sort. One would pay that cost on every search, instead of only when sorting. Last edited by chaley; 04-15-2011 at 04:55 AM. |
|
04-15-2011, 05:18 AM | #53 |
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Hmmm... not sure the sound of slowing down the display of "no search" sounds a good idea. If you have multiple secondary sort columns, is the penalty multiplied? e.g. by title/series/author?
I did find it a bit "laggy" when I was sorting each "next result group" with two calls to sort_by_named_field. If that is indicative of the extra time that will be added to every time you clear your search results it probably isn't the best to change this way. |
04-15-2011, 05:24 AM | #54 |
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Some numbers. On my 20,000 book test library
start with sorted by title Without the change: - search for 'foo'. Finds 48 records. Time: approx 2.5 seconds - sort on author: Approx 1.5 seconds - clear the search: Time: instant With the change: - search for 'foo'. Approx 2.5 seconds - sort on author: instant - clear the search: Approx 1.5 seconds These numbers are consistent with the underlying changes. One pays for a complete sort when the search is cleared. @kiwidude: there is one sort, no appreciable lag for multiple columns. The GUI knows to use the DB's multi-column sort in this case. |
04-15-2011, 05:42 AM | #55 |
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Interesting, thx for the numbers. The issue for me (and this is just my opinion) is that clearing the search is very much a part of the "99%" a user does in Calibre. Whereas in normal usage, they are far less frequently changing their sort column order.
Had this change allowed the cleared search to remain the same, speeded up small sorts but slowed down sorts of large search results then it would be brilliant Clearly there is a conflict. I can't help wondering if "despite the dragons" we are instead not better off just trying to find another way to optimise for the type of behaviour I need for duplicate finder, that won't compromise general speed in Calibre today. For instance my idea of using sort history. I know you gave an example where the view contains added books - in this instance that is perfectly fine because I am refreshing the search and those books should disappear when that happens? And even if they didn't, letting them reamain at the top by not "re-sorting" is fine too? |
04-15-2011, 05:47 AM | #56 |
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Another possibility (though maybe this gets too complicated for doing) is could a background thread be used to sort the "rest of the results"? So if I "click away" in the gui, it is just sorting my local subset. In the background it is applying that sort to the rest of the data cache. Then when I clear my search, chances are it has already sorted them?
|
04-15-2011, 05:56 AM | #57 |
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
The thread would be difficult, because threading the GUI and the database is very hard.
I backed out the changes. I was wrong when I said that adding a multisort would be complicated. It isn't. Try adding the following at the end of the books view class in gui.library.views (near line 730) and let me know if it works for you. Call it using something like: Code:
self.library_view.multisort((('title',False), ('authors', True))) Code:
def multisort(self, fields): if len(fields) == 0: return sh = self._model.sort_history for n,d in reversed(fields): sh.insert(0, (n, d)) sh = self.cleanup_sort_history(sh) self._model.sort_history = [tuple(x) for x in sh] self._model.resort() col = fields[0][0] dir = Qt.AscendingOrder if fields[0][1] else Qt.DescendingOrder if col in self.column_map: col = self.column_map.index(col) hdrs = self.horizontalHeader() try: hdrs.setSortIndicator(col, dir) except: pass |
04-15-2011, 06:04 AM | #58 | |||||
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Quote:
Quote:
Quote:
Quote:
1. Each time a user does "Find Duplicates" in a Calibre session, I remember what restriction and highlighting setting they had, and that is what I can restore to. The one exception will be if they do "Find Duplicates" while they are currently displaying duplicates (i.e. not "finished") in which case it will use the previous restriction/highlighting mode value. Perhaps I also need to remember their sort history to reapply, as once they exist duplicate mode then searching by "marked" as a primary sort would not be useful? 2. The user can exit the highlighting/restriction in any one of these ways. - Manually clearing the restriction box/toggling highlight button - Moving to the next group after merging the last groups contents - Marking the last group as a duplicate exemption - Marking all groups as duplicate exemptions - Choosing a "Clear duplicate search results" menu option (I'm open to suggestions on the name!) Quote:
That gives the user an unsorted list of books (well if they did this in the middle of finding duplicates they would still be sorted by title I guess). The user can pick a book and choose "Manage exemptions for this book" to view it's paired books and remove relationships. I guess I would also still keep the "Remove selected exemptions" menu item, so if a user wants to bulk remove for multiple books they can do so. The "Manage exemptions" dialog will just display a grid with a small subset of columns like Title, Authors, Series, Tags and maybe Date. With as you say a checkbox column allowing the user to specify which of the rows after the first one to remove, and a select all button. When I exit the "Manage exemptions" dialog, if the user made changes I refresh the search if they were showing "marked:not_duplicates". If they were showing a duplicate search result group (or anything else), there is no need to refresh. They can only get books "repartitioned" or "reappearing" by actually running Find Duplicates again. The "Manage exemptions for this book" would be available on any book that was marked "not_duplicate". So in theory they could do it without entering the "Show all duplicate exemptions" search and instead do it while reviewing their duplicate groups. However it would be a bit of a random click fest as to whether they find that "Manage" menu item enabled or not as there will be no other visual indication that a book has duplicate exemptions. I'm hapy with that. |
|||||
04-15-2011, 06:40 AM | #59 | ||||||
Grand Sorcerer
Posts: 11,866
Karma: 7036359
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
|
||||||
04-15-2011, 06:54 AM | #60 | ||||
Calibre Plugins Developer
Posts: 4,664
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Quote:
Quote:
Quote:
Quote:
|
||||
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Duplicate Detection | Philosopher | Library Management | 114 | 09-08-2022 07:03 PM |
[GUI Plugin] Plugin Updater **Deprecated** | kiwidude | Plugins | 159 | 06-19-2011 12:27 PM |
Duplicate Detection | albill | Calibre | 2 | 10-26-2010 02:21 PM |
New Plugin Type Idea: Library Plugin | cgranade | Plugins | 3 | 09-15-2010 12:11 PM |
Help with Chapter detection | ubergeeksov | Calibre | 0 | 09-02-2010 04:56 AM |