LibrarianSync: Kindle collections builder - Page 2

kaznelson · 08-30-2014, 10:33 AM

Quote:

Originally Posted by NiLuJe

That vaguely reminds me of a similar user report w/ CM... Does CM does it 'properly'?

No. On firmware >5.4.0 CM has same problems...

NiLuJe · 08-30-2014, 10:49 AM

@kaznelson: Ha! Thanks for the detective work

.

barsanuphe · 08-30-2014, 12:52 PM

@kaznelson
Thanks for all the precious information. I updated my script to add the arguments you noticed when creating a collection (first part of your post), and to use the Kindle locale too. On my pw2, this seems to fix the problem, without modifying the other differences you pointed out. Let me know if it works better for you too.
The collection_count on the ebook entries is not updated when the collections are created, maybe I'll add that later, but I'm not sure where it is used.
I'm not sure it's a good idea to make the collections "cloud collections", since their contents might not all (or at all) be Amazon ebooks.

@NiLuJe
It shouldn't be too hard to parse your json format, I might add it later when I have the time.

NiLuJe · 08-30-2014, 01:12 PM

@kaznelson/@barsanuphe: p_isArchived set to 1 for a non-Cloud collection feels weird to me. I haven't checked on an up-to-date device yet (it's obviously 0 on all my old dumps, where there wasn't any Cloud Collections support), but @kaznelson: were you checking with a Cloud Collection? Does creating a new 'local' (possibly by making sure one with the same name doesn't already exists as a CC?) collection w/ wireless off sets this to 1?

@barsanuphe: No hurry, that was mostly me thinking out loud so that I don't forget about it ;p.

As for p_collectionCount, according to the comments: -- How many collections (int) (used for filtering). So, yeah, that doesn't help much (filtering where? how? when?).

.

NiLuJe · 08-30-2014, 01:26 PM

And on an unrelated note, the list of file formats 'collectionable' might be missing some rarely used ones, here's what's the Calibre plugin checks:

VALID_EXTENSIONS = ['azw', 'azw1', 'azw2', 'azw3', 'mobi', 'pobi', 'prc', 'tpz', 'txt', 'pdf', 'pdr', 'mp3', 'aa', 'aax', 'cbz']

That should add Topaz, Print Replica, Newspaper/Magazines, Audio & Audible stuff, and Comics. Granted, the Audio/Audible/Comics stuff is useless on a K5.

. (The proper list of supported stuff is with the mimetypes in /var/local/appreg.db, IIRC).

kaznelson · 08-30-2014, 03:00 PM

barsanuphe
Thanks, principal problems are solved.

The parameter p_collectionCount for all books in LibSync collection automatically changes on 1 - enough add one book in this collection with standard method. Possibly it's necessary to set value 1 once at creation - "Home" screen will display an expected tree of collections and only root documents...

As to editing wsync.db.
I repeated all creating operations - it helped to understand that a problem in db-bases. As now LibSync works correctly, there is no need to edit wsync.db...

NiLuJe
No, i checked only local collections, without a network.

Only books and images mime types from PW2 appreg.db:

Code:

azw	application/x-mobipocket-ebook
mobi	application/x-mobipocket-ebook
prc	application/x-mobipocket-ebook
pobi	application/x-mobipocket-subscription-feed
azw3	application/x-mobi8-ebook
azw6	application/x-mobi8-images
yj	application/x-yellowjersey-ebook
azw1	application/x-topaz-ebook
tpz	application/x-topaz-ebook
pdf	application/pdf
txt	text/plain
html	text/html
htm	text/html
jpg	image/jpeg
jpeg	image/jpeg

barsanuphe · 08-30-2014, 03:49 PM

Ok, I pushed another version with isArchived set to 0 and an updated filetypes list according to what is in appreg.db.

That p_collectionCount for ebooks set to 1 is not surprising if you use generation through folders. Each book has one unique path, hence is in one collection. In any case, that's not something I update manually.

kaznelson · 08-30-2014, 07:09 PM

barsanuphe

Default value for books in LibSync collections: p_collectionCount = 0
Books at same time in collection and "Home" screen.

PS: If output in error log via "2>>":

Code:

"python generate_collections.py rebuild_from_folders 2>> log.txt"

... then wrote:

Code:

close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

NiLuJe · 08-30-2014, 08:12 PM

@kaznelson: The second 'issue' seems to be a known thing in Python-land (cf. http://bugs.python.org/issue11380).

barsanuphe · 08-30-2014, 08:42 PM

@kaznelson
OK, I understand now. I pushed a fix, let me know if everything is good on your end now.

kaznelson · 08-31-2014, 04:45 AM

barsanuphe
Perfectly well works, thanks!

But without an output "&>" don't create collections? Though, if log.txt not necessary, possible to output in /dev/null...

Quote:

Originally Posted by NiLuJe

The second 'issue' seems to be a known thing in Python-land

I thought, it's an error of LibSync and can be important...

PS: Can add in first message of barsanuphe's project link, correct a title? Or to make for Librarian a new topic?..

barsanuphe · 08-31-2014, 11:26 AM

I removed the redirection and the output, it was just for debug purposes when the ssh connection was down

I can't modify the first post, so I was thinking of making a dedicated thread at some point.

Edit: done.

NiLuJe · 08-31-2014, 11:40 AM

@barsanuphe: If you do, I can move the relevant part of this thread into it if you want

.

knc1 · 08-31-2014, 12:40 PM

Quote:

Originally Posted by barsanuphe

- - - -

I can't modify the first post, so I was thinking of making a dedicated thread at some point.

The sooner the better.

It is the common practice here to distribute things as an attachment to the first post of a thread.
And to (sometimes) reserve the second post for additional install / usage directions.

Just include a "credit" link back to this thread for those who want the prior work you started with.

NiLuJe · 09-06-2014, 07:12 PM

This is a placeholder for barsanuphe's initial release announcement in the other thread

.

----

Quote:

Originally Posted by barsanuphe

Thanks to all the (old but still relevant) information in this thread I managed to write a python script to automatically generate collections on my pw2.
Instead of looking at the files or directory structure, it uses information in a json file to create and populate collections. This way it's possible to manage collections just with one simple text file, such as:

In case someone might find this helpful, the script and KUAL menu entry can be found at https://github.com/barsanuphe/librar.../librariansync.

08-30-2014, 01:12 PM	#19
NiLuJe BLAM! Posts: 13,497 Karma: 26047188 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@kaznelson/@barsanuphe: p_isArchived set to 1 for a non-Cloud collection feels weird to me. I haven't checked on an up-to-date device yet (it's obviously 0 on all my old dumps, where there wasn't any Cloud Collections support), but @kaznelson: were you checking with a Cloud Collection? Does creating a new 'local' (possibly by making sure one with the same name doesn't already exists as a CC?) collection w/ wireless off sets this to 1? @barsanuphe: No hurry, that was mostly me thinking out loud so that I don't forget about it ;p. As for p_collectionCount, according to the comments: -- How many collections (int) (used for filtering). So, yeah, that doesn't help much (filtering where? how? when?). . Last edited by NiLuJe; 08-30-2014 at 01:15 PM.

08-30-2014, 03:00 PM	#21
kaznelson Old Kaz Posts: 208 Karma: 1064151 Join Date: May 2010 Device: PocketBook Era	barsanuphe Thanks, principal problems are solved. The parameter p_collectionCount for all books in LibSync collection automatically changes on 1 - enough add one book in this collection with standard method. Possibly it's necessary to set value 1 once at creation - "Home" screen will display an expected tree of collections and only root documents... As to editing wsync.db. I repeated all creating operations - it helped to understand that a problem in db-bases. As now LibSync works correctly, there is no need to edit wsync.db... NiLuJe No, i checked only local collections, without a network. Only books and images mime types from PW2 appreg.db: Code: azw application/x-mobipocket-ebook mobi application/x-mobipocket-ebook prc application/x-mobipocket-ebook pobi application/x-mobipocket-subscription-feed azw3 application/x-mobi8-ebook azw6 application/x-mobi8-images yj application/x-yellowjersey-ebook azw1 application/x-topaz-ebook tpz application/x-topaz-ebook pdf application/pdf txt text/plain html text/html htm text/html jpg image/jpeg jpeg image/jpeg Last edited by kaznelson; 08-30-2014 at 03:08 PM.

08-30-2014, 07:09 PM	#23
kaznelson Old Kaz Posts: 208 Karma: 1064151 Join Date: May 2010 Device: PocketBook Era	barsanuphe Default value for books in LibSync collections: p_collectionCount = 0 Books at same time in collection and "Home" screen. PS: If output in error log via "2>>": Code: "python generate_collections.py rebuild_from_folders 2>> log.txt" ... then wrote: Code: close failed in file object destructor: sys.excepthook is missing lost sys.stderr Last edited by kaznelson; 08-30-2014 at 07:42 PM.

08-31-2014, 11:26 AM	#27
barsanuphe Connoisseur Posts: 77 Karma: 40008 Join Date: Aug 2014 Location: Paris, France Device: PW2, Kobo H20, Kobo Aura One	I removed the redirection and the output, it was just for debug purposes when the ssh connection was down I can't modify the first post, so I was thinking of making a dedicated thread at some point. Edit: done. Last edited by barsanuphe; 09-03-2014 at 05:19 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Vocabulary builder and flash card feature on old kindle 2/3/dx?	dark.knight1	Kindle Developer's Corner	12	10-10-2015 02:12 PM
Vocabulary builder	bobafruit	Kindle Developer's Corner	25	09-04-2015 02:46 AM
Free (nook/Kindle/iTunes/DRM-free) Legacy Builder [Xtian Business Leadership Advice]	ATDrake	Deals and Resources (No Self-Promotion or Affiliate Links)	0	10-13-2014 03:44 AM
Vocabulary Builder for Kindle Paperwhite 1	pxpxpx	Amazon Kindle	2	02-16-2014 07:07 PM
SBPubX Builder	tmaynard	Fictionwise eBookwise	16	11-20-2008 01:35 PM

08-30-2014, 10:49 AM	#17
NiLuJe BLAM! Posts: 13,497 Karma: 26047188 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@kaznelson: Ha! Thanks for the detective work .

08-30-2014, 12:52 PM	#18
barsanuphe Connoisseur Posts: 77 Karma: 40008 Join Date: Aug 2014 Location: Paris, France Device: PW2, Kobo H20, Kobo Aura One	@kaznelson Thanks for all the precious information. I updated my script to add the arguments you noticed when creating a collection (first part of your post), and to use the Kindle locale too. On my pw2, this seems to fix the problem, without modifying the other differences you pointed out. Let me know if it works better for you too. The collection_count on the ebook entries is not updated when the collections are created, maybe I'll add that later, but I'm not sure where it is used. I'm not sure it's a good idea to make the collections "cloud collections", since their contents might not all (or at all) be Amazon ebooks. @NiLuJe It shouldn't be too hard to parse your json format, I might add it later when I have the time.

08-30-2014, 01:26 PM	#20
NiLuJe BLAM! Posts: 13,497 Karma: 26047188 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	And on an unrelated note, the list of file formats 'collectionable' might be missing some rarely used ones, here's what's the Calibre plugin checks: VALID_EXTENSIONS = ['azw', 'azw1', 'azw2', 'azw3', 'mobi', 'pobi', 'prc', 'tpz', 'txt', 'pdf', 'pdr', 'mp3', 'aa', 'aax', 'cbz'] That should add Topaz, Print Replica, Newspaper/Magazines, Audio & Audible stuff, and Comics. Granted, the Audio/Audible/Comics stuff is useless on a K5. . (The proper list of supported stuff is with the mimetypes in /var/local/appreg.db, IIRC).

08-30-2014, 03:49 PM	#22
barsanuphe Connoisseur Posts: 77 Karma: 40008 Join Date: Aug 2014 Location: Paris, France Device: PW2, Kobo H20, Kobo Aura One	Ok, I pushed another version with isArchived set to 0 and an updated filetypes list according to what is in appreg.db. That p_collectionCount for ebooks set to 1 is not surprising if you use generation through folders. Each book has one unique path, hence is in one collection. In any case, that's not something I update manually.

08-30-2014, 08:12 PM	#24
NiLuJe BLAM! Posts: 13,497 Karma: 26047188 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@kaznelson: The second 'issue' seems to be a known thing in Python-land (cf. http://bugs.python.org/issue11380).

08-30-2014, 08:42 PM	#25
barsanuphe Connoisseur Posts: 77 Karma: 40008 Join Date: Aug 2014 Location: Paris, France Device: PW2, Kobo H20, Kobo Aura One	@kaznelson OK, I understand now. I pushed a fix, let me know if everything is good on your end now.

08-31-2014, 11:40 AM	#28
NiLuJe BLAM! Posts: 13,497 Karma: 26047188 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	@barsanuphe: If you do, I can move the relevant part of this thread into it if you want .

Advert

Advert