![]() |
#1 |
Diligent dilettante
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 3,432
Karma: 48736498
Join Date: Sep 2019
Location: in my mind
Device: Kobo Sage; Kobo Libra H2O
|
KOReader crashs when trying to search Calibre metadata
After finishing a book 20 minutes ago, I decided to "Browse series" for my next book. KOReader crashed. I tried again, it crashed again. So I tried "Browse tags" and it crashed again, tried "browse tags" again, and it crashed again. That' probably why the crash log was too big to attach, hence this Google Drive link - sorry!
|
![]() |
![]() |
![]() |
#2 |
Wizard
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,074
Karma: 3581333
Join Date: Sep 2014
Location: Ukraine
Device: Kindle PW2, PW3, PW4, Voyage
|
Check for the nightly build
https://github.com/koreader/koreader/issues/9085 |
![]() |
![]() |
![]() |
#3 | |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Quote:
Seems something buried into the UI stack. We need to wait for our setDirty expert @NiLuJe |
|
![]() |
![]() |
![]() |
#4 |
BLAM!
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 13,492
Karma: 26047188
Join Date: Jun 2010
Location: Paris, France
Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E
|
Umm, I'm only seeing the crash fixed by #9085 in there? Did I miss one?
|
![]() |
![]() |
![]() |
#5 |
Diligent dilettante
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 3,432
Karma: 48736498
Join Date: Sep 2019
Location: in my mind
Device: Kobo Sage; Kobo Libra H2O
|
Thanks all - the latest nightly has fixed it.
![]() |
![]() |
![]() |
![]() |
#6 |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Huh?. I didn't expect that from that stacktrace.
![]() Anyways. Happy to see it is already fixed ![]() |
![]() |
![]() |
![]() |
#7 |
Member
![]() Posts: 13
Karma: 10
Join Date: Jan 2024
Device: Nook OG 32GB, KT3, PW5 16Gb, KT5, Android KOreader
|
Sorry to necropost, but I am having this problem too on 2024.04. Where can I find logs to troubleshoot or otherwise share?
|
![]() |
![]() |
![]() |
#8 | |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Quote:
|
|
![]() |
![]() |
![]() |
#9 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 444
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
|
This has been happening to me with my Kobo Libra 2. My database is, admittedly, quite large (390mb after compression with the KoboUtilities plugin, almost 4000 books on the device). I've been checking github, specifically the latest related bug report here, and am I right in understanding that this issue cannot be solved?
|
![]() |
![]() |
![]() |
#10 | |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Quote:
![]() No idea what's your issue. Follow the bug report template and attach logs (here or in github). Most likely your calibre json file size is above 30MB and the program switches to a utterly broken mode where it tries to parse json manually, instead of using a proper parser. That's because the "proper parser" - rapidjson- with its current lua bindings doesn't provide a way to build the result in steps. It tries to build everything in memory until the result is ready. So the hardcap of 30MiB is there to prevent users with <512MiB or even with +512MiB and high memory pressure to get an out of memory error. The proper solution is to implement a parser to use for big files, but in the mean time tweaking the hardcap https://github.com/koreader/koreader...ent-1101370638 will work up to the size of your available ram. |
|
![]() |
![]() |
![]() |
#11 |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Another related issue is that, unless you push books using wireless transfers, we consume calibre files as is, with a bunch of metadata we don't care and we don't use.
In wireless transfers we just discard these fields before writting the file, so the json ends up being much more slim. The difference isn't too much for calibre beginners (up to 10x) but grows a lot if you're one of those advanced users with toons of custom metadata. |
![]() |
![]() |
![]() |
#12 |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 444
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
|
Thank you for your help, pazos - everything is much clearer now. I had already found that link before actually and tried to tweak the hardcap but without understanding what I was doing
![]() My metadata.calibre file is 39.3MB, and changing the max_json_filesize to 40 * etc. doesn't work. The Kobo Libra 2's RAM is 512MB, so I'm not sure it's wise to try and increase that further. And you are correct in assuming that I'm an 'advanced' Calibre user ![]() I would be willing to share the log if you think that's helpful. |
![]() |
![]() |
![]() |
#13 |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Make sure those 39.3 are in fact MB and not MiB because that's more than 40x1000x1000 bytes
![]() It should be safe to raise the size to 50 x 1000 x 1000. Do that, restart the program and try again. One of the following two things will happen: 1. The program crashes and you see "./luajit: not enough memory" 2. Everything works. I hope it is the second, please let me know ![]() |
![]() |
![]() |
![]() |
#14 | |
Evangelist
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 444
Karma: 287725
Join Date: Sep 2014
Location: Europe
Device: Kobo Aura H2O/Glo HD/Libra 2, Kindle PW3/PW5
|
Quote:
I did try with 50 x 1000 x 1000 and it works perfectly now. Thank you for your patience ![]() |
|
![]() |
![]() |
![]() |
#15 | |
cosiñeiro
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 1,316
Karma: 2200073
Join Date: Apr 2014
Device: BQ Cervantes 4
|
Quote:
Hopefully you won't need to do it many times as a proper fix is in the works: https://github.com/koreader/koreader/pull/11922 |
|
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
koreader deletes calibre metadata after sending book via USB to device | Zoasa | KOReader | 4 | 06-11-2021 01:38 PM |
[Libra] KOReader not updating Calibre metadata | Uncle Robin | KOReader | 9 | 05-05-2021 01:15 PM |
KOReader Calibre Search Adjustment | Incanus | KOReader | 8 | 02-14-2021 06:46 PM |
Calibre: Metadata Search -> Wildcards possible? | bigbear77 | Library Management | 6 | 11-23-2013 05:44 AM |
Ideas on how to improve Calibre: new metadata source (goodreads) + a new cover search | MelBr | Calibre | 2 | 03-06-2013 03:25 PM |