06-30-2016, 08:05 AM | #1 |
Grand Sorcerer
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
SQLite version
I am doing some experimenting with the KoboTouch driver had an error with SQLite access to a database. After a bit of investigation, I discovered the version of SQLite that the sqlite3 library is built with is different in Windows and Linux.
The Windows build is using SQLite version 3.6.21. The Linux version of calibre I had to check with was 1.48. This has a 3.7 version of SQLite. Is there a reason for the different versions? Any chance of getting the Windows version updated? |
06-30-2016, 08:20 AM | #2 |
creator of calibre
Posts: 44,048
Karma: 22669822
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
The version of sqlite is whatever is bundled with python on windows and the system sqlite on linux.
If you want to have a modern sqlite version use the apsw module rather than the sqlite3 module (that is what calibre uses for its own database as well) Note that migrating may not be trivial as there a few differences in the interfaces of the two modules. |
06-30-2016, 09:17 AM | #3 |
Grand Sorcerer
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
I was afraid you were going to say that. I saw the use of apsw, but, there is enough mention of sqlite3 in the code, that I wasn't sure what was going on.
I had a very quick pass over the changes, and I think I might be lucky in the driver. The biggest problem might be the use of "fetchone". The plugins will probably be the painful part. |
06-30-2016, 10:20 AM | #4 |
creator of calibre
Posts: 44,048
Karma: 22669822
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Historically, calibre used to use sqlite3, but I switched to apsw for calibre 1.0 (part of the big database layer rewrite) because it has better performance, and more control over sqlite semantics (apsw is sqlite specific whereas sqlite3 has to conform to the more general DB-API).
Not to mention that one is no longer tied to whatever sqlite is shipped with python. However, as for your original issue -- I am not aware of any incompatibilities between 3.6 ans 3.7 as long as the db is not using any features that were introduced in 3.7 |
06-30-2016, 09:49 PM | #5 |
Grand Sorcerer
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Yes, it's about a database feature added in SQLite 3.7 and a possible change coming from Kobo. With the possibility of better performance from apsw, I think it's worth my time to convert over at this point. There are some things about how the driver handles the database connection that I don't like, so I'll fix that at the same time. If I do that properly, that should flow down to the plugins nicely as well.
|
07-03-2016, 09:37 AM | #6 |
Grand Sorcerer
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
For anyone that is interested, I have converted the KOBO and KOBOTOUCH driver over to apsw. It turned out to be a lot quicker that I expected. Especially when I refactored all the database connects out to a simple method. I did some other refactoring for some code that was almost the same between the two drivers.
The code is in my branch at https://github.com/davidfor/calibre.git. If there are any other Kobo users out there who want to run from source and check everything works, I would be very happy to hear the results. This is especially if you have a WiFi or the original Kobo. I don't have one of these to test with. I can do some testing of this using my Touch. I think they will be OK, but would like confirmation. This will have no effect on extended driver, my Kobo Utilities or the Annotations plugin. I will be making changes to the latter two after these changes have been released to use apsw as well. My plan is to use this for the next week and fix any problems I see. If everything goes OK, I will create a pull request next weekend. |
Tags |
sqlite3 |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Sqlite on Kobo | Lucas Malor | Kobo Developer's Corner | 6 | 03-26-2014 08:06 AM |
SQLite | Meeky | Kindle Developer's Corner | 2 | 01-01-2013 11:02 AM |
Koboreader.sqlite gundged | maxwelledison | Kobo Reader | 21 | 01-12-2011 08:24 AM |
Kindle SQLite DB | akira.spb | Kindle Developer's Corner | 2 | 10-18-2010 10:20 PM |