MTP connection issues

[aramaziakin]

As far as I can tell Calibre keeps creating files (driveinfo.calibre and metadata.calibre) that it then can't read which causes connection issues.
So I have Calibre send my books to a folder called 'Books' on an sd card in my android device, that works fine. But then for some reason Calibre keeps creating a folder called 'Document' on both the sd card and internal storage and creates the 2 files I mention before. If I don't delete these new folders Calibre throws up a connection error when I connect my device. Calibre is fine with the driveinfo and metadata files in the 'Books' folder, it is only with the ones it creates in these 'Document' folders that causes connection issues.
Is there any way to stop Calibre creating these files, or anyway to fix the files so they don't cause connection issues?

[theducks]

Those 2 files are the log of what Calibre has previously found ON THAT Drive. They are in the root of the drive regardless of where the books are.

The SD card is also a drive, so they should be there also

If they are missing, then Calibre has to rescan the whole drive(s) as the 'what is known' shortcut is missing.

MTP is iffy in some cases. One thing: Only have 1 MTP device connected at a time.
This FAQ may help (a tiny bit) https://manual.calibre-ebook.com/faq...or-kindle-fire

[kovidgoyal]

If calibre cant read files it has created previously, that indicates there are firmware issues on your device. A reboot might help, if not a reset and if that doesnt help either, then you can use one of the wireless conenction methods instead.

[aramaziakin]

Originally Posted by kovidgoyal

If calibre cant read files it has created previously, that indicates there are firmware issues on your device. A reboot might help, if not a reset and if that doesnt help either, then you can use one of the wireless conenction methods instead.

I've not had much luck with the wireless options and rebooting everything hasn't worked, is there any of figuring out what's actually going wrong here?

[theducks]

Without Calibre running (involved), can you use Windows to copy a bunch of files to a Test folder (so it is easy to clean up after the test)?
If that fails, then there may be storage integrity issues (With my devices, I still have file transfer mod, so Windows can check the drive. With MTP, I have no clue how to get that done)

[theducks]

Found something
NOT TESTED or endorsed

https://www.auslogics.com/en/article...ed-windows-10/

[kovidgoyal]

Originally Posted by aramaziakin

I've not had much luck with the wireless options and rebooting everything hasn't worked, is there any of figuring out what's actually going wrong here?

What's going wrong is calibre is making a request to the MTP subsytem on windows to do file i/o and that request is failing. This happens on some devices with flaky firmware for files that are frequently updated such as those two. If you don't want to try reset then basically deleting those two files prior to connection is the best workaround.

[aramaziakin]

Well the wireless connection suddenly started working again and in a pinch I can just take out the SD card and connect it to my computer, so I think I'll just not use mtp since it's buggy.
Thanks for the help guys.

[Ealdwulf]

Hrmm, I started experiencing an issue with transferring books to my Nook Tablet around a week ago, while it worked without issue, for many years before that. The error was also with transferring metadata.

I tried deleting the two files in the root of the Nook storage and still got the error when transferring. Further testing on my end, revealed that if I use 7.13.0 I get NO errors, but 7.14.0 does give errors.

I ended uo filing a bug report just now for this situation, #2073546. Whether this is also the reason that aramaziakin is also having the error or not, I couldn't say for certain since I have no knowledge on what version they are experiencing the problem with (or even if it is the same problem), but I wanted to at least mention it and suggest maybe trying to rollback the version of Calibre being used for testing purposes. Especially if something new that didn't happen previously.

[sushigal007]

Been looking for a solution to this problem myself for the past couple of days after getting it suddenly after years with no problems, and rolling back to 7.13.0 worked for me, many thanks.

[wanderson]

This seems like the right forum topic to discuss this issue. If not, please let me know where to redirect.

PLATFORMS:
Calibre 7.24.0 on Windows 11 / Dell Inspiron laptop
Android 13 on TCL NxtPaper 11 tablet with 128G internal & 128G SD card

GOAL:
Transfer books from Windows Calibre library to tablet's SD card

RESULTS:
Books are consistently put in internal storage, not SD card.

CONFIGURATION:

I'm enabling MTP transfer when the tablet is plugged in; this evidently seems to work, as the device information seen by Calibre sees both storage spaces:

Spoiler:

{'device_version': '1.0',
 'friendly_name': 'TCL NXTPAPER 11',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'TCL',
 'model_name': 'TCL NXTPAPER 11',
 'protocol': 'MTP: 1.00',
 'serial_number': 'E598DB9E0D2F9BCDD8844B50D861FDD2',
 'storage': [{'capacity': 128145522688,
              'capacity_objects': 0,
              'description': 'SD card',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 128142540800,
              'id': 's10001',
              'name': 'SD card',
              'rw': True,
              'type': 'removable_ram'},
             {'capacity': 113995907072,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 98783334400,
              'id': 's20001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'unknown'}

I have configured the device to use the directory "/Books" on the SD card (a pre-existing directory). However, my book transfers always go to the root of the internal storage, not the SD card. The transfer creates the internal storage directory "/Books", which did not previously exist. Interestingly, the subdirectories for the transfer are built correctly, per my device configuration template.

So, the question seems to be, "What am I doing wrong?"

All help is welcome,

Wanderson

[DNSB]

When you right click on a book in Library view and then hover on Send to Device, do you see Send to storage card A greyed out or dark? If it dark and you click on it does it send the ebook to the Books directory on the SD card?

[wanderson]

Thanks for the fast response, David!

Originally Posted by DNSB

When you right click on a book in Library view and then hover on Send to Device, do you see Send to storage card A greyed out or dark?

"Send to storage card A" is dark (activated).

Originally Posted by DNSB

If it dark and you click on it does it send the ebook to the Books directory on the SD card?

Alas, it still goes to internal storage as described above.

[kovidgoyal]

Hard to say sadly I dont have any MTP devices with an SD card anymore havent had one for years so it's possible this part of the code has bit rotted, it needs someone with the hardware to chekc the code.

[wanderson]

I thought I'd do a little more debugging to see if the MTP issue resided in the Device or in Calibre, and I came up with some potentially useful results.

I tried the same transfer to my phone (Samsung Galaxy S20+), which also has a microSD (what can I say, I like microSD devices). And the transfer succeeded, placing the book(s) into the correct place, i.e. the SDcard.

So, initially, I guessed that the firmware on the TCL tablet was at fault. I further tried to again transfer book(s) to the TCL tablet, but instead I specified that Calibre place them in the internal storage. And, as you have probably guessed by now, the books ended up on the SDcard. Calibre was evidently confusing the SDcard and the internal storage.

I reviewed the device information from each device. The TCL tablet returned:

Spoiler:

{'device_version': '1.0',
 'friendly_name': 'TCL NXTPAPER 11',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'TCL',
 'model_name': 'TCL NXTPAPER 11',
 'protocol': 'MTP: 1.00',
 'storage': [{'capacity': 128145522688,
              'capacity_objects': 0,
              'description': 'SD card',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 128142671872,
              'id': 's10001',
              'name': 'SD card',
              'rw': True,
              'type': 'removable_ram'},
             {'capacity': 113995907072,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 98897649664,
              'id': 's20001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'unknown'}

while the Samsung phone returned the following:

Spoiler:

{'device_version': 'G986U1UESBHXJ1',
 'friendly_name': 'Galaxy S20+ 5G',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'samsung',
 'model_name': 'SM-G986U1',
 'protocol': 'MTP: 1.00',
 'storage': [{'capacity': 115519483904,
              'capacity_objects': 0,
              'description': 'Internal storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 29999812608,
              'id': 's10001',
              'name': 'Internal storage',
              'rw': True,
              'type': 'fixed_ram'},
             {'capacity': 128144375808,
              'capacity_objects': 0,
              'description': 'SD card',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 92647325696,
              'id': 's20001',
              'name': 'SD card',
              'rw': True,
              'type': 'removable_ram'}],
 'type': 'phone'}

Did you see the difference? The TCL device returned the SDcard storage first, then the internal storage, while the Samsung device returned the internal storage first, followed by the SDcard. I speculate that the Calibre code assumes the internal storage is always returned first, followed by whatever external storage is available.

I did a quick scan of the MTP protocol spec (available here), and found no particular order specified in the device description transaction, but it is several hundred pages long, and I might have missed it.

My guess is that the Calibre MTP code could be made a bit more robust by actually examining the "type" and "name" fields and making decisions based on them, rather than on the order returned.

In any case, I have a short-term workaround, until the code is changed. Hopefully, the change notes will reference MTP when and if the code is corrected.

Regards,

Wanderson