CC and the cloud: discussion

[kaufman]

That sounds very much like you don't have OCD. Must be nice.

[chaley]

Originally Posted by cybmole

OK but given that calibre cloud is free (with ads). or just over £1 for ad-free pro version you ain't going to get rich competing!

But if you can write such an app AND get it accepted by amazon ( unlike c cloud) then I'll buy a copy

One reason for option 2 and the accompanying apps is to avoid competing directly with calibre cloud. I freely admit that calibre cloud does something useful that has nothing to do with CC and don't see any particular reason to offer up the same thing. Instead we want to offer something to people who prefer to use CC: the ability to fetch books from calibre libraries anywhere and add them to CC. Cloud storage like Dropbox, Copy, etc are such places, but there are others. Some people have internet-facing NAS devices, as evidenced by the popularity of things like calibre-to-opds and COPS, and these would work fine with option 2 by fetching the folder containing the books.

Being able to import a bare epub might be worth doing as part of the CC side of option 2. Doing so would let us import epubs fetched from non-calibre content servers such as the ones named above. It could be that enough people "polish" the metadata to make it at least somewhat correct, but I hesitate because there is a good chance we will get blamed if it isn't. Still thinking about this.

As for Amazon, who knows? I am sure that the base option 2 (import from folder) will be accepted because CC's scan-for-new-books does nearly the same thing today. As for the accompanying app, I suspect that Amazon doesn't include cloud apps that don't support Amazon Cloud Drive. We will see.

[chaley]

Originally Posted by kaufman

That sounds very much like you don't have OCD. Must be nice.

I have it. It just manifests in different places.

[kaufman]

Originally Posted by chaley

Being able to import a bare epub might be worth doing as part of the CC side of option 2. Doing so would let us import epubs fetched from non-calibre content servers such as the ones named above. It could be that enough people "polish" the metadata to make it at least somewhat correct, but I hesitate because there is a good chance we will get blamed if it isn't. Still thinking about this.

How is this different from the "scan for new books" option that exists now? I have never done this, but don't people run into the same issue today? Are you getting a lot of complaints about metadata now.

[chaley]

Originally Posted by kaufman

How is this different from the "scan for new books" option that exists now? I have never done this, but don't people run into the same issue today? Are you getting a lot of complaints about metadata now.

Technically it isn't different, which is one reason that option 2 interests me. However, there are some qualitative differences.

The first is the source of the books. Generally the books discovered in a CC library either came via calibre send-to-device/content server so that the metadata in the book was updated on the way out and is at least close to correct, or they came from "somewhere else" so that the metadata is unknown. The first works well. The second doesn't, and yes, we hear about it. We also hear about problems with non-epub, especially pdf where the scan produces books named "asdbwe" and the like.

A second is when the scan happens. It happens as part of a connect, which means that if a book can be matched to the calibre library the user sees the correct metadata immediately. There is no intermediate state such as will exist between an import and a connect/metadata update.

A third is that scanning is not advertised as a CC advantage/feature. It is simply something that CC does. The result: expectations are usually not set. We hear frequently hear from people who know about the scan and have expectations for it. In particular we hear from people who want the file names parsed for title/author/series because the book contains no metadata or the metadata is wrong. Down this path lies a reimplementation of calibre, something we are not interested in doing.

[kaufman]

Originally Posted by chaley

A second is when the scan happens. It happens as part of a connect, which means that if a book can be matched to the calibre library the user sees the correct metadata immediately.

Two questions:

1) how does the book get matched if the metadata is messed up?

2) What happens to the book if it isn't matched?

[chaley]

Originally Posted by kaufman

Two questions:

1) how does the book get matched if the metadata is messed up?

You must be referring to books that came from "elsewhere", not books that came from calibre to begin with. In that case, calibre does a fuzzy match based on title and author. That may or may not result in the correct match.

2) What happens to the book if it isn't matched?

It remains in CC's library with the wrong metadata. It appears in calibre's device view without the in-library check.

[kaufman]

Interesting.

Well, I'll go back to (more or less) what I said before, if you think this (option 2) is doable, it would be a nice feature to have, but if it's going to add a zillion support headaches, then it's probably not that important.

What do you think of the idea of showing a dialog allowing people to modify the title/author when a book is imported? I'm sure people will ask for full metadata editing, but if you just allowed them to do those 2, it would make it easier to match books already in calibre, or to remember what the book was later after they had uploaded it from cc to calibre.

[kaufman]

One other thing. I am convinced that you shouldn't publish the ability to get epub files because of all of the "why isn't cc working as an e-reader" complaints you will get, but I would like to suggest a concept for the import from Director function:

In options,

people can define a default import dirctory or select "ask each time"

People can select whether to delete the file after import


Then, on the main menu, have an "import new files" which would scan the directory and import all the files that were of a type that this user had selected locations for.

This would minimize the number of steps people would have to go through to import files.

[chaley]

Originally Posted by kaufman

Well, I'll go back to (more or less) what I said before, if you think this (option 2) is doable, it would be a nice feature to have, but if it's going to add a zillion support headaches, then it's probably not that important.

What do you think of the idea of showing a dialog allowing people to modify the title/author when a book is imported? I'm sure people will ask for full metadata editing, but if you just allowed them to do those 2, it would make it easier to match books already in calibre, or to remember what the book was later after they had uploaded it from cc to calibre.

Not matching calibre is not an issue with option 2 as described. The metadata.opf file contains the UUID and the title/author from the calibre library. The metadata in the book is ignored.

Originally Posted by kaufman

One other thing. I am convinced that you shouldn't publish the ability to get epub files because of all of the "why isn't cc working as an e-reader" complaints you will get, but I would like to suggest a concept for the import from Director function:

In options,

people can define a default import dirctory or select "ask each time"

People can select whether to delete the file after import


Then, on the main menu, have an "import new files" which would scan the directory and import all the files that were of a type that this user had selected locations for.

This would minimize the number of steps people would have to go through to import files.

Yes, we will do something like this. But again, be aware that we will import from a calibre directory, not a book. That directory must contain at least one format, the metadata.opf, and cover.jpg.

I might add a "listener" in CC that advertises it can handle .opf files. That would permit someone to select the opf in a file manager and hand it to CC. CC would then get the folder containing the opf and get the book + cover.

[kaufman]

My apologies, I thought we were talking about the option to just read a epub from the disk.

Just so we are clear, are you talking about a Dropbox directory on the device, or one in the cloud?

Sorry for the confusion. I have been on 17 hours of flights in the last 36 hours and am a little groggy.

[chaley]

Originally Posted by kaufman

My apologies, I thought we were talking about the option to just read a epub from the disk.

Just so we are clear, are you talking about a Dropbox directory on the device, or one in the cloud?

We are talking about two things:

1) Give CC the ability to import a book from a calibre folder on the device, where "calibre folder" means a folder containing metadata.opf, cover.jpg, and at least one format file. The choice of format will be made by applying the order specified in CC's format/folder settings. We don't care how that folder got onto the device. One option is certainly downloading such folders from Dropbox or any other cloud storage offering.

2) A separate app that will fetch "calibre folders" from the cloud. The "cloud" will include Dropbox and perhaps others like Amazon whatever. This app will be able to ask CC to do the import then delete the folder. It is not clear when we would release this app.

We have discussed importing bare epubs. That option makes me very nervous for reasons that have been given in other posts.

Sorry for the confusion. I have been on 17 hours of flights in the last 36 hours and am a little groggy.

Been there, done that. I am not fun to be around after that sort of travel.

[kaufman]

Ok, I am back on track again.

Two requests if you do 1.

1) An option to grab all of the subfolders in a folder in order to do multiple imports at once.
2) An option to have CC delete the subfolders after it does the import.

The more you do from within CC the more steps you can save for your user.


As far as the separate app goes, if you do it, I will buy it to support you, but I think its a waste of your time. Unless the app is going to recreate a lot of the searching/sorting functionality of calibre and/or CC, it doesn't seem like its going to be a lot better than just using a file manager that connects to dropbox (ie Solid Explorer).

Also, if you are going to create that extra layer, it seems like it might be better to do it as a paid add-on to CC. Its not like anyone is going to buy it unless they already have CC anyhow.

[chaley]

Originally Posted by kaufman

Ok, I am back on track again.

Two requests if you do 1.

1) An option to grab all of the subfolders in a folder in order to do multiple imports at once.
2) An option to have CC delete the subfolders after it does the import.

The more you do from within CC the more steps you can save for your user.

Both of these are reasonable.

As far as the separate app goes, if you do it, I will buy it to support you, but I think its a waste of your time. Unless the app is going to recreate a lot of the searching/sorting functionality of calibre and/or CC, it doesn't seem like its going to be a lot better than just using a file manager that connects to dropbox (ie Solid Explorer).

It would recreate some functionality offered by calibre by allowing some level of searching via data in metadata.db, such as by series or by tag. It is easy to find a book using the file manager if one knows exactly what to look for, but it is hard if you do not. An easy example: multi-author books where I don't remember who is the first author. I don't see a strong need to sort, or even to show covers.

Also, if you are going to create that extra layer, it seems like it might be better to do it as a paid add-on to CC. Its not like anyone is going to buy it unless they already have CC anyhow.

The problem here is "how?" Amazon and Google use different in-app purchasing, requiring two different apps or a licensing app. A lot of people will not buy apps that contain IAP. Licensing apps are trivially pirated. It would be harder to offer some sort of trial/demo version. It would slave the releases together. Offering it as a separate app removes these constraints, but adds the complexity of two aps. You are also right that doing so would create the possibility that someone would by the adjunct app not knowing that it requires CC. We would fight that with some text as the first line of the description. If that is ignored -- oh well.

[kaufman]

Originally Posted by chaley

The problem here is "how?" Amazon and Google use different in-app purchasing, requiring two different apps or a licensing app. A lot of people will not buy apps that contain IAP. Licensing apps are trivially pirated. It would be harder to offer some sort of trial/demo version. It would slave the releases together. Offering it as a separate app removes these constraints, but adds the complexity of two aps.

This is somewhat more work for you, but perhaps you can do it as a separate app that completely is controlled via IPC from CC. If a person tried to run it standalone it would just popup a message saying that it can't be run that way.

Another advantage of this is that you could leverage some of the searching/sorting capabilities of CC to add more functionality at little cost. So, for example, if you are grouping by a specific author in CC you could have a menu item to show all the other books by that author that are stored in the cloud (or even better all the books by that author that are not already part of the CC database). Executing that function would send a message (or however android handles IPC) to the the new app to actually get the list of matching books, which could then be shown in CC. And so on....