![]() |
#1 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
Auto loading meta data from Amazon
Hey guys, if you've been following the Sideloading and Whispersync thread, you've seen that there have been some cool new developments about getting your books (not purchased from Amazon) to sync across different devices.
It turns out that Amazon will sync your books across multiple devices if they appear to have a legit ASIN number. But if you have an Android or iOS device, it would sometimes not download the correct book cover because it couldn't associate that ASIN number with a Kindle book. So to solve that problem, I wrote a couple of scripts (one for Windows and one for *nix). They will automatically download the correct ASIN number from Amazon. If it cannot find a match, the script will just give you a random ASIN number so it will still sync between your devices. Let me walk you through doing this: -Backup all your .mobi files before trying this! -Get an Amazon Web Services account. It takes about all of 2 seconds. You'll need the API and Secret keys Windows: -Download and install Python -Download and install SetupTools -Install python-amazon-product-api -Make sure C:\python27 and c:\python27\scripts are in your PATH Environment -If you can't do the "easy_install" download and install this -Put all your .mobi files in a single folder -Download MobiPerl and extract it to the same folder your .mobi files are in -Save this file as a .bat file in that same folder: Code:
@echo off & setLocal EnableDELAYedExpansion del temp*.txt>null 2>&1 set "asin1=B00" set /a count=0 for /f %%a IN ('dir /b *.mobi') do (set /a count=count+1) echo Number of files: %count% set /a diff=((9999-%count%) - 1000) + 1 set /a divisor = 32767 / %diff% set /a diff2=(999 - 100) + 1 set /a divisor2 = 32767 / %diff2% :Random1 set /a asin2=%random% / %divisor% if /i %asin2% GTR 9999 goto Random1 if /i %asin2% LSS 1000 goto Random1 :Random2 set /a asin3=%random% / %divisor2% if /i %asin3% GTR 999 goto Random2 if /i %asin3% LSS 100 goto Random2 set /a temp1=!asin2! for /f "delims= " %%b IN ('dir /b *.mobi') do ( C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "%%b" --delexthtype 113 "%%b">null 2>&1 C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "%%b" --delexthtype 501 "%%b">null 2>&1 C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe "%%b" > temp1.txt findstr /i "longtitle" temp1.txt > temp2.txt set /p title1=<temp2.txt set title1=!title1:~11! find /i "item: 100" temp1.txt > temp3.txt for /f "tokens=* delims= " %%a in (temp3.txt) do ( set var=%%a ) set author1=!var! set author1=!author1:~33! python asin_search.py "!title1!" "!author1!" >temp4.txt set /p temp2=<temp4.txt set temp2=!temp2:~0! set filename=%%b call :movement %%b ) :movement if "!tempfile!"=="!filename!" goto :end find /c /i "b0" "temp4.txt">null 2>&1 if !ERRORLEVEL! EQU 0 goto :real :fake C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "!filename!" --exthtype 113 --exthdata "!asin1!!asin3!!temp1!" "!filename!">null 2>&1 echo Gave a fake ASIN number to: !title1!. It's new number is !asin1!!asin3!!temp1! set /a temp1=!temp1!+1 goto final :real C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "!filename!" --exthtype 113 --exthdata "!temp2!" "!filename!">null 2>&1 echo Gave a real ASIN number to: !title1!. It's new number is !temp2! :final C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "!filename!" --addexthtype 501 --exthdata "EBOK" "!filename!">null 2>&1 set tempfile=!filename! :end del temp*.txt>null 2>&1 del null* -Download this python script and save it as asin_search.py in your working directory Code:
import sys from amazonproduct import API AWS_KEY = 'A******************A' SECRET_KEY = 'h*************************************7' api = API(AWS_KEY, SECRET_KEY, "us") node = api.item_search('KindleStore', Title=sys.argv[1], Author=sys.argv[2]) for book in node.Items.Item: print '%s' % (book.ASIN) -Run the .bat file and wait for it to finish. -Copy the updated files to your kindle, android, iOS, etc. -See if it worked! *nix: -Make sure mobiperl is installed correctly (MAC) (Linux) -Install python-amazon-product-api -Put all your .mobi files in a single folder -Save the above python script as asin_search.py in your working directory -Replace with your own API and Secret keys -Save this bash script as a .sh file in your working directory Code:
#!/bin/bash count1=0 asin1="B00" FILES="*.mobi" for g in $FILES do if [ "$g" ] then count1=$[$count1+1] continue fi done echo "There is $count1 files" rand1=$[1000-$count1] while [ $rand1 -lt 1000 ] do rand1=$[ (( $RANDOM % ( $[ 9999 - 1000 ] + 1 ) ) + 1000) - $count1 ] done rand2=$[ ( $RANDOM % ( $[ 999 - 100 ] + 1 ) ) + 100 ] for f in $FILES do if [ "$f" ] then mobi2mobi --outfile "$f" --delexthtype 113 "$f"> /dev/null 2>&1 mobi2mobi --outfile "$f" --delexthtype 501 "$f"> /dev/null 2>&1 author=$(mobi2mobi "$f" | grep Author | cut -d- -f4) title=$(mobi2mobi "$f" | grep LONGTITLE | cut -c11-100) python asin_search.py "$title" "$author">asin.txt search1=`head -n 1 asin.txt` if [ "$search1" ] then mobi2mobi --outfile "$f" --exthtype 113 --exthdata $search1 "$f"> /dev/null 2>&1 echo "Gave a real ASIN number to: $title. It's new number is $search1" else mobi2mobi --outfile "$f" --exthtype 113 --exthdata $asin1$rand2$rand1 "$f"> /dev/null 2>&1 echo "Gave a fake ASIN number to: $title. It's new number is $asin1$rand2$rand1" rand1=$[$rand1+1] fi mobi2mobi --outfile "$f" --addexthtype 501 --exthdata "EBOK" "$f"> /dev/null 2>&1 rm asin.txt continue fi done -Copy the updated files to your kindle, android, iOS, etc. -See if it worked! If anyone gets some use out of this please let me know! My next project will be to get the covert art of the random ASIN numbers to work correctly on android, iOS, ect. You guys are the best ![]() Last edited by m0ngr31; 01-26-2011 at 10:02 PM. |
![]() |
![]() |
![]() |
#3 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
I just hope someone can get some use out of it
![]() |
![]() |
![]() |
![]() |
#4 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
I've made an update to the script. This will make it so it saves the cover art of the book as cover_B00*******.png (like Kerenon suggested it needed to be for android) if it cannot find the correct ASIN from Amazon. This way you won't have to live with garbage cover art on your mobile devices.
Just put Code:
mobi2mobi --savecover "cover_$asin1$rand2$rand1.png" "$f"> /dev/null 2>&1 Code:
mobi2mobi --outfile "$f" --exthtype 113 --exthdata $asin1$rand2$rand1 "$f"> /dev/null 2>&1 echo "Gave a fake ASIN number to: $title. It's new number is $asin1$rand2$rand1" --EDIT-- For the .bat file: Put Code:
C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --savecover "cover_!asin1!!asin3!!temp1!.png" "!filename!">null 2>&1 Code:
C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --outfile "!filename!" --exthtype 113 --exthdata "!asin1!!asin3!!temp1!" "!filename!">null 2>&1 echo Gave a fake ASIN number to: !title1!. It's new number is !asin1!!asin3!!temp1! Last edited by m0ngr31; 01-28-2011 at 12:06 AM. Reason: Forgot about the Windows .bat file |
![]() |
![]() |
![]() |
#5 |
Zealot
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 132
Karma: 3248
Join Date: Jun 2007
Device: sony
|
Question for you. What does it search by? I just tried one of my mobi files and I found the book in the kindle store on the website but your batch file didn't find it and gave it a fake ASIN. I'm trying to figure out what search criteria it uses so i can fine tune it in the metadata.
Thanks for great work Alex |
![]() |
![]() |
Advert | |
|
![]() |
#6 |
Zealot
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 132
Karma: 3248
Join Date: Jun 2007
Device: sony
|
Hmm in addition to that it doesn't seem to modify the data inside the mobi file. I loaded the file back into mobi2mobigui and EXTH 113 and 501 are both not there. So I may need a bit of troubleshooting assistance as my python scripting skills suck
![]() |
![]() |
![]() |
![]() |
#7 | |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 27,799
Karma: 198099188
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
Quote:
@m0ngr31 Is it possible to expand the python api search to include non-kindle store content? It might have better luck finding a match from it's pbook database. |
|
![]() |
![]() |
![]() |
#8 |
Zealot
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 132
Karma: 3248
Join Date: Jun 2007
Device: sony
|
Book i was working on exists in Amazon kindle store and has same Long Title as the amazon's book. Weird
Also for some reason its running the fake title twice and assigns 2 different fake ASINs book in question is: Moscow Sting: A Novel and here is the output from the batch file: Code:
Number of files: 1 Traceback (most recent call last): File "asin_search.py", line 7, in <module> node = api.item_search('KindleStore', Title=sys.argv[1], Author=sys.argv[2]) File "build\bdist.win32\egg\amazonproduct.py", line 489, in item_search File "build\bdist.win32\egg\amazonproduct.py", line 405, in _parse amazonproduct.NoExactMatchesFound Gave a fake ASIN number to: Moscow Sting: A Novel. It's new number is B004202788 Gave a fake ASIN number to: Moscow Sting: A Novel. It's new number is B004202789 |
![]() |
![]() |
![]() |
#9 | |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
Quote:
Also, I'm not sure why it would give 2 fake ASIN numbers. Probably crappy scripting on my part ![]() |
|
![]() |
![]() |
![]() |
#10 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
|
![]() |
![]() |
![]() |
#11 |
Zealot
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 132
Karma: 3248
Join Date: Jun 2007
Device: sony
|
m0ngr31 sent you a PM. I'm in winblows
|
![]() |
![]() |
![]() |
#12 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
I fixed the windows script. There were some serious logic errors I didn't catch because I didn't test enough haha. Just use the script in the first post.
|
![]() |
![]() |
![]() |
#13 |
Zealot
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 132
Karma: 3248
Join Date: Jun 2007
Device: sony
|
Perfecto
![]() |
![]() |
![]() |
![]() |
#14 |
Member
![]() Posts: 10
Karma: 10
Join Date: Jan 2011
Device: Kindle 3
|
It's working great on Windows here, thank you so much, just one thing - I've added in the bit to create the cover_*.png, but it's not doing creating anything - any ideas why?
|
![]() |
![]() |
![]() |
#15 |
Connoisseur
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 66
Karma: 3056
Join Date: Dec 2010
Device: Kindle Paperwhite
|
If you put the save cover part in the proper place, it will only save a cover if it can't find the ASIN number. Is it for one that can be found. If it has a valid ASIN number, your Android/iOS device should download it automatically.
Edit: I just thought of this. I did change the script a little. The modified line that you should put in is this. Code:
C:\Users\m0ngr31\desktop\kindle\mobi2mobi.exe --savecover "cover_!asin1!!asin3!!temp1!.png" "!filename!">null 2>&1 ![]() Last edited by m0ngr31; 01-28-2011 at 12:07 AM. |
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
ePub meta data | brudigia | ePub | 4 | 07-26-2010 12:58 PM |
set meta data with ebook-meta and ebook-convert | krischik | Calibre | 6 | 01-19-2010 11:40 AM |
Setting meta data with ebook_meta. | krischik | Calibre | 6 | 01-15-2010 11:17 AM |
Meta data problems | melhall | Sony Reader | 1 | 03-31-2008 10:58 PM |
PRS-500 Meta Data in Vista not available | Gamgee72 | Sony Reader Dev Corner | 2 | 09-12-2007 05:12 PM |