[GUI Plugin] English Noun Frequency

[Rellwood]

Is there a reason why the list of "bad words" which has been validated is being ignored? Also, it seems that the list of words is listed Alphabetically instead of frequency (as selected)

I am tired of seeing "arm" "chapter", "hair", "leg", etc. when I have them listed as bad words.

Am I doing something wrong?

[DaltonST]

Pictures of every Tab and one example job log?

[Rellwood]

Originally Posted by DaltonST

Pictures of every Tab and one example job log?

Sorry about that, I had to do a restart to get it to comply.

Also, is there a way to remove the obscenities and word pairs you have already made?

I don't know if this plugin was intended to be used as a kind of "peek" into what the book is about, but I am using it that way. My concern is the inability to leave in the restricted obscenities. Changing anatomy from slang to clinical or removing all together words isn't what I would like to do.

I have tried to add them to the list of "good words" but they still get overridden. Useful to separate truly "bad words" from "I can live with it".

I want to know what "bad words" are in a book without them being screened out or changed. For reasons of loaning the book or simply to know how "bad" it is before opening it. A user compiled list of words to check against would be helpful.

Heck, even a list of the words and the number of times they are used can be an indicator of the "rating level" of a book.

[DaltonST]

The purpose of ENF is explained at the beginning of the Initial Post.

You did not attach any images or a job log, so I assume that is no longer a question. Instead, you have a new question.


If you want to delete the obscenities it changes to '****', you can manually edit the .zip file.

"C:\Users\rellwood\AppData\Roaming\calibre\plugins \English Noun Frequency.zip"

Inside it is file: english_obscenities.py

Inside that file is a function: def return_english_obscenities_set():

Inside that function is the set of obscenities. Empty out the words in that list so it looks like this:

english_obscenities_set = set(list([]))

instead of english_obscenities_set = set(list(['xxx','yyy',.........]))

Do not change indentation, etc.

Back-up the original .zip, and then the revised .zip. Future versions will overwrite your personal version.

Copy your revised version into "C:\Users\rellwood\AppData\Roaming\calibre\plugins \" so it replaces the stock version.


I will put on my to-do list to add a checkbox on the first Tab to allow you to decide whether to use the words and phrases in english_obscenities_set or not. In the meantime, do the above for now.


DaltonST

[Rellwood]

@DaltonST

I believe that I am using the plugin for it's designed purpose (to get a look at what the book might be about). Although I am not needing it to translate. However, I have found the plugin to be quite useful. I am still tweaking it. I still am navigating it.

I do want to know if there is a way of adding a feature that allows specific words to always be replaced with another word? I keep laughing when I see "buttock".

Finally, if it is possible can you allow the inserted to the details page to be both html free and maybe be able to have the list be in columns instead of a long row.

Also, how can I get that view where the frequency of each word is shown? I get the list, and Job Spy shows the word followed by the arrow and the frequency number, but I don't know how that can be added. It would be nice if each output can be created into a csv or some kind of format that can be added to the book in the edit feature. Of course there is the whole editor reports.

[DaltonST]

It would be nice if each output can be created into a csv

Have you read all of the ToolTips on the first screen? It already has the above functionality across your entire Calibre ecosystem if you want to use it that way. It is in its own group-box called "Continuous Accumulation of Most Frequent Word Frequencies". You select your personal .csv file. Hover your mouse over each widget and area in the first Tab.

If you want each and every book's output to have its own .csv file, or each job's output to have its own .csv file...not interested. See the above.



DaltonST

[jefgoi]

@DaltonST

Thanks for plugin!
Is it possible to use (adopt?) it for books in Russian?

[DaltonST]

ENF can translate the top 100 nouns in a book from English to Russian if you provide it the custom dictionary for Russian. See the Original Post for a template to use.

However, "Russian Noun Frequency" would require an entirely different plug-in written by a native speaker of Russian, along with a lot of research and manual effort to carefully itemize the grammatical parts of speech for most words in Russian. Different dialects of Russian must be included (English spelling in the British Isles is different than Canada than America than Australia/New Zealand than India). Example: Indian English spells the contraction for "do not" as "dont". The rest spell it "don't". To me, "dont" means "tooth" (from the Greek), because I speak American English.

Not trivial.

If you unpack the EnglishNounFrequency.zip file, you will find many files that I had to create via lots of research to make ENF work properly and efficiently.

Spare time? Google "English Nouns", "English Verbs", "English Word Frequency", "English Word Length", "English Parts of Speech", etc.


DaltonST

[DaltonST]

Version 1.0.9 - 2018-10-11 Obfuscation of obscenities now optional.


DaltonST

[DaltonST]

Version 1.0.10- 2019-04-30 Python 3 and Calibre 3.41.3+ pdf-to-html compatibility. Minimum Calibre version now 3.41.3. Upgrade Calibre first.


DaltonST

[DaltonST]

Version 1.0.11- 2019-12-28 Technical changes after Python 3.8 testing with Calibre 4.99.2; job execution speed greatly improved.




DaltonST

[DaltonST]

Version 1.0.12- 2020-07-21 Qt tweaks.


DaltonST

[Rellwood]

Quick question,

Is there a way of having the tags populate the comments page as if they were being put in a column instead of populating them in a list?

Example:

Currently:: <center><strong> English Noun Frequency<strong><br>________________________</br> <br>word 1</br><br>Word 2 </br><br>Word 3</br>____________________________<center>


Change to: <strong> English Noun Frequency: <strong> Word 1;Word 2;Word 3;Word 4;

Basically copy over what would be in the column if a column was chosen instead of comments.

I currently am doing that manually, and it would be nice if it could update BOTH the comments and the column in that format.

I understand that it is generally for books without comments, but if there are comments then being able to have the list, but not as a long drop-down (especially if you have a lot of words) would make it much nicer.

Also, I noticed that sometimes the words come out with a "ttt" attached, it seems to happen with words that are genres, or book elements ie. "Authorttt" or "Fantasyttt"

[kcar181]

Hi Dalton curious about something. I have quite a few books that either won't return any word results at all or simply does something generic like

"Chapter, Cover, Padding, Center, Link, Navigation, Text, Title"

Is this due to the epub structure? Something I could change or convert?


Example Log:
https://pastebin.com/w5ATDxR6

[DaltonST]

Malformed content. Convert one of them to epub *again*, then Polish it. Try ENF with that copy. Also, ensure it is not an epub with scanned text embedded. That will not work.