k2pdfopt: optimizes PDFs for viewing on e-readers

[jonnyl]

What is the best way to run k2pdfopt on an Android phone? I'm assuming via Linux? Has anyone successfully been able to get it to work and would have step-by-step instructions?

[techytita]

Good question above. Hoping to get some help here.

[willus]

I don't have a test platform for compiling or running on an Android phone. I might be able to cross-compile, but I'd still need to test it. Is there a virtual Android phone simulator that will run on either a PC or a Mac Mini?

[Jamshid]

Originally Posted by willus

I don't have a test platform for compiling or running on an Android phone. I might be able to cross-compile, but I'd still need to test it. Is there a virtual Android phone simulator that will run on either a PC or a Mac Mini?

There's already a k2pdfopt port for android and last time I tested it, was based on version 2.54.
You should install Termux and install packages k2pdfopt and ghostscript.
Termux may not be available on google play and should be downloaded and installed manually.

[Quoth]

Originally Posted by willus

I don't have a test platform for compiling or running on an Android phone. I might be able to cross-compile, but I'd still need to test it. Is there a virtual Android phone simulator that will run on either a PC or a Mac Mini?

Years ago when I last wrote an Android app the development system ran on Windows or Linux and included a emulated/simulated phone plus a faster than MTP method (ADB?) to load app to also test on an actual Android phone.
All a free download from Google.
It was very like doing a Java app, except Google is a pain and keeps changing how you do I/O to SD card or storage. Really Windows for Workgroups 3.11 was better (apart from lack of Explorer desktop, as long as Win32s, 32 bit TCP/IP and 32 bit Disk etc all added), stupid changes since Android 3 and poor UI. Material Design is idiotic. But then I think Win10/Win11/Chrome/Mozilla UIs are all rubbish compared to mac OS9, Win9x, NT4.0/2K/XP UIs.

[Used1]

Is this where I can ask for help with getting the formatting down? A lot of this is way over my head. And can I post screenshots here to show what I mean?

[j.p.s]

Originally Posted by Used1

Is this where I can ask for help with getting the formatting down? A lot of this is way over my head. And can I post screenshots here to show what I mean?

Yes, but please be patient because I don't think willus visits every day. (Of course, someone else might know answers to your questions.)

You can post screenshots here, but if they are larger than 600x600 please attach them rather than embed them. (Use the "Manage Attachments" tool.) Also, please don't post deep links to images because that frequently causes problems.

[schezuk]

I wrote myself a dirty piece of code to rotate every CJK character by 90 degrees, which is designed to ignore Latin script if a word is wide enough. Not an effective implementation but with many magic numbers, though.

#! /usr/bin/env python3
# -*- coding: utf-8 -*-
import sys
import cv2
import numpy

def rotate_image(image, angle):
	(h, w) = image.shape[: 2]
	center = (w // 2, h // 2)
	M = cv2.getRotationMatrix2D(center, angle, 1.0)
	corrected = cv2.warpAffine(image, M, (w, h), flags = cv2.INTER_CUBIC, borderMode = cv2.BORDER_REPLICATE)
	return corrected

def determine_score(arr):
	histogram = numpy.sum(arr, axis = 2, dtype = float)
	score = numpy.sum((histogram[..., 1 :] - histogram[..., : -1]) ** 2, axis = 1, dtype = float)
	return score

def correct_skew(monoImg, delta = 0.1, limit = 5):
	angles = numpy.arange(-limit, limit + delta, delta)
	img_stack = numpy.stack([rotate_image(monoImg, angle) for angle in angles], axis = 0)
	scores = determine_score(img_stack)
	best_angle = angles[numpy.argmax(scores)]
	return best_angle


orgImage = cv2.imread(sys.argv[1])
myImage = numpy.ascontiguousarray(numpy.rot90(orgImage))
(my_h, my_w, _) = myImage.shape

grayImg = cv2.cvtColor(myImage, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(grayImg, (0,0), sigmaX=33, sigmaY=33)
divide = cv2.divide(grayImg, blur, scale=255)
_, myImage = cv2.threshold(divide,127,255,cv2.THRESH_BINARY)
myImage = divide


at_bs = 5
monoImg = cv2.ximgproc.niBlackThreshold(grayImg, 255, cv2.THRESH_BINARY, at_bs, -0.3, binarizationMethod=cv2.ximgproc.BINARIZATION_NICK)
monoImg = cv2.adaptiveThreshold(monoImg, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
monoImg = cv2.bitwise_not(monoImg)

angle = correct_skew(monoImg)
print(angle)
if not -0.1 < angle < 0.1:
	monoImg = rotate_image(monoImg, angle)
	myImage = rotate_image(myImage, angle)

erosion_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2, 1))
erosion = cv2.erode(monoImg, erosion_kernel, iterations = 1)
dilation_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (18, 5))
dilation = cv2.dilate(erosion, dilation_kernel, iterations = 1)
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

dilation_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 1))
dilation = cv2.dilate(monoImg, dilation_kernel, iterations = 1)


newImage = numpy.full(myImage.shape, 255, dtype=numpy.uint8)
for cnt in contours:
	x, y, w, h = cv2.boundingRect(cnt)
	
	ROI = dilation[y:y+h, x:x+w]
	# 1 1 0 1 1   # original 1 right pad
	# 1 1 1 0 1   # original 2 right pad
	ROI1 = numpy.c_[ROI, numpy.zeros(len(ROI),dtype=numpy.uint8)]
	#   1 1 0 1 1 # right shift 1
	#   1 1 1 0 1 # right shift 2
	ROI2 = numpy.c_[numpy.zeros(len(ROI),dtype=numpy.uint8), ROI]
	# 0 1 0 0 1 0 # AND 1
	# 0 1 1 0 0 0 # AND 2
	gap = cv2.bitwise_and(ROI1, ROI2) # gap between pixel
	#0 1 1 0 1 0  # reduced
	reducedArray = cv2.reduce(gap, 0, cv2.REDUCE_MAX, dtype=cv2.CV_8U)
	rcnt, length, i, left, started = [], len(reducedArray[0]), 0, 0, False
	while i < length:
		if reducedArray[0][i] >0:
			if not started:
				left = i-1
				started = True
		else:
			if started:
				rcnt.append((x+left, y, i-left, h)) # absolute coordination
				started = False
		i=i+1
	for rc in rcnt:
		_x, _y, _w, _h = rc
		if _h//_w>=2 or _w//_h>=3:
			continue
		xhigh = _x + _w//2 + _h//2
		yhigh = _y + _h//2 + _w//2
		xlow = xhigh - _h
		ylow = yhigh - _w
		if xlow <0:
			xlow = 0
			xhigh = _h
		if xhigh > my_w:
			xhigh = my_w
			xlow = xhigh - _h
		if ylow <0:
			ylow = 0
			yhigh = _w
		if yhigh > my_h:
			yhigh = my_h
			ylow = yhigh - _w
		if xlow >=0 and xhigh<=my_w and ylow >=0 and yhigh<=my_h:
			glyph = numpy.ascontiguousarray(numpy.rot90(myImage[_y:_y+_h, _x:_x+_w], -1))
			cv2.rectangle(myImage, (_x, _y), (_x+_w, _y+_h), (255, 255, 255), -1)
			newImage[ylow:yhigh, xlow:xhigh] = cv2.min(newImage[ylow:yhigh, xlow:xhigh], glyph)
cv2.imwrite(sys.argv[1]+".rotate.png", cv2.min(newImage,myImage))

[willus]

Originally Posted by schezuk

I wrote myself a dirty piece of code to rotate every CJK character by 90 degrees, which is designed to ignore Latin script if a word is wide enough. Not an effective implementation but with many magic numbers, though...

A little context here would be useful. What is your script intended to be used on? A PDF file? Does it work on any PDF?

[willus]

Originally Posted by Used1

Is this where I can ask for help with getting the formatting down? A lot of this is way over my head. And can I post screenshots here to show what I mean?

I can try to help. Have you looked at my help page or watched the "Getting started" videos (which are, admittedly, a bit dated)?

[schezuk]

Originally Posted by schezuk

As some people may not be accustomed to TTB-RTL text in mainland China, or LTR-TTB text in other regions, it may be convient to Text Reflow from one way to another. E.g.



By text reflowing it does not only mean to arrange text elements, which may be easily done by many other softwares, but to re-arrange text in scanned images.

It may be solved like this. Just rotate every square-shaped character detected 90 degree clockwise/counter-clockwise, and leave other characters be.

A workaround in similar principle for text element has been figured out by using a dedicated font with Chinese characters rotated. See LTR to TBRTL.

Originally Posted by willus

I'll put this on my feature request list, but it will be low priority because (1) it's not a common request, and (2) the implementation, even if I didn't have to try and tell western alphabets from asian characters, would be fraught with error. Determining exactly where the gaps are in the characters and which to rotate and which not to rotate would be mistake prone and would no doubt frustrate the end user. Sorry.

The code above works on extracted images rather than pdf files.
It binarizes the image, rotate it by 90 degrees, align it horizontally, and filter out noise pixels.
Reverse the color. Every white pixel will color adjacent pixels to make sure every radical of a single, square character is connected to one another. Chop the image to pieces, rotate back 90 degrees every rectangular piece (in the original image) that is square enough.