GIMP

You deskew it in GIMP? Sure, you want automation, but…

GIMP is for image processing not document manipulation. When loading a 4-page PDF, GIMP tries a hack of importing each page as a separate layer to a single image. To change pages, in principle that should be a matter of changing layers (page-up and page-down according to the menu, but it had no effect). GIMP automatically presented page-4, which is a bit odd. And I could not change layers.

Luckily page 4 happened to be the page I needed to deskew. But the rotate tool has no mechanism to fix a line that underlines a line of text followed by a mechanism that rotates as needed to make the line horizontal. We must resort to grabbing and dragging until it looks right. Eyeballing that is much more prone to human error than drawing a straight line under a line of text on the page.

When exporting the result as PDF, we can specify that each layer in the image become a page in the PDF output. And it seems to work though I did not get the impression that the page I deskewed remained in the A4 geometry that it started with because there was some kind of extra gap between the edges and the frame. Since GIMP is not designed for documents, it would not likely have a concept of A4 paper which would then need to be preserved.

ImageMagick -- perhaps

The other tool I can think of would be ImageMagick. It’s also not meant for documents but it looks like there is potential for this to be a solution. I ran display $image_of_a_page, selected the rotate transformation, and was able to draw a straight line which then became the baseline for rotation. This may be the answer but I have to fiddle with it more. ImageMagick often fails the principle of least astonishment. E.g. if you feed it some 600dpi images and simply instruct it to montage them, the result is extremely blurry. Makes no sense. The fix is that users must determine the resolution of the input images, do the math, and then explicitly specify the expected output resolution in order to have non-lossy processing. It’s bizarre how ImageMagick arbitrarily picks a fixed default resolution regardless of the input unless a user micro-manages that. So I’m not sure what kind of side-effects ImageMagick might have in this case.

Unpaper

How is unpaper “The FOSS tool to deskew” when it “often” fails to do just that?

Because it’s the only FOSS tool that even attempts to offer a deskewing function for documents (AFAIK). It seems to be the one (and only) tool for this. Everything else is hacks intended for loose images -- with functionality called “rotate” because it has no expectation of textual content. (Note that PDF is not an image format, it’s actually just a container of images and other things which can be quite dicey when working with tools not made for PDF manipulation).

The various --deskew-* options seem well explained to me.

Glad to hear you understand it! So, here are my questions:

 -dn { left \| top \| right \| bottom },...; --deskew-scan-direction { left \| top \| right \| bottom },...
         Edges from which to scan for rotation. Each edge of a mask can be used to detect the mask's rotation. If multiple edges are specified,  the
         average value will be used, unless the statistical deviation exceeds --deskew-scan-deviation. Use left for scanning from the left edge, top
         for scanning from the top edge, right for scanning from the right edge, bottom for scanning from the bottom.  Multiple  directions  can  be
         separated by commas. (default: left,right)

The opening sentence is ambiguous to readers (who don’t know how the algorithm works). What is the meaningful difference between scanning from the left and scanning from the right? Why would the scan direction make a difference? What is a mask in this context? A mask is generally used when you want to limit a function to operation within (or outside of) a specified area. But that would not seem to make sense here. Why would the default be left,right and not left,down,right,top, for example? Is top and bottom meant for landscape documents that are not rotated upright?

 -ds pixels; --deskew-scan-size pixels
         Size of virtual line for rotation detection. (default: 1500)

What is the “virtual line”? I might guess that it is the width of a line of text, but I can only guess because it is not explained. What are the algorithmic consquences of this value being too long, or too short? When deskew fails to rotate a page, how would I determine what the virtual line size should be?

  -dd ratio; --deskew-scan-depth ratio
         Amount of dark pixels to accumulate until scanning is stopped, relative to scan-bar size. (default: 0.5)

An accumulation is typically measured as a total quantity not a ratio. We can expect input docs to have an all-white left margin which has nearly zero dark pixels. So this dd ratio seems to be incomplete. We might expect there to be a certain concentration of dark pixels before we start counting the vector length. And likely a certain concentration to continue scanning, and a certain concentration to stop scanning, which I suppose would be the same as that of starting the scan. Without knowing what the scan bar size is, how can we determine what this value needs to be? And if we know the scan bar size, how is the ratio calculated? If it’s the qty of black pixels over the qty of whitespace starting from the left edge of the paper, I don’t suppose a ratio of 50% (the default) would generally be reached.

 -dv deviation; --deskew-scan-deviation deviation
         Maximum statistical deviation allowed among the results from detected edges. No rotation if exceeded. (default: 1.0)

What is a “detected edge”? Is that a line of text or an edge of the paper? I can’t tell from the man page how many edges there might be as we don’t know how the algorithm works. If It scans with a --deskew-scan-range of 5° and a --deskew-scan-step of 0.1, this seems to imply that it scans 100 times to find a single edge. Or is it finding multiple candidate edges from the same starting point and counting each of those as an edge that would then be compared for this deviation? From there, it seems unclear how many starting points it would attempt. I see no parameter for the distance between each starting point. If each starting point yields an edge, are all the edges for the whole page being considered for this deviation value? Or is it a deviation of candidate edges for just one line of text? If it’s the latter, how many lines of text are being processed?

apt search deskew returns ocrmypdf and pagetools, no idea if that all goes back to unpaper or not.

No results on oldstable but maybe Trixie has more. I often use ocrmypdf after deskewing because I thought it could not deskew. But your finding prompted me to look at the man page for ocrmypdf which reveals a --deskew option. So I will have to experiment with that.

You deskew it in GIMP? Sure, you want automation, but...

How is unpaper "The FOSS tool to deskew" when it "often" fails to do just that?

The unpaper options are not well explained.

There's https://manpages.org/unpaper and https://github.com/unpaper/unpaper/tree/main/doc

The various --deskew-* options seem well explained to me.

apt search deskew returns ocrmypdf and pagetools, no idea if that all goes back to unpaper or not.

Your logic is off-target, as this is caused by “management”, not the individual.

It is management that I was referring to. That should be obvious. The incompetence belongs to whoever makes the incompetent decision, which in this case would be high in upper management.

It’s fair to assume in this case USPS is not that incompetently wasteful.

No. It isn’t.

The USPS is being intentionally mismanaged as a step towards dismantling the pillars of US government.

A safe assumption need not be an accurate assumption. It’s about consequences. Incompetence has consequences -- and rightfully so. IOW, when the assumption is wrong, it does not obviate the purpose of my action. Therefore the assumption is safe.

Anyone wasting money like that is not competent for their job.

Your logic is off-target, as this is caused by "management", not the individual.

It’s fair to assume in this case USPS is not that incompetently wasteful.

No. It isn't.

The USPS is being intentionally mismanaged as a step towards dismantling the pillars of US government.

That would not make economic sense. Why would they hand-enter grandma’s chicken scratch hand-written return address when it is not needed for outbound routing? Anyone wasting money like that is not competent for their job.

Just finding the return address takes time in itself. It could be on the top left, or it could be a one-liner just above the destination address, or it could be on the backside, or not even supplied. They should only be looking for it when it is needed.

It’s fair to assume in this case USPS is not that incompetently wasteful. But if they are, then that incompetence is the problem (not how we choose to address our envelopes).

I would assume it gets entered into the system when they process it initially not when they need to return it.

For return service, indeed. In that very rare event, they would have to hand enter it just as they do for hand-written addresses.

Your sympathy is backwards though. Postal workers’s job security is under threat currently as people move away from postal service. Denmark eliminates postal service for the whole country this year.

(update) Also, other countries are downgrading the postal service and cutting staff in the drive toward digital transformation.

You’d probably just be making some poor postal worker have to type the information into their system manually when it fails to scan

This to me falls under reasonable loss of privacy,

A good standard for what’s “reasonable” is set out by data minimisation principles, like what you have in Europe’s GDPR Article 5.

Storing the routing data on the envelope itself is a form of data minimisation, as opposed to collecting it into a DB.

without reading (or “scanning”) the adresses, how could the postal system work?

I believe USPS has been in service for over 100 years, and demonstrated the capability of mail delivery as far back as horseback delivery (before Alan Turing was born).

It was only in the past ~15 years or so that USPS began offering a notification service that recipients can subscribe to. You get an email showing raster scans of envelopes that are out for delivery or have been delivered, for those who fancy that. But this new service does not just scan images for subscribers. Either your region offers this extra feature, or not. And if they do, then they scan /everyones/ envelopes whether they subscribe or not. Perhaps all regions offer it now.. I’m not up to date on the progress of this rollout.

pre-15 years ago (or so), the scanning was not building a database of images of envelopes (AFAIK). It was merely printing a barcode of the destination address for routing purposes. The barcode is not a reference to a DB record -- it’s an actual address encodified. So this routing info is stored on the envelope itself, not in a DB.

As for codes that becomes invisible after scanning, what would the point be?

What do you mean “codes”?

They use the fluorescent ink for barcodes. I describe the idea of using a like-colored ink for textual addresses, so the human postal worker can still read it and use it for routing, but it would be useless for mass surveillance. And to be clear, the return address is only needed for return routing. Sure, if they read the return address for routing purposes they are also likely storing it in the DB, but it’s a fair trade-off at that point because it’s a rare circumstance that return service is triggered.

You mention that there may be a privacy issue that the postal service scans recipients and senders of letters.

This to me falls under reasonable loss of privacy, without reading (or "scanning") the adresses, how could the postal system work?

As for codes that becomes invisible after scanning, what would the point be?

A code like that has very little actual value, and it honestly sounds like a classic ludicrous conspiracy theory based on a random chance.

Yeah, I meant Xorg X11. I have both, Wayland and X installed on my primary PC for years. I still use X as my daily driver for the reasons I mentioned earlier, but like exploring Wayland, too.

Btw, I like to use LyX as a WYSIWYG editor for LaTeX like documents. It has it's own format, can insert images quite well and exports into various formats, inluding PDF and LaTeX. It probably won't help much with image rendering tho.

I have no idea about X11, which I assume is what you meant by X. GIMP crashes Wayland. Or Wayland crashes GIMP and takes itself down with it. GIMP can launch on wayland and carry out /some/ tasks, but some functions and windows in GIMP are sure to trigger a crash. Usually GIMP alone goes down, but sometimes the whole wayland session goes down with it.

(edit) I should also mention that GIMP probably has no Wayland support. It’s most likely using Xwayland in my case. Xwayland is an attempt to give Wayland the x11 hooks that older software rely on (as x11 is a dying technology IIRC). The Libreoffice suite of tools have the same problem and crash on Wayland, but there is an obscure hack that fixes Libreoffice.

Gimp doesn't crash X. I know, because I still need X for a few games, java apps and some GUI automation and I recently ran Gimp in it to make neat 4k wallpapers from those newer high res square images of the sun that probe took.