5
pdfgrep: undocumented regex redefinition of “^” and “$”. Ooof! Reported 9 yrs ago and ppl still getting stung. (gitlab.com)
submitted 2 months ago by to c/bugs@sopuli.xyz
5 comments fedilink hide all child comments
 

(Apologies for publicly posting a link into Cloudflare’s walled-garden, but archive.org comes up empty.)

The linked bug report shows that astonishing but deliberate regular expression treatment was reported 9 years ago. Still today I spent time trying to work out why my regular expression was not matching only to find that the dev hi-jacked the meaning of ^ and $ for another purpose. Man page still neglects to warn users.

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 months ago (4 children)

It makes sense that pdfs don't really have 'lines' in the same way.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] [S] 2 points 2 months ago* (last edited 2 months ago) (3 children)

    If I say match a b, without the concept of a line you would be unable to even work out whether a comes immediately before a space which is immediately before b with nothing else in between. Lines are indispensible for determining sequence. Whether a pattern is a b or ^b, the tool necessarily must determine what, if anything, is to the left of b. The dev took a shortcut and skipped the effort of analysing vertical significance between tokens. The pdftotext tool proves that text lines can be determined from a searchable PDF.

    In any case, docs that lie or omit something important are always a bug. Man page says: “pdfgrep tries to be mostly compatible with GNU grep with some PDF-specific distinctions and additional options. Most notably, -n prints page instead of line numbers.” No mention of changing the meaning of ^ and $.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 2 months ago (1 child)

    You must just not have encountered a PDF that renders the page in a very strange way. The exact situation mentioned by the developer as an example in that report of a two-column PDF merging both columns into one line is very common. I don't have such a PDF at hand but in my experience this is particularly common for scanned documents that have OCR'd text content. I'm positive that pdftotext would trip up on those too, because what else could it do than show the text as it is logically laid out in the file?

    If the functionality was different, people would just be complaining about a different incongruency.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] [S] 1 point 2 months ago* (last edited 2 months ago)

    You must just not have encountered a PDF that renders the page in a very strange way.

    A scanned document is trivially rendered. So you must be referring to pure vector PDFs. With pure vector PDFs, the PDF generator is responsible for the contents. Indeed the PDF standard is about as shitty as the HTML standard and the quality of results are a product of the quality of the input. But in the case at hand, we are not really talking about managing poor quality input.

    how pdftotext handles multi-column docs

    The exact situation mentioned by the developer as an example in that report of a two-column PDF merging both columns into one line is very common. I don’t have such a PDF at hand but in my experience this is particularly common for scanned documents that have OCR’d text content. I’m positive that pdftotext would trip up on those too, because what else could it do than show the text as it is logically laid out in the file?

    The pdftotext tool has a feature to handle 2 column docs. I often have docs with the left column in one language and the right column in another. Using pdftotext, we can specify a crop box that only extracts text landing in the left or right half of the page. In the absence of that feature, the left half of a line would be language 1 and the right half would be language 2 (merged). I don’t imagine that any user would expect ^ to match the beginning of the right-hand column. But given the docs as they are, users would expect the ^ to match the beginning of the line of the left-hand column.

    the rule of least astonishment

    If the functionality was different, people would just be complaining about a different incongruency.

    Maybe but there is a principle to manage this: “the rule of least astonishment”. That is, software should be implemented to minimise astonishment. There could still be complaints if a problem is complex and difficult, but when the rule of least astonishment is followed it yields the least amount of complaints, which is not necessarily zero complaints. The case at hand seems to neglect the rule of least astonishment.

  • permalink
  • fedilink
  • source
  • parent