PDF/A and archiving: what your PDF actually contains

A PDF is a container, and the extension tells you nothing about what is inside it. Two files that look identical on screen can differ completely: one holds real text you can search and select, the other holds a photograph of text and nothing else. That difference decides whether the document is findable today. A separate set of choices, about fonts, colour and external references, decides whether it still renders in ten years. The format standardises the envelope, and almost nothing about durability comes free with it.

A PDF is a page description, not a document format

ISO 32000-1:2008, published July 2008, was the first PDF specification issued by ISO, formalising Adobe's PDF 1.7. Its scope is a viewing format: it lets people "exchange and view electronic documents independent of the environment in which they were created or the environment in which they are viewed or printed". PDF 2.0 is ISO 32000-2:2020, second edition, December 2020.

The OCRmyPDF documentation states the consequence plainly: "PDFs are page description files that attempt to preserve a layout exactly." And a warning most people miss: "A PDF page may contain multiple images, even if it appears to have only one image." Scanner software routinely splits a page into a monochrome text layer and a colour background layer for compression.

Three commands from poppler-utils settle what you actually have. pdftotext "converts Portable Document Format (PDF) files to plain text"; empty output means no text layer exists. pdffonts "lists the fonts used in a Portable Document Format (PDF) file along with various information for each font"; a file with no font rows has no text. pdfimages -list will "list the images along with various information for each image"; one page-sized image per page is a scan.

What OCR adds, and how it fails

OCR does not convert a scan into a text document. It adds a second, invisible layer on top of the unchanged pixels, drawn in text rendering mode 3, which veraPDF's formalisation of the PDF/A rules describes as the mode where "glyphs are not stroked, filled or used as a clipping boundary". OCRmyPDF is honest about the mechanism: "It uses OCR to guess the text contained in images."

Input quality drives the guess. Tesseract's own guidance says it "works best on images which have a DPI of at least 300 dpi", that "the quality of Tesseract's line segmentation reduces significantly if a page is too skewed", and that some noise "cannot be removed by Tesseract in the binarisation step, which can cause accuracy rates to drop".

The realistic failure modes are documented, not hypothetical. OCRmyPDF lists them: "It is incapable of recognizing handwriting." "It may detect gibberish and report it as OCR output." "Tesseract may struggle to analyze the natural reading order of documents." And the structural limit that matters for archiving: "Tesseract does not divide text into paragraphs or headings. It only provides the text and its bounding box. As such, the generated PDF does not contain any information about the document's structure."

The US National Archives accepts imperfect OCR and rejects destructive OCR. It takes "PDF records with uncorrected OCR'd text", but "will not accept PDF records resulting from OCR processes that alter the visible content, degrade the quality of the original bit-mapped image, or replace the original bit-mapped image with OCR'd text."

The compression that rewrites your numbers

On 24 July 2013 David Kriesel found that Xerox WorkCentre devices "randomly alter written numbers in pages that are scanned". His description of the failure is the reason scanned PDFs deserve suspicion: "This is not an OCR problem", it is "patches of the pixel data are randomly replaced in a very subtle and dangerous way", and "The scanned images look correct at first glance, even though numbers may actually be incorrect."

The cause was JBIG2 symbol matching used in its lossy mode. The encoder groups visually similar glyph patches and stores one representative for the group, so a 6 that the matcher considers close enough to an 8 is redrawn as an 8. Kriesel puts the distinction in one line: "PM&S works lossy, SPM lossless."

Two archives turned that into policy. Germany's BSI Technical Guideline TR-03138 (RESISCAN), version 1.4.1 of 23 April 2020, rules that image compression methods using symbol coding "DÜRFEN NICHT eingesetzt werden", must not be used. NARA is equally direct: it "will not accept digitized records in PDF that have been saved with lossy compression to reduce file size (e.g., JPEG, JBIG2)", and it names the scanner presets that produce it, listing "Searchable Image - Compact" among prohibited outputs and "Searchable Image - Exact" among acceptable ones. Those two menu entries differ by one word and by whether your figures survive.

What PDF/A actually standardises

PDF/A is defined as a set of prohibitions, and veraPDF publishes each one as a formalised "shall" statement. Five carry the weight.

Fonts must travel with the file: "The font programs for all fonts used within a conforming file shall be embedded within that file, as defined in PDF Reference 5.8, except when the fonts are used exclusively with text rendering mode 3." Encryption is banned outright: "The keyword Encrypt shall not be used in the trailer dictionary." External content is banned: "A stream object dictionary shall not contain the F, FFilter, or FDecodeParms keys." Active content is banned, in PDF/A-1 by name: "The Launch, Sound, Movie, ResetForm, ImportData and JavaScript actions shall not be permitted." And colour must be unambiguous, because "all colours shall be specified in a device-independent manner, either directly by the use of a device-independent colour space, or indirectly by the use of an OutputIntent."

OCRmyPDF condenses the whole family into one rule: "All fonts and resources needed to interpret the PDF must be contained within it."

Conformance levels are not quality grades

Level b promises only that the page renders the same way later. Level u adds machine-readable text: rule 6.2.11.7.2-1, which applies to levels A and U, requires that "The Font dictionary of all fonts shall define the map of all used character codes to Unicode values, either via a ToUnicode entry, or other mechanisms". Without it, text can be selected and copied out as nonsense. pdffonts reports this per font in its uni column, which reads "yes" if there is an explicit "ToUnicode" map in the PDF file.

Level a additionally requires logical structure tagging, which a plain OCR pipeline cannot deliver, because Tesseract "only provides the text and its bounding box".

PDF/A-4 abandoned that ladder. veraPDF's supported profiles are "PDF/A-1a, PDF/A-1b, PDF/A-2a, PDF/A-2b, PDF/A-2u, PDF/A-3a, PDF/A-3b, PDF/A-3u, PDF/A-4, PDF/A-4e, PDF/A-4f": no 4a, 4b or 4u exists. PDF/A-4 also reversed a 2005 decision. Its action rule reads "The Launch, Sound, Movie, ResetForm, ImportData, Hide, Rendition and Trans actions shall not be permitted", and JavaScript is no longer on that list. The newest archival PDF standard permits scripting the 2005 one forbade.

Why a file that renders today can fail later

Three mechanisms, all detectable before you archive.

Fonts that are referenced rather than embedded get substituted by whatever the future viewer has, and substitution changes glyph widths, so line breaks and table alignment move. pdffonts prints an emb column, "yes" if the font is embedded in the PDF file. Any no is a deferred rendering bug.

External resources fetched at render time simply stop resolving. That is what the ban on the F, FFilter and FDecodeParms stream keys exists to prevent.

Metadata that lies is the least obvious. Ghostscript's default PDFACompatibilityPolicy value is 0, which will "Include the feature or operation in the output file, the file will not be PDF/A compliant. Because the document Catalog is emitted before this is encountered, the file will still contain PDF/A metadata but will not be compliant." A PDF that announces PDF/A conformance in its XMP has not been checked by anything. Set the policy to 1 to drop offending features, or 2 to abort.

The check itself is free. veraPDF is "a purpose-built, open source, file-format validator covering all PDF/A and PDF/UA parts and conformance levels". For structural sanity below the standards layer, qpdf --check parses the file and reports; "An exit status of 0 indicates syntactic correctness of the PDF file."

A signature ages differently from the document

Under eIDAS, validation of a qualified electronic signature requires confirming that "the qualified certificate was issued by a qualified trust service provider and was valid at the time of signing". The operative phrase is at the time of signing, not now. Once the signing certificate expires, a verifier holding only the file cannot establish when the signature was made, or whether the certificate had already been revoked when it was made.

Timestamps close that gap. RFC 3161 defines a service that "supports assertions of proof that a datum existed before a particular time", which lets a verifier "verify that a digital signature was applied to a message before the corresponding certificate was revoked thus allowing a revoked public key certificate to be used for verifying signatures created prior to the time of revocation". An archival signed PDF therefore needs the full certificate chain, the revocation data that was current at signing, and a trusted timestamp, all embedded in the file rather than fetched later.

That evidence is itself perishable, because signing algorithms weaken. RFC 3161 says so: "any token signed by the TSA SHOULD be time-stamped again" at a later date "to renew the trust that exists in the TSA's signature". EU law names the resulting obligation. eIDAS Article 34 requires a qualified preservation service to use "procedures and technologies capable of extending the trustworthiness of the qualified electronic signature beyond the technological validity period". Long-term signature validity is a subscription, not a property of the file.

One collision to plan around: PDF/A forbids the Encrypt key, so a password-protected signed PDF cannot also be PDF/A.

Keep the original next to the derivative

PDF/A-1 forbade attachments entirely. veraPDF's note on that rule records the loosening: "This requirement was relaxed in PDF/A-2 to embed other PDF documents conforming to either PDF/A-1 or PDF/A-2, and in PDF/A-3 to allow any embedded files." ISO 19005-3:2012 is titled, in part, "Use of ISO 32000-1 with support for embedded files".

Archives are not persuaded. NARA's instruction is "Select any version of PDF/A that meets project requirements. However, do not use the attachments feature in PDF/A-3 or PDF/A-4." An embedded spreadsheet is opaque to the preservation system that stores the PDF, and it inherits the PDF's fate.

The workable pattern is sibling files. Keep the scan master, uncompressed or losslessly compressed, keep the authored source document, and treat the searchable PDF/A as a derivative you can regenerate when the OCR engine improves. NARA's preferred format list ranks PDF/A-1 and PDF/A-2 for born-digital text and accepts TIFF 6 uncompressed or Deflate for digitised paper.

The Ghostscript route needs a PDF/A definition file whose /ICCProfile entry points at a real profile for the output intent, and that path "must be a fully qualified device and path specification appropriate for your Operating System". Nothing in the pipeline confirms the result. Run veraPDF over the output and believe the validator, not the writer.