PDF is dead. Long live PDF... with Java!

© 2014, iText Group NV, iText Software Corp., iText Software BVBA© 2014, iText Group NV, iText Software Corp., iText Software BVBA

PDF is dead! Long live PDF!Java One Tutorial 29/09/2014

© 2014, iText Group NV, iText Software Corp., iText Software BVBA

Is PDF dead?

PDF is dead; Long live PDF... and Java!2



The PDF Reference Manual


Everybody uses HTML


Source:http://duff-johnson.com/2014/03/10/98-percent-of-dot-com-is-html-but-38-percent-of-dot-gov-is-pdf/


But governments also like PDF


Source:http://duff-johnson.com/2014/03/10/98-percent-of-dot-com-is-html-but-38-percent-of-dot-gov-is-pdf/

Percentage of PDF files:.org: 15%.gov: 38%.edu: 27%


Publications versus…

No need to be self-contained and may change over time

Not all content produced by the author

e.g. Advertisements

Becoming more interactive

e.g Comments on a news article


Source:The Future of PDFLeonard Rosenthol, PDF Architect at Adobe


… documents


Source:The Future of PDFLeonard Rosenthol, PDF Architect at Adobe

Needs to be self-contained

Unchanging (non-dynamic)

Able to be authenticated

Able to be secured/protected


Not counting HTML, PDF is King


Source:http://duff-johnson.com/2014/02/17/the-8-most-popular-document-formats-on-the-web/


Challenges for PDF ChallengersOffline consumption

PDF Viewers are ubiquitous (Adobe Reader is on 90% of the desk tops)

Mobile consumption

Interactivity (forms, commenting,…)

Reading books on eReaders

Attacking the weaknesses of PDF

PDF suffers from “too much” functionality

PDF was designed in a “pre-hacker” world

Not all tools are created equal

• Producers don’t use best practices (e.g. Tagged PDF)

• Not all viewers support all features (e.g. EcmaScript)



Umbrella of Standards


PDFPortable Document FormatFirst released in 1993ISO Standard since 2008

ISO 32000

PDF/Eengineering

Since 2008

ISO 24517

PDF/VTprinting

Since 2010

ISO 16612

PDF/Xgraphic arts

Since 2001

ISO 15930

PDF/Aarchive

Since 2005

ISO 19005

PDF/UAaccessibility

Since 2012

ISO 14289

Related:• XFDF (ISO)• EcmaScript (ISO)• PRC (ISO)• PAdES (ETSI)


Hello iText



Creating a PDF in 5 stepsExample 1: Hello World with iText

public void createPdf(String dest)

throws DocumentException, IOException {

// step 1

Document document = new Document();

// step 2

PdfWriter.getInstance(document, new FileOutputStream(dest));

// step 3

document.open();

// step 4

document.add(new Paragraph("Hello World!"));

// step 5

document.close();

}



A “Hello World” PDF



A simple form with three fields


Example : Filling out an existing form


Filling out a formExample 2.a: Filling out a form

public void manipulatePdf(String src, String dest)


PdfReader reader = new PdfReader(src);

PdfStamper stamper =

new PdfStamper(reader, new FileOutputStream(dest));

AcroFields form = stamper.getAcroFields();

form.setField("Name", "Raf Hens");

form.setField("Company", "iText Software");

form.setField("Country", "BELGIUM");

stamper.close();

reader.close();

}



A filled out form



Flattening out a formExample 2.b: Flattening a form






AcroFields form = stamper.getAcroFields();

form.setField("Name", "Bruno Lowagie");

form.setField("Company", "iText Group");

form.setField("Country", "BELGIUM");

stamper.setFormFlattening(true);

stamper.close();

reader.close();

}



A flattened form



Adding a watermarkExample 3: Stamping content on a PDF






PdfContentByte under = stamper.getUnderContent(1);

ColumnText.showTextAligned(under, Element.ALIGN_CENTER,

new Phrase("Watermark", new Font(FontFamily.HELVETICA, 120)),

297, 421, 45);

stamper.close();

reader.close();

}



The importance of standards

Three use cases


Speaking the same language

Not being able to understand each other is a punishment, NOT a business model!

Standards are about speaking the same language!



PDF/UA

ISO 14289: Universal Accessibility


Every one can read this



But some structure is helpful


title

list item

list item

list item

Label Content


Can every one read this?



How do we read the spider chart?


Person 2 1.7 1.4 1.3 1.9 2.1 0.8 2.2 1.8 1.6 1 1.4

Position 3 2.3 2 1.8 3.2 3.9 2.1 3.1 3.2 2.3 2 2.1

Ris

k M

anag

em

en

t

Stru

ctu

red

Fin

ance

Me

rge

rs &

acq

uis

itio

ns

Go

vern

ance

& In

tern

al C

on

tro

l

Acc

ou

nti

ng

Op

era

tio

ns

Tre

asu

ry o

pe

rati

on

s

Man

age

me

nt

Info

rmat

ion

&

Bu

sin

ess

De

cisi

on

Su

pp

ort

Bu

sin

ess

Pla

nn

ing

& S

trat

egy

Fin

ance

Co

ntr

ibu

tio

n t

o IT

M

anag

em

en

t

Co

mm

erc

ial A

ctiv

itie

s

Taxa

tio

n

Fun

ctio

nal

Le

ade

rsh

ip


Is this a better way to read the data?


Person Position

Functional Leadership 2 3

Risk Management 1.7 2.3

Structured Finance 1.4 2

Mergers & Acquisitions 1.3 1.8

Governance & Internal Control 1.9 3.2

Accounting Operations 2.1 3.9

Treasury Operations 0.8 2.1

Management Information & Business Decision Support 2.2 3.1

Business Planning & Strategy 1.8 3.2

Finance Contribution to IT Managemen 1.6 2.3

Commencial Activities 1 2

Taxation 1.4 2.1


PDF/UA to the rescue

Let’s agree on a standard way to store/interpret documentsISO 14289: Universal Accessibility

PDF/UA is a technical specification intended for developers implementing PDF writing and processing software.

PDF/UA provides definitive terms and requirements to allow people with/without disabilities the same rights.

For those equipped with appropriate software, conformance with PDF/UA ensures accessibility for people with disabilities who use assistive technology such as screen readers, screen magnifiers, joysticks and other technologies to navigate and read electronic content.



PAdES

ETSI TS 102 778: PDF Advanced Electronic Signatures


Integrity


I paid a forged invoiceand lost $40K!


Authenticity


Why am I, Emperor Constantine I, in this picture? I never transferred

authority to the Pope!


Non-repudiation


I didn’t do it!


Business requirements for signatures


Let’s agree on a standard, vendor-independent way to ensure document integrity, authentication and non-repudiation


PAdES to the rescue!

ISOISO-32000-1 (2008) based on PDF 1.7 (2006)

ISO-32000-2 will define PDF 2.0 (2016)

ETSI: TS 102 778 (2009 - 2010)PAdES 1: Overview

PAdES 2: Basic – CMS based (ISO-32000-1)

PAdES 3: Enhanced – CAdES based (ISO-32000-2)

PAdES 4: LTV – Long Term Validation

PAdES 5: XAdES based (XML content)

PAdES 6: Visual representation guidelines

ETSI: TS 103 172 (2011 - 2013)PAdES Baseline Profile



PDF/A-3 - ZUGFeRD

ISO 19005-3: Archiving

ZUGFeRD: Invoicing


Paying invoices is a pain



Processing invoices is a cost



We need a standard!

What if a vendor would provide an invoice that:Can be read by a human being? (Use PDF?)

Can be processed by a machine? (Use XML?)

What would it take to make sure that:You don’t have to do any manual work to input the amount to be paid, sales tax,…? (Structured info?)

The invoice can be preserved for the long term? (PDF/A?)



PDF/A-3 to the rescue!


Let’s agree on a standard way to archive documentsISO 19005 part 3:

PDF/A is an ISO-standardized version of the Portable Document Format (PDF) specialized for the digital preservation of electronic documents.

PDF/A-3 allows attachments that are not compliant with PDF/A, e.g. an e-mail, machine readable data such as an XML file,…


Invoice + XML Attachment



ZUGFeRD to the rescue!


Let’s agree on a standard way to exchange invoicesZUGFeRD: a pioneer to fix PDF invoicing

Developed by a Workgroup called “Forum elektronische Rechnung Deutschland” (FeRD).

Based on PDF/A-3: archiving + attachment.

Imposes an XML schema for the data added in attachment.

Data can be extracted and processed without human intervention:

• This speeds up the processing of invoices,

• Makes the process less error-prone, and

• Reduces the cost.


Conclusions

Why are standards important in business?



Standards ensure:

Clarity, as shown in the PDF/UA use case,

Security, as shown in the PAdES use case,

Interoperability, as shown in the ZUGFeRD use case,

And much more!


Creating PDF/UA, PDF/A-3, ZUGFeRD


Step 1. Create PDF

PDF



Step1. Create PDF: Initialize documentDocument document = new Document();

PdfWriter writer =

PdfWriter.getInstance(document,

new FileOutputStream("SimplePdf.pdf"));

writer.setPdfVersion(PdfWriter.VERSION_1_7);

document.open();



Step1. Create PDF: Add elementsParagraph p = new Paragraph();

Chunk c = new Chunk("The quick brown ");

p.add(c);

Image i = Image.getInstance("fox.bmp");

c = new Chunk(i, 0, -24);

p.add(c);

c = new Chunk(" jumps over the lazy ");

p.add(c);

i = Image.getInstance("dog.bmp");


p.add(c);

document.add(p);



Step 1. Create PDF: Close documentdocument.close();



Step 1. Create PDF: Result



Step 2. Create Tagged PDF


PDFTagged content

Tagged PDF



PdfWriter writer = PdfWriter.getInstance(…);


writer.setTagged();

document.open();

PDFTagged content

Tagged PDF





PDFTagged content

Tagged PDF


Step 3. Create PDF/UA


Tagged PDF MetadataFont

embeddingAlt text PDF/UA



GoalsProvide document title, author, keywords

Provide document language for better accessibility

Identify “type” of PDF (PDF/UA) to help PDF reader






writer.setTagged();

writer.setViewerPreferences

(PdfWriter.DisplayDocTitle);

document.addLanguage("en-US");

document.addTitle("English pangram");

writer.createXmpMetadata();

document.open();






GoalsMake document independent of font set of current OS

Make document independent of font selection strategy of PDF reader

Make document fully portable






document.open();

Paragraph p = new Paragraph();

p.setFont(FontFactory.getFont("FreeSans.ttf",

BaseFont.WINANSI, BaseFont.EMBEDDED, 20));

Chunk c = new Chunk("The quick brown ");

p.add(c);









Before After



GoalsProvide alternate text for non-textual content

Help conforming readers to read out loud the non-textual content






Image i = Image.getInstance("fox.bmp");


c.setAccessibleAttribute(PdfName.ALT,

new PdfString("Fox"));

p.add(c);





Step 4. Create PDF/A-3b


PDF MetadataFont

embeddingOutput intents

PDF/A-3b




PdfAWriter writer = PdfAWriter.getInstance(document,

new FileOutputStream("PdfA3b.pdf"),

PdfAConformanceLevel.PDF_A_3B);



PDF MetadataFont


PDF/A-3b



GoalsIdentify “type” of PDF (PDF/A) to help PDF reader

Similar iText code as for PDF/UA


PDF MetadataFont


PDF/A-3b




PDF MetadataFont


PDF/A-3b



GoalsMake document fully self contained and archivable

Same iText code as for PDF/UA


PDF MetadataFont


PDF/A-3b



GoalsMatch color characteristics of PDF document with color characteristics of

device on which it is intended to be rendered

Make colors device independent


PDF MetadataFont


PDF/A-3b



document.open();

ICC_Profile icc = ICC_Profile.getInstance(new

FileInputStream("sRGB Color Space

Profile.icm"));

writer.setOutputIntents("Custom", "",

"http://www.color.org", "sRGB IEC61966-2.1",

icc);

Paragraph p = new Paragraph();


PDF MetadataFont


PDF/A-3b




PDF MetadataFont


PDF/A-3b


Step 5. Create PDF/A-3a


PDF/A-3b

PDF/UAPDF/A-

3a





new FileOutputStream("PdfA3a.pdf"),

PdfAConformanceLevel.PDF_A_3A);



PDF/A-3b

PDF/UAPDF/A-

3a




PDF/A-3b

PDF/UAPDF/A-

3a



Can I make sure that my document is compliant?

iText will:

Perform a lot of checks, from technical PoV, based on the requested conformance level

Throw an exception if a requirement is not met

Not everything can be checked automatically!


PDF/A-3b

PDF/UAPDF/A-

3a




PDF/A-3b

PDF/UAPDF/A-

3a


Create ZUGFeRD invoices


PDF/A-3bXMP

metadataInvoice

attachmentZUGFeRD

PDF





new FileOutputStream("Zugferd.pdf"),

PdfAConformanceLevel.ZUGFeRD);



PDF/A-3bXMP

metadataInvoice

attachmentZUGFeRD

PDF



writer.createXmpMetadata();

writer.getXmpWriter().setProperty(

PdfAXmpWriter.zugferdSchemaNS,

PdfAXmpWriter.zugferdDocumentFileName,

"invoice.xml");

document.open();


PDF/A-3bXMP

metadataInvoice

attachmentZUGFeRD

PDF



PdfFileSpecification fileSpec =

writer.addFileAttachment("ZUGFeRD invoice",

null, "c:/invoice.xml", "invoice.xml",

"application/xml",

new AFRelationshipValue.Alternative);


PDF/A-3bXMP

metadataInvoice

attachmentZUGFeRD

PDF




PDF/A-3bXMP

metadataInvoice

attachmentZUGFeRD

PDF


The DNA of PDF

The Structure of a PDF file

Looking at a PDF from a text editor

Browsing Java objects using Java Swing


The structure of a PDF file


Header

Body

Cross-Reference Table

Trailer


Demo-time



Finding Structure in PDF


Listener for text snippetspublic class MyTextRenderListener implements RenderListener {

public void beginTextBlock() {

System.out.println("<");

}

public void endTextBlock() {

System.out.println(">");

}

public void renderImage(ImageRenderInfo renderInfo) {

}

public void renderText(TextRenderInfo renderInfo) {

System.out.println(" <");

Vector start = renderInfo.getBaseline().getStartPoint();

System.out.println(String.format(" x: %s y: %s length: %s \n Text: %s",

start.get(Vector.I1), start.get(Vector.I2),

renderInfo.getBaseline().getLength(), renderInfo.getText()));

out.println(" >");

}

}



Parsing PDFPdfReader reader = new PdfReader(src);

RenderListener listener = new MyTextRenderListener();

PdfContentStreamProcessor processor = new PdfContentStreamProcessor(listener);

PdfDictionary pageDic = reader.getPageN(1);

PdfDictionary resourcesDic =

pageDic.getAsDict(PdfName.RESOURCES);

processor.processContent(

ContentByteUtils.getContentBytesForPage(reader, 1), resourcesDic);

reader.close();



Example page to parse



Providing hints



Demo

Finding text snippets

Finding text lines

Finding text structure



Wrapping up

The Technical Roadmap for iText

Further reading

Q&A


Roadmap 2012

Mobile versus CloudAndroid version

Google App Engine (GAE) Version

Digital SignaturesUpdate to new specs

Documentation

XML Worker / XFA WorkerGeneric XML to PDF engine

Test with HTML, target XFA



Roadmap 2013

Creating Tagged PDFFrom high-level objects

From XFA Worker

Focus on standardsRewriting the PDF/A functionality

Introducing PDF/UA support

PAdES 5XML-DSig + XAdES



Roadmap 2014

Major upgrade for RUPSReading and Updating PDF Syntax

XFA Worker: Enterprise-grade solutionSupport for JavaScript

StandardsComplete PDF/UA

Complete PDF/A-2, PDF/A-3, ZUGFeRD



Roadmap 2015

Customer-driven developmentE.g. “Redaction”

Unstructured PDFsTurn our research into development

PDF 2.0Making iText ready for PDF 2.0

Next ISO Committee meeting: November 2014, Edinburgh



Books published by Manning


1st Edition: 2006• 11.500 copies

2nd Edition: 2010• 8.000 copies


New book in the making



Upcoming titles




Thank you!

Questions and answers

Visit us at booth #5712

PDF is dead. Long live PDF... with Java!

Software

Transcript of PDF is dead. Long live PDF... with Java!