Chapter 12: Existing Form Fill-in

12.1.1 SetFieldValue Method

In Chapter 9 of this manual, we already described the process of filling in a form-like document (see Section 9.2).

The approach described in that chapter had one serious drawback: individual blanks in a "form" had to be referenced by their physical coordinates. This is because the Section 9.2 code sample did not involve a real form, but a regular (static) PDF document depicting a form.

Using tools such as Adobe Acrobat, it is possible to place interactive fields on an existing document using a graphical user interface. The form creator chooses a name, size, location and other options for each field. Once a form has been created, it can be filled in either interactively by a user, or automatically using tools such as AspPDF.NET. A form field can be referenced by name, as opposed to (x, y)-coordinates, which simplifies the coding significantly.

The PdfForm object provides the method FindField which returns a PdfAnnot object representing a top-level form field with the specified name. If no field under this name is found, the method returns Nothing.

To specify a text value for a field, the method SetFieldValue of the PdfAnnot object should be used. This method accepts two arguments: a text string, and a font to be used to render the string. The SetFieldValue method uses the field's pre-defined alignment and font size parameters. It also preserves an existing field border if there is one.

As of Version 2.6, AspPDF.NET also supports the method SetFieldValueEx which only requires a single argument, the text string. This method is described below.

The SetFieldValue method can be used to set a value not only for text fields, but checkboxes, drop-down list boxes and radio buttons as well. To select or unselect a checkbox, you must specify a state name for the text string argument. The state name for the "on" state should be obtained via the FieldOnValue property. The state name for the "off" state is usually "Off".

When working with radio button groups, you need to use the PdfAnnot.Children collection to reference each individual radio button within a group. The code snippet below deselects radio button #1 (American Express) and selects radio button #2 via the SetFieldValue method.

The following code fragment fills out a credit card form generated in Section 11.2:

To make a field read-only, Bit 1 of the property FieldFlags needs to be set, as follows:

12.1.2 SetFieldValueEx Method

As of Version 2.6, AspPDF.NET supports a more convenient way to fill in forms, via the PdfAnnot method SetFieldValueEx. This method takes a single argument, the text string. SetFieldValueEx internally calls SetFieldValue with a font object it automatically creates from the information contained in the PDF form being filled.

The SetFieldValueEx method simplifies the coding as the need for a font object is eliminated. For example, the code

can now be replaced with:

Note that the SetFieldValueEx method does not make SetFieldValue completely obsolete as the former relies on the fonts embedded in the PDF form, and those may not be sufficient to display all the necessary characters. In that case, an external font should be used via the method SetFieldValue.

The emergence of Adobe Acrobat 7.0 (and above) and Designed 7.0 (and above) brought about a number of significant changes to the PDF form specifications. Two changes that are most relevant to AspPDF.NET's form fill-in functionality are a new way of naming form fields, and the Adobe XML Forms Architecture (XFA).

When you switch to PDF forms created by Version 7 and later of the Adobe Acrobat suite, you may have to make some changes to your AspPDF.NET-based applications to accommodate the new Adobe specs.

12.2.1 Field Naming

Adobe Designer 7 assigns hierarchically-built names to form fields, such as form1[0].#subform[0].Address[0]. It is these long names that have to be passed to the Form.FindField method.

If you are not sure as to the full name of a field in your PDF, we recommend opening this document in a text editor (such as WordPad) and searching for your short field name (such as "Address"). You are likely to find a fragment which looks similar to this:

The string in parentheses followed by the word /TM is likely this field's long name which you can copy and paste into your script.

To help determine the names of the fields in your form, we have provided the Form Field Finder online application which enables you to upload your PDF document to our server, and the list of full field names in the document will be instantly displayed.

12.2.2 Adobe XML Forms Architecture (XFA)

In a nutshell, XFA is Adobe's new way to describe form structure and content using XML. XFA is only briefly mentioned in the official PDF 1.6 specifications, but the full description of this new standard is available from the Adobe.com web site.

An attempt to programmatically fill in an XFA-based form (such as a form created by Adobe Designer 7) using AspPDF.NET may fail as AspPDF.NET versions prior to 3.0 lacked XFA support. As a workaround, AspPDF.NET offers the method PdfForm.RemoveXFA which completely removes the XFA information from a form, thus making it compatible with older PDF form specifications, which AspPDF does support. In most cases, the user will not even notice any difference between an XFA-based and XFA-free form.

The following code snippet demonstrates the usage of the new method:

UPDATE: XFA support was added to AspPDF.NET in Version 3.0. For details, see Section 12.7 - XFA Support of this chapter below.

The following code sample fills in a simple form created in Adobe Designer 7.0.

Click the links below to run this code sample:

http://localhost/asppdf.net/manual_12/12_fill.cs.aspx http://localhost/asppdf.net/manual_12/12_fill.vb.aspx

As of Version 2.4, AspPDF.NET is capable of filling in image fields via the method SetFieldImage of the PdfAnnot object. This method expects two arguments: an instance of the PdfImage object containing the image, and an optional parameter string or PdfParam object.

The parameters currently supported are Mode, ReadOnly and Alignment, all optional (the Alignment parameter was introduced in version 3.1.0.30169). Mode specifies the stretch mode of the image being rendered. The possible values are:

• 1 - Scale to fit (default);
• 2 - Stretch to fit;
• 3 - Do not scale or stretch.

When Mode is set to 1 (Scale-to-fit mode), the Alignment parameter can also be used. The valid values are:

• 0 - Alignment to the left or top (default);
• 1 - Alignment to the right or bottom;
• 2 - Alignment to the center.

By default, SetFieldImage makes the image field read-only. The ReadOnly parameter, if set to False, leaves the image field clickable.

SetFieldImage can only be called on image fields, all other field types will throw an exception. In most cases, calling RemoveXFA is also required for this method to work properly.

The following code sample demonstrates the use of this method:

As of Version 2.6, AspPDF.NET offers form-flattening functionality via the PdfForm method Flatten. This method has no arguments.

To flatten a form means to turn all of its interactive fields into static graphics with no possibility for further editing. Once the Flatten method is called, the PDF document stops being a form and becomes a regular static template.

The Flatten method can only be called on an already filled-in form. Calling SetFieldValue/SetFieldValueEx on the same instance of the document has no effect on the output. Therefore, in order to fill in a form and then flatten it, the form has to be filled the regular way, saved, reopened and only then flattened, as follows:

After flattening, field text information may appear distorted on some documents. For example, character spacing may become incorrect. To avoid that, a "Save State" command should be put at the beginning of each page's existing content, and a matching "Restore State" command at the end of it, as follows:

UPDATE: As of Version 3.4.0.2, as explained in the following section, the code snippet above can be replaced by

Form flattening is demonstrated by Live Demo #16 - Form Flattening.

As of Version 2.9, a new PdfForm method, Modify, has been introduced which combines and extends the XFA removal and form flattening functionality described above. The method also enables annotation flattening and JavaScript removal described below. More functions are expected to be added to this method in the future as AspPDF.NET's form-related functionality expands.

The Modify method expects a single argument: a PdfParam object or parameter string. The following Boolean parameters, all optional, are currently supported and can be used in any combination:

• RemoveXFA - using this parameter is equivalent to calling the old PdfForm.RemoveXFA method.
• Flatten - using this parameter is equivalent to calling the old PdfForm.Flatten method.
• FlattenAnnots - this parameter is useful if the PDF form being flattened contains items that are not technically form fields but field-like annotations. The following line of code flattens both the fields and field-like annotations:
  objDoc.Form.Modify("Flatten=true; FlattenAnnots=true");
• RemoveJavaScript - this parameter removes all JavaScript from the document's catalog. PDF forms created with Adobe products often contain JavaScript that displays the message "This PDF form requires a newer version of Adobe Acrobat" after the form has been flattened:

  

  The following line of code flattens the form and prevents this message from popping up by removing JavaScript from the document:

  objDoc.Form.Modify("Flatten=true; RemoveJavaScript=true");

• Reset (introduced by Version 3.4.0.2) - this parameter attempts to reset the current graphics state of all pages of the document by placing the Save State ("q") command at the beginning of every page's content and the Restore State ("Q") command at the end of it. Resetting the graphics state may be needed if the flattening procedure produces unexpected output or no output at all.

The Modify method is demonstrated by Live Demo #16 - Form Flattening.

12.7.1 XFA Overview

A traditional PDF form contains annotation objects representing form fields. Adobe LifeCycle Designer 7.0+ introduced a new generation of forms called XFA, which stands for XML Forms Architecture. Under XFA, a form layout is defined by an XML document, called template, describing subforms, fields, static text, fonts, images and other elements of the form. Below is the screenshot of a very simple template containing a submit button, two text fields, and two mutually exclusive radio buttons. To save screen space, most of the XML nodes of that template are shown in a collapsed state:

The content of the form fields is also packaged as an XML document called dataset. Both the template and dataset (as well as other information in XML format) are embedded in a PDF document. A form viewer application such as Adobe Reader merges and template and dataset together using a well-defined set of rules and produces a filled-out form.

Even a blank XFA form usually contains a property structured XML-based dataset, but this XML document just contains empty values. For example, the initial dataset for the form shown above looks as follows:

If the dataset sheet is filled with values, such as

and plugged back into the PDF form, the fields will contain the specified values when the form is viewed.

One important advantage of the XFA architecture over traditional PDF forms is that is enables dynamically growing forms, while the traditional forms are inherently static.

The Adobe XFA format is extensive. Version 3.3 of the XFA specifications is a 1,585-page document. It is available on the Adobe web site and can be downloaded from here.

12.7.2 AspPDF.NET's XFA Support

AspPDF.NET's support for XFA forms is streamlined: it enables the template and dataset data to be retrieved from a PDF form, and it also enables the modified dataset data to be plugged back into the PDF form. AspPDF.NET provides no functionality for XML parsing and creation but fortunately both classic ASP and .NET provide ample built-in support for XML management.

AspPDF.NET's XFA functionality is encapsulated in the PdfForm object via the read-only property XFATemplate and read/write property XFADatasets (note the plural form of the word "datasets".) Also, there is an auxiliary property HasXFA which returns True if the PDF form contains XFA data. If this property returns False, the form contains no XFA information and the XFATemplate and XFADatasets properties cannot be used.

The online retrieval of the template and dataset of a PDF form is implemented by Live Demo #15.

To demonstrate AspPDF.NET's XFA functionality, a sample PDF purchase order form shipped with Adobe LiveCycle Designer, Purchase Order.pdf, will be used. It can be found under the \Samples\manual_11 subfolder of the installation.

This form contains all major types of fields: text boxes, radio boxes, drop-down lists, checkboxes, and a dynamically growing list of items. This form's dataset which can be retrieved using Live Demo #15 looks as follows:

Our code sample will fill out the following fields: PO Number (<txtPONum>), Ordered by Company (<txtOrderedByCompanyName>), a Terms and Conditions radio box(<TermsConditions>), the State Tax checkbox (<chkStateTax>) and State Tax Rate (<numStateTaxRate>). It will also add three purchase order items to the item list (<detail>).

The valid values for a radio button group (1 for cash and 2 for credit in our example) can be obtained from the form's template. The relevant snippet of the template (simplified for better readability) is shown below:

Click the links below to run this code sample:

http://localhost/asppdf.net/manual_12/12_xfa.cs.aspx http://localhost/asppdf.net/manual_12/12_xfa.vb.aspx

The resultant PDF form looks as follows:

Some U.S. government agencies, U.S. Citizenship and Immigration Services (USCIS) in particular, have started employing barcode-equipped forms. These forms have a large two-dimensional PDF417 barcode on each page representing the content of this page's fields. Every time a field is modified by the user, the barcode is automatically redrawn to reflect the change. The barcode enables the processing agent to transfer the content of a paper form to the computer system instantly.

The official electronic forms I-821, N-400, G-28, I-90, I-131 and I-864 are hybrid: they are based on both the traditional AcroForm and new XFA formats. The easiest way to fill them out programmatically with AspPDF.NET is to circumvent XFA altogether and use the FindField/SetFieldValueEx methods to fill out the individual fields, but doing so breaks the barcode functionality of the form. Therefore, the PDF417 barcode has to be drawn separately according to the government specifications. Afterwards, the form can optionally be flattened. AspPDF.NET's support for PDF417 barcodes is described in detail in Subsection 13.2.2 - PDF417 of the next chapter.

The USCIS requirements for the 2D barcode can be found here: http://www.uscis.gov/forms/uscis-2d-barcode-requirements.

The following code sample fills out Page 1 of USCIS Form N-400 ("Application for Naturalization"). This form can be downloaded from the USCIS.gov web site.

With the help of the Form Field Finder application mentioned above, Form N-400 is found to have the following fields on Page 1:

For the sake of demonstration, our code sample will fill out 9 form fields as follows:

The rest of the fields will be left blank.

According to the government barcode specifications (link above), the text string encoded by the barcode for Page 1 must contain 18 values separated by the "|" character. The first three are form type ("N-400"), form revision ("09/13/13"), and page number ("1").

The other 15 values are the actual content of the form fields: A-Number ("123456789"), Eligibility - option 5 ("D"), Explanation - religious duties ("SECTION 317,INA"), Current legal name ("Smith", "John", "Frederick"), Name as it appears on resident card ("Smith", "John", "Frederick"), other names ("", "", "", "", "", "" ).

Therefore, the entire string to be encoded into a barcode is as follows:

To draw the barcode according to the specifications, the method PdfCanvas.DrawBarcode2D should be called with the following parameters (AspPDF 3.2.0.1+ required):

• X=36, Y=36 (lower-left corner coordinates, obtained empirically);
• Width=540, Height=108 (barcode size: 7.5 x 1.5 inches or 540 x 108 user units per specs);
• QZV=6, QZH=8 (vertical and horizontal extents of the quiet zone around the barcode, obtained empirically);
• ErrorLevel=5 (error correction level 5 per specs);
• Columns=30 (to roughly match the specified bar width, values of 28 and 29 are also acceptable);
• Rows=5 (to match the appearance of the barcode on the official form);
• Binary=true (optional, see below.)

It is worth noting that the barcode on the official N-400 form does not strictly adhere to the government specifications. In particular, the barcode uses the text compression of data (which is more compact) while the specifications call for byte compression. The DrawBarcode2D uses text compression by default. If byte compression is required, the parameter Binary=true should be used. The code sample below does not use it, however.

NOTE: This code sample requires that the USCIS form N-400 (file n-400.pdf) be placed in the subfolder \Samples\manual_11 of the installation. This file was not included in the installation because of its large size.

Click the links below to run this code sample:

http://localhost/asppdf.net/manual_12/12_formwithbarcode.cs.aspx http://localhost/asppdf.net/manual_12/12_formwithbarcode.vb.aspx