Data File Access API :Under the Hood
Simon Horwith
CTO
Etrilogy Ltd.
Session Topics Data File Access API General Overview
What is it? Why use it? Where to get it?
Architecture Overview DFA API Functional Specification Technology to the Rescue!
Code Overview DFA API Methods The DFAQuery Custom Tag – Putting the “I” in “API”
Viewing a Sample Application Summary
DFA API General Overview: What is it?
The Data File Access API: is a framework for storing, retrieving, and maintaining
application data in the local file system is an Application Programming Interface for ColdFusion
Developers building small to medium sized applications on the MX platform
is an alternative to the traditional approach of storing data in a relational database management system (RDBMS)
Allows developers to retrieve data as a ColdFusion Query or as XML using either SQL or XPATH!
DFA API General Overview: Why use it?
The DFA API offers many benefits over traditional RDBMS solutions:
platform neutrality portability performance XPath and XSL Support XML storage and retrieval allows for:
Data and data-definition re-purposing Web Services Flash Remoting
better flexibility and extensibility
DFA API General Overview:Where to Get It?
The DFA API is available to Macromedia DevNet subscribers. It is part of DRK 3.
Installing the DFA API from the DRK CD will install: ColdFusion Component API Custom Tag “wrapper” Sample Application Sample DFA API XML “skeleton” files
An additional application for data entry and data table creation is available at http://www.how2cf.com
DFA API Architecture Overview:Functional Specification
DFA API Goals: Create an Application Programming Interface to allow
developers to query data stored as XML or CSV using familiar syntax (SQL) as well as perform other common SQL tasks such as adding, manipulating, and deleting data
Small to medium sized application usage is recommended Performance must be comparable to that of applications
that query an RDBMS Applications using the API should be able to be migrated
from one machine to another with ease Cross-platform functionality Must be flexible and easy to extend
DFA API Architecture Overview:Technology to the Rescue!
All API functionality is housed in a single ColdFusion Component: Easy installation (just copy the cfc to the server!) Easily executed by any application on the server Public/private data and functionality API can easily be extended via component
inheritence Developers could expose API functionality and
data via Web Services API could be used to provide data to Flash
Applications via Flash Remoting
DFA API Architecture Overview:Technology to the Rescue! (cont’d)
XML is useful in the real world… Architecture uses XML to:
Define data tables (table names, columns, etc.) Map location of data files to the data table definitions Define the actual data
Each of the above (data table definitions, locations, and data) is stored in an XML file. Data table definitions and appings are loaded and stored in memory each as an XML DOM.Data table data is loaded into memory (and persisted) the first time the data is needed.
CSV support simply means converting all CSV content to XML Storing this data as XML enables:
Retrieval of data as XML Use of XPATH to retrieve data
DFA API Architecture Overview:Technology to the Rescue! (cont’d)
Query of Queries saves the day! The API must support not only XPATH but also SQL
SELECT Statements for data retrieval. Thanks to QoQ, adding support for SQL SELECT was easily accomplished by first converting the XML DOM in question to a ColdFusion Query variable and then executing the SQL against it in a Query of Queries
If the data is to be returned as XML, the result set is converted back to an XML DOM before being returned.
Performance is fantastic, performance is fantastic, performance is fantastic!!
DFA API Architecture Overview:Technology to the Rescue! (cont’d)
Locking, Memory, and File System Access Misinformation dispelled – file I/O operations
are actually very fast All required information and data operations
are stored/performed first in persistent memory in order to boost performance
Named locks are used for all data operations – Exclusive locks for all write operations to data
tables in memory and on file Read-only locks for all other data operations
Code Review – Component MethodsMethod Description
AddData Adds a row of data to a datatable
addDataTableMapping Adds a datatable mapping to the mapping file
addMasterDataTableDefinition Adds a datatable definition to the master definition file
commitDataTable Saves a datatable currently in memory as an xml file
DeleteData Deletes rows of data from a datatable
DumpIt <cfdump>s a value then does a <cfabort>. Only used for debugging and should be commented out in production.
parseDataDefinitionMappings Loads the 'datatable'-to-'data-file' mappings for an application
parseDataTableDefinitions Parses the xml file containing all available data table definitions and creates an XML DOM of all data tables
GetData Loads a datatable if doesn’t already exist, performs SQL or XPath searches, returns data as XML or a query (or as transformed xml).
getDataTableDefinitionArray Returns an array of structures that define a datatable
GetDefReturns a DOM of all available table definitions. Only used for debugging and should be commented out in production.
Code Review – Component MethodsMethod Description
GetMap
Returns the DOM of application specific table definition to directory/url mappings. Only used for debugging and should be commented out in production.
GetNextID Returns next id for a datatable when performing an insert
GetPubData Returns a DOM of all loaded datatables. Only used for debugging and should be commented out in production.
GetTableLocation Retrieves datatable location from application data definition xml mapping DOM
IsCSV Verifies whether or not a string is proper CSV format
isDataTableIDUnique Validates whether or not a data table ID is not currently in use
isDataTableInMemory Is a specific datatable in memory or able to be loaded into memory?
Code Review – Component Methods
Method Description
LoadCSV Create a datatable from CSV text or from a CSV file
PathTypeOf Determine if a string is a absolute or relative path, or a URL
removeDataTable Removes a datatable from memory
UpdateData Modifies data in an existing datatable
XmlToRS Converts a datatable xml DOM to a CFML query
The DFAQuery Custom Tag – Putting the “I” in “API”
One goal of the DFA API was to make the API as easy as possible for developers to use in their applications. This was achieved by creating a custom tag “wrapper” to shield developers from the component and to allow them to use the tag syntax they are already familiar with.
The DFAQuery Custom Tag – Putting the “I” in “API”
DFAQuery Tag Execution Modes
Mode Tag Operation
Start
Tag checks to see if there’s an active instance of the API. If so, it does nothing. If not, it checks to see if the API can be initialized and either creates an instance or throws an error.
End Tag performs the requested API operation.
Viewing a Sample Application The rolodex application is a simple sample application that
installs with the API. It uses the DFAQuery custom tag to interact with the API Component in order to allow users to query the application for rolodex members, view member details, and update member information.
A second sample application (qbe.cfm) is also installed with the API. It is a query by example interface that shows how CSV text can be parsed into a data table, queried using SQL or XPATH, and deleted on the fly.
Let’s briefly examine the underlying code and files used by these applications
http://www.how2cf.com has a third sample application that uses nothing but direct component method calls in order to create an administrative interface for data entry and data table creation!!
Summary The Data File Access API –
Serves as an example of how to “properly” architect a component based application as well as how to code an API that makes use of the best features of ColdFusion MX
Offers developers an alternative to the traditional approach to storing and manipulating data in dynamic web applications. This alternative is easily extended to meet application specific needs, is not dependent on any RDBMS platform, and allows the data tier and it’s business logic to be distributed as part of the application – all without sacrificing performance.
Top Related