Merging ODBC Data into XML: ODBC2XML

ODBC2XML is a database publisher that merges data from any ODBC data source into XML documents. Its unique architecture uses predefined XML templates that contain embedded SQL queries, and generates documents that contain the ODBC data merged into the original XML. It comes complete with a callable DLL and a command-line interface for batch processing.

ODBC2XML is distributed as shareware. You may obtain the ODBC2XML trial copy and use it freely on a single-user machine for any purpose. However, you may not redistribute this version for commercial use, and any SQL query that returns more than 15 rows will abort with an error message. To avoid these restrictions you must use the registered version. For a single-user license along with up to 1 hour of free technical support, you can register ODBC2XML online at RegNet. Additional licensing options are described below.

Immediate Order Fulfillment!
Now if you register at RegNet you will immediately be sent an email with the link to the unrestricted version.
A single-user license is just $49.95. Other licenses are also available.

A brief review of ODBC2XML is at Ronald Bourret's XML Database Middleware page.

Also, there is a Java version available as JDBC2XML.

Overview
Defining XML Templates
API Functions
Obtaining ODBC2XML
Registration & Licensing

Overview

ODBC2XML simplifies the generation of complex XML documents. It allows non-programmers to generate sophisticated XML files without the need for scripting languages, report writers, or other more complex processing tools. Using ODBC2XML, you can:

For example, let's assume that you want to publish a database table cartoon containing cartoon characters and their catchphrases:

NameCatchPhrase
Bart SimpsonDon't have a cow, man
Homer SimpsonDoh!
Bugs BunnyWhat's up, doc?
Elmer FuddBe verwwwy quiet! I'm hunting wabbits!

To publish this table in an XML file, you first need to create an XML template with processing instructions (shown in bold) that specify how to query the table and where to place the data:

<?xml version="1.0"?>
<memo>
<from>Me</from>
<to>You</to>
<re>Important Cartoon Characters</re>
<body>
<p>Here is a list of cartoon characters and the brilliant things they say:<p>
<list>
<?odbc2xml q1:select * from cartoon?>

<item><?odbc2xml q1.Name?> says '<?odbc2xml q1.CatchPhrase?>'</item>
</list>
</body>
</memo>

After running ODBC2XML, here's the result:

<?xml version="1.0"?>
<memo>
<from>Me</from>
<to>You</to>
<re>Important Cartoon Characters</re>
<body>
<p>Here is a list of cartoon characters and the brilliant things they say:<p>
<list>
<item>Bart Simpson says 'Don't have a cow, man'</item>
<item>Homer Simpson says 'Doh!'</item>
<item>Bugs Bunny says 'What's up, doc?'</item>
<item>Elmer Fudd says 'Be verwwwy quiet! I'm hunting wabbits!'</item>
</list>
</body>
</memo>

Defining XML Templates

The input to ODBC2XML is a well-formed XML template file, optionally containing processing instructions that define SQL queries and result field placement rules. All of the text defined in the input file is used to generate the output file; in the absence of any special processing instructions, the input XML file is simply copied to the standard output device.

Defining SQL Queries

The processing instruction to define an SQL query uses the following format:

<?odbc2xml query-name:select-statement?>

where query-name is the name that uniquely identifies a query (one or more alphanumeric characters), and select-statement is the text of the SQL statement to submit to the ODBC database. For example, given a table Products containing a set of product names and descriptions, the following query retrieves all such records:

<?odbc2xml q1:select * from products?>

Query Scope

Each SQL query has a scope in which it is effective, and in which its fields are available for insertion into element text and/or attribute values. This scope is exactly defined by the parse scope of the next XML element to follow the processing instruction. When the query is executed, ODBC2XML processes the elements within scope once for each record in the result set.

For example, given 3 records in the Products table, the following code will produce 3 list items:

<list>
<?odbc2xml q1:select * from products?>
<item>item text</item>
</list>

However, simply moving the processing instruction above the list tag changes the scope of the query such that 3 separate lists will be produced:

<?odbc2xml q1:select * from products?>
<list>
<item>item text</item>
</list>

Inserting Query Results into Character Data

The results of a query can be inserted anywhere within XML element text via the following processing instruction:

<?odbc2xml query-name.field-name?>

where field-name is the name of any field within the ODBC result set. This directive can only be included within the scope of the query.

For example, the following XML template file inserts the ProductName field as a list item within the memo:

<?xml version="1.0"?>
<memo>
<list>
<?odbc2xml q1:select * from products?>
<item><?odbc2xml q1.ProductName?></item>
</list>
</memo>

Placing ODBC data into text which is outside the scope of a query will result in a runtime error:

<?xml version="1.0"?>
<memo>
<list>
<?odbc2xml q1:select * from products?>
<item>item text</item>
</list>
<p>This is wrong: <?odbc2xml q1.ProductName?></p>
</memo>

Inserting Query Results into Attribute Data

ODBC data can also be merged into an element's attribute values. Since XML does not allow a processing instruction to be embedded with an attribute's text, a slightly different method is required. The ODBC2XML program recognises the following pattern within an attribute's value (after all internal entities have been resolved):

<element attr="odbc2xml query-name.field-name">

For example:

<?xml version="1.0"?>
<memo>
<list>
<?odbc2xml q1:select * from products?>
<item productname="odbc2xml q1.ProductName"></item>
</list>
</memo>

Nested Queries

ODBC2XML allows multiple SQL queries to be active at the same time. For example, the following XML template file generates a series of memos, one per product. Within each product, the features of that product are presented in a list. This requires a join between the outermost query (on the products table) and the inner one (on the features table). Within a given query, traditional SQL allows fields to be prefaced with table names as follows:

select * from products,features where features.ProductID = products.ProductID

However, since in ODBC2XML the join is separated amongst two queries (the queries need to have different XML scope), this method is insufficient. Therefore ODBC2XML allows the concept of a foreign field within an SQL query, which is identified to the SQL parser via a leading colon (:) character. This is similar to the method used in SQL precompiler products.

The following example demonstrates the linking of two related queries:

<?xml version="1.0"?>
<memos>
<?odbc2xml q1:select * from products?>
<memo>
<p>The features for the <?odbc2xml q1.ProductName?> product are:
<list>
<?odbc2xml q2:select * from features where ProductID = :q1.ProductID?>
<item><?odbc2xml q2.Feature?></item>
</list>
</memo>
</memos>

Common Problems

ODBC2XML follows the ODBC standard for quoting literal character values and for identifiers. In particular, the MS Access ODBC driver defines these characters to be the single quote (') and backwards quote (`) respectively. You should be aware of these characters if you wish to embed literal string constants into your queries, and/or reference the table names or field names with blanks in them. The following example shows all the use of the different kind of single quotes:

select `Recording Title` from Recordings where `Recording Title` = 'Sweet Home Alabama'

Similarly, the MS Access ODBC driver defines the '#' character to delimit date constants:

select * from Recordings where RecordingDate > #05/10/1999#

Also, when selecting aggregate functions (such as count(*)) and expressions (such as a+b) from record sets, the ODBC driver assigns names to the resulting columns. In particular, the MS Access ODBC Driver uses the convention Expr<N>, where N is a counter, generally beginning at 1000. It is much easier to assign a column alias within the query to avoid guessing at what naming convention the driver is using. To do this, use the ODBC SQL syntax as column-name after the expression in the select list. So to retrieve the value of count(*), use the following example:

<?odbc2xml q1:select count(*) as count from Recordings?>
<list>
<item><?odbc2xml q1.count?>
</list>

ODBC2XML does not do a validating parse; therefore any DTD provided as part of the input XML template file is not processed. Also note that XML does not provide for entity references within processing instructions; therefore entities cannot be used as part of SQL queries.

API Functions (Optional)

ODBC2XML is also provided as a DLL with a simple C-callable API. The intent of the API is to speed up server-side applications, so that each invocation of ODBC2XML need not re-initialize the XML parser, re-connect to the ODBC database, etc. A C header file is provided with prototypes for the API functions (odbc2xml.h). Also, we have included a C program containing example calls to the API functions (demo.c).

To compile the example using MS Visual C++, use the following command:

cl demo.c /link odbc2xml.lib

The ODBC2XML API is currently NOT thread-safe. Applications should make sure that API function calls are made one at a time without re-entrancy. Future versions may address this issue; feel free to contact us with your comments.

Each API function returns an integer error code under the following guidelines:

  • 0 indicates success
  • a positive number provides an error code that can be processed using o2x_error()
  • -1 indicates that an error occurred for which no code/message is available.

    Function Description
    int o2x_connect(char *dsn,char *uid,char *pwd) Establishes an ODBC connection to the named ODBC data source. Multiple connections are maintained on a stack, and can be popped off the stack using o2x_disconnect(). Once established, the connection becomes the "current" active connection.
    int o2x_load(char *infile) Loads an XML template file into memory, parses the file, but does not produce output. The file becomes the currently active file for subsequent processing by o2x_expand(). Overwrites any previously loaded template file, if any.
    int o2x_expand(char *outfile) Processes the currently active XML template, and writes the expanded XML document (containing the merged ODBC data) to the named file. Closes the file when completed. This function can be called any number of times once a template has been loaded using o2x_load().
    int o2x_disconnect(void) Disconnects the most recent ODBC connection.
    char *o2x_error(int code) Returns an error message for an error code (as returned by one of the other API functions).

    Error Handling

    ODBC2XML detects and reports a variety of XML, ODBC, and system related errors. The following XML errors are reported by the Expat parser:

    CodeDescription
    10001out of memory
    10002XML syntax error
    10003no element found
    10004not well-formed
    10005unclosed token
    10006unclosed token
    10007mismatched tag
    10008duplicate attribute
    10009junk after document element
    10010illegal parameter entity reference
    10011undefined entity
    10012recursive entity reference
    10013asynchronous entity
    10014reference to invalid character number
    10015reference to binary entity
    10016reference to external entity in attribute
    10017xml processing instruction not at start of external entity
    10018unknown encoding
    10019encoding specified in XML declaration is incorrect
    10020unclosed CDATA section

    Additional errors are described in the provided file ODBC2XML.ERR

    Obtaining ODBC2XML

    ODBC2XML is implemented as both a 32-bit Windows executable, and as a dynamic-link library (DLL). This version comes with an embedded XML parser (James Clark's expat parser), and requires Windows 95, 98, or Windows NT, and one or more ODBC drivers (not supplied). The evaluation version available on the web limits the number of records output to 10; otherwise it is the same as the registered version.

    The evaluation distribution file is available here: odbc2xml.zip and contains the following files:

    Installation

    To install ODBC2XML:

    To test the installation, you must have the MS Access ODBC Driver (not included). Then:

    The following XML file should be written to the standard output device:

    <?xml version="1.0"?>
    <memo>
    <from>Me</from>
    <to>You</to>
    <re>Products</re>
    <body>
    <p>Here are the products in the sample.mdb database:</p>
    <list>
    <item id='1' name='Expat'>an XML parser</item>
    <item id='2' name='TurboTax'>a tax package</item>
    <item id='3' name='Quicken'>a financial recordkeeping package</item>
    <item id='4' name='OmniMark'>an SGML conversion tool</item>
    </list>
    </body>
    </memo>

    Usage

    To run the command-line version of ODBC2XML, use the following command line syntax:

    odbc2xml dsn [/uid=userid] [/pwd=password] [/c] filename

    where dsn is the ODBC data source name to connect to, userid and password are optional ODBC login information, /c is a special flag to run ODBC2XML in a mode that establishes a separate database connection for each SQL statement, and filename is the name of the input XML template file. The arguments may be optionally quoted using single or double quotes, as in:

    odbc2xml 'long data source name' 'long filename'

    The output file is written to the standard output device and may be redirected using standard Windows file redirection.

    Release Notes

    Known Limitations

    Registration & Licensing

    You may use the trial version of ODBC2XML freely for any period on a single-user machine. You may not distribute copies of the DLL in an application. Registration provides you with the full production version (with no record output limitations), and up to 1 hour of free technical support, and optionally, the ability to redistribute the DLL. The following licenses are available:

    License Price Description
    ODBC2XML single-user license$49.95This license includes use single machine, where a web or network server counts as the single user. Prohibits redistribution of the ODBC2XML DLL.
    ODBC2XML 10-user license$99.95Allows use on up to 10 different machines, including servers. Prohibits redistribution of the ODBC2XML DLL.
    ODBC2XML developer's license$129.95Allows use by any number of developers at a single site, and unlimited redistribution of the ODBC2XML DLL along with applications.

    Sorry, we cannot provide support to unregistered users. Contact us for site licenses and other support plans, and for consulting services.

    You may also register via check or money order. Please include your name, address, phone number, and email contact information when registering:


    Intelligent Systems Research LLC
    5313 N. Kenmore Suite 3A Chicago IL 60640 USA
    773-989-0426 FAX: 928-438-6482
    email:info@intsysr.com
    http://www.intsysr.com

    ODBC2XML and its accompanying documentation are Copyright © 2001-2006 Intelligent Systems Research LLC. All Rights Reserved. Your use of the program indicates acceptance of the License Agreement with Intelligent Systems Research LLC. The Expat parser is distributed under the terms of the Mozilla Public License Version 1.0 (the "License").