Gibraltar

ODBC Integration

Note: Generally, this project is not being discusseded under NDA with any accounts or third parties.
Should you require permission to discuss this with a particular organization outside the company,
please contact the author.

Disclaimer: Author is a database-idiot.

Revision: 0.3
Date: May 30, 1995
Author(s): J. Allard (JAllard)
Document: httpodbc.doc
1 Overview
The goals of the Gibraltar effort include

1. establishing the NT/Internet association

2. competing effectively with Unix as an Internet publishing solution
3. offering a transport-indepdent application gateway for organization Internet access
4. shiping a BackOffice opportunity in every box

The BackOffice family offers a unique opportunity to leverage state-of-the-art products and to provide
some unique solutions in this visible server market segment. In viewing the wide array of possibilities, and
balancing our time-to-market requirement, we believe that our most leveraged opportunity is to integrate
our World-Wide-Web server efforts with SQL Server. Why SQL?

· Microsoft SQL Server represents a competitive advantage - customers have committed large volumes
of data to SQL “stores” of data
· Tight integration with SQL encourages customers to further commit to SQL in terms of $$$ and data
storage

Many “early adopters” of the Web are doing some fairly sophisticated SQL integration of their own with a
messy combination of C code, Perl scripts and shell scripts on Unix systems. Not only is this scheme not
performant, but it is generally not accessible to the average organization interested in experimenting with
the Web to reach new audiences - this approach almost certainly requires VAR assistance. Creating a
complete general-purpose SQL gateway for the target Internet Server customer is a complex endeavor
involving many components. We have chosen in the initial release to limit the scope of our problem to
simplify implementation, although this design will permit enhancements in the future. Although we will
leverage the ODBC architecture, our plan is to officially support Microsoft ODBC datasources only.

Goals
· Provide a flexible bridge between the Gibraltar Web server and Microsoft ODBC datasources
· Simple configuration/authoring model to permit future query/report tool integration (e.g., Access,
Query, Word)
· Support static, dynamic and mixed queries through GET and POST actions.

Non-goals
· Graphical query authoring
· Graphical report authoring
· Support for non-Microsoft datasources
2 Implementation

3 Basic Query Syntax (.wdg files)

The “query” file used by the httpodbc.dll is an ASCII file saved with a .wdg (web/database gateway)
extension on the server. The syntax of this file is straightforward (bold indicates required fields):
Datasource: <local ODBC System datasource>
Username: <username>
Password: <password>
MaxFieldSize: <maxsize/record in bytes>
MaxRecords: <maxrecords>
MaxConcurrentQueries: <maxqueries> (not currently implemented)
DefaultParameters: <CSV array of Parameter/Value pairs>
RequiredParameters: <CSV array of Parameters required for successful query>
Content-Type: <mime type for the results of this query>
Expires: <time, in seconds, to cache the results of this query>
Template: <fully qualified or relative path to template file or “Direct”>
SQLStatement:
+<SQL statement line 1>
+<SQL statement line 2>

The ordering and case of the fields in this file is not relevent.

An simple .wdg might look like:

Datasource: NTRaid
Username: jallard
Password:
MaxFieldSize: 2048
MaxRecords: 100
MaxConcurrentQueries: 10
DefaultParameters: Table=bugs
RequiredParameters: Assign
Template: bugtemp.htx
SQLStatement:
+SELECT * FROM
+%Table%
+WHERE (Assign=‘%Assign%’)

In this example:
NTRaid represents an ODBC System datasource as defined by the administrator using the ODBC-32
control panel applet.

jallard represents the user context that httpodbc.dll will use to connect to the datasource. In this
particular case, the user jallard has no password, and could choose to omit the Password: field altogether.
We will recommend to administrators to setup specific user accounts for use with httpodbc.dll to limit the
scope of damage one might be capable of if the .wdg file is acquired from the server. Related to this,
hidden files cannot be enumerated or transferred by any of the Gibraltar services.

2048 represents the buffer space (per record) that httpodbc.dll will allocate. The default value is 8192. We
will recommend to admins to tune this parameter to minimize overhead/query on small recordsets.
Records that exceed this value will be truncated.

100 is the total number of records which will be read/returned from httpodbc.dll on any particular query.

10 permits the administrator to “throttle” the load (to some degree) of their server by maximizing the
number of outstanding queries that the server can be processing. The default is 20.

Table=bugs assigns a default value of “bugs” to the variable %Table% referenced in the SQLStatement: at
the bottom of the .wdg. Any variable/value pairs specified on this line will be overridden by values in the
GET or POST action from the client. Note the use of Parameters is not limited to the SQLStatement.
They may appear anywhere in the .wdg file.

Placing Assign in the RequiredParameters: field requires the client to pass an assignment for the variable
Assign in it’s GET or POST action.

bugtemp.htx specifies the HTML tempate that the query’s result set will be merged through. (This permits
multiple .wdg’s to share a common HTML template). The path can be fully qualified or relative to the
location of the .wdg file.

The SELECT... statement is a standard SQL (or Access) statement which represents the query. If the SQL
statement spans multiple lines, the beginning of each new line must begin with a ‘+’. The variables
%Table% and %Assign% are passed to the server via a GET or POST action and merged by httpodbc.dll
into the query. Any variables passed from the client can be referenced anywhere within the .wdg file (for
example, in the Datasource field), they are not limited to the SQLStatement: only.

4 Basic Report Syntax (.htx files)

The .htx extension by convention specifies an HTML derived file which requires additional processing.
httpodbc.dll recognizes this extension to specify HTML merge templates referenced in .wdg files in the
Template: section. .htx files introduce the following new tags:

<%BeginDetail%>: Specifies the beginning of a result set detail

<%EndDetail%>: Specifies the end of a result set detail

<%if <v1> <op> <v2>%>: Allows for conditional inclusion of markup text
<%else%>
<%endif%>
There are four operators defined:
GT - TRUE if v1 is greater then V2
LT - TRUE if v1 is less then v2
EQ - TRUE if v1 is equal to v2
CONTAINS - TRUE if the string v1 is contained in v2

v1 and v2 can be a positive integer constant, a quoted string, a database row value or a CGI type
environment variable. All strings are case insensitive. Below are valid constructs:

<%if 5 GT 3%> ...

<%if wdg.Assign EQ “JohnL”%> ...
<%if HTTP_USER_AGENT contains “Mozilla”%> ...
<%if ColumnName EQ “1”%> ...

<%TotalRecords%>: Specifies the total number of records returned in the query

<%MaxRecords%>: Contains the MaxRecords value from the .wdg file.

<%column_value%:trunc.width>: Replaced by the value of column column_value from the .wdg

statement. The optional parameter trunc specifies the width which column_value will be truncated,
width (also optional) specifies the total width of the replacement (e.g. width-trunc specifies the
number of spaces added to a truncated field). The truncation and width parameters are currently not
supported.

<%wdg.[parameter from .wdg file]%>: Allows for the inclusion of parameters used in the .wdg file,
such as <%wdg.Assign%>. The “wdg.” scope operator is required to discriminate the variable from
possible conflicts with column names.

A simple .htm file looks like:

<title>NT Raid bugs assigned to <%wdg.Assign%></title>
 <body>
<H2>Bugs assigned to <%wdg.Assign%>:</H2><p>
Total bugs: <b><%TotalRecords%></b>
<hr>
<%BeginDetail%>
<b><%BugNum:6.8></b> priority: <%Priority%> <%Assign%:8.12> <%Title%><p>
<%if Priority LT “3”%>
Better hop on this one!
<%else%>
Ignore this bug
<%endif%>
<%EndDetail%>
<hr>
</body>

Conditionals can compile portions of the merged output. There are

5 Multi-statement Queries and Reports
To permit the ability to combine multiple Statements: in a single query, array notation is permitted in
.wdg and .htx files. The extension is straightforward (and optional for single-statement queries). In the
.wdg file, the Statement: keyword adopts an index, the .htx then requires indecies for all of its <%_*%>
keywords. For example:
Datasource: NTRaid
Username: jallard
Password:
MaxFieldSize: 2048
MaxRecords: 100
MaxConcurrentQueries: 10
DefaultParameters: Table=bugs
RequiredParameters: Assign
Template: e:\webroot\templates\bugtemp.htx
Statement[1]:
+SELECT * FROM
+%Table%
+WHERE (Assign=‘%Assign%’)
Statement[2]:
+SELECT * FROM
+%Table%
+WHERE (Assign=‘%Assign%’) AND (Priority=1)

The corresponding .htx might look like:

<title>NT Raid bugs assigned to <%wdg.Assign%></title>
<body>
<H2>Bugs assigned to <%wdg.Assign%>:</H2><p>
Total bugs: <b><%RecordCount[1]%></b><p>
Priority 1 bugs: <b><%RecordCount[2]%></b><p>
<hr>
<%BeginDetail[1]%>
<b><%BugNum:6.8></b> priority: <%Priority%> <%Assign%:8.12> <%Title%><p>
<%EndDetail[1]%>
<hr>
<b>Priority 1 bugs:</b>
<%BeginDetail[2]%>
<b><%BugNum:6.8></b> priority: <%Priority%> <%Assign%:8.12> <%Title%><p>
<%EndDetail[2]%>
</body>

Within a <%*Detail%> “block”, standard scoping rules apply to referenced variables. This implies that
conflicting column names will not be overridden. It is also possible to “nest” <%*Detail%> blocks,
although we’re not sure why this is useful. Although version 1.0 of httpodbc.dll does not permit multiple
datasources within a .wdg, the notation we’ve adopted permits this extention in future releases of
httpodbc.dll. Note: this syntax implies that the text “SQLStatement” or “Datasource” may not be the
leading text in a Statement.
6 Referencing Queries from .htm Files
The simplies way to invoke a .wdg file is from an anchor. The anchor may optionally specify the path to
the httpodbc.dll, this permits extensions other than .wdg to reference a query. The following references
are equivalent:
.
.
<A HREF=“/scripts/httpodbc.dll/queries/budget.wdg?”>Click here for the budget
summary</A>
<A HREF=“/queries/budget.wdg”>Click here for the budget summary</A>
.
.

The above example will invoke a “static query” - e.g., a query that either depends on values specified by
the DefaultParameters field, or which contains no variable expansion. A “shared static query” - e.g., a
single .wdg file shared by multiple references receives its parameters through the POST action. For
example:
.
.
<A HREF=“/queries/httpodbc.dll/queries/bugs.wdg?Priority=1>Click here for P1
bugs</A>
<A HREF=“/queries/httpodbc.dll/queries/bugs.wdg?Priority=2>Click here for P2
bugs</A>
.
.

In this example, <%Priority%> will be expanded in bugs.wdg to either 1 or 2 depending on the link the
user activates. Multiple values can be passed in using this syntax by using an ampersand as a separator.

7 Referencing Queries from HTML Forms

Queries are activated using HTML forms through the POST method, referencing the query file with the
HTML ACTION keyword. The following example uses the same bugs.wdg as the previous section, but
permits the user to override the default values for the variables: <%Server%>, <%Database%>, and <
%Table%>:
<html>
<body>
<h1>Create your own RAID query</h1>
<hr>
<form METHOD="POST" ACTION="/queries/httpodbc.dll/queries/bugs.wdg">
Server: <input text name="Server" size=36> Ex: ntraid<br>
Database: <input text name="Database" size=36> Ex: ntbug<br>
Table: <input text name="Table" size=36> Ex: bugs<br>
E-Mail Name: <input text name="Assign" size=36> Ex: davec<br>
<P>
<input type="submit" value="Do Query">
</body>
</html>
8 Dynamic Query Generation through Reports
A more complex use of variable expansion creates dynamic queries as part of a report - essentially creating
query(s) by issuing a query. The following example presents a combination of .htm, .wdg, and .htx files
which demonstrates this capability (lines of interest are italicized):

form.htm:
<html>
<body>
<h1>Simple NTRaid Database Query</h1>
<hr>
<form METHOD="POST" ACTION="/queries/httpodbc.dll/queries/bugs.wdg">
Developer’s e-mail name: <input text name="Assign" size=36> Ex: davec
<p>
<input type="submit" value="Do Query">
</body>
</html>

bugs.wdg:
Datasource: NTRaid
Username: jallard
Password:
MaxFieldSize: 2048
MaxRecords: 100
MaxConcurrentQueries: 10
DefaultParameters: Table=bugs
RequiredParameters: Assign
Template: e:\webroot\templates\bugs.htx
SQLStatement:
+SELECT * FROM
+%Table%
+WHERE (Assign=‘%Assign%’)

bugs.htx:
<title>NT Raid bugs assigned to <%wdg.Assign%></title>
<body>
<H2>Bugs assigned to <%wdg.Assign%>:</H2>
<hr>
<%_BeginDetail%><b>
<A HREF="/queries/httpodbc.dll/queries/bugdetail.wdg?BugNum=<%BugNum%>><%BugNum
%></A>
</b>pri<%Priority%> sev<%Severity%> <%Assign%> <%Title%><br>
<%_EndDetail%>

The underlined section of bugs.htx dynamically builds a “static” query within an HTML anchor. This
combination permits the user to do a query against a user, resulting in an array of bugs, with hot links on
the bug numbers. Note that the generated anchor references a different .wdg query, so the user can “drill
into” detail. In NetScape, the user would experience the following sequence:
Display of form.htm. User submits form, activating httpodbc.dll with bugs.wdg. The result set is passed
through bugs.htm which generates the following report:

User activates a link which activates httpodbc.dll with bugdetail.wdg. The bug number is passed as a
variable in the GET action. The following report is generated by passing the results through
bugdetail.htx:
Issues
Do we want to permit the admin to control the timeouts on the ODBC connection and query caches?
How do we handle errors?
- no result set
- too many queries
Are we going to allow simple templates to be included in .wdg files?