xml and sql server

sql server and xml The way to present data in a platform independent manner is through XML. The client-cursor, SQLOLEDB provider, and SQL Server internally all provide XML support.

why databases and xml? There are many reasons for integrating XML format and relational databases Most data is stored in a database for administrative and concurrency reasons XML can be used as a method of data exchange XML has become a popular data representation method XML can represent hierarchical composition of multiple relational tables XML data can be decomposed into multiple relational tables

sql server and xml choices Many facets of SQL Server and XML integration SQL Server 2000 has built-in XML functionality SQLOLEDB 2.6 provider enables XML query dialects and streamed input and output ISAPI application enables HTTP access XML for SQL Server Web Release 1 adds updategrams and XML bulk load SqlClient data provider supports extended XML function XML for SQL Web Release 2 adds client-side XML, XSD support and .NET support of SQLOLEDB 2.6 functionality

xml support in sql server 2000 SQL Server 2000 includes internal XML support SELECTFOR XML queries return the resultset as a stream of XML sp_xml_preparedocument extended stored procedure parses a VARCHAR variable containing XML, returns a document handle OpenXML function uses the document handle to produce a rectangular resultset

select for xml SQL Server 2000 has some XML query capability built in "FOR XML" on the SELECT statement is an extension to Transact-SQL Three different variations of XML output (RAW, AUTO and EXPLICIT) Produces streamed single-column XML as output Element-normal form and XDR schema/DTD inclusion is supported

for xml variations Different "FOR XML" variations server different purposes RAW mode produces one set of <row> elements, even when using JOIN AUTO mode produces hierarchies with separate nested elements for each JOINed table EXPLICIT mode produces arbitrary shapes using UNION queries and special columns and column names

elements, attributes and data values SQL column values can be stored in elements or attributes All data in elements/attributes known as element/attribute normal form XML RAW always produces attribute-normal form AUTO can produce element-normal or attribute-normal form EXPLICIT can produce mixtures of elements and attributes

XML RAW query and output

-- this query: SELECT Customers.CustomerID, Orders.OrderID FROM Customers, Orders WHERE Customers.CustomerID = Orders.CustomerID ORDER BY Customers.CustomerID FOR XML RAW -- produces this XML output document fragment <row CustomerID="ALFKI" OrderID="10643" /> <row CustomerID="ALFKI" OrderID="10692" /> <row CustomerID="ALFKI" OrderID="10703" /> <row CustomerID="ALFKI" OrderID="10835" /> <row CustomerID="ANATR" OrderID="10308" />

XML AUTO query and output

-- this query: SELECT Customers.CustomerID, Orders.OrderID FROM Customers, Orders WHERE Customers.CustomerID = Orders.CustomerID ORDER BY Customers.CustomerID FOR XML AUTO -- produces the following XML document fragment <Customers CustomerID="ALFKI"> <Orders OrderID="10643" /> <Orders OrderID="10692" /> <Orders OrderID="10702" /> <Orders OrderID="10835" /> </Customers> <Customers CustomerID="ANATR"> <Orders OrderID="10308" /> </Customers>

10

FOR XML Explicit format

-- this query: SELECT 1 as Tag, NULL as Parent, Customers.CustomerID as [Customer!1!CustomerID], NULL as [Order!2!OrderID] FROM Customers UNION ALL SELECT 2, 1, Customers.CustomerID, Orders.OrderID FROM Customers, Orders WHERE Customers.CustomerID = Orders.CustomerID ORDER BY [Customer!1!CustomerID] FOR XML EXPLICIT -- produces this output document fragment <Customer CustomerID="ALFKI"> <Order OrderID="10643"/> <Order OrderID="10692"/> <Order OrderID="10702"/> </Customer>

11

storing xml in sql server Stored procedures and UDF can assist decomposition of XML documents System stored procedure parse an XML document into a "DOM handle" OpenXML function produces a rectangular resultset from XML Nodesets of the XML document are mapped to columns with XPath mappings Can use INSERT...SELECT or UPDATE/DELETE to insert document nodesets

12

Using OpenXML and column patterns declare @idoc int declare @doc varchar(1000) set @doc ='<ROOT> <Customers CustomerID="VINET" ContactName="Paul Henriot"> <Orders CustomerID="VINET" EmployeeID="5"> <OrderDetails OrderID="10248" ProductID="11" /> <OrderDetails OrderID="10248" ProductID="42" /> </Orders> </Customers> <Customers CustomerID="LILAS" ContactName="Carlos Gonzlez"> <Orders CustomerID="LILAS" EmployeeID="3"> <OrderDetails OrderID="10283" ProductID="72" Quantity="3"/> </Orders> </Customers> </ROOT>' exec sp_xml_preparedocument @idoc OUTPUT, @doc SELECT * FROM OpenXML (@idoc, '/ROOT/Customers/Orders/OrderDetails',2) WITH (CustomerID varchar(10) '../@CustomerID', ProdID int '@ProductID', Qty int '@Quantity') exec sp_xml_removedocument @idoc

13

Decomposition using OpenXML

DECLARE @h int DECLARE @xmldoc varchar(1000) set @xmldoc = '<root> <stores stor_id="8888" stor_name="Bob''s Books" stor_address="111 Somewhere" city="Portland" state="OR" zip="97225"> <discounts discounttype="A Discount" stor_id="8888" discount="50.00"/> </stores> </root>' EXEC sp_xml_preparedocument @h OUTPUT, @xmldoc INSERT INTO stores SELECT * FROM OpenXML(@h,'/root/stores') WITH stores INSERT INTO discounts SELECT * FROM OpenXML(@h,'/root/stores/discounts') WITH discounts EXEC sp_xml_removedocument @h

14

sqloledb and xml SQLOLEDB provider in OLE DB 2.6 enables XML features XML functionality separated into SQLXMLX.DLL Command coclass supports streamed input and output Extra properties on Command enable stylesheets and database/XML mappings Two new query dialects enable XML format queries MSSQLXML wraps SQL or XPath queries in XML (requires streamed input) XPath queries are permitted as strings

15

MSSQLXML input documents

<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <sql:query> SELECT Customers.CustomerID, Orders.OrderID FROM Customers, Orders WHERE Customers.CustomerID = Orders.CustomerID ORDER BY Customers.CustomerID FOR XML RAW </sql:query> </ROOT> <ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <sql:xpath-query mapping-schema="rtest.xdr"> /Customers[@CustomerID="ALFKI"] </sql:xpath-query> </ROOT>

16

using xpath queries Support of XML XPath query language requires two pieces Mapping schema maps tables, columns, etc. to a "virtual XML document" SQLOLEDB 2.6 mapping schemas are XDR; WebRelease 2 adds support for XSD XPath query is applied to virtual document to produce output Templated and string based XPath queries supported Internally, XPath + Mapping Schema is translated to "SELECT FOR XML EXPLICIT" SQL query by SQLOLEDB

17

An XDR mapping schema <Schema xmlns="urn:schemas-microsoft-com:xml-data" xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <ElementType name="discounts" sql:relation="discounts" > <AttributeType name="discounttype" /> <AttributeType name="stor_id" /> <AttributeType name="discount" /> <attribute type="discounttype" sql:field="discounttype" /> <attribute type="stor_id" sql:field="stor_id" /> <attribute type="discount" sql:field="discount" /> </ElementType> <ElementType name="stores" sql:relation="stores" > <AttributeType name="stor_id" /> <AttributeType name="stor_name" /> <attribute type="stor_id" sql:field="stor_id" /> <attribute type="stor_name" sql:field="stor_name" /> <element type="discounts" > <sql:relationship key-relation="stores key="stor_id" foreign-relation="discounts foreign-key="stor_id" /> </element> </ElementType> </Schema>

18

Using an XML mapping schema (ADO)

Sub Dim Dim Dim

DoXPath(xmlDoc as MSXML.DOMDocument) conn as New ADODB.Connection cmd as New ADODB.Command stmOut as New ADODB.Stream

conn.Open "provider=sqloledb;uid=sa;initial catalog=pubs" Set cmd.ActiveConnection = conn stmOut.Open ' xmlDoc contains XPath query from next slide Set cmd.CommandStream = xmlDoc ' Set XPath dialect, output stream and mappings cmd.Dialect = "{ec2a4293-e898-11d2-b1b7-00c04f680c56}" cmd.Properties("Output Stream") = stmOut cmd.Properties("Base path") = "c:\schemas" cmd.Properties("Mapping schema") = "mappings.xdr" ' Return XML result in output stream cmd.Execute , , adExecuteStream End Sub

19

Using an XML mapping schema - XPath query and results

-- query input <ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"> <sql:xpath-query mapping-schema="rtest.xdr"> /stores </sql:xpath-query> </ROOT> -- returned document <root> <stores stor_id="6380" stor_name="Eric the Read Books" /> <stores stor_id="7066" stor_name="Barnum's" /> <stores stor_id="7067" stor_name="News & Brews" /> <stores stor_id="7131 stor_name="Doc-U-Mat: Quality" /> <stores stor_id="7896" stor_name="Fricative Bookshop" /> <stores stor_id="8042" stor_name="Bookbeat"> <discounts discounttype=Usual" stor_id="8042" discount="5.00" /> </stores> </root>

20

sqlxmloledb provider Web Release 2 includes new OLE DB provider SQL Server 2000 handles "FOR XML" inside SQL Server; SQLXMLOLEDB provider exposes analogous functionality on client Preprocesses "FOR XML" queries so only raw SQL is submitted to SQL Server Post-processes SQL resultset in XML format on client SQLXMLOLEDB is an OLE DB service provider - currently only supports SQLOLEDB as underlying provider Enables SQL Server 2000-like support for SQL Server 7

21

iis support for database driven xml Any version of SQL Server can directly export XML through IIS SQL Server 2000 uses native "FOR XML" clause and SQLOLEDB provider HTTP GET and POST supported Query templates enable fixed and parameterized queries XPath and SQL supported Transactional XML Updategrams in WR1 Client-side XML is supported in WR2 through SQLXMLOLEDB provider Client-side XML enables using "FOR XML" queries with SQL Server 7 or 2000

22

Using FOR XML and SQL Server 2000

Submit through SQLOLEDB

TDS SQL Server XML

Client HTTP

SQLISAPI.DLL
Add stylesheet

23

Using SQLXMLOLEDB provider

Format SQL from XML Submit by SQLXMLOLEDB

Client HTTP

SQLISAPI.DLL TDS
Format output XML Add stylesheet, DTD, schema

SQL Server

24

updategrams Updategrams use XML format to update SQL Server Supported in Web Release 1 through ISAPI program Insert/Update/Delete consist of XML-format before and after images SQLOLEDB converts these XML format to SQL Batching of updates and transactions are supported Web Release 2 supports .NET Diffgrams

25

Updategram in element-centric mode

<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"> <updg:sync > <updg:before> </updg:before> <updg:after> <Employees> <FirstName>Nancy</FirstName> <LastName>Davolio</LastName> </Employees> </updg:after> </updg:sync> </ROOT>

26

Updategram in mixed mode

<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"> <updg:sync > <updg:before> </updg:before> <updg:after> <Employees FirstName="Nancy" > <LastName>Davoliio</LastName> </Employees> </updg:after> </updg:sync> </ROOT>

27

xml bulk loader Web Release 1 includes XML bulk load facility OpenXML requires too much memory for bulk load of large XML documents Bulk load exposed through separate COM object Bulk load analogous to import through DTS or BCP

28

XML Bulk Load through the bulk load class

set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad") objBL.ConnectionString = "provider=SQLOLEDB;uid=sa;pwd=;database=pubs" objBL.ErrorLogFile = "c:\error.log" objBL.Execute "c:\SampleSchema.xml", "c:\SampleData.xml" set objBL=Nothing

29

sqlclient, datasets and diffgrams Three integration points in native .NET libraries SQLISAPI program can be used directly as a web endpoint SqlClient data provider exposes "FOR XML" results using SqlCommand.ExecuteXmlReader Resulting XmlReader can be used for DataSet input producing multiple DataTables DataSets can export two variations of Diffgram format Entire DataSet and changes Only changed rows

30

Using SQLXML through a ExecuteXmlReader

SqlCommand cmd = new SqlCommand( "select * from authors for xml auto", new SqlConnection( "server=localhost;uid=sa;database=pubs")); cmd.Connection.Open(); XmlTextReader rdr; rdr = (XmlTextReader)cmd.ExecuteXmlReader(); DataSet ds = new DataSet(); ds.ReadXml(rdr, XmlReadMode.Fragment);

31

Using SQL Server 2000 ISAPI application

DataSet ds = new DataSet(); XmlTextReader rdr = new XmlTextReader( "http://localhost/northwind/template/modeauto1.xml"); ds.ReadXml(rdr);

32

Producing DiffGrams SqlDataAdapter da = new SqlDataAdapter( "select * from sample", "server=localhost;uid=sa;database=pubs"); DataSet dsAll = new DataSet("all"); DataSet dsChanges = new DataSet("changes"); da.Fill(dsAll, "sample"); DataTable t = dsAll.Tables["sample"]; DataRow r = t.NewRow(); r[0] = 4; r[1] = "Shemp"; t.Rows.Add(r); t.Rows[1]["name"] = "CurleyJoe"; t.Rows[2].Delete(); dsAll.WriteXml("c:\\all.xml", XmlWriteMode.DiffGram); dsChanges = dsAll.GetChanges(); dsChanges.WriteXml("c:\\changes.xml", XmlWriteMode.DiffGram);

33

Microsoft.Data.SqlXml Web Release 2 includes SQLOLEDB-XML functionality SqlXml data provider exposes Command, Parameter and DataAdapter All SQLOLEDB 2.6 and above functionality supported MSSQLXML and XPath queries XSL transforms DataAdapter enables integration with DataSet and DiffGrams

34

Using SqlXmlCommand and XSD mapping schema

void doXPath() { Stream strm; SqlXmlCommand cmd = new SqlXmlCommand( "provider=sqloledb;uid=sa;database=northwind"); cmd.CommandText = "x:Employee[@EID='1']"; cmd.CommandType = SqlXmlCommandType.XPath; cmd.RootTag = "ROOT"; cmd.Namespaces = "xmlns:x='urn:myschema:Employees'"; cmd.SchemaPath = "c:\\xml_mappings\\MySchemaNS.xml"; strm = cmd.ExecuteStream(); StreamReader sw = new StreamReader(strm); Console.WriteLine(sw.ReadToEnd()); }

35

Using SqlXmlCommand and XSD mapping schema

<!-- this mapping schema and previous code --> <schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:mapping-schema" xmlns:emp="urn:myschema:Employees" targetNamespace="urn:myschema:Employees"> <complexType name="EmployeeType"> <attribute name="EID" msdata:field="EmployeeID" type="ID"/> <attribute name="FName" msdata:field="FirstName" type="string"/> <attribute name="LName" msdata:field="LastName"/> </complexType> <element name="Employee" type="emp:EmployeeType" msdata:relation="Employees"/> </schema> <!-- produces this output --> <?xml version="1.0" encoding="utf-8" ?> <ROOT> <y0:Employee xmlns:y0="urn:myschema:Employees" LName="Davolio" EID="1" FName="Nancy"/> </ROOT> 36

Using XSLT and SqlXmlCommand output

Stream strm; SqlXmlCommand cmd = new SqlXmlCommand(NorthwindConnString); cmd.CommandText = "select FirstName, LastName from Employees For XML Auto"; cmd.RootTag = "root"; // get output Stream strm = cmd.ExecuteStream(); // line up transform and document XmlTextReader reader = new XmlTextReader(strm); XPathDocument xd = new XPathDocument(reader, XmlSpace.Preserve); XslTransform xslt = new XslTransform(); xslt.Load("c:\\xml_mappings\\MyXSL.xsl", null); XmlTextWriter writer = new XmlTextWriter( "c:\\xml_mappings\\xslt_output.html", System.Text.Encoding.UTF8); // transform stream output xslt.Transform(xd, null, writer);

37

summary SQL Server 2000 includes native XML support SQLOLEDB provider contains additional support ISAPI DLL exposes SQL/XML as HTTP endpoints Updategrams and Bulk Load added in Web Release 1 Native .NET support in Web Release 2

38