October 2006
An Introduction to Archaeological Vector Graphics and SVG
Online Archaeology - MIDAS XML and Google Maps
Introduction
Welcome to newsletter number seven.
In this issue Holly Wright continues a theme from past newsletters on the use of SVG for archaeological drawing publication. One of the most interesting recent developments is the availability of the Google Maps API. Users can easily integrate interactive maps into their websites. In the second article Steve White shows how to integrate MIDAS with Google maps to produce interoperable data sets.
Unfortunately this may be the last newsletter to be sent out in this format. As usual I have had problems in getting enough material together for this newsletter, hence the long delay since the last one went out. I am looking at alternatives to emailing out a newsletter at ever increasing intervals. One possibility would be to put articles directly on my website at www.archweb.co.uk.
Feedback and ideas would be very welcome. Please get in contact with me if you have any thoughts on future format and especially if you have any relevant material that you want published, be it articles, news or information.
Since the last newsletter I have upgraded my website to the Drupal content management system. You should be able to log in to the site using your existing username and password. If you have forgotten them you can request a new one by using the email address this newsletter is sent to. Registered users can add comments to the archived newsletters on the site.
Mark Bell
An Introduction to Archaeological Vector Graphics and SVG
Holly Wright, University of York
Readers of the XML Newsletter will be familiar with Scalable Vector Graphics (SVG) from Mike Rains' discussion in issue four, and this is a further introduction to it's potential use for archaeology. It is based on the article Archaeological Vector Graphics and SVG: A case study from Cricklade featured in the current issue of the online journal Internet Archaeology (http://intarch.ac.uk/journal/issue20/1/index.html), which discusses some of the ways vector graphics are used in archaeology, and outlines the development and features of SVG, which are then demonstrated in the form of a case study.
The full article details the way in which large-scale plan and section drawings originally created on Permatrace were digitised by Guy Hopkinson, for use in Jeremy Haslam's Internet Archaeology publication Excavations at Cricklade, Wiltshire, 1975 (http://intarch.ac.uk/journal/issue14/haslam_index.html). They were designed as an exercise in 'retrospective publication', to illustrate how traditional forms of visual recording might be digitised for online publication. Hopkinson went on to publish his methodology jointly with Internet Archaeology editor, Judith Winters in Problems with Permatrace: a note on digital image publication ( http://intarch.ac.uk/journal/issue14/hopkinson_index.html), and generously made the drawings available for further research using SVG. The goal of the work discussed in Archaeological Vector Graphics and SVG: A case study from Cricklade, was to use this set of images and re-create them online using SVG, while maintaining the same functionality built into them by Hopkinson, as interpreted for Haslam's publication.
The following is a brief, general discussion based on the article, of the importance of vector graphics to archaeology, and the development of SVG.
Archaeology and Vector Graphics
Archaeologists have been using vector-based digital tools for many years (Eiteljorg 1989), and today they are used routinely in many areas of archaeological work. This method of organising visual information using points, lines and polygons is a unique system that can inform the archaeological process. Unlike raster images, which are made up of an array of disparate pixels (Jones 1997), vector images exist within a defined set of coordinates (Eisenberg 2002, 2). The ability of a vector image to 'know where it is' is significant for a spatial discipline like archaeology. Vector graphics are resolution independent, and display the same clarity at any level of magnification (Laaker 2002, 7). Vector graphics can also be used to organise drawings into layers (Watt 2002, xvii), and the pan and zoom features make different forms of analysis possible as well (Eiteljorg 1989). Raster and vector graphics are not competing forms of visualisation; rather they are different formats for different purposes and are sometimes most powerful when used in combination.
A vector image may be created in layers that can be turned on or off, and the user controls the way elements are displayed. The ability to pan or zoom to create different views of an image without loss of resolution may be necessary in order to interpret the image. The image may reference a database and interaction with the database through the image may be the most effective way of accessing the associated data. The image may be generated by a database itself and only exists based on a user-defined query. All of these functions are beyond the capabilities of any static raster image, but are fundamental to much of the vector-based work currently undertaken in archaeology.
The publication of large plan and section drawings has always been problematical in archaeology, both in print and on the Web. Using vector graphics can address some of these problems, but it is important to consider the inherent differences associated with use of this technology as well. For example, a hard copy of a section drawing may be over a metre long, but once digitised into vector format it can be presented at any level of detail or in its entirety without loss of resolution, and the user can choose which areas are of interest. This is consistent with the trend towards publishing more completely in archaeology, and not just synthesising conclusions. (Clarke et al. 2003, 402; Livingood 1996). Although only one of many decisions archaeologists make during the analytical process, by using vector graphics in this way an author or editor would not have to pick and choose which details of an image to use, and so a layer of subjectivity may be removed.
Use of layers in archaeological vector drawings can be very helpful as an aid to interpretation as well. They may help simplify a complex drawing, or highlight areas that may be of particular interest, without having to choose a detail (Judith Winters pers. comm. July 2003). Because layers can be turned on or off, the user can decide which is most important, and in what combination (Eiteljorg 1989). This again demonstrates a way that vector graphics can give more control to the user. Layers can show how a site or section is organised or how it has changed over time; but it can also illustrate the process that went into creating a drawing. In particular, this type of visualisation can be used as an important tool to help identify and communicate uncertainty in archaeological data, and provide alternative interpretations (Ryan 2002).
The Development of SVG
Scalable Vector Graphics (SVG) is something entirely new. Far from being an inexpensive substitute for Flash, SVG was created to describe vector graphics in XML, and therefore belongs to the growing family of open-source World Wide Web Consortium (W3C) technologies that are actively shaping the way information is presented on the Web (Winter and Neumann 2003). As with any other discipline, those working in archaeology should use and promote solutions that consider the larger issues in Web development, and SVG falls into this category. To understand SVG and how it fits with other W3C recommendations, it is important to explore how SVG was created.
SVG was developed to address the lack of an alternative to the raster images that dominate the Web (Watt 2002, xvii). Raster is the native, and therefore most appropriate format for images such as photographs, but even with file compression, the most common image file formats, .gif and .jpg, creates larger files that are slow to load, with limited image quality. XML is an entirely text-based language that creates comparatively small file sizes, so calling on its extensible nature, developers looked for a way to develop a graphics format that could be text based as well. For example, SVG is written in XML, so the SVG markup for a circle is simply <circle></circle> which makes it perfectly understandable to humans. A circle, marked up in SVG, with a radius of 50 pixels and a black outline of three pixels looks like:
<circle r="50" style="stroke-width: 3: black: fill: none;"/ >
Because vector graphics render from text, they can be recalculated using different variables each time they are loaded (Watt 2002, xvi). By changing the radius from r="50" to r="100", the size of the circle will become larger, but it will retain all of its other characteristics. This allows users to pan and zoom around an image at any magnification without loss of image quality, because the vector image just re-calculates each time. This ability is completely basic to the workings of a vector graphic, but impossible for a raster image.
The commercial sector was quick to see the potential for defining a graphic specification for XML. Adobe and Microsoft both submitted proposals to the W3C in 1998, the same year XML became an official recommendation. Adobe's submission was called Precision Graphics Markup Language (PGML) and Microsoft's was Vector Markup Language (VML) (Story 2000). Macromedia's Flash specification was also released at this time in a non-XML binary format. Macromedia chose to throw their support behind Microsoft's bid against Adobe, their traditional rival. The SVG working group took both specifications under consideration, and chose to merge the two technologies into what would become SVG. Through the working group, a wide variety of vendors were able to contribute to the process of creating the specification. These included AutoDesk, IBM, Netscape, Apple, Sun Microsystems, Xerox, Corel, Visio, Hewlett-Packard and Quark. Despite SVG's subsequent adoption as an official W3C recommendation, Microsoft continued to develop VML. The result of this is a browser plug-in that is only compatible with Microsoft Internet Explorer 5 for Windows (Cagle 2002, 10).
The development of SVG also set historic precedents for the W3C. As stated by Chris Lilley, chair of both W3C SVG working groups, SVG was
'a demonstration of the process that enables competing companies to come together in a vendor-neutral space and work on commonly agreed, open specifications for the benefit of the Web in general and to grow the market. SVG was the first specification to not only have a test suite, but also publish the results of testing on named implementations. During the Candidate Recommendation phase, implementers and content creators gave a large amount of valuable feedback that helped to improve the clarity and technical accuracy of the specification. As a result, compared to other specifications at an equivalent level of maturity, SVG was extremely well implemented by the time it became a W3C Recommendation on September 4, 2001.' (Watt et al. 2003, xxiv).
The initial reasons for creating SVG may seem quite modest, but its XML heritage has resulted in a tool that is powerful and full of possibilities. As such, it is important to remember that some of its most compelling features, such as platform independence and interoperability with other XML technologies, are not exclusive to SVG, they are just part of membership in the XML family.
One of the primary complaints about Flash is that documents are created in the proprietary .swf file format, which is binary and therefore not meant to be read or accessed by people (Laaker 2002, 13). Users and developers can view SVG, and the contents are understandable to humans and not just computers (Jackson 2002). SVG allows selective display of elements in an image. Because vector graphics can be created in layers that can be turned on and off, those layers can be preserved in SVG, and interacted with using a scripting language like JavaScript (or more accurately, ECMAScript) (Watt et al. 2003, 17). The text that makes up an SVG graphic is also available to search engines able to read XML, so if a vector graphic contains text it is still recognisable as such even if it is embedded into an image (Watt 2002, 95).
This ability to recognise the textual parts of an SVG image also allows for internationalisation of its content. If the settings in a browser are set for a particular language, the SVG image will display the same graphical elements, but the textual elements that are appropriate for that language will be chosen and displayed (Watt et al. 2003, 19). Browser detection and SVG will also allow for greater flexibility in the future for designing accessible websites. SVG's resolution independence already means users with visual limitations can scale images to a level of magnification that is comfortable. For users that require images with different colour contrasts or text only, SVG should be able to serve a version of the same page to meet their needs. Rather than designing Websites for an accessible lowest common denominator, developers could create one site that can be viewed (or not, in the case of audio browsers) in a variety of ways. This will be somewhat in the future, however, since accessible SVG browsers and plug-ins will have to be created first (Watt et al. 2003, 510). SVG can also create graphics that are data-driven and generated dynamically from a server. This can be done in a variety of ways using existing programming languages like PHP, PERL, ASP or JSP, but the result is a visualisation that can be created 'on the fly' based on user criteria (Watt et al. 2003, 695-98).
While XML is a rapidly growing specification, all web browsers do not natively support it, but this is changing. The most popular browsers that do offer such support are Mozilla's Firefox 1.5. (anon. 2006a), and Opera 8 (anon. 2006b). These are both available for a wide variety of operating systems, including the most popular: Windows, Macintosh and Linux. Testing of these two browsers using Windows XP and Apple OSX revealed quite uneven implementation using Firefox, while Opera was much more solid. Because SVG is non-proprietary there are several companies and organisations with an interest in furthering its development, and they have added SVG support to their products or created plug-ins for other browsers. Adobe's SVG Viewer plug-in is the most widely used, with the broadest distribution and compatibility with all major browsers or platforms.
This short introduction can only touch on the power and versatility of SVG. It is important to note, however, that although designed for the Web, XML has also been quickly adopted for other uses. As a non-proprietary way to handle data-intensive tasks, XML can be shared across platforms and programs, and has been taken up enthusiastically by developers working in areas that may have nothing to do with the Web. This will almost certainly be the case for SVG as well.
References
anon. (2006a) 'Mozilla SVG Project Website', http://www.mozilla.org/projects/svg/.
anon. (2006b) 'Opera Website: SVG Features', http://www.opera.com/features/svg/index.dml.
Cagle, K (2002) SVG Programming: The Graphical Web. Berkeley: Apress.
Clarke, A, Fulford, M, and Rains, M (2003) Nothing to Hide - Online Database Pubication and the Silchester Town Life Project. IN Doerr, M, and Sarris, A (eds) Proceedings of the 29th CAA conference held at Heraklion Crete, Greece, April 2002. Hellenic Ministry of Culture.
Eisenberg, J D (2002) SVG Essentials. Sebastepol: O'Reilly and Associates, Inc.
Eiteljorg II, H (1989) 'Computer-Assisted Drafting and Design: New Techniques for Old Problems', http://csanet.org/inftech/cadbklt.html.
Jackson, D (2002) 'SVG On the Rise', http://www.oreillynet.com/pub/a/javascript/2002/06/06/svg_future.html.
Jones, S J (1997) 'Raster And Vector Images - An Important Distinction', http://csanet.org/newsletter/spring97/nl059707.html.
Laaker, M (2002) Sams Teach Yourself SVG in 24 Hours. Indianapolis: Sams Publishing.
Livingood, P (1996) 'Electronic Publication in Archaeology', http://www-personal.umich.edu/~patrickl/sthesis_2.htm.
Ryan, N (2002) 'Documenting and Validating Virtual Archaeology', http://www.cs.kent.ac.uk/people/staff/nsr/cvro.
Story, D (2000) 'Extensible Graphics With SVG', http://www.oreillynet.com/pub/a/network/2000/04/28/feature/svg.html.
Watt, A (2002) Designing SVG Web Graphics. Indianapolis: New Riders Publishing.
Watt, A, Lilley, C, Ayers, D J, George, R, Wenz, C, Hauser, T, Lindsey, K, and Gustavsson, N (2003) SVG Unleashed. Indianapolis: Sams Publishing.
Winter, A M, and Neumann, A (2003) 'Vector-based Web Cartography: Enabler SVG', http://www.carto.net/papers/svg/index_e.shtml.
Online Archaeology – MIDAS XML and Google Maps
By Steve White
webmaster@online-archaeology.co.uk
http://www.online-archaeology.co.uk/GoogleMap
Objectives
This application attempted to use the freely available Google Maps API as a neutral interface for displaying archaeological data. In order to ensure that the application was interoperable with other heritage applications data should be both exported and imported using the MIDAS XML schema.
Technology
Database: SQL Server 2000
Web Programming: VB.NET, Javascript, AJAX, CSS
Thesauri
The application uses the full English Heritage Thesauri dataset.
Database
The database schema is outlined below. Individual records (Items) are all related to a single Category. The Category table then needed to be linked to the relevant Thesauri Term and a flag field to show that a relationship exists.
Since the application will also display data from the database that is not part of any thesaurus, such as the latest archaeological digs, or museums with archaeology, the IsFISH (bit field) flag determines whether the Category is capable of being exported to MIDAS XML.
GM_Category
This holds all the Categories. Categories have a relationship to a ParentCategoryID because TreeView is used to display the Categories. The IsFISH bit field can be seen as well as the THE_TE_UID field for linking a Category to a Term.
|
ColumnName |
DataType |
Length |
Allow Nulls |
|
CategoryID |
INT |
4 |
No |
|
CategoryName |
NVARCHAR |
255 |
No |
|
ItemCount |
INT |
4 |
No |
|
ParentCategoryID |
INT |
4 |
No |
|
CategoryDescription |
NVARCHAR |
255 |
Yes |
|
Moderated |
BIT |
1 |
No |
|
DateAdded |
DATETIME |
8 |
No |
|
DateModerated |
DATETIME |
8 |
Yes |
|
AddedBy |
NVARCHAR |
150 |
Yes |
|
ModeratedBy |
NVARCHAR |
150 |
Yes |
|
IsActive |
BIT |
1 |
No |
|
MarkerColour |
VARCHAR |
10 |
Yes |
|
IsFISH |
BIT |
1 |
No |
|
THE_TE_UID |
INT |
4 |
No |
GM_EH_Terms
This holds the Terms from all EH Thesauri.
|
ColumnName |
DataType |
Length |
Allow Nulls |
|
THE_TE_UID |
INT |
4 |
No |
|
TERM |
VARCHAR |
50 |
No |
|
INDEX_TERM |
VARCHAR |
1 |
No |
|
SCOPE_NOTE |
VARCHAR |
255 |
Yes |
|
CLA_GR_UID |
INT |
4 |
No |
|
STATUS |
VARCHAR |
1 |
No |
|
IMAGE_EXISTS |
VARCHAR |
1 |
Yes |
Exporting data to the MIDAS XML standard
In the admin section of the application a single Category is selected and the IsFISH field can be edited. In order to edit, the relevant Thesaurus is first chosen from a drop-down list.
The selected drop-down value is then passed to an AJAX TreeView. Then the Term can be selected by clicking a node in the TreeView. The DataKey for each node is THE_TE_UID and once a node has been selected in the TreeView, clicking ‘Update’ will set the IsFISH bit field to 1 and update the THE_TE_UID in the GM_Category table.
Now we have a relationship in the database:
GM_Category.THE_TE_UID = GM_EH_Terms.THE_TE_UID
To create the XML file for the Category, clicking ‘Create FISH XML’ will build an XML file in server memory and save the file to a directory on the server. The file is then instantly available to be downloaded or used by anyone.
The XML file is built by passing the CategoryID to a function like this:
Dim FileName, XMLPath, TType As String
Dim FType As String = ViewState("FISHType").ToString()
‘ FType is the name of the Thesurus selected in the drop-down
XMLPath = "../../XML/"
FileName = Server.MapPath(XMLPath) & lblCatName.Text & ".xml"
Try
' which type of XML do we need
Select Case FType
Case 1 ' MONUMENT TYPE
TType = "eh_tmt2"
oUtil.BuildMonumentXML(CategoryID, FileName, TType)
Case 128 ' MDA OBJECT TYPE
Case 129 ' MAIN BUILDING MATERIALS
Case 143 ' MARITIME CRAFT TYPE
Case 225 ' AIRCRAFT TYPE
Case 365 ' DEFENCE OF BRITAIN
TType = "dob_98"
oUtil.BuildMonumentXML(CategoryID, FileName, TType)
Case 546 ' COMPONENTS
End Select
lblError.Text = "FISH XML has been generated"
Catch ex As Exception
lblError.Text = ex.ToString
End Try
Here we are sending the CategoryID, FileName and TType to a utility function that will do all the work for us.
Public Shared Sub BuildMonumentXML(ByVal CategoryID As Integer, ByVal FileName As String, ByVal TType As String)
Dim oDB As New GoogleMapDB
Dim DT As DataTable = oDB.GetCategoryItems(CategoryID).Tables(0)
‘ fill a DataTable with all Items in the Category
Try
' Use an XmlTextWriter to write the XML data to a string...
Dim sw As New StringWriter
Dim writer As New XmlTextWriter(FileName, Encoding.GetEncoding("ISO-8859-1"))
writer.Formatting = Formatting.Indented
writer.WriteStartDocument()
' write out <monuments>
writer.WriteStartElement("monuments")
writer.WriteAttributeString("xmlns", "http://www.heritage-standards.org/midas/schema/1.0")
writer.WriteAttributeString("xmlns:xsi", "http://www.w3.org/2001/XMLSchema-instance")
writer.WriteAttributeString("xsi:schemaLocation", "http://www.heritage-standards.org/midas/schema/1.0 http://195.74.122.210/~fish/midas/schema/1.0/midas_monument.xsd")
' start <meta>
writer.WriteStartElement("meta")
' start <title>
writer.WriteStartElement("title")
writer.WriteString("Online Archaeology - UK Archaeology Resource data in MIDAS XML format")
writer.WriteEndElement()
…./
End Sub
We use an XmlTextWriter to construct the MIDAS XML file which saves it to the XMLPath.
Importing data to the MIDAS XML standard
To import XML into the system, open the application and click ‘External’.
You can now see the ‘Load external MIDAS XML’ textbox. Paste in a URL of an XML file that uses MIDAS, such as:
http://heritage-standards.org/midas/examples/monument_example.xml
The Javascript function then sends the request to a proxy on the server to handle the request. This ensures that cross-domain XMLHTTP requests can be done via Javascript. The XML is then parsed and data is loaded onto the interface.
Once we have the data in the XMLHTTP response object we need to parse the data.
In Javscript we load the parser depending on the client:
var dom;
if(oBrowser == "FF") {
var parser = new DOMParser();
dom = parser.parseFromString(XmlRequest.responseText, "text/xml");
}
else if(oBrowser == "IE") {
dom = GXml.parse(XmlRequest.responseText);
}
Then we add the MIDAS ‘monuments’ element to a local var:
var monuments = dom.getElementsByTagName("monument");
Next we loop through the monuments collection and process each
record:
for (var j = 0; j < monuments.length; j++) {
var oItem = monuments[j];
// strip out whatever the application needs from MIDAS
}
We can then assign various nodeValues from the individual ‘monument’ elements to local vars in Javascript which we use to populate the GoogleMap with markers and tooltips and InfoWindows.
Since the MIDAS XML schema uses the National Grid Reference as a spatial reference, we need to convert it on the fly in Javascript to Latitude and Longitude, which is how the Google Maps API plots spatial references on the interface.
Once more XML files become available using MIDAS schema it would be a simple process to provide a method to overlay different sources on top of each other and toggle them on/off using Javascript.
This client-side method of retrieving data from MIDAS documents has only been tested on small MIDAS datasets. If the datasets where large then this method would cause the client browser to freeze. For larger datasets it would make more sense therefore to use server-side code to parse the MIDAS XML and return the data back out via the same XMLHTTP client-side call. This could be achieved by doing all of the server-side processing in the server-side page we are using for a proxy call to get the XML from another domain.
The Google Maps API is still in beta, meaning that it has not been formally released as a product. As such, it is still being developed, and new features are still being added weekly.
END OF NEWSLETTER NUMBER 7
This newsletter is © copyright Mark Bell and the individual authors, 2006. Please contact the editor before reproducing material from this newsletter.