This post is the fifth in a series of posts providing a reading
course of the TEI Guidelines. It starts with

1. a basic one on Introducing XML and Markup then
2. on Introduction to the Text Encoding Initiative Guidelines then
3. one on the TEI Default Text Structure then
4. one on the TEI Core Elements then
5. one looking at at The TEI Header.

None of these are really complete in themselves and barely scratch the surface but are offered up as a help should people think them useful. This sixth post is looking at how to represent primary source documents, including transcription, linking transcriptions to facsimiles, and genetic editing. Already in the core module of the TEI a number of elements are defined specifically for encoding primary sources. If you’ve got this far then you’ve already read about those, for example unclear or the choice element and its component parts abbr/expan, sic/corr, orig/reg. Some of these are further supplemented with additional elements if the ‘transcr’ module (the ‘Primary Sources’ chapter) is included in your schema. For example, the addition of am to abbr to record the abbreviation marker and ex inside expan to mark an editorial expansion.   Other elements provided if the ‘transcr’ module is included in the TEI ODD file
that created your schema include:

addSpan am damage damageSpan delSpan ex facsimile fw handNotes handShift
line listTranspose metamark mod  redo restore retrace sourceDoc space subst substJoin
supplied surface surfaceGrp surplus transpose undo zone

Annotating the activities of transcription and the relationship of this transcript with the original source document is at the heart of this chapter. This has several aspects including: more detailed encoding of the act transcription, the creation of digital facsimiles, and recording the writing process.

Transcription

As you already know from reading about it the choice element is a way of encoding multiple transcriptional interpretations at a single point in a text. For example, an
abbreviation with abbr and its expansion with expan, or an apparent error with sic and its editorial correction with corr, or an original reading with orig and a regularised form with reg.
These children of choice are repeatable, so it is possible to encode an abbreviation with multiple possible expansions (hint one could use the @cert attribute to indicate which of these expansions is more certain).

What could be encoded as:

could also be encoded as:

As mentioned above this chapter also adds am to abbr to record the abbreviation marker and ex inside expan to mark an editorial expansion. The abbreviation marker may or may not be present but is the thing in the original text which indicates to you that the word should be interpreted as an abbreviation. In this case, although NATO is more commonly abbreviated as an initialism with no ‘.’ marking the individual letters, here it has these. We could encode this, using ex inside expan to mark the expanded portions of text as:

This form of markup is different from the superficially similar to subst (added in this module) which contains add and del to record additions and deletions. The prime difference is that whereas in choice only one of the child elements is truly transcribing something in the text, with subst both the deletion and addition are present to be transcribed.   There are also elements to record damage to the source, text that has been supplied by the editor, is considered surplus to editorial requirements, or for recording unusual space in the document. There is also a way to note a change of scribal hand, using the handShift element.

Digital Facsimiles

In many scholarly editions the provision of digital images acts a facsimile or surrogate of the original document to such a degree as to enable primary source research without recourse to the original object. Although the TEI stands for the Text Encoding Initiative, it is indeed possible to have a TEI document which does not contain a text element. Inside the TEI, after the teiHeader there must be either a facsimile, sourceDoc, or text element. But you could have a document which at this point only had a facsimile and no transcribed text.   The facsimile element contains images rather than text, and these can appear either directly as graphic elements, or be organised by surface elements for each surface and zone to
specify sections on those surfaces. The surfaceGrp element can be used to group multiple surfaces together (e.g. recto and verso of a folio, or indeed gatherings).
A basic facsimile element may have looked like:

These instead could be grouped as individual surfaces:

or with zones and coordinates:

These can be given as x/y coordinates for the upper left and lower right to draw a rectangular bounding box. The @xml:id attributes in these examples can be pointed to from the page breaks in the transcription of the text (if provided):

Or from any other element, such as a division:

Recording the Writing Process

Linking a transcription to a zone in a facsimile by pointing to it with the @facs attribute is one way to relate the text to images. Another does not prioritise the final text, but the process which was undertaken to create this text. In this case the transcription can be made in the sourceDoc element. When surface, zone and line (for transcription of topographic lines on the document) are used inside sourceDoc they are for transcriptions of the text as they appear as units on the physical document without the semantic interpretation that we find in transcriptions that use the text element. (For example, deciding that they form paragraphs or speeches by particular characters.)

Of course, all of the surfaces, zones and lines can have coordinates on them or use the @points attribute for a series of coordinates for non-rectangular areas.   There are other elements, for use in recording the text which also relate to the process of writing. These include metamark which records any symbol which indicates how it should be read rather than forming part of the content. (For example, an arrow ‘moving’ a paragraph above the one which proceeds it in the document.) A general mod element can also be used to record a modification in the document without the semantic interpretation of some of the other transcriptional elements.

Additional elements are available that help to record the process of writing the document, including the restore element to indicate a deletion that has been marked as reverting to a previous version by cancelling some textual interaction. This is used for comparatively simple cases, whereas the more general undo element can be used to indicate any form of cancellation. If a cancellation is then marked as being reaffirmed or reasserted in some manner then a redo element can be used. There is also a way to record the act of transposition by using a transpose, sometimes gathered in a listTranspose, to point to the elements that are transposed.

A retrace element can be used where writing has been overwritten, usually with the intention of clarifying or fixing the text. This is sometimes a distinct phase in the production of the text. Any distinct stages in the text, such as campaigns of revision or editing phases, can be recorded using the listChange and change elements. These are not provided in this chapter, but the Header, where they are used in revisionDesc to record stages of revisions in the creation of the electronic file. When used in the creation element in the header it instead records phases of development of the text itself.

Questions about Encoding Primary Sources

As usual I’ve got some self-assessment questions for you to test that you’ve read the chapter carefully.

1. What is the difference between a surface and a zone?
2. What are the options for child elements of the TEI element?
3. Can a zone be larger than its parent surface?
4. How can you point from a surface element to a page break rather than the other way around?
5. What do you use to break up textual transcription inside a line element?
6. Think of an example of metamark used in documents your familiar with. How would you encode it?
7. Why might you use a g element inside an am element?
8. What is a substJoin element used for?
9. What is the difference between using damage and unclear with textual content?
10. How is line different from zone? When would you use line?
11. How would you record how large an unexpected space was?
12. Can you think of a reason why recording stages of production of the texts you are interested in
    might benefit your own work?

1. a basic one on Introducing XML and Markup
2. an Introduction to the Text Encoding Initiative Guidelines
3. and one on the TEI Default Text Structure
4. and one on TEI Core Elements

None of these are really complete in themselves and barely scratch the surface but are offered up as a help should people think them useful.

This fifth post is looking at The TEI Header.

The <teiHeader> is an essential part of every TEI file; it is where you record metadata for the digital text you are creating, document what you have done and why, as well as put additional information which may be useful in understanding or interrogating this file.

The <teiHeader>, often just casually referred to as ‘the header’, is in some ways the most important part of your TEI file. Without it we can’t know what the file consists of, what you were trying to do when you created it, what we are allowed to do with it, or anything else about this electronic file. A digital file without proper metadata is only of very limited use. However, the provision of basic metadata need not be an onerous task only completed by well qualified librarians and bibliographers: you too can provide decent metadata for your digital text.

At its very minimal the TEI requires that the header have a <fileDesc> element and that in turn this have child elements for a  <titleStmt> (information about the title of the digital file), a <publicationStmt> (information about the publication of the digital file), and a <sourceDesc> (information about the source of the digital file even if newly created).

As siblings to the <fileDesc> one could also have the elements <encodingDesc> (to store information about the encoding of the digital text), <profileDesc> (a text profile of additional information), or <revisionDesc> (to store information about major revisions).

The <fileDesc> Element

Inside <fileDesc> you can store all sorts of information about the file. The RelaxNG Compact Syntax for this content model (excluding its membership in attribute classes) is:

(titleStmt, editionStmt?, extent?, publicationStmt,
seriesStmt?, notesStmt?), sourceDesc+

This means that there is:

• a required <titleStmt> which allows you to record one or more  <title> (required) and responsibilities such as <author>, <editor>, <funder>, <meeting>, <principal>, <sponsor>, or general purpose <respStmt> followed by
• an optional <editionStmt>, to record information about this digital edition followed by
• an optional <extent> element to give a place for information about size followed by
• a required <publicationStmt> to record necessary information about the publication of the digital file either as prose paragraphs or structured information on the <distributor>, <authority>,  <availability>, <address>, <date>, <publisher>, <pubPlace> or one or more <idno> element. This is followed by
• an optional <seriesStmt> gives a place for relating this digital file to a series of any sort of which it might be a part
• an optional <notesStmt> gives a place for any notes relating to the file not encoded elsewhere
• and after all of this at least one <sourceDesc> is required to record information concerning one or more sources for this electronic file. This can contain either prose paragraphs or more structured information about the bibliographic sources in a variety of formats.

Note that the required elements inside <fileDesc> are <titleStmt> (itself with a required title), <publicationStmt>, and <sourceDesc>.

And that is it! That is all that is required for a valid and useful <teiHeader>.

The <encodingDesc> Element

But of course, sometimes we don’t want to only record the minimal amount of information, we may wish to record other things. As mentioned above after the <fileDesc> we can also have an  <encodingDesc> (to store information about the encoding of the digital text), <profileDesc> (a text profile of additional information), or <revisionDesc> (to store information about major revisions).

The <encodingDesc> element is where one can store information about what decisions were made in the encoding of the text. Like many metadata categories in the TEI this can either be given as prose paragraphs or more structured forms concentrating on the following:

• when the header module (required) is loaded:
  • information about an application which has edited the TEI file: <appInfo>
  • taxonomies defining any classificatory codes used elsewhere in the text: <classDecl>
  • details of editorial principles and practices applied during the encoding of a text: <editorialDecl>
  • a geographic coordinates declaration: <geoDecl>
  • a list of definitions of prefixing schemes used in data.pointer values: <listPrefixDef>
  • a project description: <projectDesc>
  • a declaration specifying how canonical references are constructed for this text: <refsDecl>
  • a description of the rationale and methods used in sampling texts in the creation of a corpus or collection: <samplingDecl>
  • information about the language in which style information used to describe the original object is supplied: <styleDefDecl>
  • detailed information about the tagging applied to a document: <tagsDecl>
• when the gaiji module is loaded:
  • information about nonstandard characters and glyphs: <charDecl>
• when the iso-fs module is loaded:
  • a feature system declaration comprising one or more feature structure declarations: fsdDecl
• when the tagdocs module is loaded:
  • a specification of the schema the document is intended to validate against: <schemaSpec>
• when the textcrit module is loaded:
  • a declaration of the method used to encode text-critical variants: <variantEncoding>
• when the verse module is loaded:
  • a metrical notation declaration: <metDecl>

Of course, these are all optional or instead of using structured elements you can just use the <p> element (or if the linking module is loaded the <ab> element) to provide one or more prose paragraphs.

The <profileDesc> Element

After the <encodingDesc> it is possible to have a <profileDesc> element to record various non-bibliographic aspects of a text. The information recorded again depends on what modules are loaded when creating your schemas. This allows metadata categories including:

• when the header module (required) is loaded:
  • a record of the calendaring system used in the dating elements:  <calendarDesc>
  • information about the creation of a text: <creation>
  • a description of the languages, sublanguages, registers, or dialects, represented within a text: <langUsage>
  • a collection of information describing the nature or topic of a text in terms of a standard classification or keywords scheme: <textClass>
• when the corpus module is loaded:
  • information about identifiable speakers or other participants (of any sort) in the text: <particDesc>
  • a record of the setting(s) within which a language interaction takes place:  <settingDesc>
  • a description of a text in terms of its situational parameters: <textDesc>
• when the transcr module is loaded:
  • documentation of the different hands identified within the source texts: <handNotes>
  • a list of transpositions, each of which is indicated at some point in a document typically by means of metamarks: <listTranspose>

Unlike <encodingDesc> you cannot provide just paragraphs inside <profileDesc>, however, you can do so inside many of its child elements.

The <revisionDesc> Element

The final component of the <teiHeader> is an optional single <revisionDesc> which summarises the the revision history of the file. Inside <revisionDesc> you usually place a series of change elements ordered so the most recent is at the top. The change element has a both dating attributes like @when to provide the date of the change as well as a @who attribute to point to information (such an author, editor, or more general respStmt in the <titleStmt>.

And that is the <teiHeader>!

Ok, there are indeed lots more that can be said about each of those individual grandchildren in the XML hierarchy, and some aspects, such as the description of manuscripts and early printed books (using <msDesc>) even gets a chapter of its very own (Manuscript Description) that I’ll cover in another post. But this is meant to be a series of blog posts as a reading course of the TEI Guidelines. So below are some basic questions you should be able to answer if you’ve read the TEI Header chapter.

Questions About the <teiHeader> Chapter

1. What are the four major components of the <teiHeader>?
2. Inside <titleStmt> inside a <fileDesc> what element would you use to record who transcribed a manuscript?
3. What is the difference between a new edition of your file and a revision of it? How would you document each of these?
4. Where would you put general notes about your text?
5. What element would you use inside <sourceDesc> to provide a manuscript description? What about a script for a spoken text? What about the recordings used to produce a transcription?
6. Inside the <editorialDecl> how do you indicate whether end-of-line hyphenation has been retained in a text?
7. What is the rendition element used to describe? What global attribute do you use to reference it from the text?
8. What elements do you need to construct an arbitrarily-deeply nested taxonomy?
9. If you were writing a computer program which modified a TEI file, where in the <teiHeader> would you store information about how your program had modified the file?
10. How (and where) would you indicate that approximately 80% of a text was in Latin and 20% was in English?
11. How do you provide information about a date that is in a non-Gregorian calendaring system?
12. The TEI Guidelines can not enforce the provision of all possible metadata. What information do you think should be provided as a minimum? What would you include as recommended components of the <teiHeader> for your own project? How might this differ if you aren’t encoding just one document but hundreds or thousands of them?

Encoding Your Own Material

Continue encoding your own material, but this time return to the <teiHeader> and improve it as much as you can. Think about those aspects that might be useful for you to encode to be able to find this text amongst many others; think about those aspects of the text that might be helpful for you to encode for those that wish to study texts like this in large collections through examining their metadata through (semi)automated means. Hopefully but doing so you’ll make better use of the <teiHeader>.

1. a basic one on Introducing XML and Markup
2. an Introduction to the Text Encoding Initiative Guidelines
3. and one on the TEI Default Text Structure

None of these are really complete in themselves and barely scratch the surface but are offered up as a help should people think them useful.

This fourth post is looking at Chapter 3 of the TEI P5 Guidelines: Elements Available in All TEI Documents. This is, of course, a terrible name for this chapter.  It has been called this or something similar for quite a number of versions of the TEI so probably not worth campaigning to change it until a major new version of the TEI is on the cards.  The reason it is a bad name, of course, is that you cannot guarantee that the elements listed in this chapter are available in all TEI Documents. Every use of the TEI is through some form of customisation. Even if someone uses one of the pre-prepared schemas generated by the TEI Consortium, they are all the result of a TEI customisation stored in an TEI ODD file.

Go and read the chapter Elements Available in All TEI Documents and answer the following questions to force yourself to make sure you’ve read it.  While you do so imagine how you might use these common elements in a document you would like to encode.  This will be useful because the assignment following reading the chapter will be to encode a small amount of material from your chosen document.

Elements Available in All TEI Documents: Questions

1. What is the difference between paragraphs, ‘phrase-level’ elements, ‘chunks’  and ‘inter-level’ elements? (Give an example element name of each!) Is this way of describing elements useful?
2. Think about the hyphens (and hyphen like symbols) that occur in the documents you are interested in.  How do they function? Is there a difference in how you might encode them if they are at the end of line?
3. Highlighting is often how texts indicate that a segment of text has a feature or characteristic that is different, in some way, from the surrounding text. Think about the way text you are interested in is highlighted: What does it use colour, special marks, or characters to highlight? List what the text is trying to convey through this highlighting and how you would mark it using TEI elements.
4. How do you mark a bit of Lingua Latina that appears in the middle of some English text?
5. Quotation marks, another form of highlighting, are used to indicate a wide variety of things.  What is the difference between <q>, <said>, <mentioned>, <soCalled>, <quote>, and <cit>? Can you think of instances in the material you are interested in where you might use these?
6. The <term> element is used to mark technical terms. Why might you wish to mark technical terms? Which might be useful to mark in your material?
7. When might the @cert attribute be useful in your encoding?
8. The <choice> element enables you to present two or more conflicting editorial choices at the same time — does this mean that software processing this needs always to choose just one of these? The <choice> element enables us to group: <abbr> (abbreviations) with their <expan>, a <sic> (apparent error) with a <corr> (corrected form), and an <orig> (original form) with a <reg> (regularization). When might it be useful to have multiple <expan>, <corr>, or <reg>? Is there something fundamentally different between abbreviations and expansions compared to the other two sets of elements concerning which is the original?
9. How do you indicate that some material is missing because you cannot read it? What if you want to provide your guess as to what the material is?
10. We’ll skim over looking the <name> element in detail because there is a whole chapter about more detailed names; but, the <name> element has a @type attribute, what types of name occur to you? When there are specialised forms of this, such as <persName> (personal name, which will be introduced in a later blog posting) why might you want to use the simpler <name> element?
11. How would you encode your own address using the more semantically-rich forms rather than <addrLine>?
12. As with names, there are more complex discussions to have about <date> elements in a later post; But using as precise attributes as possible how would you encode the following dates (try it out in oXygen making sure your document remains valid):
    • The date text: “17 March 1999″
    • From 17 March 1999 to April 2013
    • The phrase “the thirteenth century”
    • A single date where you know it did not occur before 1971 and certainly could not have happened after the 1st of January
    • The 17th March when you do not know the year
13. What is the difference between a <ptr/> and a <ref> and why might you prefer one over another?
14. Lists are very ubiquitous in texts of most periods and cultures: How would you encode a list from a text of your choice? When might you encounter nested lists?
15. The <note> element can appear many places: what different types of notes can you envision using if you were encoding a modern edition of your favourite text?
16. The <graphic/> element enables you to point to an image to include at this point. Why might this be a bit limited? (Hint: The <figure> element is defined in chapter 14.)
17. What are milestone elements? What is the main difference between <milestone/> and <pb/>, <gb/>, <cb/>, <lb/>? Can you think of instances when you would use <milestone/>? What might you do if you want to record that a line-break is artificially breaking a word?
18. There are three main forms of bibliographic citation <bibl>, <biblFull>, and <biblStruct>: Why might you choose <bibl> over <biblStruct> (<biblFull> is used a lot less frequently)? What kind of elements are allowed inside them (compare using their reference pages) and how might that inform your decision to use them?  Try to encode the bibliographic reference for an academic journal article of your choice using both <bibl> and <biblStruct> … of these which do you prefer and why?
19. What is the difference between <biblScope> and <citedRange>?
20. How would you mark up this simple piece of drama from Hamlet?
    

    QUEEN GERTRUDE: Came this from Hamlet to her?
    LORD POLONIUS: Good madam, stay awhile; I will be faithful.
    [Reads]
    Doubt thou the stars are fire;
    Doubt that the sun doth move;
    Doubt truth to be a liar;
    But never doubt I love.

Encoding Your Own Material

That is an awful lot of questions above!  Sorry! If you still have time left then try to encode a small amount of material that you are interested in creating a valid TEI XML file. (If you don’t, well, do it next time you get a chance before moving on to the next blog post!) Where appropriate encode:

• The structure of the text including any paragraphs and lists
• Forms of highlighting, colours, or what you feel the highlighting is indicating
• Quotations and citations if they exist
• Notes, both existing, and editorial notes you wish to make
• Expand some abbreviations if there are some using  <choice>, correct any errors
• Mark any page breaks, line-breaks (if not encoding metrical lines), gathering breaks, column breaks, etc.
• In a <p> in the <sourceDesc> element in the header make a note of the date of the material using as precise a date as you can
• The list of elements created by the core module (and chapter) are at the bottom of the chapter; are there features you want to encode which are not covered by these? Make a list of them and think about what chapter may enable you to encode these.
• What other problems or limitations in encoding your text do you find? Are these problems likely to be unique? Try to find a good TEI way of solving them!

Next time, we’ll move on to looking at the <teiHeader> element and how to make better use of it.

In this post we look at the overall basic structure of a TEI File. In many ways this is much more concrete than the infrastructure of the TEI where it is possible to get lost in the differences between TEI ODD files and the schemas generated from them, or modules, model classes, and attribute classes. Instead here we’re looking at the markup that is part of almost every TEI file, its default text structure. Readers may notice that the ‘Default Text Structure’ chapter of the TEI Guidelines comes after two that I’ve skipped: ‘The TEI Header‘ (chapter 2) and the slightly inaccurately named ‘Elements Available in All TEI Documents‘ (chapter 3). Have no fear if you are following this set of blog posts, I will be returning to chapter 3 next and then chapter 2, I just feel it is good to get a sense of a TEI file as a whole before learning about all the core elements and metadata.

A Basic TEI File Structure

A basic TEI file might look like this image below.

In this image the element names are in blue and XML comments (delineated by <! -- comment --> ) are in green.

An XML file always should start with an XML Declaration (here at the top in purple). After that we have a <TEI> element in the TEI Namespace (http://www.tei-c.org/ns/1.0). Inside all <TEI> elements the TEI Guidelines require there to be a <teiHeader> element. In order for this to be a real and valid TEI P5 file, there are some elements which would need to appear inside the <teiHeader> element, but I’ll talk about those in another post.

After the <teiHeader> element you can have one or more optional <facsimile> or <sourceDoc> elements. These are for recording image facsimile information, or for a non-interpretative transcription method sometimes used for creating genetic editions.

After these we have a <text> element. Technically this is optional if you have <facsimile> or <sourceDoc> elements but really for most introductory uses of the TEI it is probably a good idea to have a <text> element. If you do use one it has to come last.

Inside a <text> element you can optionally have a <front> element. This is for containing front matter like titlepages or prefaces, anything that comes before the main body of the text.

The <body> element is required, because whatever text you are creating (whether a transcription of ancient clay tablets, medieval manuscripts, modern web-pages or teaching slides) it will have a body of some sort. Inside <body> you might get divisions (the <div> element) or just paragraphs (the <p> element) or a wide variety of other things. (We’ll talk more about these in a bit.)

The <back> element which follows the <body> element, as with <front>, is optional but is intended for back matter such as indexes, appendices, bibliographies, addenda, etc.

Now one of the things you might notice about this is it brings to bear certain assumptions of the TEI. This default text structure reflects the assumption that most text-bearing objects can be transcribed and editing in a way which resembles something that we might usually associate with a codex-like structure. (e.g. front matter, the main body stuff, then stuff that comes after). Our association of this with an assumed codex structure probably is a bit misplaced. For example, manuscript rolls, for example, often have optional ‘stuff at the top’ then ‘the main body stuff’, then optional ‘stuff at the end’ and many other cultures and methods of writing text on objects also have such systems. People have used the TEI to successfully encode a huge variety of texts from different times and cultures so it is unlikely that this structure will impose too much of a semantic burden on your own use of it.

 The TEI Default Text Structure Chapter

This is a long chapter which covers a lot of ground. It looks at the default text structure of the TEI (that I’ve tried to explain briefly above), and then investigates the kind of things that happen inside the <text> element. This includes looking at the types of divisions available inside the <body>, <front> and <back> elements and the elements available inside these divisions. It includes ways of encoding groups of texts (such as anthologies and collections), virtual divisions that can be automatically generated such as tables of contents. It also looks at the <front> element, title pages, and the <back> element.

Read this chapter and in order to make sure you have, answer these questions:

• How might you decide whether a text is unitary or composite?
• Personally I have a strong preference for almost always using un-numbered divisions <div> rather than numbered ones <div1>. In what circumstances might numbered ones be more appropriate to use?
• Why does the TEI not use numbered headings (c.f. HTML where there are elements <h1>, <h2>, <h3>, etc.) but just a <head> element?
• If you were digitising my love letters (who knows why?!), how would you mark up the closing bit at the end of a letter where I say:

With love and cuddles,
James
xxx

• When would you use <group> element rather than have separate TEI files?
• What is a <floatingText> element used to indicate? Try to think of examples from your own area of work?
• Do the texts you work with have front matter that you would encode in the <front> element? How would you encode it? How do you decide to encode something as front matter rather than as the body of the file?
• On a title page how would you encode a title that has several parts to it?
• Are there differences between what is allowed in <front> and what is allowed in <back>? Why is this the case?

Try it out

I always think, if possible, it is good to have practical exercises to reinforce things you have learned. If you have time try this:

• Start up the oXygen editor
• Create a new document by going to File ? New and double-click to expand ‘Framework templates’ scroll down inside it and do the same to open ‘TEI P5′. Inside this select ‘All’, and click on ‘Create’ to open a new document.
• Ignoring the schema declarations at the top you should get a file which looks something like this:

• Assuming you’ve not turned off automatic document checking, you should have a happy green square in the upper right-hand corner of the editor, near where a scrollbar would appear if our document was longer. This tells you not only that it is well-formed but also valid according to the rules of the tei_all schema.
• Delete the entire paragraph element (including <p> and </p> tags) that says:

<p>Some text here.</p>

• Does that happy green square disappear? Is it angry and red? If document checking is turned on the opening <body> tag should be underlined in red, that happy green square should now be red and there should be a red line part way down the right-hand side indicating where the error is in the document.
• At the bottom of the screen there will be an error message, in this case saying ‘element “body” incomplete’ because it is expecting one of any number of elements.
• Instead of replacing this paragraph, let’s instead add a division. Move to inside the <body> element between the opening tag and the closing </body> tag where the paragraph was previously. Press the < key and wait a second; oXygen should be helpful and give a drop down list of the elements allowed by the TEI at this point. Scrolling up and down this list can give you a sense of the vast array of things you could be encoding at this point, but is also a bit of a mixture because you can have texts with divisions or without them at this point. Select the <div> element and notice what oXygen does.
• oXygen should have added both an opening and closing division tag: <div></div> . Move the cursor between these two tags and press enter a couple times to get some space.
• Add a <head> element and inside it put the text content “My First Heading”.
• After the closing </head> tag, add a paragraph using the <p> element and the text “My first paragraph.”
• In all cases make sure you only stop when you have a happy green square indicating that your document is well-formed and valid.
• Your <body> element should now look something like:

• Add at least one more division after this. (If you had a document with only one division, you don’t really need to use the <div> element at all.) Inside this second division, try nesting a sub-division!
• If you do your <body> element might look something like:

• Save your document.
• The oxygen-tei framework comes complete with some transformations to other formats. From the oXygen menus choose Document ? Transformation ? Configure Transformation Scenario(s) and select ‘TEI P5 XHTML’ and click on ‘Apply associated’ (though this may be slightly different if you are using a different version of oXygen).
• You should get a minimal HTML rendering of your file appearing in a browser. Note some of the information that the transformation has added. Try some other transformations or changing the document and seeing the effect.
• Think about the nature of your own materials and how you might structure them if encoding them according to the default text structure of the TEI!

I’ve intentionally glossed over the introduction of many of the core TEI elements (such as <p>), but don’t worry we will survey these next time!

Go on to Self Study (part 4) TEI Core Elements next!

The next step I’d recommend  is to learn a bit more about the Text Encoding Initiative and the Guidelines it produces.

• Start with:  http://www.tei-c.org/Support/Learn/intro.xml. This has some basic concepts that are useful in understanding the TEI and the Guidelines that are the main output of the TEI.

Questions:

1. In what markup language did documents using TEI P1 to TEI P3 use?
2. How was this changed for TEI P4 and then TEI P5?
3. In what way is the TEI ‘extensible’?

• Next read The TEI Infrastructure chapter: http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ST.html. This is the first main chapter of the TEI P5 Guidelines and describes the infrastructure and framework for the whole of the TEI is based on.

Questions:

4. What does ‘ODD’ stand for? What can one generate from a TEI ODD file?
5. What is a TEI module? What is the relationship between modules and chapters?
6. What language does one use to define a TEI schema?
7. Why might a single project use more than one schema at different stages in their project workflow?
8. What is an attribute class? The att.global attribute class provides @xml:id and @n attributes to every element in the TEI; what is the difference between these two attributes? When might it be useful to use @n to number verse lines? When might this be a silly waste of time?
9. What is the @xml:lang attribute for?
10. What is the difference between the @rend, @style, and @rendition attributes?
11. What is @xml:space for?
12. What is a TEI model class, and what do members of the same class share?
13. Why are model and attribute classes a good idea?
14. What is a TEI datatype?

Note: If you are confused about modules vs model classes vs attribute class the following blog post might help: http://blogs.it.ox.ac.uk/jamesc/2008/09/01/modules-vs-model-classes-vs-attribute-classes/

• Next, familiarise yourself with the table of contents of the TEI Guidelines: http://www.tei-c.org/release/doc/tei-p5-doc/en/html/index-toc.html
• And then browse http://www.tei-c.org/release/doc/tei-p5-doc/en/html/REF-ELEMENTS.html which contains a complete list of elements provided by the TEI.
• Choose a couple elements which you think you might know what they are used to encode and click on them to explore their reference pages. For example http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-address.html
• The information on this page seem confusing but it lists:
• the element’s definition; which ‘module’ it comes from
• what attributes it has (and if they come from attribute classes)
• what model classes the element might claim membership of (which controls where it is allowed to appear in your document)
• a list (by module) of these elements which are allowed to contain this element
• a list (by module) of which elements this element is allowed to contain
• a declaration of the content model of the element (which can be toggled between Relax NG compact syntax and XML syntax)
• one or more examples
• possibly some additional notes on usage.

Questions:

• The address element does not define any attributes of its own. How does this compare in layout to the availability element? What attribute does this (at time of writing) define for itself rather than getting it from a class?
• The address element has two examples; what is the difference between them?
• If you click on the ‘Show all’ link in one of the examples what do you get? Notice, for example how address is used inside the publicationStmt element to give the address of the publisher of the electronic text.

This is a very basic survey of some of the initial things you might want to learn before diving into the Guidelines in more detail.  I plan to continue this with similar directed reading and questions on some of the topics they cover in the future. In fact, the next post in this series is http://blogs.it.ox.ac.uk/jamesc/2013/01/31/self-study-part-3-the-tei-default-text-structure/ which looks at the TEI’s Default Text Structure.

rhyme=”(a*)a*(a*)b(c*)c*(c*)bddee(f)fg(h)hg/”

But I need at any particular point to be able to get at least 2 things from this string:

1. The documented rhyme above for the current <rhyme> element that I’m processing
2. Whether the current rhyme is an internal (parentheses) or a feminine (asterisk) rhyme or not.
3. The set of rhymes for the current line
4. Whether the current line has any internal (parentheses) or feminine (asterisk) rhymes or not.

So the first step with this is to tokenize the given rhyme scheme.  I do this as an XSLT function and if I want to output it I could have something like:

 <xsl:variable name="rhyme">
(a*)a*(a*)b(c*)c*(c*)bddee(f)fg(h)hg/
</xsl:variable>
<tokenized-rhymes>
  <xsl:copy-of select="jc:tokenizeRhymes($rhyme)"/>
</tokenized-rhymes>

Here, inside some unseen template, I’ve got a variable with the rhyme scheme in it, and I’m getting a copy-of the output of a function I’ve created called jc:tokenizeRhymes(). This isn’t a very difficult XSLT function it just consists of some xsl:analyze-string as so:

<xsl:function name="jc:tokenizeRhymes" as="item()*">
<xsl:param name="rhyme"/>
<xsl:variable name="rhymes">
<list>
    <xsl:analyze-string select="$rhyme" regex="\(*[a-zA-Z]\**\)*">
        <xsl:matching-substring>
            <item>
                <xsl:value-of select="."/>
            </item>
        </xsl:matching-substring>
        <xsl:non-matching-substring/>
    </xsl:analyze-string>
</list>
</xsl:variable>
<xsl:copy-of select="$rhymes"/>
</xsl:function>

All this does is have a function which takes a single parameter (rhyme), and creates a variable containing a list with a bunch of items inside. To do this is uses a regular expression on xsl:analyze-string which looks optionally for an opening parenthesis \(* then any letter from a-zA-Z optionally an asterisk \** follow by an optional closing parenthesis \)* … see, simple. The output from this lookst like:

  <list>
         <item>(a*)</item>
         <item>a*</item>
         <item>(a*)</item>
         <item>b</item>
         <item>(c*)</item>
         <item>c*</item>
         <item>(c*)</item>
         <item>b</item>
         <item>d</item>
         <item>d</item>
         <item>e</item>
         <item>e</item>
         <item>(f)</item>
         <item>f</item>
         <item>g</item>
         <item>(h)</item>
         <item>h</item>
         <item>g</item>
      </list>

Well then, getting the current rhyme when I’m processing a rhyme is fairly easy then. I just create a variable $rhymePosition (the current number of rhymes I’m on) and then can call another function jc:getCurrentRhyme with that and the rhyme variable.

<xsl:variable name="currentRhyme">
  <xsl:value-of select="jc:getCurrentRhyme($rhyme, $rhymePosition)"/>
</xsl:variable>

The jc:getCurrentRhyme function is fairly straightforward as well. It looks like:

<xsl:function name="jc:getCurrentRhyme" as="item()*">
   <xsl:param name="rhyme"/>
   <xsl:param name="currentRhyme" as="xs:integer"/>
   <xsl:variable name="rhymes" select="jc:tokenizeRhymes($rhyme)"/>
   <xsl:copy-of select="$rhymes/list/item[$currentRhyme]"/>
</xsl:function>

It takes two parameters, the $rhyme and the $currentRhyme (which is an integer of how many rhymes there are so far in the <lg> including the one we are processing). It then creates a new variable $rhymes which has the output of the jc:tokenizeRhymes above. Then getting the current rhyme from the list is easy because we know its number so we just make a copy of the <item> we’ve created in that variable by using xsl:copy-of and filtering it by the number $currentRhyme. (This is why we made sure that this parameter was an integer.)

In order to check whether these are internal or feminine rhymes it is now very straight-forward, we just test the $currentRhyme we’ve created above to see whether it contains($currentRhyme, ‘)’) or contains($currentRhyme, ‘*’).

In order to get all the rhymes for this line, we need to re-process this tokenized list somewhat. We want to group those items which have parentheses together with the letter which follows them, splitting on each non-parenthesised letter (optionally having an asterisk). It took me awhile to get my brain around that but eventually I came up with:

<xsl:function name="jc:groupRhymes" as="item()*">
<xsl:param name="rhyme"/>
<xsl:variable name="rhymes" select="jc:tokenizeRhymes($rhyme)"/>
<xsl:variable name="groupedRhymes">
  <list>
   <xsl:for-each-group select="$rhymes/list/item"
      group-ending-with="*[matches(., '^[a-zA-Z]\**$')]">
     <item>
      <list>
       <xsl:for-each select="current-group()">
        <item>
         <xsl:value-of select="."/>
        </item>
       </xsl:for-each>
      </list>
     </item>
    </xsl:for-each-group>
  </list>
</xsl:variable>
<xsl:copy-of select="$groupedRhymes"/>
</xsl:function>

This function takes in the parameter $rhyme and tokenizes it using the earlier function, so now we have a list with some individual items in. It then creates a new list and uses xsl:for-each-group to select all the tokenized items. It creates groups ending with any item where the content matches a full line going from start to finish of a letter followed by an optional asterisk. This means each group will end with a normal rhyme letter and any internal rhymes (in parentheses) will be included in that group. For each group it puts out a new item with a nested list and makes each rhyme in that line an item in that nested list. This might seem overkill to some, but having the extra nesting, regardless of whether there are 1, 2, or 20 rhymes in the line just makes things easier. So this output from this looks like:

<list>
<item>
    <list>
        <item>(a*)</item>
        <item>a*</item>
    </list>
</item>
<item>
    <list>
        <item>(a*)</item>
        <item>b</item>
    </list>
</item>
<item>
    <list>
        <item>(c*)</item>
        <item>c*</item>
    </list>
</item>
<item>
    <list>
        <item>(c*)</item>
        <item>b</item>
    </list>
</item>
<item>
    <list>
        <item>d</item>
    </list>
</item>
<item>
    <list>
        <item>d</item>
    </list>
</item>
<item>
    <list>
        <item>e</item>
    </list>
</item>
<item>
    <list>
        <item>e</item>
    </list>
</item>
<item>
    <list>
        <item>(f)</item>
        <item>f</item>
    </list>
</item>
<item>
    <list>
        <item>g</item>
    </list>
</item>
<item>
    <list>
        <item>(h)</item>
        <item>h</item>
    </list>
</item>
<item>
    <list>
        <item>g</item>
    </list>
</item>
</list>

Which, admittedly, is fairly verbose. But you can now have a function that just gets the individual line’s items that you are interested in which would look something like:

<xsl:function name="jc:getCurrentLineRhymes" as="item()*">
  <xsl:param name="rhyme"/>
  <xsl:param name="currentLine" as="xs:integer"/>
  <xsl:variable name="rhymes" select="jc:groupRhymes($rhyme)"/>
  <xsl:copy-of select="$rhymes/list/item[$currentLine]"/></xsl:function>

Which when called with something like:

 <xsl:copy-of select="jc:getCurrentLineRhymes($rhyme, 4)"/>

(where ’4′ here usually would be a variable containing the current line number) it will produce something like:

<item>
 <list>
  <item>(c*)</item>
  <item>b</item>
 </list>
</item>

Which a simple string test using contains() can again tell you whether there are any feminine (asterisk) rhymes or internal (parentheses) rhymes, etc.

Hurrah! See that wasn’t that difficult after all. In this case it makes a good example of using XSLT2 functions to call other functions to break the overall task down into manageable more object oriented-like tasks which can be re-used for a variety of purposes. (There are a lot of efficiencies which could be implemented here… the jc:getCurrentLineRhymes and jc:getCurrentRhyme are almost identical, except that one uses jc:groupRhymes() and the other uses jc:tokenizeRhymes(). This could be one function which tests a parameter to see which is intended.

The whole XSLT stylesheet is available from https://github.com/jamescummings/conluvies/blob/master/xslt-misc/tokenize-rhyme-test.xsl.

Elena Pierazzo started off the panel by introducing what it would cover. It looked and the differences and the similarities in teaching the TEI in a range of contexts: from a dedicated intensive workshop targeted at professionals to the teaching of TEI as part of a related academic course. These have differences in aims, methodologies, and overall coverage and the syllabus of each of these types of teaching might cover different chapters of the TEI Guidelines.

The panel discussed which of these approaches seemed most successful, and what was meant by success when teaching the TEI. The question of whether the TEI works better as a tool to solve a problem researchers are currently facing (e.g. a digital edition of a manuscript, a dictionary, a corpus…) or as a method for approaching analysis or tool for modelling concepts? Throughout the panel these two types of teaching were contrasted to see what might be learned to benefit the other pedagogical form.

Marjorie Burghart contrasted the similarities and differences between the BA and MA level training provided in Lyon as compared to Elena’s examples. She insisted on the importance of embedding TEI teaching in other disciplines, giving the example of one of her courses where students are taught editorial techniques as a whole, from the historical developments of diplomatics and philology to their digital translation. The central message being that not all TEI teaching is done in “TEI courses” or even those specifically about digital technology, some of it occurs in academic field-related courses that happen to include sections about the TEI.

James Cummings briefly surveyed the types of TEI training provided at the University of Oxford, noting the Digital.Humanities@Oxford Summer School (http://digital.humanities.ox.ac.uk/dhoxss/) evolved out of many years of TEI Summer Schools and now always included a week-long TEI workshop. The introductory TEI workshop in such a context tends to cover a large amount of the TEI Guidelines, giving a broad but shallow and intensive overview. He mentioned that they also did bespoke training for individual research projects where the whole of the TEI was not taught, but just a brief overview followed by specific training in the aspects that project would be using.

Julia Flanders provided a description of the workshops they teach at Brown University and the workshops provided at DHSI.org, and how these compare to those in Oxford and differ from those that form part of larger academic courses. She discussed various approaches to teaching the underlying concepts and how existing tools such as Roma might be improved to facilitate this. She suggested that introductory tools which allowed ‘Finger painting with semantics’ should be created to allow people to play with the concepts of data modelling in a user friendly manner.

There was much wide-ranging discussion with the audience with many interesting points made and questions raised.  Several participants mentioned that they used text encoding generally, and the TEI in specific, to teach different things. That is, the process of learning the TEI helps students to understand more about other topics (e.g. the nature of text). Michael Gavin commented that it would be nice to have a survey of both TEI courses and courses which include the TEI in higher education. Marjorie mentioned that Marjorie mentioned that Florence Clavaud (EnC) had started a similar survey for France / Europe, and that it would be good to get in touch with her.

TEI is taught in a variety of different ways, and the more teaching of it the better, but what has to be closely examined by providers is why any particular course is being offered. Is it to induct a large number of people into a basic understanding of the scope and coverage of the TEI Guidlines? Is it to teach them the practical skills to undertake the work on one particular research project? Or is it to teach them other more ethereal concepts, of which the TEI is one practical and concrete example?  The teachers on this panel had all taught in a variety of these kinds of contexts and the differences in approach and coverage made for an interesting comparison.  As the TEI continues to grow and be used more pervasively as the de facto standard for the encoding of digital texts (especially in academic contexts) then the community will need to continue to improve its teaching and organization of teaching.  One promising sign is the network of Digital Humanities training institutes (see those referenced at http://digital.humanities.ox.ac.uk/dhoxss/) which are slowly cooperating to produce a consistent pedagogical basis while retaining their unique character and experiences.

The @rend attribute

The TEI provides a @rend attribute which indicates an interpretation of how the element in question was rendered or presented in the source text. It has nothing to say about what should be done with the element in any particular output from processing or displaying the TEI text. The assumption that many people make is that processing TEI means outputting HTML designed to help you read the text, but this is certainly not necessarily the case. The TEI text might have any number of outputs, just for reading it might be HTML, ePub, PDF, DOCX, and many more, moreover those encoding the texts might not be intending to read it but process it for other forms of text analysis in any number of formats. While individual projects can provide project documentation on how they intend certain elements to be presented in particular forms of output, other people processing those texts could choose to do something completely different.

@type and @rend values and their whitespace

During a TEI-L discussion concerning why the @type attribute did not allow spaces it was explained that this is because the @type attribute does not contain free text, but a special token that categorises the element in some way. Moreover, the recommended practice is for projects to customise the TEI to constrain the choices available for the value of the @type attribute on some elements and document in their customisation exactly what those special tokens mean. @type attribute values are a datatype of data.enumerated which means that they are “expressed as a single XML name taken from a list of documented possibilities”. That means that this value has to obey the rules of what it means to be an XML name, and it should be from a set list that the project has documented (preferably in its TEI customisation, but possibly just in prose documentation preserved with the TEI file). Most elements that have a @type attribute get it from claiming membership in the att.typed attribute class, and if a secondary type classification is allowed they also get @subtype.

The discussion moved on (possibly because I referenced my earlier post on @rend) to the difference with the @rend attribute and using CSS inside it. However, with the @rend attribute though the situation is slightly more confusing. It allows 1 to infinity occurrences of the datatype data.word in it. A data.word datatype “defines the range of attribute values expressed as a single word or token.” As I’ve discussed elsewhere, this means if someone marks up a text using:

<hi rend=”It looks a bit like that other one”>text</hi>

This actually has 8 tokens “It”, “looks”, “a”, “bit”, “like”, “that”, “other”, “one”. The point is that the whitespace between these words in the attribute make these each separate values or tokens, not a phrase. The encoder might just have written:

<hi rend=”big bold beautiful”>text</hi>

or indeed

<hi rend=”largeStyle42″>text</hi>

The data.word datatype says that “Attributes using this datatype must contain a single ‘word’ which contains only letters, digits, punctuation characters, or symbols: thus it cannot include whitespace.”

Some encoders believe that the TEI should reverse its decision on free text in attributes and allow @rend to contain “It looks like that other one” and this not to be a set of discrete tokens. Personally, I disagree and feel that would be a retrograde step.

@rend values and their order

Other than defining it as a set of data.word occurrences the TEI does not dictate what the @rend values should look like. In my opinion it would be wrong if the TEI try to codify all the possible rendition values that appear in every sort of text. Moreover, describing the way something appears in a text is always an interpretative process and two separate encoders looking at the same text, or looking at it for different reasons, might perceive it in very different ways. In fact the Guidelines explicitly say:

“These Guidelines make no binding recommendations for the values of the rend attribute; the characteristics of visual presentation vary too much from text to text and the decision to record or ignore individual characteristics varies too much from project to project.” (http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-att.global.html)

Some encoders believe that it is a shame that the TEI has not defined a syntax by which they should specify the @rend attribute values. I disagree because I feel that the greatest flexibility should be given to projects and sub-communities to customise and constrain such values for themselves. It could be argued that the TEI has indeed provided a syntax, but in a very general way, that these are whitespace separated tokens containing only letters, digits, punctuation characters or symbols. The point is that these are entirely meant to be intended as magic tokens that individual projects can decide for the meaning for their own use (and document). If I put in the magic token ‘bold’ it might mean in my project something different than it means in yours.

It came out in the TEI-L discussion that some encoders believe that the order of @rend values provided should be important, as if they are making a phrase. Others tend to put the most important rendition classification first, and still others always provide different types of classification in the same order. I find these all prone to human inconsistency and so I choose to believe that they are an unordered set of values that could be entered in any order. i.e. that:

<hi rend=”big bold beautiful”>text</hi>

should be understood to be semantically equivalent to:

<hi rend=”beautiful big bold”>text</hi>

My beliefs here are, perhaps unduly, influenced by long and painful experience in processing hand-encoded texts (which also influences my beliefs on the value of automatic and semi-automatic up-converting markup). In my encoding projects I recommend that no special significance be granted based on the order of the tokens present in the @rend value. The TEI, I think sensibly, allows individual projects to do what they want but does specify that these are individual tokens.

Some projects decide to put various standard presentation-description formats, e.g. Cascading StyleSheets, into the @rend attribute. I personally feel that this is misguided and sloppy. Partly this is because I suspect that some of them are actually encoding for a particular output format (rather than documenting what the original source looked like) and this is the wrong place to store this information. Partly this is because such presentation-description formats often use significant whitespace (which then means an abuse of the data.word datatype). And partly this is because I feel there is a better and easier way to do this more consistently using the @rendition attribute.

@rendition and <rendition> really aren’t extreme

As with many other things in the TEI, the Guidelines provide a simple use-case (@rend’s magic tokens) and a more complex system (@rendition). The @rendition attribute allows you to point to a <rendition> element up in the header where you can use any form of free text to describe how this was rendered in the original source. This means that instead of putting a set of magic tokens or classifications like “largeStyle42″ an encoder can completely transparently point to a fuller description using the standard URI fragment pointing mechanism that is common throughout the TEI recommendations. Thus instead of writing:

<hi rend=”largeStyle42″>text</hi>

And having it documented somewhere what this meant. The encoder can point to a <rendition> element by its @xml:id attribute and have a fuller description there. For example this could be:

<hi rendition=”#largeStyle42″>text</hi>

and while that doesn’t look much different the URL fragment ‘#largeStyle42′ points to a place inside the TEI file’s <teiHeader> (specifically inside the <tagsDecl> element) where there is a better description:

<rendition scheme=”free” xml:id=”largeStyle42″>This text is really big, bold, and beautiful</rendition>

Okay, admittedly that might not be a very useful description. But the point with the ‘free’ scheme is that it is free text. It can be any prose, in any language, and way of describing it. The @scheme attribute also allows for ‘css’ for those people wishing to use cascading stylesheet language, ‘xslfo’ for those wanting to use extensible stylesheet language formatting objects, and ‘other’ for those using another set rendition description language. So ‘#largeStyle42′ could point to something using CSS that looked like:

<rendition scheme=”css” xml:id=”largeStyle42″>
font-weight:bold;
font-size: 75pt;
font-family:”brushstroke”, fantasy;
color:#002147;
</rendition>

If a more precise description (in whatever language) is able to be provided for ‘largeStyle42′, then this can be changed at a later date. Equally this could be broken up into multiple <rendition> elements and you can have:

<rendition scheme=”css” xml:id=”bold”>font-weight:bold;</rendition>
<rendition scheme=”css” xml:id=”big”>font-size:75pt;</rendition>
<rendition scheme=”css” xml:id=”beautiful”>font-family:”brushstroke”, fantasy;</rendition>
<rendition scheme=”css” xml:id=”oxBlue”>color:#002147;</rendition>

and in the text:

<hi rendition=”#big #oxBlue #bold #beautiful”>text</hi>

Moreover, because @rendition is one of the TEI’s many pointing elements it does not need to point to a <rendition> element in the very same file! Instead a project could centralise all their rendition information to a single place. So that might look like:

<hi rendition=”renditionFile.xml#largeStyle42″>text</hi>

or indeed

<hi rendition=”http://www.example.com/renditionFile.xml#largeStyle42″>text</hi>

Some encoders feel that pointing to a <rendition> element is a lot harder than just sticking some tokens into the @rend attribute. Others argue that as part of the process of hand encoding users should be able to add whatever they want to @rend, and for this to be valid because rationalising these in advance is more difficult than doing so afterwards. Or indeed that it is more convenient to encode unusual variants ‘in-line’ rather than pointing back to the header. Both of these are good points, and have some truth to them. In the first case, it depends on the level of specification needed. Most encoders in my experience use very general and imprecise @rend categorisations. That is, they could have a rend value of ‘big72pt’ but they tend to just use ‘big’ (or small/medium/large/x-large).

How much time and energy one wants to spend worrying about specifying @rend and/or @rendition values depends on how important to your project that that this information is documented and done so in a consistent manner. If it is just that you want record whether something is in one of a handful of different colours, sizes, or styles, then you probably just want to agree a project specification of @rend values (and what they mean) for your TEI customisation.

Other @rend issues

Some encoders believe that there is no formal way of indicating what syntax you have used for your @rend values. I disagree because I believe these are magic tokens which are most properly documented in the TEI customisation. This enables an encoder to give a free text description for every magic token used in @rend attribute values, and moreover if they wish it enables a project to constrain it to be just this set of values. If a project is using a specified syntax inside their @rend attribute values (so-called ‘rendition ladders’ are one such format) then this should be documented inside the <encodingDesc>, perhaps in prose or perhaps the TEI will add a mechanism in response to the TEI-L discussion which enables categorisation and description of the taxonomy of @rend attribute values.

Changing @rend

My arguments here are based on my own views and understanding of the current (P5 2.0.2) version of the TEI Guidelines. However, these are subject to change (both my views and the Guidelines). I’ve often been told that the TEI recommendations seem like dictates coming down from on high saying “do it this way”, but that is really not how I view the TEI Guidelines or the community that creates them. The TEI is an open source project which takes solicitations for bug and feature requests from anyone and everyone. This can be from someone encoding their very first TEI document, reading the Guidelines for the first time, or it can be from those with a long history of experience with the TEI. Each and every bug and feature request should be considered on its own merits by the TEI Technical Council elected by the TEI community. [Note: there is scope for electoral reform, but this is a very different topic.] The recommendations of the TEI are not a fixed quantity but an evolving record of the concerns and experience of the community that produces it. In many ways hearing what users new to the TEI have difficulty with, or where they find the Guidelines confusing is more valuable in the long run than some of the more arcane technical discussions.

However, some people are unable to participate in such training and prefer self-directed learning. What should they do? There are lots of resources online such as TEI By Example and the TEI Guidelines. Where to start?

When people are taking an Introduction to TEI workshop I usually introduce markup but move onto TEI and XML very quickly because in such intensive workshops time is limited. Instead, when people are undertaking self-directed learning I think they should use the time they have to learn more about HTML and then XML before starting to learn about the TEI vocabulary of XML itself.

There is so much reading that is possible to suggest for an initial exploration of XML and Markup.  I would suggest at least looking at:

• http://www.w3schools.com/html/ (basic section and xhtml section)
• http://www.w3schools.com/xml/ (basic section and page on namespaces)
• http://www.tei-c.org/release/doc/tei-p5-doc/en/html/SG.html
• http://en.wikipedia.org/wiki/XML
• http://www.tei-c.org/About/Archive_new/ETE/Preview/guidelines.xml
• http://www.tei-c.org/About/Archive_new/ETE/Preview/principles.xml
• http://www.youtube.com/watch?v=NLlGopyXT_g
• http://en.wikipedia.org/wiki/RSS

as a good start.

If I were to suggest a series of assignments someone might undertake based on this reading it would be to do the following, writing up answers to the questions.

1. Read the W3Schools HTML basic section and XHTML section, do the HTML and XHTML quizzes
2. Read the W3Schools XML basic section and XML Namespaces page, do the XML quiz
3. Read the TEI Guidelines Gentle Introduction to XML; and the wikipedia article on XML.
4. How does XML differ from HTML? Why might it be more powerful to describe what some piece of data is, rather than say how it should be presented?
5. Download and install the oXygen XML editor (you can get a 1 month free trial license, otherwise costs $64 USD)
6. Choose a very short (1 page) sample of a document you are interested in.
7. Create a list of the overall structural aspects you feel define this sort of document. Create a list of any of data-like entries (like names or dates) in the document. Create a list of presentational aspects of the document that you think important to record.
8. Funding challenge part 1: Hypothetically, imagine you had funding to mark up several thousand pages of this material. Look at the list of aspects you would like to record. Why is each one important? What benefit does recording each of these things give those wanting to use or understand the text (or culture from which it originates)? Which would you choose to markup? How consistently can you mark up this feature? Such document analysis should be done long before any project starts (or asks for funding).
9. Funding challenge part 2: An uncaring government has slashed its funding for higher education research projects and has reduced your project’s funding by 50%! What would you do? Will you mark up only 50% of the material? If so, how do you decide which parts? Will you only mark up certain aspects? If so, which ones and why?
10. Using the ‘Text’ (code view) mode of the  oXygen XML editor create a well-formed XML file of your sample document with elements and attributes that you have invented yourself. What difficulties do you encounter doing this?
11. Why might it be better for communities of users to agree on elements, what they mean, and how they should be used?
12. What are the central ideas of Michael Wesch’s youtube video? How do they relate to the nature of XML and how it is used?
13. Read the wikipedia article on RSS, and find an RSS feed to subscribe to in google reader to see its application.
14. Does order really matter in an XML document?  What is the difference between:
    

    <list><item n=”1″>item 1</item><item n=”2″>item number 2</item></list>  and
    <list><item n=”2″>item number 2</item><item n=”1″>item 1</item></list>

    And how much difference does this make when viewing XML as a data storage format rather than a presentational one?

15. Join the TEI-L mailing list and start lurking.

This certainly isn’t exhaustive, but with a bit of support, I suggest someone undertaking this would be much better placed to start learning about TEI P5 XML from the online sources available.

The next post in this series is an Introduction to the Text Encoding Initiative Guidelines.

The definition of @rend is “(rendition) indicates how the element in question was rendered or presented in the source text.” but it is very often used by some encoders to signal to processing how you want the output to appear.  In the remarks on the values allowed for the attribute it says:

may contain any number of tokens, each of which may contain letters, punctuation marks, or symbols, but not word-separating characters.

The point here being the ‘word-separating characters’ part. So although you can say <hi rend=”It looks a bit like that other one”>text</hi>, this actually has 8 tokens “It”, “looks”, “a”, “bit”, “like”, “that”, “other”, “one”. Sometimes people stick CSS or CSS-like rendition information into @rend so have values like “text-align: right”. Which I would say was wrong… or at least saying that there are two classifications applicable to its rendition in the source material, one that it is “text-align:” and another that it is “right”.  Of course they could solve this just be deleting the space “text-align:right” would be better, or even “text-align:right; font-size:large;” if you wanted to add another token.  However, even better would be to use @rendition to point to at least one @xml:id of a <rendition> element in the header.  This allows you to specify exactly what scheme you are using (e.g. CSS) and to give multiple statements for one classification.

Why does this matter you might ask? Well, of course, it doesn’t really — they are all magic tokens of one sort or the other to be interpreted (or not) by your processing for whatever reason you are undertaking the encoding. The <rendition> method is the most detailed in documenting precisely how you are interpreting the rendition in the original document.

However, the reason it matters to me is that there are NO attributes in the TEI which allow free-text.

By that I mean that all attributes are assigned to one datatype or another, and in none of them can you just type sentences of prose and have it be semantically meaningful.  This is as a result of the long War on Text-Bearing Attributes that was undertaken in the run-up to the first release of TEI P5. This took as one of its many principles that because any bit of free text might have a need to use a non-Unicode character, and that the TEI’s method for documenting non-Unicode characters was to use its <g> element, that you couldn’t have free-text attributes because you can’t use an element inside an attribute value. This is the reason for the creation of many new child elements like <desc> which are intended to contain free text concerning the nature of the element that contains them.

In the case of the @rend attribute it allows one to infinity of the data.word datatype.  This data type, even in P5 1.0.0 “defines the range of attribute values expressed as a single word or token.”  Thus when people put space separated characters into it, they are really putting in multiple tokens.  The war of text-bearing attributes attempted to limit the places where people were able to do this by the use of datatypes and the removal of free text in attribute values.

This helps to highlight the difference between syntactic and semantic validity. Just because your document validates against a schema, does not mean that it is semantically valid.  You can put the text of a title inside an <author> element and vice-versa and there is no way your schema can know that you have done this.

So really, I’ve posted this post so I can point to it later when people ask me about spaces in @rend and similar datatype kerfuffles.