TOBAR NA GAEDHILGE 1.7

• What the results may look like

We may begin by finding sentences containing a selected Gaelic word, and we will do one example each from the Munster, Connacht and Scottish texts. Thereafter, we will draw our examples from the Ulster texts, which form by far the largest part of the material stored.

Figure 1: We looked for examples in the Munster texts of the word cábóg (a country person). We found two examples in Pádraig Ua Maoileoin, Na hAird Ó Thuaidh, and we show the second example here. Page and line reference is given to the published book.

The navigation panel (at upper right) allows us to move around the retrieved examples. The panel at the lower right allows the form in which the sentence is displayed to be modified. All the options are described under Figure 6 below.

Figure 2: We looked for examples in the Connacht texts of the sequence of words lúb gaoil (blood relationship), and we found two examples, both in Séamus Mag Uidhir, Fánaidheacht i gConndae Mhuigheo. We show the first sentence here.

Figure 3: We looked for examples in the Scottish texts of words beginning with càr. With a request like this for words matching a general pattern, clearly more than one word may match — we refer to this situation as matching a disjunction of words. The sentences for each matching word are presented (over all the relevant books) before presenting the sentences for the next matching word. Here, we show a sentence containing the matching word càraich (fix).

Besides viewing complete sentences as above, two other ways of viewing results  are provided, which are more compact for large quantities of text. Output from these methods is shown in the next two figures.

Figure 4: Uses of the word saoghal (life) shown as its frequency in the various Ulster texts.

Scrolling may be necessary to reveal the information for all the texts. If the request matched a disjunction of words, a menu option AthFhocal (next word) will proceed to the next matching word. A menu option Réidh (finished) leaves the displayed results and returns to the search screen.

Figure 5: A keyword-in-context index of the word-form athair (father) in Séamus 'ac Grianna, Thiar i dTír Chonaill. The first batch of 43 occurrences is shown.

The navigation panel at the upper right allows several options. First of all, we may move up and down through the concordance lines, which are in screenfuls of a size which depends on the size of the window (43 lines in this illustration). We may move down a screenful (Síos), up a screenful (Suas), to start of text (Suas go bárr), to end of text (Síos go bun). However, these options will not take us into another text, nor (if the request matched several word in a disjunction) will they take us into another matching word; they move only within the current text, for the current matching word.

Second, we may move, forwards only, through the entire collection: AthShlámán (next batch), AthLeabhar (next book), AthFhocal (next matching word, if the request matched several in disjunction), Réidh (finished). If we wish to examine all the examples from the beginning again, we need only choose Réidh and then click OK without altering our previous choices. This second set of options are provided with keyboard shortcuts, which may prove convenient for repeated use.

Cóipeáil (copy) copies the current display to a textfile, which by default, is called samplaí.txt and is placed in the My Documents folder, and the copied material is appended to it. Comhad Cóipeála (copy file) allows the name and location of the file to be changed, and also the mode from append to overwrite (but it will revert to appending after overwriting once).

Réidh (finished) leaves the displayed results and returns to the search screen.

Figure 6: Returning now to display by sentences, we give an example of the word oidhreógach (ice) from the Ulster texts. The example shown is from Seosamh 'ac Grianna, Pádraic Ó Conaire agus Aistí Eile.

The panel at the lower right allows the text of the sentence to be shown in a choice of ways: plain and uncorrected (Foillsighthe); plain but corrected (Lom); including mark-up (Marcáilte); or as a list of the words by which it is indexed (Innéacsáilte). The Innéacsáilte option allows you to see which index terms will fetch this sentence. This will allow you to examine how the text has been tokenized and indexed. The panel also controls the display of the sentence in other languages, when this is possible (see later) — any language with its name shown in in italics is unavailable for this sentence. The choice of display modes applies only to the first language, i.e. the language of the index used; for other languages, the text is shown plain.

To explain what is meant by "correction": this is limited to obvious errors. An isolated spelling variant which is at odds with consistent usage in the rest of the same book may also be corrected; but beyond this, we make no attempt at normalization of forms which are not clearly in error. Corrections to Gaelic text may also include restoring the wording of the manuscript where known. And correction of a text in a language other than Gaelic may be used to bring that text closer to the edition used by the Gaelic translater. None of these correction processes can be guaranteed to have been applied exhaustively. The corrections are applied to the indexes; and to the displayed sentences under the Lom option. The exact book text will always be displayed under the Foillsighthe option.

As an example, the misprint cómhhartha occurs in Ben-Hur on page 337. If the relevant sentence is shown under the Lom option, we will see it as cómhhartha. Under the Foillsighthe option, we will see it as cómhartha. Under the Marcáilte option, we will see the whole markup as cómh[h]artha. And under the Innéacsáilte option, we will see that it is indexed as cómhartha.

The navigation panel at the upper right allows several options.

First of all, we may move around the examples. We may move down a sentence (Síos), up a sentence (Suas), to start of text (Suas go bárr), to end of text (Síos go bun). However, these options will not take us into another text, nor (if the request matched several words in a disjunction) will they take us into another matching word; they move only within the current text, for the current matching word.

Second, we may move, forwards only, through the whole collection: AthAbairt (next sentence), AthLeabhar (next book), AthFhocal (next matching word, if the request matched several in disjunction), Réidh (finished). If we wish to examine all the examples from the beginning again, we need only choose Réidh and then click OK without altering our previous choices. This second set of options are provided with keyboard shortcuts, which may prove convenient for repeated use.

Cóipeáil (copy) copies the current display to a textfile, which by default, is called samplaí.txt and is placed in the My Documents folder, and the copied material is appended to it. Comhad Cóipeála (copy file) allows the name and location of the file to be changed, and also the mode from append to overwrite (but it will revert to appending after the first copy). The darker elongated panel shown above is the result of clicking Comhad Cóipeála.

Réidh (finished) leaves the displayed results and returns to the search screen.

If you accidentally lose the navigation panel from the screen while examining the results, a reminder labelled Preab-liosta! will appear from underneath it.  Clicking on this reminder will recover the navigation panel.

Multi-word retrievals may require the creation of workfiles. These will be placed in the temporary workspace folder, if possible. In the unlikely event that this is not possible, you will be prompted for the name of a folder to hold workfiles; you could, for example, place them on a memory stick.

In displaying results by sentence, the translation of the sentence into other languages may be shown, if it is stored. To do this, tick the names of the desired languages, visible on the sentence-level displays above, labelled Béarla (English) and Fraincis (French) and Gearmáinis (German). If the name of the language is italicised, this means that a translation in that language is not available for the sentence to be displayed. At present, translations or originals are held only for texts in the Ulaidh collection, and only for only a proportion of this: English, 40%; French, 19%; German, 7%.

Figure 7: Sentences containing the word creafadaigh (shaking); the first of two examples found in Seosamh 'ac Grianna, Seideán Bruithne/Amy Foster. English and French translations are available and are shown.

When a sentence is shown in several languages, the choice of display mode between plain (Lom); including mark-up (Marcáilte); as a list of the word-forms by which it is indexed (Innéacsáilte); or plain and uncorrected exactly as printed (Foillsighte) applies only to the language of the index used. In other languages the sentence is shown plain.

• What Gaelic texts can be searched

Gaelic is found in several slightly different forms, and the texts are organized into collections to reflect this and to keep each collection fairly homogeneous in language.  The five collections supplied are (as of 2014/08/24):

• Ulaidh (Ulster)           Index: Gaedhilg. 17 authors;  62 books;  49,362 word-forms;  3,960,160 word-tokens
• Connachta                 Index: Gaedhilge.  3 authors;   3 books;  10,066 word-forms;    139,184 word-tokens
• Mumhain (Munster)  Index: Gaolainn.   2 authors;   2 books;   6,755 word-forms;    101,386 word-tokens
• Alba (Scotland)          Index: Gàidhlig.    4 authors;   5 books;    8,025 word-forms;   115,996 word-tokens
• Oirthear (Eastern)     Index: Gaodhlag.   1 region;    1 book;    3,447 word-forms;   48,569 word-tokens

At the present stage of development, the Ulaidh collection is much larger than any of the others.

A different division into collections would, of course, be possible.

The identities of the texts in all collections are listed in full below.

Searching may be restricted to any chosen subset of the texts of a collection, by deselecting temporarily individual texts or authors.

Each collection has a pre-compiled index associated with it, made from the words found in the relevant books, and in which requests for words are looked up. The statistics just given for the collections refer to these indexes — indexes of words found in the Gaelic texts. This does not mean that the words in the index are exclusively Gaelic words; rather they will reflect faithfully what the Gaelic texts contain.

Much more detail will be given later about what may be found in the indexes.

Figure 8: This is the program's opening screen, and the first task is to choose the desired collection and index.

The program should show a list of the available collections, as in blue above — together with some statistics of the highlighted collection (below) and a pick list of its index (on the right). If there are no collections listed, you may be in the wrong folder, and you can browse (using Cuirtear Lorg) to a different folder. The indexes available to the highlighted collection are shown in brown. When you have marked the desired collection and chosen your index to it, click on Isteach to enter the collection/index.

Amach is to exit the program.

Treoir is for help. The help file supplied with Tobar na Gaedhilge is in Windows Help format (.hlp). Support for Windows Help File format was discontinued by Microsoft in Windows Vista and later. In order to use Windows Help files in these versions of Windows, download and install the Microsoft upgrade of WinHlp32.exe for your version of Windows from http://support.microsoft.com/kb/917607

• Requesting Gaelic words

When a collection has been selected, and Isteach clicked, the display changes to that shown in the next figure, which allows you to type words, among other things.

Figure 9: Requesting Gaelic words.

Before entering our own words, however, first notice that this screen allows you to go back and change to a different collection and index, by using the Athruigh button on the Cnuasacht panel. And also, that you may see which authors and books are included in the current collection and index by using the Athruigh button on the Leabharthaí panel, and you may choose to select temporarily a subset of those books. (Note that selecting a subset of a collection is not reflected in the pick-list of words, which remains that for the whole collection.)

From the Radharc panel, you may choose your display mode for the results: frequencies (Minicidheacht), a keyword-in-context concordance (KWIC) or sentences (Abairteacha). Samples of each form of results have already been shown above.

And now we come to the Focal panel, where the desired word or words may be typed into the box provided, or may be inserted there by double-clicking them on the pick list, which is a displayed segment of the collection's index. The pick list accommodates itself to the existing contents of the box, as a guide to what words are available.

As an alternative to typing it into the box, a search word may also be chosen by double-clicking it on the pick list.  The word will be appended to anything already in the box (and not highlighted).

In the word-form box you may put:
• a word, such as oidhreógach or saoghal or athair, as used in our previous examples
• two or more words occurring together, either consecutively or within the same sentence (eg. lúb gaoil)
• any word may contain a wild-card (*), that is, an asterisk which matches any number of letters, including none.  For example, all words with a particular stem may be sought (eg. beir*), or all words with a particular termination (eg. *stin).

If the words box contains more than one word (ie. there is a space within it), you are asked to choose between seeking the words directly adjacent and in the given order; or within the same sentence in any order.

It is even possible to give one or more of the words as simply the asterisk (*), which matches any word; the search is then assumed to be a consecutive one. (But avoid giving * as the final word, as the search will be slow.) As we will see below (under demutation) a hyphen is, in most circumstances, counted as a separate word, so search for sean-bhean as three words: sean - bean (as well as sean bean and seanbhean to cover any unhyphenated instances).

You may tick the Gan beinn ar an tsíneadh fhada checkbox if you want to include words which differ from that requested only by the presence or absence of an accent, eg. comhradh with this checkbox ticked will match comhrádh, cómhradh and cómhrádh as well. Ticking this box also includes words which differ from that requested by the addition of a hyphen, apostrophe or period — though most Gaelic words containing apostrophes or hyphens are indexed as two or more separate words anyway, as explained under decompounding below.

To type accented vowels, use your normal method of doing this under Windows. For information about keying accented letters under Windows, look here, or see the section "Keyboards layouts" near the end of this file. (But you will not require the support for dotted consonants offered by these keyboards, as lenition is always indicated by suffixing the letter h in Tobar na Gaedhilge.)

When all this is complete, you may click the OK button to produce the results.

Further hints on the selection of search words will come in the next section of this document.

• More about what to search for in a Gaelic index

Here are some pointers regarding what kinds of words are worth requesting.

When a word is requested, it is matched against a pre-compiled index of words from the chosen collection.  A scrolling alphabetic listing of the current index is shown, and will indicate what words are available. For Gaelic, this index consists of words which are aggregated in a number of ways to increase coverage:

• lowercased: the words in a Gaelic index have been converted to lowercase by replacing any capital letters by small letters; this even applies to proper names. Any capital letters you include in your request will also be so converted.

• decliticised: common enclitics, such as d' in d'ól, or 's in 'seadh, or -sa in agam-sa, are treated as separate words in the index (d' + ól; 's + eadh; agam + - + sa), and should also be detached in your request. Enclitics are normally signalled in running text by a hyphen or an apostrophe. But when there is no overt signal in similar cases (eg. agamsa, seadh), the splitting in the index will have been performed manually and is unlikely to have been exhaustive.

A number of common contracted words have been indexed under their parts, e.g. 'na (from ina) under ' and a; 'na (from chun an) under 'n and a'; 'na (from chun na) under ' and na; ab (from a ba) under a and b, or under a and b'; gurab under gur and a and b; and many other similar cases. This aspect is to be made more rigorous.

• decompounded: very few words containing a hyphen have been admitted to the indexes — a list of these can be obtained by searching for *-*. Rather, most hyphenated words have been treated atomistically in the indexes, and are found by seeking their parts, including the hyphen, eg. leith-phighinn by seeking the three items leith and - and pighinn, with checking of the "consecutive" option ("Díreach i ndiaidh a chéile").

• demutated: initial mutations are removed from words in the index; so, for example, fear, fhear and bhfear are all indexed as fear, while t-olc, n-olc and holc are all indexed as olc—but, where the mutation is permanent, it is retained, e.g. chugam, thart (in one of its senses), (go) dtí. You may have noticed the benefits of demutation and decliticisation in our athair example above. An initial mutation does not leave any trace in the index; and this is also true of any hyphen which is nothing but part of an initial mutation. When typing words of Gaelic to be searched for, remember to remove initial mutations, unless they are a permanent part of the word. Removal of initial mutations may seem counter-intuitive when requesting a sequence of words (eg. ár athair), but it is nonetheless required.

But the words in a Gaelic index are not lemmatized, i.e. terminally inflected forms, such as fear, fir, feara, must be searched for separately—although the wild card may often be used to advantage to retrieve the several related forms.

You may choose (by ticking Gan beinn ar an tsíneadh fhada) to make the search insensitive to accents. A suitable example might be searching together for words like comhartha and cómhartha. Both words are help separately in the index, and with this option selected a request for either of them will fetch both.

Finally, note that the index which will be searched is based on the words of the text which have been subjected to a limited and controlled degree of correction, as explained just below Figure 6 above.

• Searching in other languages

As well as searching the textbase for Gaelic words, indexes have been created in three additional languages, based on the words found in translations or originals in these languages of some of the Gaelic texts. As stated earlier, these translations/originals are at present confined to the Ulaidh collection, and cover only a proportion of it: Béarla (English), 40%; Fraincis (French), 19%; and Gearmáinis (German), 7%. (The German index remains somewhat experimental.)

Figure 10: Search of the English index of the Ulaidh collection for the word bunch. An example is shown from Ben-Hur, and the English and Gaelic and French and German of the sentence is displayed.

Figure 11: Search of the French index of the Ulaidh collection for the word accroché. An example is shown from Iascaire Inse Tuile, and the French and Gaelic and English and German of the sentence is displayed.

Figure 12: Search of the German index of the Ulaidh collection for the word Knurren. An example is shown from Scairt an Dúthchais, and the German and Gaelic and English and French of the sentence is displayed.

To search using another language, select the Ulaidh collection, and then the appropriate language. You will then have a further choice between Foirmeacha (word-forms) and Lemmata (lemma-types), because a rough and ready lemmatization has been applied to the English, French and German texts, resulting in two indexes for each of these languages. The statistics for these indexes are, at 2014/08/24:

• Ulaidh (Ulster)           Index: Béarla (foirmeacha). 6 Gaelic writers; 16 books;     37,770 word-forms;    1,594,407 word-tokens
•                                   Index: Béarla (lemmata).  6 Gaelic writers;   16 books;   29,979 lemma-forms;    1,592,465 lemma-tokens
•                                   Index: Fraincis (foirmeacha). 3 Gaelic writers;  8 books;    31,651 word-forms;    751,717 word-tokens
•                                   Index: Fraincis (lemmata).  3 Gaelic writers;   8 books;    15,616 lemma-forms;    750,874 lemma-tokens
•                                   Index: Gearmáinis (foirmeacha). 3 Gaelic writers;  3 books;   25,729 word-forms;   271,439 word-tokens
•                                   Index: Gearmáinis (lemmata).  3 Gaelic writers;   3 book;    16,983 lemma-forms;   271,291 lemma-tokens

The word-form indexes for the additional languages are lowercased, even for proper names, except for German, where the initial letter of a noun (or a name) retains its case — actually, the initial letter takes its case from the lemma we have assigned to it, which is usually the same thing (lemmatization is discussed below). Enclitics are separated (eg. English 's, 've, n't, French l', m', German 's (gibt's), 'n (so'n)). Hyphenated words are generally decomponded, eg. French garde-robe; but this policy has not been consistently applied to English, where eg. decompounded water-tight is found as well as unitary water-tight and watertight. In the German index, decompounding of (hyphenless) words, eg. weitergehen, has not been attempted.

When typing a word request into the forms index of an additional language, any uppercase letters are automatically converted to lowercase, except for the German index, where the initial letter of a word remains as typed.

Those searches used a word-forms index, but for these additional languages there are also the lemma-forms indexes. Using the lemmatized English index, a request for the man will match the words man or men; while, using the French lemmatized index, a request for homme will match the words homme or hommes; or using the German lemmatized index, a request for Mann will match words Mann, Mannes, Manne, Männer, Männern. (There is no immediate prospect of a Gaelic lemmatized index.)

Figure 13: A KWIC list of the examples of the lemma listen in Gadaidheacht le Láimh Láidir, according to our English lemmatized index. The corresponding material, in Gaelic and any other languages in which it is available, may be inspected, one example at a time, in the sentence display mode.

Figure 14: A KWIC list of the examples of the lemma abandonner (to abandon) in Ben-Hur, according to our lemmatized French index. The corresponding material, in Gaelic and any other languages in which it is available, may be inspected, one example at a time, in the sentence display mode.

Figure 15: A KWIC list of the examples of the lemma Baum (tree) in Scairt an Dúthchais, according to our lemmatized German index. The corresponding material, in Gaelic and any other languages in which it is available, may be inspected, one example at a time, in the sentence display mode.

It is important to understand, however, that our lemmatization of English and French and German has been performed automatically, using the Tree Tagger. This software is among the best of its kind, and lemmatization would have been impractical without it, but, as with all statistical operations, a percentage of errors is inevitable, and many remain despite much manual post-checking.

In making our lemmatized indexes, an initial capital has been retained in some words, mostly names (as well as for nouns in the German index); but when typing a word-request into a lemma index, no changes are made to the case typed. Therefore it will make a difference to the results whether the letters you type in your request to a lemmatized index are small letters or capitals. Keep an eye on the scrolling alphabetic list for guidance on what lemmas are available and when you should use a capital letter.

Also, foreign words are excluded from lemma indexes, and this is the main cause of the discrepancies between the counts of word-tokens and lemma-tokens in the indexes. It also makes the lemma indexes slightly more "mono-lingual" than the form indexes.

In the context of the additional languages, the Gan beinn ar an tsíneadh fhada box (accent-insensitive box) requires further explanation. Actually, the function of this box is more correctly described as to include among the retrieved words any which differ from the requested word only
• by the presence or absence of any accent, not just the acute, or
• by case differences, or
• by the intrusion of non-alphabetic non-punctuation characters such as £, %, period (indicating abbreviation), apostrophe (indicating elision), hyphen (indicating anonymisation), etc.
But some of these distinctions are already neutralized in the indexes, as we have explained, with the result that the practical effect of this box is to include words differing in accents when using either a forms or a lemmas index; and to include words differing in case when using either a lemmas index or the German forms index.

When displaying a sentence found through a lemmatized index, the Innéacsáilte display option shows the sentence in the first language as a list of the lemmas with which it is indexed.

Many lemmas are ambiguous, e.g. in English: pack or stamp or well or lie or bound or back; or in French: pas or tendre or vague or fin. Ideally, we would like to retrieve only the desired sense of an ambiguous lemma, and, at version 1.5, we have tried to separate the senses by part of speech, using four broad categories of N (noun), V (verb), J (adjective) and Z (other). Thus for example a request for English lemma well will be asked to choose between N, V and Z; a request for French lemma vague will be asked to choose between J and N. This may help in many cases, but not in others; for the English lemma lie, for example, a more useful division would be into recline and untruth, rather then into noun and verb. Further changes in this area may be expected in future versions.

• Translation equivalents

A related innovation is the calculation of translation equivalents. Given a word in the source language, this consists of a listing of the relatively most common words in the corresponding segments of the target language. This will clearly be more effective using lemmas than using words, so it is offered only with lemmatized English or lemmatized French as the source index. At present the target produced is a list of words (Gaelic, English or French), though again lemmas would be preferable but are not yet available. This technique has potential, but is limited at present by the amount of text available (English/Gaelic: about 1.6 million words; French/Gaelic: about 750,000 words; English/French: about 750,000 words). Results with the present amounts of text range from interesting to comical.

Where a lemmatized source index (English or French) is in use, translation equivalents may be chosen as a fourth output display mode, named Freagar-fhocla. The calculation, for a selected source-language lemma, may take a few moments. The resulting display is a list of target-language words, each accompanied by a score, and sorted on these scores (the user may have it re-sorted alphabetically on the words themselves). The scores — which are not raw word counts — may range from 99,999,999 down to 100,000. They measure how many times as common the target-language word is in the neighbourhood of the source-language lemma than in the target language corpus on average.

Figure 15: Search for Gaelic words collocating with the English lemma child, in the Ulster texts.

The chosen source-language lemma defines a set of sentences in the source-language corpus — those sentences in which it occurs — and a corresponding set of sentences in the target-language corpus — those sentences which translate them. This "select part" of the target-language corpus is studied, looking for words (freagar-fhocla, word-equivalents) which are more frequent there than in the target-language corpus on average.

If the source-language word is uncommon (read: selects less than one-thousandth part of the source language corpus), a warning is issued that the results may not be statistically useful, but no impediment is placed on calculating them.

If a target-language word turns out to be equally frequent in the select part as on average, it is given a score of 100,000; if it turns out to be twice as frequent in the select part as on average, it is assigned 200,000; and so on. Words less frequent in the select part than on average are discarded as uninteresting, so that 100,000 is the minimum score among those retained. At the other end of the range, the score 99,999,999 is assigned to any word which is 100 times or more as common in the select part as on average.

Even if a word falls within the range 100000..99999999, it is still omitted from the displayed list if its absolute frequency is small. This is intended to overcome "accidental" collocations, which will disappear naturally as more text is added, but may mask more significant data while they remain. A suitable empirical lower cutoff for absolute frequency of a word-equivalent is found to be the square root of one-tenth of the frequency of the source-language word.

Results are still poor enough with the amount of text available, but will improve as the quantity increases. Even at the present time, however, it may be of interest to input English lemmas from the following list, and to compare the results with the content of existing English–Irish dictionaries, noting what is found in the dictionaries but absent in the corpus, as well as what relevant equivalents are found in the corpus but not in the dictionaries: smoke, minute, also, yet, dog, ice, bee, garden, help, interest, gravel, cave, busy, cell, kitchen, open.