ARTICLES

 

Which Defining Model Contributes to More Successful Extraction of Syntactic Class Information and Translation Accuracy?

 

Watter definiëringsmodel dra by tot meer geslaagde onttrekking van inligting rakende sintaktiese kategorie en vertaalakkuraatheid?

 

 

Bartosz Ptasznik

University of Warmia and Mazury, Olsztyn, Poland (bartosz.ptasznik@uwm.edu.pl)

 

 

---

ABSTRACT

Definitions in English monolingual learners' dictionaries are the central focus of the paper. Metalexicographers have had a consuming interest in the following three types of definitions: analytical definitions (or classical definitions), full-sentence definitions (also called contextual definitions) and single-clause when-definitions. The use of when-definitions, the role of which is to define abstract nouns, has raised questions and doubts as to their efficacy on correct part of speech recognition of the definiendum, or item being defined, in light of the problems related to the substi-tutability of headwords and parts of definitions (lack of general category words in this definition format). By and large, existing research has substantiated the superiority of the classical definition-type over single-clause when-definitions with respect to the accuracy of word class identification. The current experiment attempts to further delve into the subject of part of speech recognition with regard to the three aforementioned defining formats - in previous studies only data from analytical and single-clause when-definitions were collated, since contextual definitions were not included in the study design. The study was conducted on a group of 120 advanced-level Polish university students of English. The subjects were tested on their ability of correct extraction of syntactic class information and translation accuracy of abstract noun headwords as regards the three predominant definition-types in English lexicographic practice.

Keywords: Learners' Dictionaries, Definitions, Monolingual Dictionaries, Analytical Definitions, Full-sentence Definitions, Single-clause When-definitions

---

OPSOMMING

Die hooffokus van hierdie artikel is definisies in Engelse eentalige aanleerderswoordeboeke. Metaleksi-kograwe het 'n intense belangstelling in die volgende drie tipes definisies gehad: analitiese defini-sies (of klassieke definisies), volsindefinisies (ook genoem kontekstuele definisies) en enkel-bysin when-definisies. Die gebruik van when-definisies, wat die definiëring van abstrakte selfstandige naamwoorde ten doel het, het vrae en twyfel laat ontstaan oor hul effektiwiteit in die herkenning van die korrekte woordsoort van die definiendum, oftewel die item wat gedefinieer word, met inag-neming van die probleme wat verband hou met die vervangbaarheid van trefwoorde en dele van definisies ('n gebrek aan algemenekategoriewoorde in hierdie definisieformaat). Met betrekking tot die akkuraatheid van woordsoortherkenning, het bestaande navorsing in hoofsaak die groter geslaagdheid van die klassieke definisie-tipe bo die enkel-bysin lü/ien-definisies bevestig. Die huidige eksperiment poog ook om verder ondersoek in te stel na woordsoortherkenning met betrekking tot die drie bogenoemde definisietipes - in vorige studies is slegs data van analitiese en enkel-bysin lülien-definisies met mekaar vergelyk, aangesien kontekstuele definisies nie deel gevorm het van die studiedoelwit nie. Hierdie studie is uitgevoer op 'n groep van 120 gevorderde vlak Poolse universiteitsstudente van Engels. Die proefpersone is met betrekking tot die drie hoof-definisietipes in die Engelse leksikografiese praktyk getoets op hul vermoëns om inligting rakende sintaktiese kategorie korrek te onttrek asook op hul vertaalakkuraatheid van abstrakte selfstandige naamtrefwoorde.

 Sleutelwoorde: Aanleerderswoordeboeke, Definisies, Eentalige Woorde-boeke, Analitiese Definisies, Volsindefinisies, Enkel-bysin When-definisies

---

 

 

1. How can words be defined?

It goes without saying that meaning is the main reason why people decide to consult dictionaries. This rather unsurprising fact had been long since established in one of the earliest dictionary use questionnaires, which were conducted on Polish and American students (Tomaszczyk 1979), as well as French students of English (Béjoint 1981). In brief, people use dictionaries with a view to learning or acquiring newly encountered lexical items. Significantly, the meanings of words are conveyed to dictionary users through the use of definitions in dictionaries. Consequently, this means that definitions are one of the key features of dictionaries, as without them dictionaries would simply not be able to serve their primary function - which is providing English learners with pertinent information about word meanings.

So how can words be defined? According to Richards and Taylor (1992), there are various strategies that can be adopted. Synonyms, antonyms¹, taxo-nomic² definitions (for example, when we define the word "rugby" as "a sport"), as well as definitions by exemplification or function are only a handful of defining strategies employed by those of us who attempt to explain to someone the meaning of a given word. But there is no denying that the implementation of such simple and basic defining techniques by lexicographers might not be a satisfactory method in most cases - the meanings of words have to be explained more scrupulously if an average student is to fully grasp the meaning of a word. Hence, dictionary compilers need to strive to enhance the quality of their dictionary's definitions (the present context here applies to English monolingual learners' dictionaries) if users are not to become discontented with the level of the dictionaries.

The most important types of definitions that are commonly applied in professional lexicographic practice as regards English monolingual learners' dictionaries have been discussed in the following section.

 

2. Types of definitions in English monolingual learner's dictionaries

The analytical definition (also called a classical definition, or Aristotelian definition) is the most basic and standard type of definition in lexicographic practice; hence, "traditional definition" is yet another common term for this specific definition-type. The following constituent parts form an analytical definition: 1) defi-niendum; 2) definiens; 3) genus³ proximum; and 4) differentiae specificae. In simple terms, the definiendum is the word (or term) that is being defined, the definiens is the "right-hand side, defining, part of the definition" (Adamska-Sataciak 2012: 324), the genus proximum is the general category (superordinate) under which the item being defined can be classified, while the term differentiae specificae apply to the specific or distinguishing features of the definiendum. An example of the analytical⁴ definition has been demonstrated below:

house⁵- a building⁶ that someone lives⁷ in, especially one that has more than one level and is intended to be used by one family (Longman Dictionary of Contemporary English, 6th edition)

Dziemianko and Lew (2013: 156) rightly notice that substitutability is an essential and inherent characteristic of the traditional definition-type. Put another way, the word class of the superordinate (general category word) needs to match the syntactic class of the definiendum - as can be seen from the example above, both "building" and "house" are nouns. Last but not least, classical definitions lend themselves to defining especially concrete nouns, as well as, for example, verbs of motion⁸ and verbs of making or creating (Atkins and Rundell 2008: 415).

In general, substitutability can be perceived as an advantageous defining strategy. Some metalexicographers are of the opinion that substitution seems to be the right strategy that allows one to grasp the meaning of definitions (Fischer 1991) and also this definition-type makes it possible to describe "meanings (...) with precision and economy" (Atkins and Rundell 2008: 439). From a different perspective, research (Deese 1967; Miller 1985; Miller and Gildea 1985; Fischer 1991; Nesi and Meara 1994, Nesi 2000; Nesi and Haill 2002)

has demonstrated that analytical definitions tend to result in incorrect word class recognition of the items that are being defined. One possible explanation for this phenomenon is the so-called "kidrule⁹ strategy" (Miller and Gildea 1985) - a simple strategy based on the reasoning that specific fragments (words) of definitions can replace the definiendum in different contexts. Not surprisingly, adhering to this strategy may prove to be an erroneous choice on numerous occasions.

The full-sentence definition is a signature feature¹⁰ of the Collins Cobuild Advanced Learner's Dictionary and has become a defining characteristic of the dictionary since the 1980s. An example of this type of definition is shown below:

resurgence - if¹¹ there is a resurgence of an attitude or activity, it reappears and grows (Collins Cobuild Advanced Learner's Dictionary, 8th edition)

The full-sentence definition (also called a contextual definition) has the following structure (Lew and Dziemianko 2006a: 228): 1) hinge (if); 2) co-text1 (there is a); 3) topic (resurgence); 4) co-text2 (of an attitude or activity); 5) matching framework (it); and 6) gloss (reappears and grows). In short, a full-sentence definition consists of two parts: left-hand side and right-hand side parts of the whole definition. The left-hand side part of the definition most importantly has a hinge (if/when), the item being defined (topic) and the surrounding environment or context in which the word being defined most typically appears (co-text). As for the right-hand side of the definition, there is a matching framework (it - this specific pronoun refers to the topic) and a gloss (it explains the meaning of the topic) - this suggests that the right-hand contextual frame of the definition provides learners with an explanatory comment as to what the word being defined specifically means. As for the initial part of the definition, one discovers more about the context of the item being defined; as an example, in this case we learn that the abstract noun resurgence most probably frequently occurs with the preposition of¹²(colligational preferences). All things considered, this is a real advantage of contextual definitions - as the name suggests, these definitions provide us with some context in which the given word routinely functions (we are provided with grammatical and collocational information of lexical items) and consequently we hone our linguistic production skills. An added advantage would be that dictionary users learn what words mean and how they are used in a more naturalistic setting - the words being defined are incorporated into sentences. As for the drawbacks, a few have been noted. One problem is that full-sentence definitions are lengthy - they tend to be longer than analytical definitions. Furthermore, Piotrowski (1994: 127) contends that on the contrary to traditional definitions, contextual definitions hinder the process of substitutability of full-sentence definitions for the item being defined. Rundell (2006: 326) contributes to the discussion by emphasizing the restrictiveness¹³ of full-sentence definitions, as they primarily describe only the most frequent instances of word use, or in other words they demonstrate only the most common contexts in which a given item appears, excluding the less common ones despite their undisputed significance and relevance in various different situations.

The third type of definition format which has left its mark in English monolingual learners' dictionaries is the so-called single-clause when-definition¹⁴, which can be characterized as being shorter than its double clause counterpart (full-sentence definition), it is used in folk-defining and conversation, and also sporadically in spontaneous defining as demonstrated in Fabiszewski-Jaworski's research (2011). Most importantly, however, it is perceived by lexicographers as an appealing and straightforward method of defining abstract nouns in dictionaries (for example, Longman Dictionary of Contemporary English, Cambridge Advanced Learner's Dictionary, Cambridge Learner's Dictionary). More specifically, Lew and Dziemianko (2012: 997) describe the single-clause when-defining style as "a stand-alone relative clause introduced with the relative word when". An example of the single-clause wzen-definition for the entry "rebirth" is illustrated below:

rebirth - when something or someone becomes alive again after dying (Longman Dictionary of Contemporary English, 6th edition)

The schematic structure of the single-clause definition format above is the following (Lew and Dziemianko 2006a): 1) the word when; 2) co-text2 (something or someone); and 3) gloss (becomes alive again after dying). The question which perhaps ought to be addressed¹⁵ is how does the single-clause when-definition of a word affect one's ability to correctly identify the syntactic category of this word. Put another way, the analytical definition-type is structured in a convenient way for dictionary users - learners can easily discover the word class of the item being defined through a simple and uncomplicated analysis of the general category word in the classical definition. However, the single-clause when-definition does not permit one to extract this type of information. Of course, the more aware language learners would most probably contrive to find this information in the entry itself, as dictionary entries are designed in a user-friendly way, and it is not rocket science to know that information about an item's part of speech is located at the very beginning of an entry. Nevertheless, the definition itself does not contain pertinent information about a given word's syntactic category and hence it would be interesting to see how this peculiarity of the single-clause when-definition influences learners' accuracy of part of speech recognition.

The following section elaborates on the topic of part of speech recognition and single-clause when-definitions in the context of empirical research that has been conducted in this field (mainly Dziemianko and Lew's studies). Also, the subject of syntactic class recognition and the single-clause when-defining format is the area of interest of this paper (the present study).

 

3. Empirical studies on syntactic category recognition of nominal headwords

Dziemianko and Lew conducted a series of studies (2006a; 2006b; 2013) dealing with the effect of definition-type on part of speech recognition in English monolingual learners' dictionaries. 129 native speakers of Polish representing an upper intermediate and advanced English level participated in their first experiment (2006a). The design of the study included 20 test items (10 abstract nouns and 10 distracters¹⁶ - 5 low-frequency verbs and 5 adjectives) which appeared with their definitions, half¹⁷ of which were analytical definitions and the remaining half were single-clause when-definitions (there were two different test versions, each test had a different assignment of definition-types to the target items, each subject was given only one test version). Instead of using the actual headwords selected, pseudo words replaced the original items in the study in order to remove information about a word's part of speech that could have possibly been obtained from the morphological structure of words. The verbs and adjectives were included in the design of the experiment with a view to achieving some variation as to the part of speech of the items tested. As for the procedure, the subjects had to provide the Polish equivalents of the target items (the items appeared only with their definitions) and form English sentences with these items. Each student had 45 minutes to complete the tasks. Importantly, statistical significance was achieved in both types of tasks - the students managed to correctly identify a target item's part of speech 66.7% of the time in the case of analytical definitions and 33.2% of the time when dealing with single-clause when-definitions, while in the sentence formation task, the subjects met with success in 53.6% and 26.6% of the cases when being assisted by analytical and single-clause when-definitions respectively. Despite a definite advantage of the analytical definition format, Lew and Dziemianko stressed the need for more research¹⁸ as the microstructure adopted for their study lacked example sentences and, most importantly, grammatical information, which normally is incorporated into dictionary entries and creates an opportunity for dictionary users to discover an unknown word's syntactic class (part-of-speech labels).

A follow-up study (Dziemianko and Lew 2006b) was done on 238 Polish students of English who primarily had an intermediate level of proficiency in English. There were minor differences between this experiment and the previous one with respect to study design. First of all, in the present study a richer microstructure was incorporated into the dictionary entries - part-of-speech labels were included, as well as example sentences, syntactic codes and usage labels. Existing research (Bogaards and Van der Kloot 2002; Dziemianko 2006) has shown that grammar codes and examples have some significance when it comes to acquiring part-of-speech information from dictionary entries. Second, the task was different - this time the subjects were to complete a 45-minute multiple-choice task and they were provided with the possible answers (three Polish equivalents¹⁹ which were of a different part of speech - adjectives, nouns, verbs), which means that the subjects were given semantic information. In addition to this, there was no compose-sentence task, this meaning that the students were supposed to devote all their attention to syntactic class identification only. On the whole, the present subjects were exposed to a more naturalistic environment in light of having access to a more elaborate microstruc-ture (however, there was no information about phonetic transcription), nevertheless, the task focusing solely on syntactic class recognition could also be perceived as an artificial one, not having much in common with natural and everyday dictionary consultation. Interestingly, the effect of defining-style format was statistically nonsignificant. Regardless of this finding, it is important to mention that analytical definitions slightly outperformed single-clause when- definitions - word class identification scores amounted to 86.1% for the classical defining model and 85.4% for the single-clause when-defining model. These results suggest that Polish intermediate-level students can remain boastful about their dictionary reference skills - they possess the ability to successfully extract syntactic information from part-of-speech labels found in the micro-structure of entries, and this skill allows them to dexterously compensate for when-definitions lacking in this specific type of information. Moreover, although there was only a slight difference, analytical definitions once again proved to be superior to the less common single-clause when-definitions with respect to the accuracy of correct syntactic class identification. In conclusion, Dziemianko and Lew felt the need for conducting yet another study - only this time they wanted to enrich the microstructure of entries by including information about phonetic transcription, as well as create a "less syntax-focused task²⁰" (Dzie- mianko and Lew 2006b: 862).

In their third study (2013), Dziemianko and Lew departed from adhering to identical microstructure designs that were applied in previous studies. To be more precise, they incorporated information about phonetic transcription in-between the lemma sign and syntactic class label, the aim of which was to minimize the salience of part-of-speech labels that was present in the second experiment. Also, more improvements were introduced in comparison with their first two studies through the implementation of "explicit grammatical information, style labels and examples of usage" (Dziemianko and Lew 2013: 164), all of this being done with a view to exposing the subjects to the most naturalistic environment of dictionary consultation possible. Furthermore, the subjects were asked to complete a 30-minute meaning-based task - provide the translation (single-word equivalent) of the English item into their native language. In the second study, the students were given the answers - they were provided with three Polish equivalents which were all of a different part of speech. The Polish equivalents were derivatives from the same root, which meant that the task was explicitly syntax-based. In other words, the subjects could have easily discovered that the task was grammar-oriented, or focusing on one's ability of part of speech recognition. The present meaning-based task was different in this respect. Apart from the modifications mentioned above, the study design did not differ much from the paradigm selected for the earlier studies. 134 subjects participated in the experiment who were Polish learners of English (upper-intermediate - advanced level of proficiency in the English language). Once again, the analytical definition-type achieved a higher score (90.1%) than the single-clause when-definition (87%) and this difference was found to be statistically significant²¹, however, in reality this was only a marginal difference of three percentage points and hence Dziemianko and Lew contend that the effect of defining style on part of speech recognition of abstract nouns is rather small. In their view, advanced dictionary users have enough reference skills to acquire syntactic class information from part-of-speech labels located within entries, rather than from definitions themselves.

Needless to say, a more complete microstructure plays an "important compensatory role²² (...) in POS identification" (Lew and Dziemianko 2012: 1002). Nevertheless, the findings also suggest that not only part-of-speech labels but also example sentences are a reliable source of information about the syntactic class of words - "syntactic class labels and examples obviously attract users' attention and offset the apparent syntactic emptiness of iien-definitions (Dzie-mianko and Lew 2013: 169). On the whole, Dziemianko and Lew²³ are of the opinion that the use of single-clause iien-definitions with a richer microstruc-ture in English monolingual learners' dictionaries is reasonable, but perhaps excluding this definition-type from dictionaries might be a more shrewd decision in light of the fact that the subjects in the experiment were advanced students of English whose reference skills could have been rated as above average - this meaning that less advanced students could still perhaps encounter some difficulty with respect to the extraction of part-of-speech information from single-clause iien-definitions embedded in abstract noun entries of richer microstructures. This view is in line with Atkins and Rundell's stance on single-clause iien-definitions (this defining format ought not to be applied in English monolingual learners' dictionaries).

In the present context, perhaps the results of one more study should be briefly discussed. Fabiszewski-Jaworski and Grochocka (2010) experimented the effect of definition-type on part of speech recognition on 150 upper-intermediate-level (level of proficiency in English) native speakers of Polish. The task was to provide Polish equivalents of English target items appearing with either analytical or single-clause iien-definitions. Not surprisingly, analytical definitions scored significantly higher (33.3%) than single-clause iien-defini-tions (26.2%) with respect to the accuracy of part of speech recognition. Most importantly, Fabiszewski-Jaworski and Grochocka explain that the inclusion of iien-definitions in dictionaries seems to be a logical solution especially when dictionary compilers encounter problems with finding the right general category words required for defining abstract concepts.

In the following section, the author discusses the current study's research design.

 

4. Research design

Dziemianko and Lew's studies inspired the present paper and served as a paradigm for the research design in the current experiment. The aim was to test the usefulness of the most common definition-types in English monolingual pedagogical dictionaries for advanced learners. Two research questions were addressed:

(1) Which defining-model (analytical definitions vs. full-sentence definitions vs. single-clause iien-definitions) contributes to more successful extraction of syntactic class information from abstract noun entries?

(2) Which defining model (analytical definitions vs. full-sentence definitions vs. single-clause when-definitions) contributes to higher translation accuracy with respect to abstract noun entries?

Similarly to previous studies, the analytical and single-clause when definition-types were selected for the analysis as intuition suggests that the genus and differentia model of the classical definition is conducive to leading to more effective extraction of part-of-speech information from abstract noun entries rather than the single-clause when-defining format. As for two-clause²⁴when-definitions, it is possible that this specific definition-type can be beneficial to dictionary users with respect to the accuracy of part-of-speech recognition as its inherent nature provides learners not only with some general contextual information but also grammatical information, which is a key factor in deriving pertinent part-of-speech information from dictionary entries. However, as no attempt had been previously made to test the effectiveness of the full-sentence definition format on the accuracy of part of speech recognition, the contextual definition-type was added to the design of the present study and a null hypothesis that there would be no relationship or association among the three groups (analytical definitions/full-sentence definitions/single-clause when-definitions) was assumed to be true. Put another way, the effect of definition-type (independent variable) on both syntactic class recognition and also translation accuracy would turn out to be nonsignificant - a separate one-way GLM ANOVA was run for each dependent variable - (1) syntactic class recognition; and (2) translation accuracy. The data were calculated in SPSS (version 25).

24 test items were selected for the study - 12 abstract nouns, 6 low-frequency verbs and 6 low-frequency adjectives. The verbs (ensnare, glisten, lash, wring, yank, devour) and adjectives (obnoxious, adamant, comely, concomitant, egregious, fecund) which were used in the study were the distracters (data were not collected from these items) and their role was to avoid having subjects focus solely on the part of speech of the items (grammar-oriented tasks), as this was primarily supposed to be a semantic-based²⁵ task. Also, another function of the distracters was to reduce the saliency of the target items in the whole study. The 12 nouns (dexterity, disturbance, omen, deceit, disquiet, peculiarity, quirk, abstraction, compliance, legislation, apprehension, infatuation) used in the experiment were replaced with pseudo words (tiezon, menave, conluse, sardy, reprive, tortex, overlar, arouch, vargin, mortap, sharpeg, barrex) - the aim of such a study design being that any morphological information carried by nouns had to be removed from context, so that the subjects would not take advantage of their existing knowledge about the language, or derivational information about words, which would most probably allow them to easily identify the part of speech of the test items. Nonexistent words, however, did not replace the verbs and adjectives used in the study. The pseudo words (nouns) were generated by a nonword-generating program, called WordGen²⁶ (Duyck, Desmet, Verbeke and Brysbaert 2004). All of the test items (24 items) were assigned random locations within specific test versions thanks to the Random Integer Generator tool available online and free of charge. There were three different test versions. Each test version²⁷ consisted of 12 nouns (pseudo words), 6 verbs and 6 adjectives. Four nouns in one test version appeared with analytical definitions, the other four nouns with full-sentence definitions and the remaining four with single-clause when-definitions. The assignment of specific definition-types to nouns was rotated across different test versions and as a result a cross-balanced design was achieved (confounding effect of item and subject were reduced to a minimum, each subject was exposed to a specific defining style of the target items ^Λ Α of the time). Each single test item (headword) in the test formed an entry with its own microstructure. In order for the aim of the study to be met, a richer microstructure²⁸, which was based on the one applied in Dziemianko and Lew's most recent study (2013), was adopted for the experiment. To be more precise, the headword/test item (appearing in boldface font) was followed by pronunciation information (prepared by the author of the study), part-of-speech labels, grammar codes, usage labels, definitions and example sentences. The aim of the incorporation of such a microstructure was to create a naturalistic²⁹ environment for dictionary use consultation, which of course can never be fully achieved under artificial experimental conditions. The lexicographic data for the definitions and examples used in the study were taken from various English monolingual learners' dictionaries: LDOCE online, COBUILD8, OALD9, CALD4, MED2 and MWALED. Sporadically, the author had to slightly modify some of the definitions³⁰ and example sentences. Moreover, the pronunciation and grammatical information in entries was based on lexicographic information from LDOCE online.

There were 120 subjects (males and females) who were native speakers of Polish. Their English proficiency level had been assessed as upper-intermediate or advanced, and they were third-year and fourth-year students of English at a Polish university (University of Warmia and Mazury in Olsztyn). The subjects were asked to provide a one-word Polish equivalent of the English items in the spaces provided (this was not a multiple-choice task). The subjects had 45 minutes to complete the task during their regular class at the university. Prior to the experiment, they were briefly instructed by the experimenter (the subjects were instructed orally and they were provided with a Polish instruction in written form) and the subjects were also told how much time they would have for the completion of the whole task.

As for the grading system, the subjects' answers were assessed separately for syntactic class recognition and translation accuracy. In the case of the former, subjects would receive a score of "1" only when being able to provide the correct part of speech of the target item - the word written down in the answer sheet had to be a noun. The meaning of this noun was not taken into consideration but only its grammatical category. In the case of the latter, this time not the part of speech but the meaning of the word was most important. The answers that were considered to be correct were not only the Polish equivalents of the target item found in the dictionary, but also other answers had to be taken into account as being possibly correct in the present context, as the subjects were supposed to infer what the correct equivalent was only on the basis of the type of information which they had access to in the task given - more specifically, the microstructure of entries. The Polish equivalents of the test items, which were marked as either correct or incorrect in the experiment (part of speech recognition and translation accuracy), are listed in Table 1 in the following section.

 

5. Results

The mean results for syntactic class identification accuracy by definition-type are illustrated below in Figure 1.

Figure 1 confirms the superiority of the analytical definition-type over the single-clause iien-defining format. The subjects managed to correctly select the part of speech of a headword when dealing with analytical definitions in 86.6% of the cases, while when being exposed to single-clause iien-definitions the accuracy rate amounted to 81.8%, which means that there was a difference of almost 5 percentage points (4.8%) between the two defining styles. As for full-sentence definitions, altogether the subjects achieved a score of 83.7%, which indicates that contextual definitions only slightly outperformed single-clause iien-definitions by approximately 2 percentage points (1.9%), but fared worse than the classical definition by almost 3 percentage points (2.9%). The success rate for individual items ranged between 66%-95% - the test items compliance (vargin) and dexterity (tiezon) having an average of 66% and 95% respectively.

The statistical analysis revealed that the effect of definition-type on syntactic class identification accuracy was nonsignificant³¹ (one-way GLM ANOVA, F(2,1437)=2.09, p=0.124). Moreover, there was very little practical significance and the effect size was very small (η2 = 0.003). Hence, the null hypothesis was not rejected.

The mean results for translation accuracy by definition-type are demonstrated in Figure 2.

 

 

Once again it was the analytical definition which contributed to the highest scores. Figure 2 reveals that the analytical definition-type helped the subjects in the translation tasks (English into Polish) more than the remaining two types of definitions. Exposure to analytical definitions led to successful translations in 69.7% of the cases, which was only a marginally better score than the 68.7% success rate of single clause iien-definitions. Interestingly, the single-clause iien-defining style proved more beneficial than the two-clause iien-definition format in the translation tasks, outperforming full-sentence definitions by almost 7 percentage points (translation accuracy amounted to 68.7% for analytical definitions, whereas for full-sentence definitions it amounted to only 61.8%). Overall, translation accuracy ranged between 42%-91% for individual test items - the test items abstraction (arouci) and appreiension (siarpeg) had a mean of 42% and 91% respectively.

In this case, the null hypothesis was rejected. A separate ANOVA was run for translation accuracy and the analysis showed that there was a statistically significance³² difference among the groups (one-way GLM ANOVA, F(2,1437)=4.02, p=0.018), which strongly suggested that one or more pairs of treatments were significantly different. Once again, there was little practical significance and the effect size proved to be very small (η2 = 0.006). The Tukey HSD test revealed that there was a statistically significant difference between analytical and full-sentence definitions with respect to translation accuracy (p=0.025<.05), while the difference between full-sentence and single-clause iien-definitions approached statistical significance (p=0.061>.05). No statistical significance was noted between the analytical and single-clause when-definition types (p=0.899>.05).

Given the subjects' answers in the test, the most frequent Polish equivalents of the test items that were provided by the subjects have been listed in Table 1.

 

6. Discussion and conclusions

To reiterate, Dziemianko and Lew's series of studies (2006a; 2006b; 2013) functioned as the paradigm for the current research. In brief, each consecutive experiment endeavored to correct the previous study design and as a result adopt a more satisfactory and practical design, one which would eliminate the artificiality of the tasks at hand and at the same time provide a more natural dictionary use environment. Innovations involved including a rich er micro-structure with not only the definitions of words but also examples, syntactic class labels, grammar codes, usage labels and information about pronunciation. Phonetic transcription was located in-between the headwords and part-of-speech labels, which increased the level of naturalness of the tasks by minimizing the saliency of syntactic class labels. Moreover, Dziemianko and Lew shifted away from applying grammar-oriented tasks and opted for using meaning-based tasks in their most recent experimental design. Hence, the aim of the present study was to test the general usefulness of the most common defining styles in pedagogical monolingual dictionaries for learners of English with respect to syntactic class identification and translation accuracy, by drawing from Dziemianko and Lew's conclusions. Significantly, as no significant existing study had previously incorporated full-sentence definitions into its design, contextual definitions were included in the present experiment and were treated as one of the three levels of the independent variable - definition-type - the two other levels being the default analytical and single-clause when-definitions. Also, another modification in the study design involved the introduction of an additional dependent variable - translation accuracy. Therefore, the effect of definition-type on the subjects' accuracy of translation from English into Polish was tested.

The current study demonstrates that analytical definitions hold a clear advantage over full-sentence and single-clause when-definitions (Research question 1: Which defining-model contributes to more successful extraction of syntactic class information from abstract noun entries?). This finding is in line with Dziemianko and Lew's three studies (2006a; 2006b; 2013), as well as Fabiszewski-Jaworski and Grochocka's (2010) experiment, where in all four studies the classical definition format proved superior to the less common and newer in pedagogical dictionaries for learners of English single-clause when-definition. In the present study, in the context of part-of-speech identification accuracy, exposure to analytical definitions led to success in 86.6% of the cases, whereas headwords which were defined through full-sentence and single-clause when-definitions achieved a score of 83.7% and 81.8% respectively. Notwithstanding the lack of a statistically significant difference among the three groups, these numbers suggest the analytical definition's dominance, especially over single-clause when-definitions, bearing in mind that classical definitions outperformed single-clause when-definitions with regard to syntactic class identification accuracy in all of the earlier studies. It seems, then, that the oldest and most traditional way of defining words is most beneficial to dictionary users. The genus and differentia paradigm allows for the substitutability of the genus expression and the headword (when the genus proximum and the headword are of the same word class), which most apparently suits the user more than any other existing and practiced-in-lexicography defining model. The data also suggest that two-clause when-definitions are slightly more useful when it comes to the extraction of part-of-speech information from entries than their single-clause counterparts, this finding perhaps being unsurprising as full-sentence definitions are lengthier definition-types which contain more contextual information (also grammatical information) about the word being defined.

The mean part-of-speech identification accuracy by definition-type of Dziemianko and Lew's studies (2006a; 2006b; 2013) and the present study is summarized in Table 2.

Similarly to Dziemianko and Lew's study (2013), the study participants in the current experiment were not asked to underline any fragments of the entry during entry consultation and hence it was not possible to draw any conclusions or infer which specific parts of entries were most useful for the part-of-speech recognition task. However, a detailed analysis of Table 2 indicates that the syntactic class recognition scores ranged between approximately 33%-67% in the first study and 82%-90% in the second, third and present study. Consequently, the evidence supports the conclusion that a more sophisticated micro-structure enhances the extraction of syntactic class information from entries regardless of the given definition-type at hand, in light of the fact that studies 2 and 3, as well as the current study, all incorporated a more elaborate micro-structure into the experimental design. As a result, the data demonstrate that it is highly likely that the presence of, for example, syntactic class labels and example sentences in dictionary entries enhances the process of extracting correct part-of-speech information from abstract nominal headwords. This finding confirms the conclusion from Dziemianko and Lew's study (2013) that complete entries allow more proficient dictionary users to acquire pertinent information about a word's part of speech even when being exposed to the syntactically impoverished single-clause iien-definitions. As for full-sentence definitions, there is some likelihood that a higher amount of contextual information in this definition format, for example, grammatical information, compensates for the absence of the abundance of such information from single-clause iien-defi-nitions and perhaps for this reason success rates for the contextual definition-type slightly exceeded the scores for the single-clause iien-definition by barely 1.9%. However, the lack of such empirical evidence in other studies does not allow to arrive at any correct conclusions. On balance, despite everything said, it would seem wise to agree with Fabiszewski-Jaworski and Grochocka (2010) that perhaps the analytical definition is the safer option for lexicographers while single-clause when-definitions can be helpful when the genus and differentia model of the classical definition does not stand up to its expectations. The same should possibly apply to full-sentence definitions, nevertheless, it must be admitted that there were only marginal differences in the experiment among all three means (range of 81.8%-86.6%), which in turn implies that the three aforementioned definition-types are comparably effective with respect to the accuracy of part of speech recognition.

As far as translation accuracy is concerned (Research question 2: Which defining model contributes to higher translation accuracy with respect to abstract noun entries?), exposure to full-sentence definitions (61.8%) led to the lowest scores of all three types of definitions (69.7% - analytical definitions; 68.7% - single clause when-definitions) analyzed in the study. First and foremost, single-clause when-definitions fared worse than the analytical definition by only 1 percentage point. There is no doubt that such a minor difference cannot have any significance at all in the present context. This finding perhaps is indicative of the fact that both analytical and when-definitions are equally beneficial defining styles in relation to meaning-oriented tasks such as translating from the target language to one's mother tongue. In other words, traditional definitions are more valuable for learners than single-clause when-definitions when it comes to identifying the correct part of speech of a headword, however, this definite advantage of the classical definition seems to be counterbalanced in more meaning-related tasks. Second, it is possible that the non-sub-stitutability of full-sentence definitions hinders the process of correct translation (despite the fact that the word class of the Polish equivalents in the present experiment was not taken into account when assessing and assigning scores to the Polish equivalents in the translation task), a drawback generally (but only by principle) non-existent in analytical definitions, which heavily rely on the genus and differentia model. Nevertheless, when-definitions also suffer from this inconvenience and yet this defining style contrived to achieve comparable if not almost identical results in comparison with the Aristotelian defining style. One possible explanation is that Polish learners of English do not really tend to define words in their native language through the use of contextual definitions (Mikoiajczak-Matyja 1998), and also this specific definition-type is not widely applied in Polish lexicography - hence the problems that the subjects in the present study might have encountered in the target language. Another possibility is that full-sentence definitions could be perceived as too long or wordy, whereas analytical definitions and single-clause when-defini-tions seem to be shorter and concise, or simply more straightforward, lacking in more complex syntactic structures. Moreover, we cannot count out the learner variable. English learners differ from one another in many respects - cultural background, intelligence levels, motivation, linguistic abilities, etc. Some of these factors may have influenced the scores achieved in the translation task. Finally, the lexicographic information at hand might have played its part in the present experiment. Some concepts expressed in the subjects' second language could have been either easier or more difficult to render into Polish by the study participants.

One of the findings from Dziemianko and Lew's study (2013) was that single-clause when-definitions which fit the pattern when + indefinite pronoun (someone/something) had a rather negative effect on part-of-speech recognition accuracy of abstract nouns, especially when being compared to when-defini-tions which take the when + personal pronoun/noun phrase structure. Of the two types of definitions mentioned above, it was the latter that contributed to higher syntactic class identification success rates, or results that were nearly as successful as the scores of the analytical definition-type in the experiment. In the current study, the item analysis of single-clause when-definition scores reveals a marked tendency for the when + personal pronoun/noun phrase when-definitions to outperform their when + indefinite pronoun counterpart. The data³⁴are gathered in Table 3.

The mean for part of speech identification accuracy of the when + personal pronoun/noun phrase definition-type was by approximately 7.8 percentage points higher (83.8%) than the mean of the when + indefinite pronoun definition (76%). Likewise, taking into consideration translation accuracy, the when + personal pronoun/noun phrase definition pattern had better scores by as many as 22.8 percentage points (78.3%) than the when + indefinite pronoun single-clause when-definition (55.5%). As illustrated above in Table 3, the when + personal pronoun/noun phrase defining style clearly dominated the when + indefinite pronoun definition format within specific test versions (there were three different test versions). In the case of part of speech identification accuracy, the item overlar (quirk) was one exception, as this item achieved an accuracy of 85%, while the item menave (disturbance) had an accuracy of only 77.5% in test version no. 3. A similar pattern occurred with respect to the accuracy of translating target items from English into Polish, only this time it occurred in test version no. 2 - the test item tortex (peculiarity) had an accuracy of 70%, while the item mortap (legislation) performed worse by about 2.5% (67.5%). By and large, these observations indicate that the when + indefinite pronoun definition-type may indeed be the inherently weaker defining style of the two defining single-clause when-formats discussed above. In order to see whether this is actually true, an additional study would be needed - one with an experimental design suited for testing and comparing the effectiveness of the when + indefinite pronoun and when + personal pronoun/noun phrase defining styles.

To sum up, more empirical research is required if any right conclusions are to be reached about the role of full-sentence definitions on syntactic class recognition or translation accuracy. At the present moment, it seems like it would be worthwhile to undertake research into the usefulness of the single-clause when-defining style in the context of part-of-speech extraction of information from entries. More specifically, the single-clause when + personal pronoun/noun phrase and when + indefinite pronoun (someone/something) defining models could be tested experimentally and contrasted with one another.

 

Endnotes

1 . Synonyms and antonyms - definitions which are one-word equivalents of the item being defined.

2 . Taxonomic definitions involve classifying words into classes or subclasses.

3 . The term genus proximum is also called the "genus expression" (Atkins and Rundell 2008: 414).

4 . For more information about analytical definitions see Adamska-Salaciak (2012).

5 . Defniendum.

6 . Genus proximum.

7 . Differentiae specificae - the distinguishing features of this specific building are: 1) it is a building that someone lives in; 2) it is a building that has more than one level; 3) it is intended to be used by one family.

8 . Atkins and Rundell (2008) mention that the genus expression "walk" can be used for defining words such as "stroll" or "tiptoe" (verbs of motion), while the genus expression "copy" can be the superordinate for the words "reproduce" or "photocopy" (verbs of making or creating).

9 . "Kidrule" - a rule applied by children - hence the term "kidrule".

10 . From a historical point of view, the Collins Cobuild Dictionary is also known for: (1) being the first corpus-based English learners' dictionary; and (2) its grammatical column.

11 . Hinge - usually if/when form the hinge in full-sentence definitions.

12 . Another common grammatical pattern is resurgence in.

13 . This problem has been termed by Rundell as "overspecification".

14 . The single-clause when-definition begins with the word "when".

15 . This subject has been the focus of a few studies conducted by Anna Dziemianko and Robert Lew (this topic is also the primary focus of the present paper).

16 . The distracters appeared only with analytical definitions and they were not included in the analysis.

17 . Half of the definitions of the ten target items (abstract nouns).

18 . One suggestion was to include a richer microstructure in the next study, while the other one was that it seemed to "be worthwhile to further extend the scope of the study and compare when-definitions not only with analytical, but also contextual ones" (Lew and Dziemianko 2006a: 237). The current study extends the scope of Dziemianko and Lew's studies (2006a; 2006b; 2013) by introducing the full-sentence definition into the design of the study.

19 . These equivalents were derivatives from the same root.

20 . The intention was to reduce the salience of part of speech information (syntactic labels) by having this type of information separated from the lemma sign through the inclusion of phonetic transcription information.

21 . However, the effect size was small and hence there was little practical significance.

22 . This is especially true of single-clause when-definitions (Dziemianko and Lew 2013: 169) and not necessarily analytical definitions. When comparing Dziemianko and Lew's second (2006b) and third study (2013), the success rate tripled for ithen-definitions in a rich-microstructure environment.

23 . Also, Dziemianko and Lew (2013) notice that single-clause ithen-definitions can especially be misleading when the following structure of these definitions is applied: when + indefinite pronoun (someone/something). They imply that whenever possible single-clause when-defini-tions should perhaps adopt a different structure: ihen + personal pronoun/nominal phrase. Importantly, single-clause ihen-definitions which take the ihen + indefinite pronoun (someone/something) structure decrease part-of-speech recognition scores even in more elaborate microstructures.

24 . Full-sentence definitions in other words.

25 . In this regard, the study was no different from Dziemianko and Lew's contribution (2013) to the topic of study.

26 . WordGen is a simple tool which is based on the CELEX and Lexique lexical databases. Its main function is to select words and generate nonwords not only in the English language, but also in Dutch, German and French.

27 . By contrast, in Dziemianko and Lew's three studies there were always 20 test items: 10 abstract nouns, 5 verbs, 5 adjectives.

28 . See Appendix at the end of the paper to acquire more information about the microstructure applied in the present experiment.

29 . The order of specific types of information appearing in the microstructure of entries that was adopted was also based on Dziemianko and Lew's study (2013) with a view to avoiding having a typical syntax-based task. Hence, pronunciation information separated the lemma sign from the part-of-speech labels and in this respect a more natural setting of dictionary use was created for the subjects in the current study.

30 . For example, due to the lack of a full-sentence definition for a specific word in various dictionaries, the author invented his own example of such a definition, on the basis of the lexicographic data in the dictionary. One example includes the test item disturbance (menave). The following definition for the noun entry disturbance can be found in the Oxford Advanced Learner's Dictionary of Current English (9th edition): "actions that make you stop what you are doing, or that upset the normal state that something is in". This definition was changed into: "a menave (disturbance) is an action that makes you stop what you are doing, or that upsets the normal state that something is in".

31 . The effect was nonsignificant at the significance level of 0.05.

32 . The effect was statistically significant at the significance level of 0.05.

33 . Study 1 - Lew and Dziemianko (2006a); Study 2 - Dziemianko and Lew (2006b); Study 3 - Dziemianko and Lew (2013).

34 . As this analysis was not the primary aim of the paper, it has been mentioned and elaborated on in the Discussion and Conclusions section.

 

References Dictionaries

Delacroix, L. (Ed.). 2014. Longman Dictionary of Contemporary English. Sixth edition. Harlow: Longman. [LDOCE6]        [ Links ]

Delacroix, L. (Ed.). 2014. Longman Dictionary of Contemporary English. Sixth edition (free online version). Harlow: Longman = https://www.ldoceonline.com/. [LDOCE online]        [ Links ]

Deuter, M. (Ed.). 2015. Oxford Advanced Learner's Dictionary of Current English. Ninth edition. Oxford: Oxford University Press.         [ Links ]

Gillard, P. (Ed.). 2003. Cambridge Advanced Learner's Dictionary. First edition. Cambridge: Cambridge University Press. [CALD1]        [ Links ]

Gillard, P. (Ed.). 2004. Cambridge Learner's Dictionary. Second edition. Cambridge: Cambridge University Press. [CALD2]        [ Links ]

McIntosh, C. (Ed.). 2013. Cambridge Advanced Learner's Dictionary. Fourth edition. Cambridge: Cambridge University Press. [CALD4]        [ Links ]

Perrault, S.J. (Ed.). 2008. Merriam-Webster's Advanced Learner's English Dictionary. Springfield: Mer- riam-Webster. [MWALED]        [ Links ]

Rundell, M. (Ed.). 2007. Macmillan English Dictionary for Advanced Learners. Second edition. Oxford: Macmillan Education. [MED2]        [ Links ]

Sinclair, J. (Ed.). 2014. Collins COBUILD Advanced Learner's Dictionary. Eighth edition. London: HarperCollins. [COBUILD8]        [ Links ]

Summers, D. (Ed.). 2003. Longman Dictionary of Contemporary English. Fourth edition. Harlow: Longman. [LDOCE4]        [ Links ]

Walter, E. (Ed.). 2005. Cambridge Advanced Learner's Dictionary. Second edition. Cambridge: Cambridge University Press. [CALD2]        [ Links ]

Other literature

Adamska-Sataciak, A. 2012. Dictionary Definitions: Problems and Solutions. Studia Linguistica Universitatis Iagellonicae Cracoviensis 129: 323-339.         [ Links ]

Atkins, S. and M. Rundell. 2008. The Oxford Guide to Practical Lexicography. Oxford: Oxford University Press.         [ Links ]

Béjoint, H. 1981. The Foreign Student's Use of Monolingual English Dictionaries: A Study of Language Needs and Reference Skills. Applied Linguistics 2(3): 207-222.         [ Links ]

Bogaards, P. and W.A. Van der Kloot. 2002. Verb Constructions in Learners' Dictionaries. Braasch, A. and C. Povlsen (Eds.). 2002. Proceedings of the Tenth EURALEX International Congress, EURA-LEX 2002, Copenhagen, Denmark, August 13-17, 2002. Vol. II: 747-757. Copenhagen: Center for Sprogteknologi, Copenhagen University.         [ Links ]

Deese, J. 1967. Meaning and Change of Meaning. American Psychologist 22(8): 641-651.         [ Links ]

Duyck, W., T. Desmet, L. Verbeke and M. Brysbaert. 2004. WordGen: A Tool for Word Selection and Nonword Generation in Dutch, English, German, and French. Behavior Research Methods, Instruments, & Computers 36(3): 488-499.         [ Links ]

Dziemianko, A. 2006. User-friendliness of Verb Syntax in Pedagogical Dictionaries of English. Tübingen: Max Niemeyer Verlag.         [ Links ]

Dziemianko, A. and R. Lew. 2006b. When You are Explaining the Meaning of a Word: The Effect of Abstract Noun Definition Format on Syntactic Class Identification. Corino, E., C. Marello and C. Onesti (Eds.). 2006. Proceedings of the 12th EURALEX International Congress, Torino, Italy, September 6-9, 2006: 857-863. Allesandria: Edizioni dell'Orso.         [ Links ]

Dziemianko, A. and R. Lew. 2013. When-Definitions Revisited. International Journal of Lexicography 26(2): 154-175.         [ Links ]

Fabiszewski-Jaworski, M. 2011. Spontaneous Defining by Native Speakers of English. Akasu, K. and S. Uchida (Eds.). 2011. ASIALEX 2011 Proceedings. Lexicography: Theoretical and Practical Perspectives. Papers Submitted to the Seventh ASIALEX Biennial International Conference, Kyoto Terrsa, Kyoto, Japan, August 22-24, 2011: 102-109. Kyoto: The Asian Association for Lexicography.         [ Links ]

Fabiszewski-Jaworski, M. and M. Grochocka. 2010. Folk Defining Strategies vs. Comprehension of Dictionary Definitions: An Empirical Study. Kernerman, I. and P. Bogaards (Eds.). 2010. English Learners' Dictionaries at the DNSA 2009: 89-105. Tel Aviv: K Dictionaries.         [ Links ]

Fischer, U. 1991. How Students Misunderstand Definitions: Some Evidence for Contextual Representations of Word Meanings. Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society, 7-10 August 1991, Chicago, Illinois: [no pagination]. Hillsdale: Lawrence Erlbaum.         [ Links ]

IBM SPSS Statistics Package. Version 25.

Lew, R. and A. Dziemianko. 2006a. A New Type of Folk-inspired Definition in English Monolingual Learners' Dictionaries and its Usefulness for Conveying Syntactic Information. International Journal of Lexicography 19(3): 225-242.         [ Links ]

Lew, R. and A. Dziemianko. 2012. Single-clause when-definitions: Take Three. Fjeld, R.V. and J.M. Torjusen (Eds.). 2012. Proceedings of the 15th EURALEX International Congress, 7-11 August 2012, Oslo: 997-1002. Oslo: Department of Linguistics and Scandinavian Studies, University of Oslo.         [ Links ]

Mikotajczak-Matyja, N. 1998. Definiowanie pojgc przez przecigtnych uzytkowników jgzyka i przez leksy-kografów. Poznan: Sorus.         [ Links ]

Miller, G. 1985. Dictionaries of the Mind. Proceedings of the 23rd Annual Meeting of the Association for Computational Linguistics: 305-314. Chicago: University of Chicago.         [ Links ]

Miller, G. and P. Gildea. 1985. How to Misread a Dictionary. AILA Bulletin, 1985: 13-26.

Nesi, H. 2000. The Use and Abuse of EFL Dictionaries: How Learners of English as a Foreign Language Read and Interpret Dictionary Entries. Tubingen: Max Niemeyer Verlag.         [ Links ]

Nesi, H. and R. Haill. 2002. A Study of Dictionary Use by International Students at a British University. International Journal of Lexicography 15(4): 277-305.         [ Links ]

Nesi, H. and P. Meara. 1994. Patterns of Misinterpretation in the Productive Use of EFL Dictionary Definitions. System 22(1): 1-15.         [ Links ]

Piotrowski, T. 1994. Z Zagadnien Leksykografi. Warsaw: Panstwowe Wydawnictwo Naukowe.         [ Links ]

Random Integer Generator. https://www.random.org/integers/.

Richards, J. and A. Taylor. 1992. Defining Strategies in Folk Definitions. Perspectives (City University of Hong Kong) 4: 1-8.         [ Links ]

Rundell, M. 2006. More than One Way to Skin a Cat: Why Full-sentence Definitions have not been Universally Adopted. Corino, E., C. Marello and C. Onesti (Eds.). 2006. Proceedings of the 12th EURALEX International Congress, Torino, Italy, September 6-9, 2006: 323-338. Allesandria: Edizioni dell'Orso.         [ Links ]

Tomaszczyk, J. 1979. Dictionaries: Users and Uses. Glottodidactica 12: 103-119.         [ Links ]

 

 

APPENDIX: THE STUDY - SAMPLE 5 TEST ITEMS (VERSION 1)

INSTRUKCJA: Test zawiera 24 wyrazy w języku angielskim (są to raczej wyrazy trudne). Każdy wyraz wraz z podaną informacją o tym wyrazie tworzy tzw. hasło słownikowe. Twoim zadaniem jest przetłumaczyć wyrazy angielskie na język polski. Uwaga - polskie odpowiedniki angielskich wyrazów muszą być wyrazami jednowyrazowymi!