Lost in Text. A Cross-Genre Analysis of Linguistic Phenomena within Text

Chiara Buongiovanni•, Francesco Gracci•, Dominique Brunato�, Felice Dell’Orletta�

• University of Pisa{c.buongiovanni, f.gracci1}@studenti.unipi.it

�Istituto di Linguistica Computazionale “Antonio Zampolli” (ILC–CNR)ItaliaNLP Lab - www.italianlp.it

{dominique.brunato, felice.dellorletta}@ilc.cnr.it

Abstract

Moving from the assumption that formal,rather than content features, can be used todetect differences and similarities amongtextual genres and registers, this paperpresents a new approach to the linguis-tic profiling methodology, which focuseson the internal parts of a text. A casestudy is presented showing that it is possi-ble to model the degree of variance withintexts representative of four traditional gen-res and two levels of complexity for each.1

1 Introduction

The combined use of corpus-based and computa-tional linguistics methods to investigate languagevariation has become an established line of re-search. The heart of this research is the so-called‘linguistic profiling’, a technique in which a largenumber of counts of linguistic features automat-ically extracted from parsed corpora are used asa text profile and can then be compared to av-erage profiles for groups of texts (van Halteren,2004). Although it has been originally developedfor authorship verification and recognition, lin-guistic profiling has been successfully applied tothe study of genre and register variation, followingBiber’s claim that “linguistic features from all lev-els function together as underlying dimensions ofvariation, with each dimension defining a differentset of linguistic relations among registers” (Biber,1993). By modeling the ‘form’ of a text throughlarge sets of linguistic features extracted from rep-resentative corpora, it has been possible not onlyto enhance automatic classification of genres (Sta-matatos et al., 2001), but also to get a better un-

1Copyright c©2019 for this paper by its authors. Use per-mitted under Creative Commons License Attribution 4.0 In-ternational (CC BY 4.0).

derstanding of the impact of features in classifyinggenres and text varieties (Cimino et al., 2017).

This paper moves in this framework butpresents a new approach of linguistic profiling,in which the unit of analysis is not the documentas a whole entity, but the internal parts in whichit is articulated. In this respect, our perspectiveis similar to the one proposed by Crossley et al.(2011), who developed a supervised classificationmethod based on linguistically motivated featuresto discriminate paragraphs with a specific rhetori-cal purpose within English students’ essays. How-ever, differently from that work, we focus on Ital-ian and enlarge the analysis to four traditional tex-tual genres and two levels of language complexityfor each. The aim is i) to explore to what extent theinternal structure of a text can be modeled via lin-guistic features automatically extracted from textsand ii) to study whether the variance across differ-ent parts of a text changes according to genre andlevel of complexity within genre.

2 Corpora and approach

Our investigation was carried out on four genres:Journalism, Educational writing, Scientific proseand Narrative. For each genre, we selected thetwo corpora described in Brunato and Dell’Orletta(2017), which represent a ‘complex’ and a ‘sim-ple’ language variety for that genre, where thelevel of complexity was established according tothe expected reader. Specifically, the journalisticgenre comprises a corpus of articles published be-tween 2000 and 2005 on the general newspaperLa Repubblica and a corpus of easy-to-read arti-cles from Due Parole, a monthly magazine writ-ten in a controlled language for readers with ba-sic literacy skills or mild intellectual disabilities(Piemontese, 1996). The corpus belonging to theEducational genre is articulated into two collec-tions targeting high school (AduEdu) vs. primaryschool (ChiEdu) students. For the scientific prose,

the ‘complex’ variety is represented by a corpusof 84 scientific articles on different topics, whilethe ‘simple’ one by a corpus of 293 Wikipedia ar-ticles, extracted from the Italian Portal ‘Ecologyand Environment’. For the Narrative genre, wetook a dataset specifically developed for researchon automatic text simplification. It consists of 56texts covering short novels for children and piecesof narrative writing for high school L2 students ar-ranged in a parallel fashion, i.e. for each originaltext a manually simplified version is available. Forour study, the original texts and the correspondingsimplified versions were chosen as representativeof the complex variety and the simple variety, re-spectively.

All corpora were automatically tagged by thepart-of-speech tagger described in Dell’Orletta(2009) and dependency parsed by the DeSR parser(Attardi et al., 2009) to allow the extraction ofmore than 80 linguistic features, on which we re-lied to investigate our research questions. Thesefeatures (detailed in Section 3) capture linguis-tic phenomena of a different nature, with a fo-cus on morpho–syntactic and syntactic structure,and were selected since they were proven effec-tive for genre classification in previous works, aswell as in other scenarios all focused on the analy-sis of the ‘form’ of the text rather than its content,such as linguistic complexity, readability assess-ment (Collins-Thompson, 2014), native languageidentification (Malmasi et al., 2017).

As a preliminary step for the analyses, all doc-uments were split into a fixed number of sec-tions, where each section is composed by a cer-tain number of paragraphs, roughly correspondingto the three main parts of the rhetorical structureof a text (i.e. introductory, body and conclud-ing paragraphs). According to the literature, forsome genres, such as academic writing, the dis-tinction into paragraphs is quite rigid and followsthe so-called ‘five-paragraphs’ format (Crossley etal., 2011) which adheres to the rhetorical goalsof the document, i.e. the first and the last para-graph correspond respectively to the introductionand the conclusion, and the three middle ones tothe body part. However, based on a preliminaryinvestigation of our corpora we preferred to definea six-section subdivision in order to avoid flatten-ing too much the distinctions across genres. Thecorpora under analysis indeed are made by docu-ments which are very different in terms of average

length: for instance, scientific articles are on av-erage longer than others (184 sentences per docu-ment) and this reflects the fact that the body partis more dense and possibly articulated into moremiddle paragraphs. For each document, the sixsections are thus composed by an average numberof sentences that depends on the document length,ranging from 2 sentences per section, for the short-est documents, to ∼35 for the longest ones. Ac-cording to this choice, documents shorter than sixsentences were discarded, thus we finally reliedon a corpus of 1168 documents (see Table 1 fordetails). As a result of the stage, we representedeach section of a document as a vector of features,whose values correspond to the average value thateach feature has in all sentences included in thesection.

In order to understand whether and to what ex-tent the different parts of a text represent distinc-tive varieties with a peculiar linguistic structure,we carried out two statistical analyses. First, weassessed whether the difference of the feature val-ues in each section was statistically significant.Specifically, we performed a pairwise comparisonbetween each section and the following one (i.e.1/2, 2/3, 3/4 etc), as well as between the first andthe last section (i.e. 1/6); the latter was deliber-ately aimed at verifying whether our set of featuresalone is able to distinguish between the introduc-tory and the closing part of a document, the twomore distant sections of a text which are supposedto have a more codified structure. Secondly, weverified whether there is a correlation between thevalues of features in the two sections under com-parison. For both analyses, all data were calcu-lated across and within genre. The cross-genreanalysis was focused on genre only, thus consid-ering the two corpora representative of the com-plex and simple variety as a unique one for eachgenre. In the second scenario, the two corporawere kept distinct to investigate if there is an effectof genre that is preserved despite language com-plexity changes.

3 Linguistic features

The set of features extracted from previously iden-tified sections are distinguished into three differ-ent categories, according to the level of annotationfrom which they derive.

Raw Text Features: they include the averageword and sentence length (char tok and n tokens

Genre Corpus Initial dataset Analyzed datasetN◦ Doc Tokens N◦ Doc Tokens Avg sentence/section

Journalism Repubblica (Rep) 318 232.908 304 230.789 5.1DueParole (2Par) 321 73.314 303 71.228 2.1

Educational High-schools educ. materials (AduEdu) 70 48.103 69 47.854 3.9Primary schools educ. materials (ChilEdu) 60 23.192 52 22.382 3.5

Scientific Prose Scientific articles (ScientArt) 84 471.969 84 471.883 35.9Wikipedia articles (WikiArt) 293 205.071 249 200.681 4.9

Narrative Terence&Teacher-original versions (TT orig) 56 27.833 53 25.931 4.2Terence&Teacher-simplified versions (TT simp) 56 25.634 54 23.866 4.3

Table 1: Statistics about the corpora used in the study.

in Table 2), calculated as the number of charactersper token and of tokens per sentence, respectively.

Morpho-syntactic Features: i.e. distributionof unigrams of part-of-speech distinct into 14coarse-grained pos tags (cpos ) and the 37 fine-grained tags (pos ) according to the ISST-TANLannotation.

Syntactic Features: these features model gram-matical phenomena of different types, i.e:- the probability of syntactic dependency types e.g.subject (dep subj), direct object (dep dobj), mod-ifiers, calculated as the distribution of each typeout of the total dependency types according to theISST-TANL dependency tagset;- the length of dependency links, i.e. the av-erage length of all dependency links (each onecalculated as the number of words occurringbetween the syntactic head and the dependent)(avg links l) and of the maximum dependencylink (max links l);- the order of constituents with respect to the syn-tactic head: as a proxy of canonicity effects, itis calculated the relative position of the subject,object and adverb with respect to the verbal headand the position of the adjective with respect to thenominal head;- the parse tree structure, in terms of featurescalculating: the depth of the whole parse tree(sent depth) (in terms of the longest path fromthe root of the dependency tree to some leaf); thewidth of the parse tree (sent width), measured asthe highest number of nodes placed on the samelevel; the average number of dependents for allverbal and nominal heads (avg dependent);- subordination features: within the group of syn-tactic features, a in–depth analysis was devotedto model subordination phenomena by measuring:the average distribution of subordinate clauses forsentence (avg sub clause), the percentage of sub-

Figure 1: Average sentence length in the 6 sectionsacross genres.

ordinate clauses with respect to the main clause (%sub main) and the percentage of embedded sub-ordinate clauses, i.e. subordinate clauses depen-dent on other embedded subordinate clauses (%sub minor); for each type, it is also calculatedthe average depth (subord depth) and weight (sub-ord width) of the parse tree generated by the sub-ordinate clauses and their relative order with re-spect to the clause on which they depend.

4 Data Analysis

Table 2 illustrates the main findings we obtained.Specifically, it shows all features which turned outto have a statistically significant variation in atleast one of the six pairwise comparisons, or a cor-relation score > 0.3 according to the Spearman’scorrelation coefficient. A first clear result is thatthe higher number of features varying in a statisti-cally significant way occurs in the journalistic andscientific genre, both considered as whole (i.e. rowg for each feature) and with respect to the languagecomplexity variety (rows s and c). The oppositetrend is reported for educational texts, which isprobably due to the heterogeneous nature of this

Figure 2: Distribution of lexical parts-of-speech in the four genres.

genre that includes documents of different textualtypologies (course books, pieces of literature etc.).

If journalism and scientific prose are the twogenres with the highest internal variance, the com-parison between sections allows us to get a betterunderstanding of this data. Specifically, for bothgenres, the majority of significant variations areobserved between the first and the second sectionand between the first and the last one. This sug-gests that the introduction is a stylistic unit witha peculiar linguistic structure with respect to thebody and the conclusion. It is characterized e.g.by shorter sentences (Figure 1), likely due to thepresence of the title in both newspaper and sci-entific articles, and by a distinctive distributionof Parts–of–speech (Figure 2). With this respect,this data are consistent with other studies in theliterature, e.g. (Voghera, 2005), and also withprevious findings we obtained on the same cor-pora (Brunato et al., 2016), showing that scientificprose and newswire texts rely more on the nominalstyle. However, with the proposed approach, wewere able to go further in this analysis, highlight-ing that noun/verb ratio is always higher in the firstsection than all other ones. Besides, at least fornewspaper articles, this feature appears as a genremarker which is not affected by language com-plexity, since the same tendency is observed whenthe ‘simple’ and the ‘complex’ corpus are ana-lyzed independently. The same does not hold forother features related to syntax and, in particular,

to the use of subordination. In this case, the ‘shift’between the introduction and the subsequent partof texts yields significant variations only for arti-cles of Repubblica. Specifically, the first sectioncontains less embedded sentences (sent depth: 1stsect: 5.55; 2nd sect: 7.76), and a lower presence ofsubordinate clauses, which appear as structurallysimpler e.g. in terms of depth (subord depth: 1stsect: 1.67; 2nd sect: 3.5) and width (subord width:1st sect: 0.94; 2nd sect: 1.97). Conversely, for thesimple variant of this genre (i.e. the articles of theeasy-to-read newspaper 2Parole), we do not ob-serve significant changes affecting these features;this is not particularly surprising since subordina-tion is always less represented in this corpus withrespect to all the other ones.

Leaving aside the similar tendencies character-izing the introduction, Journalistic and Scientificprose show a different behavior when we focuson the internal structure of text. While in thiscase much fewer features vary in a significant way,the majority occurs in the journalistic genre only,especially between the second and the third sec-tion. Again, they concern a different distributionof morpho-syntactic categories but also some syn-tactic features related to subordination. Accordingto these data, we can conclude that the journalisticgenre has a more rigorous structure and that it ispossible to capture the boundaries between differ-ent parts by using linguistic features that are notrelated to the content of the article.

features Journalism Scientific Prose Narrative Educational1/2 2/3 3/4 4/5 5/6 1/6 1/2 2/3 3/4 4/5 5/6 1/6 1/2 2/3 3/4 4/5 5/6 1/6 1/2 2/3 3/4 4/5 5/6 1/6

Raw text features

n tokensg XX X∗ - ∗ - ∗ - ∗ XX XX - - - ∗ - ∗ XX - - - - - X∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗s XX - - - - - XX - - - - XX - ∗ - - - - X - ∗ - ∗ - ∗ - ∗ X∗ - ∗c XX - - ∗ - ∗ - ∗ XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - - X - ∗ - - ∗ - - -

char tokg - ∗ - ∗ - ∗ - ∗ - ∗ X X - - ∗ - ∗ - X - - ∗ - ∗ - - - - ∗ - ∗ - ∗ - ∗ - ∗ - ∗s - ∗ - ∗ - ∗ - ∗ - XX X - - ∗ - - XX - - ∗ - ∗ - ∗ - - X∗ - ∗ - ∗ - ∗ - ∗ - ∗c - - ∗ - ∗ - ∗ - ∗ - - ∗ X∗ - ∗ - ∗ - ∗ - ∗ - - ∗ - ∗ - - - - ∗ - ∗ - ∗ - ∗ - ∗ - ∗

Morpho-syntactic features

cpos ADJg - - - - - - X X - ∗ - - ∗ - - - - - - - - ∗ - ∗ - - ∗ - ∗s X - - - - - XX - - ∗ - - X - - - - - - XX - - - - - ∗c X - - - - X - - ∗ - ∗ - ∗ - ∗ - ∗ - - - - ∗ - ∗ - ∗ - - ∗ - ∗ - - ∗ -

cpos ADVg XX∗ X∗ - - - XX XX∗ - - - ∗ - XX - - - - - X - - - ∗ X - -s XX - - - - XX XX∗ - - - - XX - - - - - X X - - ∗ X - -c XX∗ - ∗ - - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - - - - - - ∗ - - -

cpos CONJg XX X - - - XX XX - - - - XX - - - X - - - X X - - -s XX - - - - XX XX - - - - XX - - - - - - - X - - - -c XX - - - - XX X∗ - ∗ - ∗ - ∗ - ∗ X∗ - - ∗ - - - ∗ - - - - - - -

cpos NOUNg XX∗ X∗ - ∗ - ∗ - ∗ XX XX - - - ∗ - XX XX - - - ∗ - XX∗ - - ∗ - ∗ - ∗ - ∗ - ∗s XX∗ - ∗ - - - ∗ XX XX - - - - XX X - - - ∗ - XX X - ∗ - ∗ - - - ∗c XX∗ XX∗ - ∗ - ∗ - ∗ XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ X∗ - - - ∗ - X∗ - - - ∗ - ∗ - ∗ -

pos PROP Ng XX∗ - ∗ - ∗ - ∗ - ∗ XX∗ XX - ∗ - ∗ X∗ - ∗ XX X - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - ∗s XX∗ - ∗ - ∗ - ∗ - ∗ XX∗ XX - ∗ - ∗ - ∗ - ∗ XX - - ∗ - ∗ - ∗ - ∗ - X∗ - ∗ - ∗ - ∗ - ∗ - ∗c XX XX - - ∗ - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - ∗ - ∗ - ∗ - ∗ - - ∗ - ∗ - ∗ - ∗ - - ∗

cpos VERBg XX X - - - XX XX - - - - XX X - - - - - - - - X X -s XX - - - - XX XX - - - - XX X - - - - - - - - X XX -c XX XX - - ∗ - ∗ XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ X - - - - - - - ∗ - ∗ - - -

pos AUXg X∗ X∗ - ∗ - ∗ - ∗ X XX∗ - - - - XX - - - - - - - - - - X -s - ∗ - ∗ - ∗ - ∗ - ∗ - XX - - - - XX - - - ∗ - - - - - - - - - ∗c XX∗ XX∗ - ∗ - ∗ - ∗ XX - ∗ - - - ∗ - - ∗ - - - - ∗ - - - - - - - -

Syntactic features

dep dobjg X X - - - XX XX - - - - XX X X X - - X - X - - - Xs - X - - - XX XX - - - - XX X XX - ∗ - - X - XX - - ∗ - Xc XX - - - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - - - - - - - - -

dep subjg - - - - - - XX - - - - XX X - - - - - - - ∗ - - - -s - - - - - - XX - - - - XX - - ∗ - - - - - ∗ - ∗ - - ∗ - -c X - - - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ X - - - - - - - - - - -

max links lg XX - X∗ - ∗ - XX XX - - - ∗ - XX - - - - - - ∗ - - ∗ - ∗ - ∗ - ∗ - ∗s XX - - - - X XX - - - - XX - - - - - - - - - ∗ - - ∗ - ∗c XX - X - ∗ - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - - - ∗ - - ∗ X - - -

avg links lg XX - X - - XX XX - - - - XX - - - - - - - - - - - ∗ -s X - - - - - XX - - - - XX - - - - - - - - - ∗ - - ∗ -c XX - X - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - - - - ∗ - - - ∗ - - ∗ - - - -

sent depthg XX - ∗ - ∗ - ∗ - ∗ XX XX - - - X∗ XX - ∗ - - ∗ - ∗ - ∗ - - ∗ - ∗ - ∗ - ∗ - ∗ - ∗s - - - X - X XX - - X - ∗ XX - ∗ - - - ∗ - ∗ - - - ∗ - ∗ - ∗ -∗ X∗c XX - - ∗ - ∗ - ∗ XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - ∗ - ∗ - - - ∗ - ∗ - - ∗ - ∗ - ∗

sent widthg XX X - - - XX XX - - - - XX - - - - - - - ∗ - ∗ - - - ∗ -s XX - - - - - XX - - - - XX - - - - - - - ∗ - ∗ - - - ∗ - ∗c XX - X - ∗ - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - ∗ - - - - ∗ - - ∗ X - ∗ - -

avg dependentg XX X - ∗ - ∗ - ∗ XX XX - ∗ - ∗ - ∗ X∗ XX X - ∗ - ∗ - - ∗ X - ∗ - ∗ - ∗ - ∗ - ∗ - ∗s X - - - - ∗ - XX - - - - XX - - - ∗ - - ∗ X - - ∗ - ∗ - ∗ - ∗ - ∗c XX X - ∗ - ∗ - ∗ XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - ∗ - ∗ - - ∗ X - ∗ - ∗ X - ∗ - ∗ - ∗

Subordination features

avg sub clauseg XX X∗ - ∗ - - ∗ XX XX - - - - XX X - - - ∗ - ∗ XX - - - - - ∗ - ∗s - - - - - XX XX - - - - XX - - - - ∗ - ∗ X - - - - - ∗ - ∗c XX - - - ∗ - XX - ∗ - ∗ - - ∗ - ∗ - ∗ XX X - - - - - ∗ - - - - -

subord depthg XX X - - - XX XX - - - X XX - ∗ - - ∗ - ∗ - ∗ - - - - ∗ - ∗ - -s - - - - - X XX - - - - XX - - - - ∗ - ∗ XX - - - ∗ - ∗ - ∗ Xc XX - - - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - ∗ - - - X - - - -

subord widthg XX X - - - XX XX - - - X XX - - - - - - - - - ∗ - - -s - - - - - X XX - - - - XX - X - - ∗ - ∗ X - X - ∗ - ∗ - ∗ -c XX - - - - XX - ∗ - ∗ - - ∗ - ∗ - ∗ - - - - ∗ - - - - - - - ∗ -

% sub maing XX X - - - XX XX - - - - XX - - - - - ∗ XX - - - - - -s - - - - - XX XX - - - - XX - - - - - ∗ XX - X∗ XX∗ - ∗ X∗ -c XX - - - - XX - ∗ - ∗ - ∗ - ∗ - ∗ - ∗ - - - - - - - X - - - -

% sub minorg XX - ∗ - - ∗ - XX - - - - - X - - - - X - - - ∗ X - - Xs - - - - - X - - - - - X - - - - - ∗ - ∗ - - X - - ∗ - ∗c XX - - - - XX - ∗ - ∗ - - ∗ - - X - - - ∗ - - - - - - ∗ - X

Table 2: A set of linguistic features resulting as significant in at least one pairwise comparison. XX meanshighly statistically significant (p < 0.001), X statistically significant (p < 0.05), - no significance; ∗ correlationrelated to the Spearman’s rank correlation coefficient (rho > 0,3), g=global corpus, s=simple variety of thecorpus, c=complex variety of the corpus.

5 Conclusion

In this paper we have presented a novel approachto the study of language variation, which re-lies on the prerequisites of the linguistic profil-ing methodology but with the specific purpose ofmodeling the stylistic form of the different partswithin a text. A cross-genre investigation on fourtraditional genres in Italian, and two levels of com-plexity for each, showed that morpho-syntacticand syntactic features are differently distributedacross subsections of texts belonging to a spe-cific genre and language variety. This approachhas important implications for research on genrevariation since it suggests that the characteriza-tion of texts and texts varieties should benefit byinspecting corpora from this fine-grained perspec-tive. A better understanding of linguistic phenom-ena characterizing the introductory, middle andconclusive parts of a text is also highly relevantnot only to enhance automatic genre classificationbut also for other natural language processing ap-plications devoted to modeling style: e.g. in edu-cation, as a component of intelligent tutoring sys-tems able to provide detailed feedback to studentsin writing courses or for the automatic generationof texts with the stylistic properties of a specificgenre and level of complexity.

Acknowledgments

This work was partially supported by the 2-year project ADA, Automatic Data and docu-ments Analysis to enhance human-based pro-cesses, funded by Regione Toscana (BANDOPOR FESR 2014-2020).

ReferencesGiuseppe Attardi, Felice Dell’Orletta, Maria Simi,

Joseph Turian. 2009. Accurate dependency pars-ing with a stacked multilayer perceptron. In Pro-ceedings of EVALITA 2009 - Evaluation of NLP andSpeech Tools for Italian 2009. Reggio Emilia, Italy,December 2009.

Douglas Biber. 1993. Using register-diversified cor-pora for general language studies. ComputationalLinguistics, 19(2), 219–242.

Dominique Brunato and Felice Dell’Orletta. 2017. Onthe order of words in Italian: a study on genre vscomplexity. International Conference on Depen-dency Linguistics (Depling 2017), 18-20 September2017, Pisa, Italy.

Dominique Brunato, Felice Dell’Orletta, SimonettaMontemagni and Giulia Venturi. 2016. Mon-itoraggio linguistico di Scritture Brevi: aspettimetodologici e primi risultati. A. Manco e A.Mancini (eds.), Scritture Brevi: segni, testi e con-testi. Dalle iscrizioni antiche ai tweet, Collana distudi Quaderni di AION-Linguistica, Universita diStudi di Napoli “L’Orientale”, Napoli, 149–176.

Andrea Cimino, Martijn Wieling, Felice Dell’Orletta,Simonetta Montemagni, Giulia Venturi. 2017. Iden-tifying Predictive Features for Textual Genre Classi-fication: the Key Role of Syntax. Proceedings of4th Italian Conference on Computational Linguis-tics (CLiC-it), 11-13 December, 2017, Rome.

Kevyn Collins-Thompson. 2014. Computational As-sessment of text readability. Recent Advances in Au-tomatic Readability Assessment and Text Simplifica-tion. Special issue of International Journal of Ap-plied Linguistics, 165:2, John Benjamins PublishingCompany, 97-135.

Crossley, S.A., Dempsey, K., McNamara, D.S. 2011.Classifying paragraph types using linguistic fea-tures: Is paragraph positioning important? Journalof Writing Research.

Felice Dell’Orletta. 2009. Ensemble system for part-of-speech tagging. In Proceedings of EVALITA 2009- Evaluation of NLP and Speech Tools for Italian2009, Reggio Emilia, Italy, December 2009.

S. Malmasi, E. Keelan, A. Cahill, J. Tetreault, R. Pugh,C. Hamill, D. Napolitano, and Y. Qian 2017. Areport on the 2017 native language identificationshared task. In Proceedings of the 12th Workshopon Building Educational Applications Using NLP.

Maria Emanuela Piemontese. 1996. Capire e farsicapire. Teorie e tecniche della scrittura controllata.Napoli, Tecnodid.

Efstathios Stamatatos, Nikos Fakotakis and GeorgeKokkinakis. 2001. Automatic text categorization interms of genre and author. Computational Linguis-tics, (26) 471–495.

Hans van Halteren. 2004. Linguistic profiling for au-thor recognition and verification. In Proceedingsof the Association for Computational Linguistics(ACL04), 200207.

Miriam Voghera. 2005. La misura delle categorie sin-tattiche. In Chiari Isabella / De Mauro Tullio (eds.)Parole e numeri. Analisi quantitative dei fatti di lin-gua, Aracne, Roma, 125–138.