دانلود کتاب پردازش زبان طبیعی برای CORPUS LINGUISTICS

Corpus linguistics has entered a golden age, driven by both the amount and the range of language that is now available for linguistic analysis. Corpus data is able to represent a population’s usage at scale, bypassing the limitations which made introspection so important in the 1950s. But this wide availability of language data requires that linguists have the methods available to analyze it. And while there has been a surge of advances in natural language processing and computational linguistics, these advances have become increasingly disconnected from corpus linguistics and linguistic theory. This Element brings natural language processing and corpus linguistics together, showing how computational models can be used to answer both categorization and comparison problems. These computational models are presented using five case studies that will be introduced in the next section, ranging from syntactic analysis to register analysis to corpus-based sociolinguistics. The goal here is to show how to use these computational models, what linguistic questions they can answer, and why it is important to scale up corpus linguistics in this way. A linguist can use this Element to learn how to use natural language processing to answer linguistic questions they are already familiar with. And a computer scientist can use this Element to learn about the linguistic assumptions and limitations behind computational methods, matters that are too often disregarded within natural language processing itself. A categorization problem is about assigning a predefined label to some piece of language. At the word level, this could involve asking whether a particular open-class word is a noun or a verb. At the sentence level, this could be asking what kind of construction a particular sentence represents. At the document level, this could be asking whether a particular speaker represents New Zealand English or Australian English. All of these questions can be answered using a text classifier. This is a type of supervised machine learning in which we as linguists define the categories that we are interested in. A comparison problem is about measuring the relationship between two observations. At the word level, this could be asking whether two nouns like cat and dog belong to the same semantic domain. At the sentence level, this could be asking whether two tweets have a similar sentiment. At the document level, this could be asking whether two articles are examples of a similar style. These questions can be approached using a text similarity model. This is a type of unsupervised machine learning in which we as linguists only control the representations being used, not the set of labels used for annotation.