دانلود کتاب یادگیری ماشین برای اقتصادسنجی

Econometrics and machine learning (ML) share many statistical tools, as we will see in Chapter 2. However, the philosophies and goals of these two approaches often differ in subtle ways. To draw the contours of the two fields and give the reader an idea of the questions that animate them, we will first exaggerate their differences. We remind the reader that the reality is much more nuanced: the purpose of this textbook is to see how we can harness the forces of one to achieve the goals of the other. The first point of divergence lies in the purpose of the two approaches. Econometrics, first and foremost, aims to quantify a precisely defined effect. For instance: what is the impact of a minimum wage increase on employment? Are there peer effects among groups of students? What is the average wage gap between women and men? In this sense, the econometrician focuses on one or a few parameters of interest aimed at summarizing the effect they seek to measure. We are particularly interested in statistical inference, i.e., building confidence intervals and testing hypotheses. More specifically, over the last three decades, empirical economics has focused on measuring causal effects, not just correlations (Angrist and Pischke, 2010), an effort crowned by the Nobel Prize awarded to Joshua Angrist, Guido Imbens, and David Card in 2021, as well as the one awarded to Esther Duflo two years earlier. This paradigm is studied in Chapter 3. The crux of most economics articles is to demonstrate the rigor of their identification strategy, i.e., to prove that the measured effect is due only to the highlighted causal variable, excluding other parasitic phenomena. Hence, the interest in laboratory, field, or natural experiments, or the search for exogenous variation in a particular policy, i.e., generated by a cause independent of the phenomenon of interest. On the other hand, the goal of ML is to build a model that allows one to obtain the best possible predictive performance for a given problem, often by respecting a computational constraint when calling the model, also called runtime or inference time performance. Thus, the model must generate predictions within a defined timeframe. ML researchers often talk about algorithms rather than models, to stress that this process is based on a series of instructions that lead to a prediction, regardless of their nature, rather than on a single statistical model. ML is therefore used to respond to different problems than those of econometrics, such as constructing song or movie recommendation systems, matching job-seekers to firms, translating documents, predicting the next data point in a time series, categorizing products, recognizing patterns in images, retrieving documents based on their content, etc. The term artificial intelligence (AI) is often used as a synonym for machine learning. This term underlines that a machine replaces the human in performing a cognitive task and that its implementation can be carried out on a very large scale at a very small marginal cost – the main fixed costs consisting of training the algorithm and then making it available. As an aside, these costs are far from negligible, so much so that training large language models (LLMs) like the one that powers ChatGPT from scratch can run to well over a few million dollars. Machine learning is an area in which computer science is ubiquitous, and comparing ML algorithms with traditional algorithms can help to understand the paradigm differences. Traditional algorithms consist of fixed rules, established a priori by a human, that the machine simply executes; whereas training an ML algorithm consists of using datasets that correctly associate inputs with outputs to teach the computer the implicit rules underlying these associations, regardless of the exact nature of these rules, as long as they produce relevant responses for the (human) end-user. In the case of analyzing text data, this is the difference between using a regular expression (Chapter 12) and a modern language model (Chapter 14). It is noteworthy that machine learning, and deep learning in particular, have achieved their most impressive successes in well-defined tasks characterized by a favorable signal-to-noise ratio. Such tasks are those that most humans are capable of performing: recognizing an object in an image, finding the synonym of a word, recommending a movie that a friend will enjoy, etc. However, two main roadblocks hinder their automation on a large scale: on the one hand, they are greedy in cognitive resources, and on the other hand, they are challenging to reformulate as standard predictive tasks due to the unstructured nature of the input data. The lack of structure in the input data makes it challenging to create features (explanatory variables) or to integrate them into conventional statistical frameworks because they do not neatly fit into a table, as opposed to tabular data social scientists are used to tackling. This is the case for images, for example, which are characterized by integer tensors representing pixels, but each pixel taken separately does not contain any information. Text is another case of unstructured data which can be represented as a sequence of integers that are uniquely mapped to tokens (pieces of words), which cannot be easily represented in a table because documents vary in length and word occurrences do not follow an auto-regressive process. Therefore, the capability to scale a task achievable by humans and to incorporate unstructured and highly complex data into a mathematical model fuels the current enthusiasm around ML. Its performance on tasks difficult to perform by a human being, because they might suffer from too high a level of noise or flagrant non-stationarities such as the prediction of macroeconomic series or stock prices, is yet to be demonstrated. To nuance this first difference between the two fields, we recall that forecasting is a well-known econometric subfield whose similarities with machine learning are very easy to see (Chapter 11). Nevertheless, econometric forecasting, because it often deals with complex phenomena, imposes a heavier constrained model. The goal is to counteract an unfavorable signal-to-noise ratio by injecting theoretical priors into models.