The ORE of Politics will be available for subscription in late September. Speak to your Oxford representative or contact us to find out more.

Dismiss
Show Summary Details

Page of

PRINTED FROM the OXFORD RESEARCH ENCYCLOPEDIA, POLITICS (politics.oxfordre.com). (c) Oxford University Press USA, 2016. All Rights Reserved. Personal use only; commercial use is strictly prohibited. Please see applicable Privacy Policy and Legal Notice (for details see Privacy Policy).

date: 24 September 2017

The Empirical Promise of Game Theory

Summary and Keywords

Game theory is a set of mathematical tools used to analyze the strategic interaction between decision makers. Proponents of game theory have offered different perspectives about its potential benefits in the study of politics: It is a rigorous apparatus that can offer a solid foundation for the scientific enterprise; it offers predictions that could be tested with statistical analysis; it can account for the essence of unique cases and could be tested with qualitative evidence. Critics of game theory, in political science and international security specifically, argued in the 1990s that it had generated few empirical insights. Two decades later, game-theoretic approaches to international security remain a robust research program, but their prevalence remains limited. It is important to evaluate the potential benefits of game theory and the contributions that it has made to international security, so as to devise appropriate strategies to maximize its empirical purchase. The controlled comparison approach, using qualitative evidence on a medium number of cases, appears especially promising.

Keywords: methodology, international security, game theory, war, democratic peace, audience costs, nuclear proliferation, empirical international relations theory

Introduction

Political science has long experienced a love/hate relationship toward game theory, admiring the rigor, deductive logic, and sophistication of game-theoretic arguments but questioning its empirical purchase. In the 1990s, just as the foundations for the current game-theoretic approaches to international security were being laid out, some scholars heavily criticized game theorists for failing to provide new empirical insights, and feared the consequences of their growing influence (Green & Shapiro, 1994; Walt, 1999a).

Two decades later, game-theoretic approaches remain a robust research program in international security, but there has been no increase in the frequency of game-theoretic publications in international security (Maliniak, Oakes, Peterson, & Tierney, 2011, pp. 451–452). Worse, game theory’s appeal as uncovering causal processes is being questioned, given advances in statistical approaches under the “identification revolution.”1 Have game-theoretic approaches to international security reached their full potential?

This article argues that dominant perspectives on the usefulness of game theory, either praising its rigor and generality, marrying it with large-N evidence, or with qualitative evidence on unique cases, have not maximized its empirical purchase. Game-theoretic approaches are most useful in tracing the causal effect of strategic interactions on political outcomes, and should be combined with deep qualitative evidence of a medium number of cases using a “controlled comparison” approach (George, 1979). With this approach, the researcher specifies a universe of cases relevant for the study of the phenomenon of interest, and uses qualitative evidence to trace the effect of certain factors (or independent variables) on the outcome of interest (or dependent variable). Game theory is especially well suited to harness such a method, since it specifies the outcome of a strategic interaction (an equilibrium) and can analyze the effect of changes to the strategic environment on the phenomenon of interest (using “comparative statics”). Married with a controlled comparison approach, game theory could shed new light on empirical patterns and maximize its usefulness to the field of international security as a whole.

The rest of the article is structured as follows. The next section, “Existing Debates on the Role and Usefulness of Game Theory,” reviews existing debates on the role of game theory in political science and international security. The following section, “Correlates of Game-Theoretic Approaches to International Security,” provides some correlates on the use of game-theoretic approaches in the study of international security. Then, the section entitled “A Case for Game Theory” reviews the controlled comparison approach and argues that it would maximize the empirical purchase of game theory in international security. Finally, the section entitled “Characterizing the Contributions of Game Theory” evaluates the contributions of game-theoretic work on three substantive areas of international security: the causes of war, domestic politics and interstate conflict, and nuclear politics, offering suggestions for future research.

Existing Debates on the Role and Usefulness of Game Theory

The role and usefulness of game theory in political science as a whole and in international security in particular has long been the subject of heated debate. Specifically, critics have shed doubt on the theory of rational choice, or the hypothesis of rational decision making underlying most game-theoretic models, and questioned its empirical purchase.2

In an influential book, Donald Green and Ian Shapiro fault rational choice theorists for being too “inward-looking” (Green & Shapiro, 1994). They have focused too much on rationalizing political behavior and on addressing controversies raised by members of their community. They have generated too few novel insights. Much empirical evidence is at odds with the theory, and its limited empirical purchase is especially alarming given its growing influence in the field. According to Green and Shapiro, the share of papers published in the American Political Science Review using rational choice theory increased steadily from 0% to 32% between 1952 and 1992 (Green & Shapiro, 1994, p. 3).

Analyzing the field of security studies, Stephen Walt reached similar conclusions (Walt, 1999a). According to Walt, rational choice theorists have focused too much on precision and logical consistency, at the expense of originality and empirical validity. Early applications of the methodology, by Thomas Schelling for example, were simple and insightful (see, e.g., Schelling, 1960). More recent work, however, focused too much on formal proofs and derivations. This greater mathematical complexity has undermined potential dialogues with non-game theorists, leading game theorists to “rediscover” old insights: the “old wine in new bottles” problem. Concurrently, because game theorists have paid insufficient attention to empirical validity, they have clung to inferior theories for too long.3

In response to these critiques, game theorists have presented a series of arguments;4 this article focuses on three of them.

A first response is to praise the theoretical contributions of game theory. Game theory places a premium on transparency, rigorous deductive logic, and internal consistency, properties that are essential for the construction of an applied science (see, e.g., Bueno de Mesquita & Morrow, 1999, p. 57; Niou & Ordeshook, 1999, p. 86). The actual empirical evaluation of theories could come later; at a minimum, a healthy field of academic inquiry should allow for division of labor and specialization (see, e.g., National Science Foundation, 2002, p. 24).5

A second response is to marry formal models with large-N statistical tests (see, e.g., Lewis & Schultz, 2003; Signorino, 1999). This view has been promoted chiefly by the “Empirical Implications of Theoretical Models” (EITM) initiative of the National Science Foundation (National Science Foundation, 2002). According to the initial report setting up the initiative, “A schism has developed between those who engage in formal modeling that is highly mathematical, and those who employ empirical modeling which emphasizes applied statistics” (National Science Foundation, 2002, p. 1). The Foundation has sought to narrow that gap through summer institutes and workshops, encouraging theoretical work to develop testable hypotheses and explanations of stylized facts, and encouraging statistical work to identify general correlations and test theoretical hypotheses.6

A third response is to combine formal models with qualitative evidence on unique cases, producing what is called an “analytic narrative” (Bates, Greif, Levi, Rosenthal, & Weingast, 1998). Proponents of this approach “seek to cut deeply into the specifics of a time and place, and to locate and trace the processes that generate the outcome of interest” (Bates et al., 1998, p. 12). For example, they have offered narratives on the efficacy of the podesta system in Medieval Genoa and the origins, operations, and impact of the International Coffee Organization.

Recently, there has been a resurgence of interest in the potential role of game theory, especially in its ability to highlight causal mechanisms, using process tracing (see, e.g., Goemans & Spaniel, 2016; Lorentzen, Fravel, & Paine, 2016).7 Yet despite the prominence of the three approaches just described here, there has been no systematic evaluation of their relative effectiveness.8 What we do know is that, two decades after Green and Shapiro (1994) and Walt (1999a), there is no evidence of a domination of political science or international relations by game-theoretic approaches. The most exhaustive study on the topic, Maliniak et al. (2011, pp. 451–452), finds that game-theoretic tools were used in 13% of articles in international relations between 1990 and 2007, and their prevalence has actually been on the decline in security journals since the late 1980s.9 Similarly, they conclude that formal methods do not correlate with a significantly greater impact in the field as measured by the citation count. Whether or not game theory has maximized its full potential remains an open question. This article argues that it has not.

Correlates of Game-Theoretic Approaches to International Security

This section documents how game-theoretic tools have been married with other methods in international security. The analysis begins with data from the Teaching, Research, and International Policy (TRIP) project (Maliniak, Oakes, Peterson, & Tierney, 2007; Maliniak et al., 2011; Maliniak, Powers, & Walter, 2013), more specifically the version in Maliniak et al. (2013). This dataset contains information on publications from a large set of papers in the top 12 peer-reviewed journals in International Relations between 1980 and 2007.10 Of particular interest, the dataset identifies a paper’s issue area, its methodology (i.e., whether it uses formal methods, whether it uses qualitative methods, and whether it uses quantitative methods), as well as its impact, using its citation count as measured by Thomson Reuters Web of Knowledge.

First, Figure 1 shows the prevalence of game-theoretic papers in international security, replicating the analysis of Maliniak et al. (2007).

The Empirical Promise of Game TheoryClick to view larger

Figure 1: The Fraction of Papers in International Security Using Formal Models.

In short, the fraction of papers using formal methods in international security has stayed below 30% since the late 1980s, averaging about 16% for the whole period and 13% in the 2000s (Figure 1).

Second, Table 1 looks at the various methodologies that have been used by formal-theoretic papers.

Table 1: The Frequency of Quantitative and Qualitative Methods in International Security, Non-Formal vs. Formal Papers.

The Empirical Promise of Game Theory

The table shows that formal-theoretic papers are as likely as non-formal papers to be combined with quantitative methods (in about 35% of the cases), but they are sharply less likely to be combined with qualitative methods, using them in only 17% of papers, compared with 38% of non-formal papers. In short, while non-formal papers are about as likely to use quantitative and qualitative methods, formal-theoretic papers are half as likely to use qualitative methods as they are to use quantitative methods. This is somewhat surprising, given the EITM’s concern about a “gap” between formal and statistical methods. If anything, in international security, the gap is wider between formal and qualitative methods.11

Third, the paper characterizes the particular way in which formal-theoretic papers use qualitative methods, coding two new variables. The first documents whether a paper offers a significant amount of qualitative evidence to describe a single case (or describing multiple cases with the same value for the dependent and independent variables). The second variable documents whether a paper contains a significant amount of qualitative evidence to describe more than one case.12 These two variables are rough proxies for the two qualitative approaches described above. In short, the first variable captures whether a paper is using an “analytic narrative” approach (though many pre-date the presentation of the term by Bates et al., 1998), while the second variable captures whether a paper is using a “controlled comparison” approach, trying to explain how the model can make sense of variations in empirical patterns.13 Looking at the evidence, very few papers use a controlled comparison approach: Of the 202 papers using formal-theoretic tools in international security, only 7 used a controlled comparison approach.

Fourth, the paper characterizes the impact of formal-theoretic publications. We know that in international relations as a whole, the presence of formal-theoretic tools, everything else equal, does not significantly correlate with the size of a publication’s impact, as measured by its citation count (Maliniak et al., 2013, p. 902; we can also verify that this is true in international security). What we do not know much about is the nature of a publication’s impact, or the distribution of the types of papers that cite a game-theoretic publication. To address this issue, this paper collects some information about the nature of the citations of game-theoretic papers (see Figure 2).14

The Empirical Promise of Game TheoryClick to view larger

Figure 2: Characterizing the Impact of Papers in International Security Using Formal Models.

Figure 2 shows the share of citations of a formal-theoretic paper that uses formal methods. Interestingly, more than 40% of the citations of a formal-theoretic paper are themselves formal theoretic. This is quite high, given that formal-theoretic papers comprise only 16% of all papers in international security.15

Taking stock, a review of papers published in top journals in international security suggests that qualitative methods have been used relatively infrequently by formal-theoretic papers. In addition, formal-theoretic papers have rarely used qualitative methods to explain variation across more than one case. At the same time, formal-theoretic papers have mainly impacted other formal-theoretic papers. These pieces of evidence are broadly consistent with Green and Shapiro’s (1994) view that the game-theoretic community has been inward looking and/or that the non-formal community has been skeptical of the insights generated by the game-theoretic literature.

A Case for Game Theory

The effort of building game-theoretic models of social phenomena may best be seen as a “dialogue” between theory and evidence (Powell, 1999a, p. 24), aiming to achieve the twin goals of internal consistency and external validity. The question is whether the three main approaches reviewed in this article, that is, a focus on the abstract value of game theory, an argument for the statistical evaluation of game-theoretic models, and a case for using game theory in analytic narratives, are most conducive to a dialogue between theory and evidence. This article argues that they are not.

Praising abstract theoretical work is not sufficient to generate empirical insights. Many steps must be taken to translate an abstract mathematical framework to its empirical applications, and the game theorist would benefit from being directly involved in these steps. Generality can take many different forms. By engaging the empirical world, the game theorist could make a more informed choice among different, and sometimes competing, general models. Ultimately, the dialogue is most productive if game-theoretic tools are used in conjunction with empirical analysis. Certainly, scholars have methodological comparative advantages, yet no observation is purely free of theory (Popper, 2000 [1959], p. 106). When confronting empirical observations, scholars are inspired by theoretical puzzles; they try to make sense of the complexity of the empirical world with their own theoretical arguments. To state that empirical scholars would “document empirical patterns” while game theorists would “explain them” undermines the theoretical work involved in the empirical enterprise.

When searching for the optimal method of engaging the empirical world, statistical methods also have their limits. They are helpful in controlling for confounding factors and identifying general patterns, but fundamentally they are aimed at documenting whether certain variables are correlated, not explaining why they are related.

To capture the strategic thinking of different actors, qualitative evidence appears especially promising. However, the analytic narrative approach has its limits. The fundamental goal of an analytic narrative is to capture the essence of a unique case. Yet how do we know what is “essential” about a unique situation? Any narrative of a case will simplify the situation and leave out some details. The analytic narrative would then be faulted for missing important details of the case at hand.

Qualitative evidence can offer powerful insights on the causal processes underlying empirical patterns, but it would best be served using a controlled comparison approach (George, 1979).16 In this exercise, the theorist spells out the empirical phenomenon that he/she seeks to explain, specifies a universe of cases relevant for the study of the phenomenon, investigates the strategic thinking of actors in these different cases, and how they may be influenced by key features of the international system. Relative to the analytic narrative, the controlled comparison approach acknowledges that it would at best offer a limited explanation for the empirical pattern, rather than a complete account of a unique case.17 However, by using case studies as anchors for one another, it can trace the effect of changes to the strategic environment on the outcome of interest. In this exercise, the theorist is focused in capturing both the outcome of a strategic interaction and the process generating the outcome.

The controlled comparison approach has a long tradition in the social sciences, but its impact on game-theoretic approaches to international security has been limited. Game-theoretic approaches are actually ideal complements to the controlled comparison approach. After describing the behavior of strategic actors in a given setting (by solving for the “equilibrium” of a game), formal theory offers predictions about the relative probability of different outcomes under different settings (offering “comparative statics” for the outcome of interest). Formal theory thus allows for a comparison of strategic calculations in different cases, offering an explanation for variations in empirical patterns. For example, the controlled comparison approach could compare interstate crises to understand why some degenerated into war and others did not, looking at the effect of the balance of power, regime type or alliances on the calculations of leaders, their perception of the credibility of threats, etc.

Certainly, implementing the controlled comparison approach presents its own challenges. Space constraints are tighter in journal articles, the outlet currently preferred by formal theorists. Yet if the emphasis is on identifying causal processes, authors could devote more attention to the relevant qualitative evidence. Some mathematical derivations could be moved to an (online) appendix, and further exploration of the qualitative evidence could be saved for a book-length project. Focusing on causal mechanisms and the empirical purchase of formal models could broaden the appeal of game-theoretic work.

Characterizing the Contributions of Game Theory

The previous section made a case for marrying formal-theoretic tools with qualitative evidence, using a controlled comparison approach. Now the article reviews three issue areas and the different ways in the literature harnessed game-theoretic methods, evaluating the associated debates and offering suggestions for future work.18

Causes of War

Understanding the causes of war has been a central concern for international security scholars, including those who have used game-theoretic tools (for an early contribution, see, e.g., Bueno de Mesquita & Lalman, 1992). For example, game theorists have explained how, using the shadow of future interactions, potential enemies may cooperate (see, e.g., Axelrod, 1984; Langlois & Langlois, 1996).

The study of the causes of war has also produced what is considered by game theorists as their central baseline model: the “bargaining” model of war (Fearon, 1995). This simple, powerful model conceptualizes war as a costly alternative to peaceful diplomacy, or bargaining. The paper thus posits the basic “puzzle of war”: “Why does war occur, when it is costly and destructive?” It proposes three answers: private information, that is, one side has some information about its resolve or capabilities, which it has an incentive to misrepresent, and as a result the other side may press too hard in diplomatic exchanges, causing war; commitment problems, that is, a rising power cannot commit to refrain from using its future power, inducing a declining power to strike preventively; and issue indivisibilities, that is, the issue in dispute between two states cannot be split, constraining diplomatic settlements.19 Given the influence of the bargaining model, it would be impossible to give an exhaustive review of its critiques. This paper focuses on two points of contention.

A first debate has been on the usefulness of the bargaining model of war. If the model accurately describes fundamental features of the international system, then it cannot explain variations in the occurrence of war (e.g., Gartzke, 1999; Lake, 2010–2011). According to the theory, elements that could explain war were unobservable to other states, and hence it will remain unobservable for researchers. As a result, as Gartzke (1999) puts it, war remains “in the error term.”

A second debate has centered on the relative importance of the three mechanisms. For example, Kirshner (2000, p. 144) noted that issue indivisibility played an important role in key situations such as, say, confrontations between Israelis and Palestinians over the control of Jerusalem. Gartzke (1999, p. 573), for his part, favors information problems as arguably the most important for the study of international conflict. In contrast, Fey and Ramsay (2007) and Slantchev and Tarar (2011) debate the logical consistency of one type of information problem, that is, uncertainty about the balance of power that generates mutual optimism, as a possible cause of war.20

Debates on the relative importance of these different mechanisms began with Fearon (1995, pp. 381–382) itself, which suggested that the problem of issue indivisibility was “less compelling” than the other two. Powell (2006) went further in arguing that issue indivisibility is a special case of commitment problems. Intuitively, states could replicate the “lottery of war” through peaceful diplomacy, but the problem is that states could not commit to abide by the results of the lottery. Both Fearon (2004, p. 90) and Powell (2006) argue for the importance of commitment problems as a cause of war. Debs and Monteiro (2014), for their part, argue that commitment problems could not in and of themselves cause war, when shifts in the balance of power are a result of the states’ decision, unless there are information problems. Put differently, if it takes time and effort to impart a shift in the balance of power, then enemies could issue threats, promising to strike before the shift takes place, and deterring the enemy from attempting to shift the balance of power in the first place. Only when enemies are uncertain of such militarization attempts could deterrence fail, and war ensue.

Assessing these debates, one should keep in mind the following. The fact that the bargaining model of war begins from fundamental features of the international system is a virtue, not a flaw (Debs & Monteiro, 2014, p. 23). An alternative model that focuses on contingent aspects of the international system may not be a good starting point, and a model that relies on erroneous features certainly would not be. Any theory uses simplifying assumptions, and assumptions that accurately capture important features of the international system are especially promising.

At the same time, there are limits in debating the relative merits of different mechanisms on purely logical grounds. The bargaining model is a promising foundation for an applied theory of international security, given its general and abstract qualities. However, to fulfill this promise, theorists should build upon a close reading of the empirical phenomena they wish to explain. After identifying which friction is most important in a specific set of cases, the theorist should specify conditions under which this given friction is more or less severe, and thus more or less likely to lead to war. Observable features of a strategic interaction can be traced to the outcome of interest, through a model’s comparative statics. Herein lies the empirical promise of formal theory.

This methodology would be especially fruitful in understanding the duration of conflict.

For example, Powell (2006) argues that commitment problems may be the most promising avenue for understanding why long-lasting civil wars persist: States keep fighting because none can commit to refrain from using its power if the other side lays down its arms (see also Fearon, 2004, p. 290). Powell (2013) then specifies conditions under which a government may try to defeat the opposition quickly, as a function of the size of benefits that a group would obtain after a consolidation of power, or what Powell calls “contingent spoils.” Powell (2013) concludes that if contingent spoils are small, the state will try to buy off the opposition peacefully; if contingent spoils, the state attempts to consolidate power militarily by defeating the opposition. Together, Powell (2006, 2013) provides an applied theory of civil conflict, illustrating the benefits of complementing debates on abstract causes of war with an engagement of the empirical evidence.

Other perspectives on the duration of conflict are also promising. In his study of interstate wars, Weisiger (2013) similarly argues that commitment problems are particularly difficult to resolve, and concludes that ``wars triggered by commitment problems tend to be of longer duration and greater intensity, testing his claims using quantitative and qualitative evidence. By contrast, Langlois and Langlois (2009, 2017) argue that information problems can account for wars of long duration; belligerents refrain from an agreement because they are concerned that making a generous offer would be interpreted as a sign of weakness. These studies highlight the benefits of the bargaining framework in capturing important features of conflict. Further work, especially harnessing the controlled comparison approach, would be fruitful.

Domestic Politics and Interstate Conflict

If the canonical bargaining model of war abstracts away from the effect of domestic politics on international conflict, game theorists have nevertheless engaged such topics, focusing on two particular lines of research: the “democratic peace” and the theory of audience costs.

The Democratic Peace

The democratic peace is the proposition that democracies rarely, if ever, fight wars against one another (Russett, 1993).21 The most prominent game-theoretic explanation for the democratic peace is given by the “selectorate model” of Bueno de Mesquita, Morrow, Siverson, and Smith (1999). This paper argues that polities significantly differ in the size of their enfranchised citizenry, or “selectorate,” and the extent of support needed for a leader to remain in power, or the size of their “winning coalition.” Democracies have a large selectorate and a large winning coalition. In such systems, leaders survive in office by offering public goods. By contrast, dictatorships have small selectorates and small winning coalitions. Leaders in such regimes remain in office by offering private goods to members of their winning coalition, who are less likely to receive such favors from a challenger. Assuming that the outcome of war is a public good, democracies will be reluctant to fight wars they do not expect to win. Two democracies are unlikely to believe they would both win a war against each other, and thus are unlikely to fight one another.

Other explanations of the democratic peace have been offered by game theorists.22 For example, Debs and Goemans (2010) argue that polities differ in the importance of violence in leadership replacement. Violence is more important in dictatorships than in democracies. As a result, dictators are especially concerned about the consequences of losing office, they are more reluctant to make concessions on the international scene, and they may inadvertently preclude the possibility of diplomatic resolution to interstate conflict.

In part, the democratic peace proposition is appealing for game theorists because it appears as a robust “stylized fact” in search of an explanation, in a field where generalizations may otherwise be elusive, given standard limitations on observational data. As Levy (1988, p. 662) famously put it, “This absence of war between democracies comes as close as anything we have to an empirical law in international relations.” Some scholars, in line with the priorities of the EITM initiative, would view this situation as ideal for building a game-theoretic analysis of international security: Large-N studies establish empirical regularities, game-theoretic approaches offer an explanation for the finding, and generate additional predictions.

Yet it is important to note that the empirical observation of the democratic peace was not produced in a theoretical vacuum. In fact, theoretical reflections on the democratic peace much pre-date its observation in the data, going back at least to the work of Immanuel Kant (see Kant, 2006 [1795], and the discussion in Doyle, 1983). As a result, new arguments on the democratic peace, whether game theoretic or not, had to develop a broader set of empirical lessons, for example, by making sense of other interconnected empirical regularities.23

The Theory of Audience Costs

One game-theoretic approach on the effect of domestic politics on international conflict, which is especially sophisticated in its discussion of causal mechanisms, is the theory of audience costs (Fearon, 1994a). According to the theory, leaders who may be punished for backing down in interstate crisis gain important leverage when they do issue threats on the international scene: Doing so reveals their resolve, and induces their enemy to make concessions.24

In its initial formulation, the theory nevertheless left some questions unanswered: Why would democracies be more effective in holding leaders accountable? And why would domestic audiences punish leaders for backing down? Game theorists have tackled these questions in an effort to build a more complete rational theory. For example, Schultz (1998, 2001) argued that the presence of a domestic opposition enhances the ability of democracies to hold their leaders accountable. According to Schultz, the domestic opposition, informed of their country’s resolve, would have an incentive to call the bluff of its government for electoral purposes. Thus, when it does not, and instead supports the government, the credibility of democratic threats would be enhanced. Smith (1998), for his part, argued that domestic audiences would punish a leader for backing down because backing down revealed the leader’s incompetence (see also Ramsay, 2004).25 Put differently, a leader who backs down reveals that he or she could not prosecute a war effectively. Rational audiences would prefer to replace this leader with a more competent leader, who would be more effective in fighting conflicts when they do occur, and may extract more favorable concessions from enemies to avoid conflict.

More recently, the theory of audience costs has come under attack. Looking at the historical record, some claim that domestic audiences did not especially care about whether their leader backed down—instead they cared about the right policy being implemented (Snyder & Borghard, 2011). Democratic leaders rarely used threats as coercive devices (Trachtenberg, 2012), and threats issued by democracies were no more successful than those issued by non-democracies (Downes & Sechser, 2012).

Some could view these developments as exposing the limits of a game-theoretic approach. In fact, the opposite is true. The vigorous debates generated by the theory of audience costs, reaching beyond the game-theoretic community, are a sign of its originality and empirical purchase. The theory offered specific predictions about the outcome of interstate crises and the behavior of the actors involved. For example, Schultz (2001) explained how the Liberal Party’s support of the British government during the Fashoda crisis of 1898 allowed it to coerce the French government, and how, a few months later, the Liberal Party’s opposition to the British government’s plans in South Africa led to the Boer War of 1899. This precision in the causal mechanisms, and the claims that such mechanisms could explain variations in the empirical record, offered an opportunity for non-formal theorists to engage the theory and debate its empirical relevance. Empirical studies have further probed the mechanism of audience costs, using both experimental methods (see, e.g., Chaudoin, 2014; Kertzer & Brutger, 2016; Tomz, 2007) and structural models on observational data (Kurizaki & Whang, 2015). Other analyses have expanded the purview of audience cost theory, investigating how audiences in non-democracies could also impose cost on their leaders for their foreign-policy failures (see, e.g., Weeks, 2008, 2014; Weiss, 2013, 2014).

This dialogue between theory and evidence, and between different methodological approaches, is conducive to the growth of knowledge. One way forward for this literature is to think differently about the role of threats in public debates. Threats, much like other statements in public debates, are ways in which leaders engage their domestic audience, to convince them about appropriate policy (Debs & Weiss, 2016). Backing down is thus a retreat from the policy that a leader previously labeled as appropriate. More work should be done to understand how leaders can convince their audiences of the appropriate foreign policy (see, e.g., Levendusky & Horowitz, 2012).

Nuclear Politics

Game-theoretic approaches to international security have also contributed to our understanding of important questions in nuclear politics. From the early days of the nuclear age, game-theoretic approaches have provided valuable insights on the management of nuclear crises (see, e.g., Fearon, 1994b; Powell, 1990, 2015; Schelling, 1960, 1966). In particular, Fearon (1994b, pp. 255–257) explained how strategic selection in a crisis can explain the deterrent success of threats by nuclear powers: They are likely to be challenged on peripheral issues, which are potentially of secondary interest, but they will be effective in deterring enemies if they do signal their resolve.

More recently, game-theoretic tools have also been applied to the study of the process of nuclear proliferation. In the last two decades, the literature on nuclear proliferation has mostly focused on non-security considerations, investigating the role of the economic preferences of ruling coalitions (Solingen, 1994, 2007), the psychology of leaders (Hymans, 2006), and technological assistance (Fuhrmann, 2009, 2012; Kroenig, 2009, 2010). This is due in part to the limits of previous security approaches. By focusing on the benefits of nuclear weapons for the would-be proliferator (e.g., Thayer, 1995, p. 486), such arguments over-predicted proliferation: Too many states should have benefited from acquiring nuclear weapons, and too few acquired them. Methodologically, the literature has also embraced statistical analyses of proliferation (see, e.g., Jo & Gartzke, 2007; Singh & Way, 2004).

Certainly, recent studies have been insightful, and contributed to our understanding of proliferation. Yet the fact that existing security approaches over-predicted proliferation does not mean that one should focus on non-security considerations. Nuclear weapons are, after all, weapons. Instead, one could focus on the security dimensions of proliferation by placing it in its strategic context, that is, by looking not only at the incentives of would-be proliferators, but at the interaction of the potential proliferator, its enemies and allies. Game theory is especially well suited to analyze such scenarios.26 Methodologically, we may also want to combine such a strategic analysis with qualitative evidence. Few states seriously considered a nuclear-weapons option. It would be particularly helpful to understand the strategic situation of actual proliferators, and compare them with states that attempted to acquire nuclear weapons, but were not successful, and others that debated the option but did not consider it productive. The controlled comparison approach, using game-theoretic tools, appears especially fruitful.

In recent work, Debs and Monteiro (2014, 2017) and Monteiro and Debs (2014) modeled the problem of proliferation as a costly investment with delayed returns. Intuitively, it takes time and effort before a nuclear-weapons program comes to fruition. The acquisition of nuclear weapons could ultimately provide security benefits to a potential proliferator. By the same token, however, nuclearization would impact negatively the security of the new nuclear state’s enemies. A nuclearization attempt could thus be thwarted by a potential proliferator’s enemies, which may launch a preventive strike, or threaten to do so. Similarly, allies of the potential proliferator may be concerned about proliferation, and offer reassurances, so as to obviate the need for an independent deterrent; or use coercive means, so as to press the potential proliferator into abandoning its nuclear ambitions.

Using this approach, Debs and Monteiro argue that there is a positive correlation between the conventional capabilities of the potential proliferator and the likelihood of its successful nuclearization. The stronger is the potential proliferator, the more effectively it can shield itself from the counter-proliferation attempts of enemies. For example, the Soviet Union proceeded unimpeded to acquire nuclear weapons in 1949, given its significant advantage in conventional forces on the European continent, which undermine the prospects of an American preventive strike. By contrast, Iraq never completed its nuclear-weapons program, as it was the target of preventive strikes and sanctions by its more powerful enemies, Iran, Israel, and the United States. Next, Debs and Monteiro argue that the presence of an ally weakens the relationship between the conventional capabilities of a potential proliferator and the odds of nuclearization. Weak potential proliferators may now possess the opportunity to proliferate, as was the case with Pakistan in the 1980s; and strong potential proliferators may be convinced that they do not need their own deterrent, as was the case with South Korea in the same period.

Thus, as we see, the theory produces new predictions about the empirical patterns of proliferation, and it is supplemented by qualitative evidence on a medium-N set of countries, using the controlled comparison approach. As this article argues, this approach is especially conducive to a conversation with a broad set of scholars, whether or not they use game theory.

Conclusion

This article analyzes various approaches for the use of game-theoretic tools in the construction of applied theories, with a focus on international security. Scholars have typically made three arguments in favor of game theory: It provides abstract and general lessons; it can be used in conjunction with large-N techniques, providing testable hypotheses and making sense of empirical regularities; it can be used to rationalize the realization of unique cases. Instead, this article argues that game-theoretic tools would best be used in a controlled comparison approach, using qualitative evidence on a medium number of cases to evaluate the strategic thinking of decision makers. Looking at game-theoretic analyses between 1980 and 2007, the article finds that game-theoretic studies have disproportionately impacted other game-theoretic studies, and they have rarely used qualitative methods, especially the controlled comparison approach. In a survey of three substantive areas (the causes of war, domestic politics and interstate conflict, and nuclear politics), the article finds that game-theoretic tools, when combined with qualitative evidence using the controlled comparison approach, have had a broad appeal and triggered rich and interesting debates.

Looking ahead, it is important to refine the standards of best practices for combining qualitative and game-theoretic methods.27 Ultimately, multiple approaches can be harnessed to fulfill the empirical promise of game theory. Of these, embedding game-theoretic analyses with qualitative work, using the controlled comparison approach, seems especially fruitful.

Acknowledgments

I thank Songying Fang, Catherine Langlois, Nuno Monteiro, Jack Paine, Ryan Powers, participants at the 2017 annual meeting of the International Studies Association, as well as William Thompson and two anonymous reviewers for very helpful comments. I thank Daniel Maliniak for assistance with the TRIP dataset; and Tyler Bowen, Gautam Nair, and Jiahua Yue for excellent research assistance. All remaining errors are mine.

References

Ashworth, S., Berry, C. R., & Bueno de Mesquita, E. (2015). All else equal in theory and data (big or small). PS: Political Science & Politics, 48(1), 89–94.Find this resource:

Axelrod, R. (1984). The evolution of cooperation. New York: Basic Books.Find this resource:

Bas, M. A., & Coe, A. J. (2012). Arms diffusion and war. Journal of Conflict Resolution, 56(4), 651–674.Find this resource:

Bas, M. A., & Coe, A. J. (2016). A dynamic theory of nuclear proliferation and preventive war. International Organization, 70(4), 655–685.Find this resource:

Bas, M., & Schub, R. (2016). Mutual optimism as a cause of conflict: Secret alliances and conflict onset. International Studies Quarterly, 60(3), 552–564.Find this resource:

Bates, R. H., Greif, A., Levi, M., Rosenthal, J.-L., & Weingast, B. R. (1998). Analytic narratives. Princeton, NJ: Princeton University Press.Find this resource:

Bennett, A., Barth, A., & Rutherford, K. R. (2003). Do we preach what we practice? A survey of methods in political science journals and curricula. PS: Political Science and Politics, 36(3), 373–378.Find this resource:

Blainey, G. (1988 [1973]). The causes of war (3d ed.). New York: Free Press.Find this resource:

Bueno de Mesquita, B., & Lalman, D. (1992). War and reason: Domestic and international imperatives. New Haven, CT: Yale University Press.Find this resource:

Bueno de Mesquita, B., & Morrow, J. D. (1999). Sorting through the wealth of notions. International Security, 24(2), 56–73.Find this resource:

Bueno de Mesquita, B., Morrow, J. D., Siverson, R. M., & Smith, A. (1999). An institutional explanation of the democratic peace. American Political Science Review, 93(4), 791–807.Find this resource:

Bueno de Mesquita, B., Morrow, J. D., Siverson, R. M., & Smith, A. (2003). The logic of political survival. Cambridge, MA: MIT Press.Find this resource:

Chaudoin, S. (2014). Promises or policies? An experimental analysis of international agreements and audience reactions. International Organization, 68(1), 235–256.Find this resource:

Clarke, K. A., & Primo, D. M. (2012). A model discipline: Political science and the logic of representations. Oxford: Oxford University Press.Find this resource:

Dafoe, A. (2011). Statistical critiques of the democratic peace: Caveat emptor. American Journal of Political Science, 55(2), 247–262.Find this resource:

Debs, A. (2016a). Living by the sword and dying by the sword? Leadership transitions in and out of dictatorships. International Studies Quarterly, 60(1), 73–84.Find this resource:

Debs, A. (2016b). Optimism and war. Yale University Press.Find this resource:

Debs, A., & Goemans, H. E. (2010). Regime type, the fate of leaders, and war. American Political Science Review, 104(3), 430–445.Find this resource:

Debs, A., & Monteiro, N. P. (2014). Known unknowns: Power shifts, uncertainty, and war. International Organization, 68(1), 1–31.Find this resource:

Debs, A., & Monteiro, N. P. (2017). Nuclear politics: The strategic causes of proliferation. New York: Cambridge University Press.Find this resource:

Debs, A., & Weiss, J. C. (2016). Circumstances, domestic audiences, and reputational incentives in international crisis bargaining. Journal of Conflict Resolution, 60(3), 403–433.Find this resource:

Diermeier, D. (1995). Rational choice and the role of theory in political science. Critical Review, 9 (1–2), 59–70.Find this resource:

Downes, A. B., & Sechser, T. S. (2012). The illusion of democratic credibility. International Organization, 66(3), 457–489.Find this resource:

Doyle, M. W. (1983). Kant, liberal legacies, and foreign affairs. Philosophy and Public Affairs, 12(3), 205–235.Find this resource:

Dunning, T. (2008). Improving causal inference: Strengths and limitations of natural experiments. Political Research Quarterly, 61(2), 282–293.Find this resource:

Farber, H. S., & Gowa, J. (1995). Polities and peace. International Security, 20(2), 123–146.Find this resource:

Fearon, J. D. (1994a). Domestic political audiences and the escalation of international disputes. American Political Science Review, 88(3), 577–592.Find this resource:

Fearon, J. D. (1994b). Signaling versus the balance of power and interests: An empirical test of a crisis bargaining model. Journal of Conflict Resolution, 38(2), 236–269.Find this resource:

Fearon, J. D. (1995). Rationalist explanations for war. International Organization, 49(3), 379–414.Find this resource:

Fearon, J. D. (2004). Why do some civil wars last so much longer than others? Journal of Peace Research, 41(3), 275–301.Find this resource:

Fearon, J. D. (2011). A simple political economy of relations among democracies and autocracies. Stanford University Press.Find this resource:

Fey, M., & Ramsay, K. W. (2007). Mutual optimism and war. American Journal of Political Science, 51(4), 738–754.Find this resource:

Fey, M., & Ramsay, K. W. (2012). Mutual optimism and choosing to fight. Princeton University Press.Find this resource:

Fiorina, M. P. (1995). Rational choice, empirical contributions, and the scientific enterprise. Critical Review, 9(1–2), 85–94.Find this resource:

Fuhrmann, M. (2009). Spreading temptation: Proliferation and peaceful nuclear cooperation agreement. International Security, 34(1), 7–41.Find this resource:

Fuhrmann, M. (2012). Atomic assistance: How “Atoms for Peace” programs cause nuclear insecurity. Ithaca, NY: Cornell University Press.Find this resource:

Gartzke, E. (1999). War is in the error term. International Organization, 53(3), 567–587.Find this resource:

Gartzke, E. (2007). The capitalist peace. American Journal of Political Science, 51(1), 166–191.Find this resource:

George, A. L. (1979). Case studies and theory development: The method of structured, focused comparison. In P. Gordon Lauren (Ed.), Diplomacy: New approaches in history, theory, and policy (pp. 43–68). New York: Free Press.Find this resource:

Goemans, H., & Spaniel, W. (2016). Multimethod research: A case for formal theory. Security Studies, 25(1), 25–33.Find this resource:

Gowa, J. S. (1999). Ballots and bullets: The elusive democratic peace. Princeton, NJ: Princeton University Press.Find this resource:

Granato, J., Lo, M., & Wong, M. C. (2010). A framework for unifying formal and empirical analysis. American Journal of Political Science, 54(3), 783–797.Find this resource:

Granato, J., & Scioli, F. (2004). Puzzles, proverbs, and omega matrices: The scientific and social significance of empirical implications of theoretical models (EITM). Perspectives on Politics, 2(2), 313–323.Find this resource:

Green, D. P., & Shapiro, I. (1994). Pathologies of rational choice: A critique of applications in political science. New Haven, CT: Yale University Press.Find this resource:

Green, D. P., & Shapiro, I. (1995). Pathologies revisited: Reflections on our critics. Critical Review, 9(1–2), 235–276.Find this resource:

Hyde, S. D. (2015). Experiments in international relations: Lab, survey, and field. Annual Review of Political Science, 18, 403–424.Find this resource:

Hymans, J. E. C. (2006). The psychology of nuclear proliferation: Identity, emotions, and foreign policy. New York: Cambridge University Press.Find this resource:

Jackson, M. O., & Morelli, M. (2007). Political bias and war. American Political Science Review, 97(4), 1353–1373.Find this resource:

Jo, D. J., & Gartzke, E. (2007). Determinants of nuclear weapons proliferation. Journal of Conflict Resolution, 51(1), 167–194.Find this resource:

Kant, I. (2006 [1795]). Toward perpetual peace: A philosophical sketch. In P. Kleingeld (Ed.). & D. L. Cloclasure (Trans.), Toward perpetual peace and other writings on politics, peace, and history (pp. 67–109). New Haven, CT: Yale University Press.Find this resource:

Kertzer, J. D., & Brutger, R. (2016). Decomposing audience costs: Bringing the audience back into audience cost theory. American Journal of Political Science, 60(1), 234–249.Find this resource:

Kirshner, J. (2000). Rationalist explanations for war? Security Studies, 10(1), 143–150.Find this resource:

Kroenig, M. (2009). Exporting the bomb: Sensitive nuclear assistance and proliferation. American Political Science Review, 103(1), 113–133.Find this resource:

Kroenig, M. (2010). Exporting the bomb: Technology transfer and the spread of nuclear weapons. Ithaca, NY: Cornell University Press.Find this resource:

Kuhn, H. W. (2004 [1944]). Introduction. In J. Neumann & O. Morgenstern (Eds.), Theory of games and economic behavior (Sixtieth-Anniversary ed., pp. vii–xiv). Princeton, NJ: Princeton University Press.Find this resource:

Kurizaki, S., & Whang, T. (2015). Detecting audience costs in international disputes. International Organization, 69(4), 949–980.Find this resource:

Lake, D. A. (2010–2011). Two cheers for bargaining theory: Assessing rationalist explanations of the Iraq war. International Security, 35(3), 7–52.Find this resource:

Langlois, C. C., & Langlois, J.-P. P. (1996). Rationality in international relations: A game-theoretic and empirical study of the U.S.-China case. World Politics, 48(3), 358–390.Find this resource:

Langlois, C. C., & Langlois, J.-P. P. (2009). Does attrition behavior help explain the duration of interstate wars? A game theoretic and empirical analysis. International Studies Quarterly, 53(4), 1051–1073.Find this resource:

Langlois, C., & Langlois, J.-P. (2016, February). Empirical testing of formal models. Oxford Bibliography. Oxford University Press. Published online. Available at http://www.oxfordbibliographies.com/view/document/obo-9780199743292/obo-9780199743292-0166.xml.Find this resource:

Langlois, J.-P. P., & Langlois, C. C. (2017). War can be in the signaling effect: War and bargaining with bilateral uncertainty. Paper presented at the 58th Annual Convention of the International Studies Association, Baltimore, MD, February 22–25.Find this resource:

Levendusky, M. S., & Horowitz, M. C. (2012). When backing down is the right decision: Partisanship, new information, and audience costs. Journal of Conflict Resolution, 74(2), 323–338.Find this resource:

Levy, J. S. (1988). Domestic politics and war. Journal of Interdisciplinary History, 18(4), 653–673.Find this resource:

Lewis, J. B., & Schultz, K. A. (2003). Revealing preferences: Empirical estimation of a crisis bargaining game with incomplete information. Political Analysis, 11(4), 345–367.Find this resource:

Lijphart, A. (1971). Comparative politics and the comparative method. The American Political Science Review, 65(3), 682–693.Find this resource:

Little, A., & Pepinsky, T. (2016). Simple and formal models in comparative politics. Chinese Political Science Review, 1(3), 425–447.Find this resource:

Lorentzen, P., Fravel, M. T., & Paine, J. (2016). Qualitative investigation of theoretical models: The value of process tracing. Journal of Theoretical Politics (forthcoming).Find this resource:

Maliniak, D., Oakes, A., Peterson, S., & Tierney, M. J. (2007). The international relations discipline, 1980–2006. Paper prepared for the Annual Meeting of the American Political Science Association, Chicago, Illinois.Find this resource:

Maliniak, D., Oakes, A., Peterson, S., & Tierney, M. J. (2011). International relations in the U.S. academy. International Studies Quarterly, 55(2), 437–464.Find this resource:

Maliniak, D., Powers, R., & Walter, B. F. (2013). The gender citation gap in international relations. International Organization, 67(4), 889–922.Find this resource:

Martin, L. (1999). The contributions of rational choice: A defense of pluralism. International Security, 24(2), 74–83.Find this resource:

Monteiro, N. P., & Debs, A. (2014). The strategic logic of nuclear proliferation. International Security, 39(2), 7–51.Find this resource:

National Science Foundation. (2002). EITM: Empirical implications of theoretical models. Report of the Political Science Program, Directorate for Social, Behavioral and Economic Sciences.Find this resource:

Niou, E. S., & Ordershook, P. C. (1999). Return of the Luddites. International Security, 24(2), 84–96.Find this resource:

Oneal, J. R., & Russett, B. (1999). Is the liberal peace just an artifact of Cold War interests? Assessing recent critiques. International Interactions, 25(3), 213–241.Find this resource:

Ordeshook, P. C. (1995). Engineering or science: What is the study of politics? Critical Review, 9(1–2), 175–188.Find this resource:

Osborne, M. J. (2004). An introduction to game theory. New York: Oxford University Press.Find this resource:

Popper, K. R. (2000 [1959]). The logic of scientific discovery. London: Routledge.Find this resource:

Powell, R. (1990). Nuclear deterrence theory: The search for credibility. Cambridge, U.K.: Cambridge University Press.Find this resource:

Powell, R. (1999a). In the shadow of power: States and strategies in international politics. Princeton, NJ: Princeton University Press.Find this resource:

Powell, R. (1999b). The modeling enterprise and security studies. International Security, 24(2), 97–106.Find this resource:

Powell, R. (2006). War as a commitment problem. International Organization, 60(1), 169–203.Find this resource:

Powell, R. (2013). Monopolizing violence and consolidating power. The Quarterly Journal of Economics, 128(2), 807–859.Find this resource:

Powell, R. (2015). Nuclear brinkmanship, limited war, and military power. International Organization, 69(3), 589–626.Find this resource:

Ramsay, K. W. (2004). Politics at the water’s edge. Crisis bargaining and electoral competition. Journal of Conflict Resolution, 48(4), 459–486.Find this resource:

Russett, B. (1993). Grasping the democratic peace: Principles for a post-Cold War world. Princeton, NJ: Princeton University Press.Find this resource:

Schelling, T. C. (1960). The strategy of conflict. Cambridge, MA: Harvard University Press.Find this resource:

Schelling, T. C. (1966). Arms and influence. New Haven, CT: Yale University Press.Find this resource:

Schultz, K. A. (1998). Domestic opposition and signaling in international crises. American Political Science Review, 92(4), 829–844.Find this resource:

Schultz, K. A. (2001). Democracy and coercive diplomacy. Cambridge, U.K.: Cambridge University Press.Find this resource:

Shepsle, K. A. (1995). Statistical political philosophy and positive political theory. Critical Review, 9(1–2), 213–222.Find this resource:

Signorino, C. S. (1999). Strategic interaction and the statistical analysis of international conflict. American Political Science Review, 93(2), 279–297.Find this resource:

Singh, S., & Way, C. R. (2004). The correlates of nuclear proliferation: A quantitative test. Journal of Conflict Resolution, 48(6), 859–885.Find this resource:

Slantchev, B. L., & Tarar, A. (2011). Mutual optimism as a rationalist explanation of war. American Journal of Political Science, 55(1), 135–148.Find this resource:

Slater, D., & Ziblatt, D. (2013). The enduring indispensability of the controlled comparison. Comparative Political Studies, 46(10), 1301–1327.Find this resource:

Smith, A. (1998). International crises and domestic politics. American Political Science Review, 92(3), 623–638.Find this resource:

Snyder, J., & Borghard, E. (2011). The cost of empty threats: A penny, not a pound. American Political Science Review, 105(3), 437–456.Find this resource:

Solingen, E. (1994). The political economy of nuclear restraint. International Security, 19(2), 126–169.Find this resource:

Solingen, E. (2007). Nuclear logics: Contrasting paths in East Asia and the Middle East. Princeton, NJ: Princeton University Press.Find this resource:

Tarrow, S. (2010). The strategy of paired comparison: Toward a theory of practice. Comparative Political Studies, 43(2), 230–259.Find this resource:

Thayer, B. A. (1995). The causes of nuclear proliferation and the utility of the nuclear proliferation regime. Security Studies, 4(3), 463–519.Find this resource:

Tomz, M. (2007). Domestic audience costs in international relations: An experimental approach. International Organization, 61(4), 821–840.Find this resource:

Trachtenberg, M. (2012). Audience costs: An historical analysis. Security Studies, 21(1), 3–42.Find this resource:

Walt, S. (1999a). Rigor or rigor mortis? Rational choice and security studies. International Security, 23(4), 5–48.Find this resource:

Walt, S. (1999b). A model disagreement. International Security, 24(2), 115–130.Find this resource:

Weeks, J. L. (2008). Autocratic audience costs: Regime type and signaling resolve. International Organization, 62(1), 35–64.Find this resource:

Weeks, J. L. P. (2014). Dictators at war and peace. Ithaca, NY: Cornell University Press.Find this resource:

Weisiger, A. (2013). Logics of war: Explanations for limited and unlimited conflicts. Ithaca, NY: Cornell University Press.Find this resource:

Weiss, J. C. (2013). Authoritarian signaling, mass audiences, and nationalist protest in China. International Organization, 67(1), 1–35.Find this resource:

Weiss, J. C. (2014). Powerful patriots: Nationalist protest in China’s foreign relations. New York: Oxford University Press.Find this resource:

Wolford, S. (2015). The politics of military coalitions. New York: Cambridge University Press.Find this resource:

Notes:

(1.) For a discussion of the role of the role of experiments in political science and international relations in particular, see, respectively, Dunning (2008) and Hyde (2015). For a discussion of the relationship between the game-theoretic and the causal-inference traditions, see Ashworth, Berry, and Bueno de Mesquita (2015).

(2.) To be clear, there are differences between game theory and the theory of rational choice. The theory of rational choice posits that decision makers have well-defined preferences and choose the best action available given their preferences, while game theory analyzes the strategic interaction between decision makers (Osborne, 2004, pp. 1–9). Since most game-theoretic models assume that decision makers are rational, the theory of rational choice is best seen as a critical element of game theory, and any criticism of the theory of rational choice challenges the foundation of game-theoretic analyses.

(3.) Interestingly, skepticism toward game theory and mathematical approaches is not specific to political science. As the economist Harold Kuhn explained, the Theory of Games and Economic Behavior of John von Neumann and Oskar Morgenstern was “too mathematical for the economists,” and it took more than a quarter century before it convinced the economics profession of its broader applicability (Kuhn, 2004, p. xii).

(4.) See, e.g., Bueno de Mesquita and Morrow (1999), Diermeier (1995), Fiorina (1995), Niou and Ordeshook (1999), Ordeshook (1995), Powell (1999b), Shepsle (1995), and the responses in Green and Shapiro (1995) and Walt (1999b). See also National Science Foundation (2002).

(5.) For the idea that it would be elusive to test theoretical models with data, and instead models should be thought of as maps, which can serve different purposes, see Clarke and Primo (2012).

(6.) While proponents of the EITM approach did not shun qualitative work, the priority lays in bridging the gap between formal and statistical approaches. Granato and Scioli (2004, p. 314) explained that it would be a “mistake to consider EITM as an exclusively ‘quantitative’ or mathematical enterprise,” but nevertheless conceded that “the emphasis of EITM is on the linkage between formal and empirical analysis and is a natural with a quantitative approach.” Qualitative approaches should best be seen as “a complement to quantitative analysis.” In 2010, Granato, Lo, and Wong (2010, p. 783) again motivated the EITM enterprise by describing the “important disconnect [that] exists between the current use of formal modeling and applied statistical analysis.”

(7.) For another defense of game theory, see Little and Pepinsky (2016).

(8.) Lorentzen et al. (2016) document that game-theoretic papers on international relations and comparative politics, published between 2006 and 2013, do use qualitative methods at a significant rate. Here, I document that game-theoretic papers on international security, published between 1980 and 2007, use qualitative methods much less than non-game-theoretic papers. Also, I evaluate the effectiveness of the different ways in which qualitative methods have been used by game-theoretic papers, and make a case for the controlled comparison approach.

(9.) See also Martin (1999), and Bennett, Barth, and Rutherford (2003).

(10.) The dataset contains more than 3,000 articles in the following: American Journal of Political Science; American Political Science Review; British Journal of Political Science; European Journal of International Relations; International Organization; International Security; International Studies Quarterly; Journal of Conflict Resolution; Journal of Peace Research; Journal of Politics, Security Studies, and World Politics. This set of papers represents about two-thirds of the publications in these outlets over the time period.

(11.) If we restrict attention to the period between 1980 and 2001, i.e., before the publication of the EITM report, results are practically the same.

(12.) These variables are coded as follows. First, the piece focuses on papers using “qualitative” methods rather than “descriptive” methods, according to the TRIP dataset. In short, a piece presenting qualitative evidence is coded as using “qualitative” methods if the qualitative evidence is used for theory building or theory testing, and it is coded as “descriptive” otherwise (Maliniak et al., 2007, p. 44). Second, the piece determines whether the amount of evidence on a case is “significant” if the paper reserves a section or subsection to discuss this evidence. This is a minimum threshold for the qualitative evidence to be taken on its own terms (though there is no minimum limit on the length of the section or subsection).

(13.) The analytic narrative approach is restricted to extensive form games, and aims to capture the sequence of events in a particular case (Bates et al., 1998, p. 9).

(14.) The paper first collects the list of a publication’s citations using the Social Sciences Citation Index. Then, it identifies the citations that are themselves in the TRIP dataset. Finally, it records whether these citations are game-theoretic or not, as measured by the TRIP dataset.

(15.) The figure leaves out the last years of the sample, 2006 and 2007, since there would be too little time to collect enough citations within the sample. There was one paper published in 2006 with citations in the dataset: One citation was formal, the other was not; there was one paper published in 2007 with citations in the dataset: It had one, non-formal, citation.

(16.) For discussions in comparative politics, see, e.g., Lijphart (1971), Tarrow (2010), and Slater and Ziblatt (2013).

(17.) Some proponents of the analytic narrative approach do compare different cases. For example, Avner Greif briefly compares Pisa with Genoa (Bates et al., 1998, pp. 37–38), and Jean-Laurent Rosenthal compares fiscal institutions in England and France (Bates et al., 1998, pp. 64–108). However, as a matter of principle the method eschews the comparative approach: “In effect, our cases selected us, rather than the other way around” (Bates et al., 1998, p. 13).

(18.) This review is obviously limited in scope, both in the number of topics and the set of papers reviewed for each topic. For an annotated bibliography of formal papers and their empirical testing, see Langlois and Langlois (2017).

(19.) The bargaining model of Fearon (1995) analyzes the strategic interaction between two states. For recent studies of the interaction between a greater number of states, see, e.g., Bas and Schub (2016), Debs and Monteiro (2017), and Wolford (2015).

(20.) For initial formulations of mutual optimism, see Blainey (1988 [1973]). For later discussions of Fey and Ramsay (2007) and Slantchev and Tarar (2011), see Fey and Ramsay (2012), and Debs (2016b).

(21.) For debates on the democratic peace, see, e.g., Dafoe (2011), Farber and Gowa (1995), Gartzke (2007), Gowa (1999), and Oneal and Russett (1999).

(22.) See, e.g., Debs and Goemans (2010), Fearon (2011), and Jackson and Morelli (2007).

(23.) Bueno de Mesquita et al. (1999) and Debs and Goemans (2010) both generate additional predictions. Bueno de Mesquita et al. (1999, p. 791) argue that their model accounts for six empirical regularities of the foreign policy behavior of democracies and autocracies, while Debs and Goemans (2010) argue that their theory is consistent with hitherto unknown differences in the war-proneness of non-democratic regimes. See, also, Bueno de Mesquita et al. (2003) and Debs (2016a), respectively, for additional predictions derived from the framework of the initial pieces.

(24.) Clearly, the theory of audience costs appears as a prime candidate to explain the democratic peace, which has arguably enhanced the theory’s appeal. However, proponents of the theory of audience costs have been very cautious about its ability to explain the democratic peace (see, e.g., Fearon, 1994a, pp. 582, 586; Schultz, 1998, p. 840; Schultz, 2001, p. 11).

(25.) To be sure, bluffing is part of an optimal strategy. Rational audiences would not punish bluffing per se, but what bluffing reveals: i.e., that the leader would be relatively incompetent in prosecuting a war, and another leader could prosecute the war more effectively.

(26.) See, e.g., Bas and Coe (2012, 2016), Debs and Monteiro (2014, 2017), and Monteiro and Debs (2014).

(27.) For a recent step in this direction, see Lorentzen et al. (2016).