Once I was asked what the “comparative method” meant for comparative political science (read taking doctoral comprehensive exams). The following was my long-winded, smart-ass response. Enjoy!
To answer the question whether the “comparative method” is a distinct method within the discipline of political science requires that we review some of the history of this method, and in a way, the history of comparative politics as a whole, for they are intimately bound together. I am tempted to provide a quick quotation from someone like Arend Lijphart and be done with the question, but this would only be grasping onto a single argumentation of what the comparative method is, and it would be a dishonest answer since the way that Lijphart defines the method is something that I doubt I shall ever espouse, as in using it as the only method in my own research.
Therefore, in answering the question whether the comparative method is distinct, I will begin at the beginning of comparative methodology in general in the study of politics so that we can see when and where this notion of the Comparative Method (capitalized letters, naturally) is said to have emerged. More than a few trade-offs—that is, strengths and weaknesses—are applicable here to the comparative method.
Additionally, in order to know how the comparative method is distinct, more than a few scholars will be mentioned, but I shall also highlight three scholars in particular as examples of utilizing a particular method at various points in this essay: 1) Arend Lijphart; 2) Charles Tilly; and 3) Daniel Posner.
To state the conclusion first: the comparative method is neither wholly distinct from other methods within political science, nor is it entirely the same as other methods within political science.
The comparative method is better thought of as a collection of comparative methods—meaning method with an “s”—and, in the light of the scope and history of the sub-field of comparative politics as a whole, we should not do wrong to conceive of these comparative methods as belonging to something like a tradition of comparative methodology, rather than as a mono-concept of the method.
Some History of Comparative Methodology
When did comparative politics, or, more specifically, the comparative method start?
Some like to look back to Aristotle of 300s BC comparing Athens to other city-states. Others like to think of Montesquieu in the 1740s and his comparing of the ancien régime of France to the stodgy Parliament that was emerging in England at the time and to the famed despotism of the near “Orient.” Others still, like Robert Putnam, wish to look back to Tocqueville’s in-depth comparison and “case study” of America to the France of the 1830s. One could add Mill of 1840s and his A System of Logic which would become better well known for political analysis again after Theda Skocpol used it in her work on social revolutions. One could also conjure up someone like Max Weber in the early 1900s and his empirical work and methodology. But as much as we may like to think of the above examples as being within the tradition of comparative methodology within political science in general, one may worry that this is simply imputing a contemporary idea of the comparativist and comparative method(s) onto these thinkers and scholars of the past.
Comparative methodology in the way that we think of it today as a subfield of political science did not really emerge until after the World War II.
This observation becomes all too clear especially if we remember that even scholars like Gabriel Almond were still referring to comparative politics as a “movement” up until the late 1960s. Some have described this first decade or so after the war in the subfield as reflecting a “general optimism of the period.” This may be so, but one should not forget that this comparative methodology of political science was very reactionary to the work done by scholars before the war.
Reacting against what? They were reacting against “institutionalism.” This institutionalism can be thought of as the exclusive approach to comparative methodology before the war, which emphasized the study of law and constitutions, the state and the government and their legislative instruments. Sometimes this meant comparing different constitutions. Sometimes this meant trying to understand their legal evolution in one national state. Woodrow Wilson comes to mind as an example of someone working in this tradition before World War I. As for the interwar period, and as an example of the “methodology” of this time, one can be reminded of Edward McChesney Sait and his now largely forgotten work Political Institutions: A Preface. Just to get something of the tenor of that pre-WWII methodology in comparative politics, I want to quote Sait directly:
Apart from their history and their environment political institutions cannot be studies with much profit.
Overall, we should not be too harsh to conclude that most studies at this time were legalistic in nature and questions were formed without any intention for predictive power. During American Political Science Review’s (APSR) centennial publication, Mark Blyth argued that the comparative methodology’s
inability to predict any of the great events of the previous decade [i.e, before the 1950s] had proven a serious embarrassment.
This was an embarrassment that the next approach of comparative methodology within political science would desperately seek to avoid.
The next dominate approach and methodology—or “paradigm” if one wishes to use Khun’s language within the social sciences—of firmly modern political science and comparative politics was called “behavioralism.” This behavioralism can be defined as having at least three elements that differed greatly from the study of comparative politics before World War II:
- Political behavior as a research focus;
- A methodological appeal for science;
- And a focus on pluralism, especially in spite of any “old” concepts of the state.
V. O. Key in an address to the annual meeting of the American Political Science Association (APSA) on 4 September 1958 asked his colleagues to make sure that new graduate students received “more, and far more rigorous, research training.” This normally meant more use of survey methodology, which would fit better with the tallying of people’s individual “behavior.”
Arend Lijphart has one of the best examples of someone who tried to standardize the mentality of survey methodology into his definition of the comparative method, or as he put it:
The comparative method is then nothing but the statistical method under relatively unfavorable, but improvable circumstance.
By defining the comparative method of comparative politics in this light, Lijphart could then recommend four ways to make the comparative method more statistical (and hence more scientific) by the following:
- Increasing the number of cases;
- Combing variables and/or categories;
- Considering the inclusion of more diachronic cases;
- And omitting variables of marginal weight.
Thus, as was implied above in the introduction, one might be tempted to define the comparative method in the way that Lijphart did and quickly end the discussion.
But there is a problem in doing so.
For myself, I’ve always viewed Lijphart’s definition of the comparative method as one of the more depressing, mainly because it is a definition that will always place “the method” of comparative politics as a second best alternative to larger-N studies which use more and more quantitative numeration that go beyond the usual mean, median, mode, and range and toward correlation coefficients and much more complicated statistical methods. However, reading reviews of Lijphart’s most recent, second addition of his Patterns of Democracy these past month, I was struck how many contemporary political scientists thought that Lijphart did not go far enough in this study of 36 democracies with quantitative testing and that the book was much too qualitative. I am fairly agnostic about all of this, but what I shall say is that Lijphart did need to have those 300 pages of qualitative explanation around the few statistical tables he provides, because without those the content validity of many of those concepts—operationalizing “degrees of dominance and balance of power” is just one example out of many—would not have been defendable let alone understandable. And of course this is all debatable how high the content validity can be when one wants to index numerically a concept like “balance of power” within the constitutional makeup of a democracy.
Either way though, what is important here is to understand that behavioralism, in this sense of a firmly established acceptance of statistics as a tool for comparative methodology, is still very much alive today and taking a basic class on survey methodology is requirement for most political science graduate programs in America. But let us return to the behavioralism of the 1950s-60s so that we can continue to see how this has affected comparative methodology as a whole.
In moving toward an objective survey of American pluralism, behavioralism also sought to distance itself from any notion of the state . Instead, comparativists were asked to think of the “social system“—the preference for the word “system,” as opposed to state and society or, born from the concept of “systems analysis” and first carried over into the study of politics in Ronald McKean’s Efficiency in Government through Systems Analysis.
A perfect example can be found from the comparativist scholar Seymour Martin Lipset when he called for
generalized theoretical categories…to have a valid relationship to characteristics of total social systems.
Theories like functionalism, modernization theory, and political culture became areas of concern within comparative politics. And, of course, these were applied to the more “exotic” places, thereby increasing comparative politics’ global reach via programs like area studies programs and others.
However, behavioralism was not to dominate comparative methodology forever. Toward the end of the 1960s, there seemed to be something of a backlash against behavioralist methodology throughout political science, with even the head of the APSA at the time David Easton calling for a new “post-behavioral revolution.” Despite this call for change, one cannot help but notice that possibly Robert Dahl was correct when he said that the behavioral approach
will not disappear, then, because it has failed. It will disappear rather because it has succeeded.
Scholar Example: Arend Lijphart
As an example of behavioralism as a distinct method within comparative politics, I should like to emphasize an older work by Lijphart in the early 1960s. In his article on UN General Assembly voting patterns, Lijphart was adamant that his method of investigation was to have nothing to do with past historical or legal analyses. Lijphart argued that he was interested in the study of political behavior—in understanding “bloc-like behavior.” He was clear that
[t]he purpose of this essay is…to suggest an alternative method not handicapped by the weaknesses of techniques so far employed.
The break with past comparative methodology before the war is palpable. Unlike reviewing legal documents for how roll call votes work in the UN General Assembly, he spends the majority of his argument concerning himself with issues of measurement.
Lijphart begins his study of “The Identification of Blocs” with a typical literature review, pointing out “major weaknesses” in all past studies of voting within the UN. This initial section though does not yet go into the methodological argument that Lijphart will extend for the majority of the article. Interestingly, Lijphart is more concerned with concept creation here. Essentially, he finds past scholars’ attempts to define “bloc” as unsatisfactory because they fail to “provide a precise definition of terms and to distinguish between different kinds of groupings in the Assembly.” The second section entitled “The Measurement of Bloc Cohesion” is where we learn of Lijphart’s commitment to define the UN General Assembly voting bloc by “the degree to which bloc members consistently vote together.” There is definitely a precision here that was not to be found in any of the terms that McChesney Sait would have used in his study of institutions in the 1930s.
Lijphart begins by borrowing terminology from the American Government subfield and then synthesizing them with past work on the Assembly. But of these past quantitative studies on voting behavior, Lijphart has agrees most Thomas Hovet, Jr. Lijphart agrues that Hovet’s equations of “identical vote” and “solidarity vote” have potential before eventually settling in on the “Rice–Beyle Method” to measure bloc voting in the Assembly, which Lijphart refashions for his own uses into what he terms the “Index of Agreement.”
The clearest example of behavioralism here is when Lijphart “operationalizes” exactly what he means by bloc voting in the UN General Assembly, by moving onto the final section of the article entitled “An Example: Voting Alignments on Colonial Issues, 1956-1958,” where he begins to collect data to run through his equation, borrowing the tools of American psephology for the international arena.
What I find amazing about all of this is not so much the technicality of Lijphart’s study but that he had to calculate his data without the help of a contemporary software program like SPSS or Stata. It is easy to understand why, then, he chose to narrow his final empirical test to a sample of only 44 roll-call votes in the UN General Assembly, particularly since every voting country had to be weighted against every other voting country to be able to unearth any potential patterns of bloc voting.
Overall, Lijphart’s study in APSR portrays all of the three characteristics mentioned above (i.e., political behavior, scientific methods, and pluralism) for behavioral-type epistemology. If one reads a new publication of Lijhpart’s like the second addition of his Patterns of Democracy in 2010, you shall see that Lijhpart still follows these three criteria.
The question is, then, does comparative politics still utilize Lijphart’s recommended method as the method of comparative methodology today. The answer to this is neither a full affirmation or negation. But we need to show why this is the case.
Against Behavioralism as the Core of Comparative Methodology
From the middle 1960s-1980s, there was a backlash against behavioralism. Some of the backlash was against theories which seemed to not be able to explain current events as authoritarian dictatorships appeared and coups d’état occurred in Africa and Latin America and Asia. Some of the backlash was against survey methodologies that did not seem to take into consideration, or to take seriously, either context or history. Indeed, it is this latter backlash that is important for this essay question on comparative methodology.
Institutionalism as a comparative methodology seemed to be coming back into vogue for some within comparative politics. But unlike the institutionalism before World War II (and like Dahl correctly predicted) some aspects of behavioralism would continue to imbue the subfield.
For example, this “neo”-institutionalism still had traces of the importance of pluralist democracy in its focus on “developmentalism.” This would include voter behavior and political parties, yet also contain a renewed interest on how these affected the state. This would also concern how the state constrained, enabled, and affected society and how powerful people constrained the state. The challenges of elites in these developing countries and “democratization” would also become more important to the subfield. And, of course, there were also some comparativits who never really took behavioralism too serious to begin with and always maintained a historical focus to their work.
To the above changing overarching theories of how to do political science and comparative politics was also those who wanted their “comparative method” to be one of “thick description,” ranging from just one case to a few detailed and very descriptive cases.
Here, I am talking of those in comparative politics who were influenced by the anthropologist Clifford Geertz. Comparativists who were also something of area specialists were probably the most influenced. In brief, what Geertz called “webs of significance” spoke of the construction of religious beliefs and their practices, along with cultural customs, social interactions, attitudes and behavior—the social milieu on the local and micro level—and its importance for politics.
When comparative methodology incorporated his ideas, we tended to pack it into concepts of “political culture” or, say, “political environment” as a whole.
For example, anyone who had already been enchanted by the macro study in The Civic Culture now had a reason to go even deeper into single cases at the local level of how culture affects politics and political outcomes. For someone who enjoys studying Thai national politics, a piece like Lucien Hanks‘ “Merit and Power in the Thai Social Order” and its importance for area specialists on Thailand in the 1960s and 1970s is an article I always associate with this type of thick description. Of course, what Geertz was arguing for here was for a kind of “interpretation” that went beyond the, though qualitative, but still very positivistic, five-nation comparative study of Almond and Verba.
To the forays into both culture as a subject and/or “independent variable” and an interpretive method à la Geertz we also need to talk about those who, when turning away from behavioralism, sought out more historical methods as being the core of comparative methodology.
Of course, if we think of comparative methodology in the way that someone like Max Weber would have, then history as a “method” is always important. And we see this influence today in methods like “process tracing”, which, honestly, I still think is a rebranding of historical narrative, but made much more palatable for the political scientist. Maybe we could call it contextualizing of causation or contextualizing of social scientific generalizations?
I know that someone like William H. Sewell, Jr. as a historian has no problem pointing out
that the underlying causal structures themselves undergo mutations or transformations over the course of historical time.
Paul Pierson has championed this idea in connection with “path dependence.” In this definition, we can talk of a “critical juncture” being the start of a historical process, whereby as time proceeds, the path upon which the social convention or institution has been set, increases the likelihood that that the convention or institution will itself continue and decreases the likelihood, or makes it harder for, the convention or institution to change back to the way things were before this seminal event of the critical juncture.
Mahoney and Rueschemeyer flesh out these and many other ideas in their edited volume and how they can still meet the demands of “generalization” or “knowledge accumulation” or even how we might be able to use what are essentially tools of the historical side of social science in a comparative manor across cases. In this sense, Mahoney and Rueschemeyer are not completely rejecting the comparative method of Lijphart, but instead are incorporating some interesting elements that go beyond survey methodology.
Thus “comparative historical analysis” (CHA) can be defined as having three main components by Mahoney and Rueschemeyer:
- A “fundamental” concern for “casual configurations that produce major outcomes of interest;
- A focus on “historical sequences” and the unfolding of these “processes over time”;
- And a demand that the comparison and contrasting of cases be both “systematic and contextual”.
What CHA avoids to do though, at least for Mahoney and Rueschemeyer, is try to proscribe any particular theory (say, Marxism or modernization) or any type of interpretation like Geertz or “the nihilistic devil of postmodernism“, and because of this, CHA is not so much different from the methods argued by Skocpol in her Vision and Method in Historical Sociology. One difference, though, between Skocpol and this newer variant of CHA by Mahoney and Rueschemeyer is that alongside
temporal processes and path dependence, conceptual formation and measurement, and strategies of causal inference ranging from historical narrative and process tracing to Boolean algebra and fuzzy-set analysis
one can also add statistical methods where appropriate to bluster one’s comparative argument—and hence we see that behavioralism and Lijphart’s concept of the comparative method still finds its incorporation into a more eclectic comparative methodology.
Furthermore, CHA in a wholistic sense is not like Lieberman’s “nested analysis” of “in-depth” and or historically-minded research, which is supposed to fit into a larger statistical survey. Rather it should be noted that CHA would work the other way round, whereby the statistics can be “nested” in particular case of historical contextualization: thus not historical processes nested in statistics, but statistics nested in processes.
It is important to understand though for the purposes of this essay that this “historical turn” really does stretch from the time of behavioralism up until today. As will be mentioned with rational choice, all of these methods overlap to make what is comparative methodology today.
Scholar Example: Charles Tilly
After having explored some of the methods of this more narrative form of comparative methodology, I now want to turn to a specific example in the work of Charles Tilly.
A popularly cited example of both his method and his hypothesis on state formation is the theory argued by Charles Tilly, which stated that the national states of Europe were formed both from the “internal” state-making processes of government—or what he described as the “quintessential protection rackets with the advantage of legitimacy”—and the “external” processes of war. These processes overlap into four general activities by the government of the following:
- “War making”, meaning the elimination/neutralization of rivals outside one’s territory;
- “State making”, meaning the elimination/neutralization of rivals inside one’s territory;
- “Protection”, meaning the elimination/neutralization of the enemies of one’s clients;
- And “extraction”, meaning the acquirement of resources (e.g., taxes) to carry out the first three activities.
In a somewhat idealized sequence, Tilly argues that some feudal lord made war so well that he became the sole ruler of a space of land/territory, but that war-making in turn meant that he had to increase extraction of resources from the population within his territory; thus, almost as a by-product, this sequence
created organization in the form of tax-collection agencies, police forces, courts, exchequers, account keepers; thus it again led to state making.
This process between war and state-making is probably best remembered by an oft-quoted line from Tilly’s earlier work that in Europe:
War made the state, and the state made war.
I agree with Centeno about the above quotation:
There is a causal ambiguity in Tilly’s famous aphorism: which came first, states or wars?
But for most comparativists, we generally use something of the following causal formula for this argument that goes: war → extraction → repression → state formation. Today we refer to this as the “bellicist” theory or hypothesis of state formation. Tilly would expand this argument by added another over-arching element of “commitment” after about 1800 by both those in the government and the people within the territory. In Tilly’s own words:
Nationalism in the sense of heightened commitment to a state’s international strategy appeared rarely before the nineteenth century, and then chiefly in the heat of war.
But even with this modification, Tilly kept the core of his theory of state formation in war-making.
After what I have just described above using Tilly’s bellicist hypothesis, we should step back and see how Tilly went about testing this hypothesis as compared with how Lijphart went about testing his hypothesis on UN General Assembly voting. The first thing is the absolute breadth of time that Tilly is dealing with as compared with Lijphart’s study. I cannot think of a way of utilizing Lijphart’s behavioralism to describe approximately five-hundred years of history without resorting to the type of comparative historical analysis that Tilly is using.
Indeed, this is the main point: not all questions and/or puzzles of the comparativist can be answered adequately utilizing only the method of comparative politics as argued by Lijphart.
This is not to mean that for contemporary studies of state formation after the Second World War some form of survey methodology cannot be used. Rather, the key here is simply that the study is mixed with the type of descriptive narrative employed by Tilly.
For example, one additional extension of Tilly’s argument outside of Europe can be found in Taylor and Botea’s argument for Afghanistan and Vietnam. They argue that they think that Tilly’s theory holds for Europe during the medieval and early modern periods and that cases like North Korea and Israel can be cited to illustrate how it can still be used effectively today . However, Taylor and Botea point out that the two national states just mentioned are unique in that
there is much less consensus on whether the war-making/state-making nexus still functions in the contemporary, post-World War II world.
To test if this is a valid claim or not, the two scholars choose two of what should be proof-positive cases of Afghanistan and Vietnam, since both have been embroiled in the longest international wars of the post-WWII period (Afghanistan from 1978-2014 and Vietnam from 1946-1989). If Afghanistan or Vietnam fail to have strong states, then the bellicist theory would seem to make less sense for the post-WWII era. Obviously, one need not review their argument to already know that Afghanistan has a very weak state (measured by ability to tax or control of territory), whilst Vietnam is a very strong states (again using the same measurements).
So the question becomes, what other factors do Taylor and Botea use to explain why the Vietnamese state is so much stronger than the Afghan state despite both countries having gone through more than 30 years of international conflict apiece. They identify two variables as possible explanations:
- A “core ethnic group”;
- And 2) a combination of war with revolution, “facilitated by the promulgation of a unifying ideology.”
Looking at the second factor first, we see that this does not seem too help us too much since both Vietnam (communism/nationalism) and Afghanistan (Islamism) have currents of strong ideology that persist amongst the citizenry. Therefore, Taylor and Botea put their hopes on ethnic homogeneity as being a necessary but not sufficient condition that must be present before the onset of international conflict. Vietnam has the Kihn (Viet) which make up more than 85 percent of the population and can trace a history in that area back for two millennia easily, whilst the Afghan state has 14 official ethnic groups, of which the Pashtuns make up about 40 percent, the Tajik about 20 percent, the Hazara about 10 percent, and so on and has varied considerably since the time of Alexander’s conquest two and a half millennia ago.
Again, returning to a discussion of comparative methodology, I would argue that if this is the case, that some kind of ethnic majority is a prerequisite in the post-WWII world to keep Tilly’s argument working, then a new study ought to be undertaken, researching cases that vary along this factor against Vietnam which includes a statistical test of all post-WWII cases.
Using Tilly and other scholars who have cited him is useful to this discussion because it shows how not only did Tilly use a method that was distinct from behavioralist comparative methodology, but that today scholars within comparative politics will often use both of these methods as a mixed-methods formation to better make their arguments.
Methods beyond the Narrative
Aside from the culture and the historical methods mentioned above, the beginnings of more economic deductive logic or “rational” theories from elsewhere in political science—very much contra the survey methods of behavioralism—would slowly make themselves felt in comparative politics as well. I have not much to say about rational choice here, aside from that it theoretically posits man as something of Homo economicus and seeks to understand political outcomes from this micro level of the individual and his or her choices, which by the theory ought to reach an optimal “parity” stemming from a rational actor’s maximization of utility in the person’s choices.
One can also raise the level of analysis either to that of institutions or heads of organizations, even states, or continue to think of the rational actor, but “within” an institutional setting. This latter aspect is where we sometimes hear of “bounded” rationality, which can take on a few forms, but it generally means a chink in the armor of or a softening of Homo economicus and his or her supposed maximization of utility, so that researchers in this tradition can confront questions about other values that are not totally reducible to “self-interest.”
I will say this about rational choice: it is much more purely deductive than even survey methodology of behavioralism, hence William H. Riker calling it a “positive” political theory. I suppose it is odd to see political science and some within the subfield of comparative politics move toward rational choice, when economists are abandoning many of the premises of “classical” economics for “behavioral” economics.
The most damning aspect of Green and Shapiro’s critique on rational choice in the 1990s was simply that even though those who make models of game theory do well on the deductive side, they often do very poorly on the testing side using real empirics from the real world. Either way, like other “turns” in comparative methodology, rational choice overlaps and still has its defenders and practitioners today.
For myself, I can say that there are times when thinking of political actors as acting rationally does make sense. It is, rather, just how does the theory explain those times of irrationality?—Or are we to assume irrational goals can be worked toward in a rational way?
Back to the discussion at hand, what then was the main comparative method of comparative methodology of the 1980-1990s after the review of history and culture and rational choice above? Because of the multifariousness of methods and focus areas, it becomes harder to categorize this period under any one epithet.
Was this then a time of flowering theories, methods, and questions where they all worked off each other, or would we do better to use Weber’s Methodenstreit to describe this period after behavioralism?
The Comparative Method as a “Messy Center”
By the middle 1990s, many in the subfield thought that the diverging interests seemed to create a “messy center” for any notion of the comparative method. Disagreements on proper methodology seemed to be a never ending source of headache not just for the subfield but for all of political science. A great example for this comes from a symposium that was held during 1993-94 at the Center of International Studies at Princeton University entitled “The Role of Theory in Comparative Politics” and included guests like Peter Evans, Peter Katzenstein, Adam Przeworski, Susanne Rudolph, James Scott, and Theda Skocpol. Each participant had an argument to make and it is worth pausing here to see where comparative methodology was at this time in the 1990s.
Evans seemed quite okay with the “eclectic” messy center and argued that neither “values and symbols” nor rational choice would overrun the diverse methodological center of comparative politics. Like Mahoney and Rueschemeyer a decade later, Evans also argued that building knowledge is important, as is prediction, or as Evans does well to say:
The desire to predict is part of social science, not because we are positivists but because social scientists share with everyone else the desire to know what is likely to happen to them and how they might be able to improve prospective outcomes.
Katzenstein made the argument that he simply does not care about, nor put his own work under, any particular “professional flag” of comparative methodology like institutionalism, cultural studies, or rational choice. Instead he says we tend to do our best research in comparative politics (or international relations) when we focus on “political problems” and especially “intellectual puzzles” in politics, most often “driven by real-world events.”
For myself, when reading an argument like this by Katzenstein, it give me more confidence to argue that the comparative method is something that really does not exist and that we should really be arguing, as this essay does, that comparative methodology has become more of a plural concept than a single, mono-concept.
Continuing with this symposium, for Przeworski, though he is probably better known in comparative methodology as a champion of statistical methods, interestingly, he argues that it is best to be a “methodological opportunist”—even to the point that
[i]f deconstruction is needed, I will even try deconstruction.
At the very least, the gist of Przeworski’s overall argument is that though we often construct comparisons from real world cases, thinking counterfactually about un-empirical realities is also important for comparativists, so that often times
matching the observable cases may just exacerbate the selection bias. Instead, we must theorize about the mechanism by which the observations are generated ad then use this knowledge to compensate for the nonrandom nature of the observable world.
Rudolph argues that we use the insights of both Weber and Foucault when necessary in comparative methodology, which in a way is another argument like the above that the “messy center” of comparative methodology is not necessarily a bad thing. Scott has one of the more honest arguments for the center of comparative methodology, which for him is that he simply does not know or does not do very well at describing what he enjoys doing when he thinks theoretically about empirical comparative cases.
The best advice Scott has for anyone in comparative methodology is that
[i]f half of your reading is not outside the confines of political science, you are risking extinction along with the rest of the subspecies.
Skocpol emphasized what she likes to do when she does comparative politics which she called “macroanalytic comparative history.” But for others in the subfield, she argues that all of us should have some core and that
the purpose of comparison should be partly to explore and test hypotheses from a variety of theoretical perspectives and partly to notice and hypothesize about new causal regularities.
Skocpol seems to make one of the strongest arguments out of this group for the importance of not just applying theory, whatever that may be, to real-world empirics, but that we should be concerned with these “causal regularities” when we speak of the comparative method.
How might I sum up this symposium then? Assuming that Skocpol, Scott, Rudolph, Przeworski, Katzenstein, and Evans is about as an eclectic a mix as is possible, the only real center or conclusion for comparative methodology in the middle 1990s seemed to be that “real-world” and “problem-driven” work is important, as is using empirical data with theory and not just theory by itself.
Causation is also important to all of the above and for comparative methodology at this juncture. However, not everyone agreed about the supremacy of a generalization of those posited causations, meaning that causation can be local or context-dependent and not always a “pattern” or “regularity.”
As for the theoretical center of comparative methodology at this stage? Was it rational choice, culture, or institution, or something else entirely? Interestingly, all seemed okay with the “messy center” and none wanted to be the tyrant who said that one vantage point must be the core of comparative political methodology. In fact, it almost seemed like something of a faux pas with this group of scholars to say one method must be king.
Fighting for the Core of Comparative Methodology
However, across political science in general, some have tried to argue for a narrower methodological center to our research. This argument has also affected the way we think about the comparative method.
These scholars tried to base any and all studies with a stricter understanding of “causal inference”—a reallocation of quantitative statistical survey methods back onto those who preferred qualitative and/or “single” case studies. But this may have upset more people than truly centered either the subfield or political science as a whole. For myself, I have to admit that if it were the case that qualitative work had to follow exactly the prescriptions in Designing Social Inquiry: Scientific Inference in Qualitative Research, then I should be tempted to give up getting a PhD in political science altogether. This is because some of my favorite research projects and arguments that utilize comparative methodology break one or many of the rules of this particular textbook.
To just name one out of many methods that Designing Social Inquiry rejects out of hand because they cannot be “observed” would be the construction of an argument around counterfactual cases or conditionals. James D. Fearon makes the argument that “counterfactual conditionals,” or what is sometimes called “thought experiences,” have a long tradition in both comparative politics and international relations even if they are not always articulated as such at the time of a researcher’s explanation of causation.
In a world where particular arguments based on a testing of a hypothesis cannot be made for either practical or moral reasons,
support for a causal hypothesis in the counterfactual strategy comes from arguments about what would have happened [and are made credible by] invoking general principles, theories, laws, or regularities distinct from the hypothesis being tested [and by] drawing on knowledge of historical facts relevant to a counterfactual scenario.
See also David Lewis on this point of counterfactuals. Additionally, both the publications by Levy and Brady and Collier, along with many other “tools” for doing more qualitative work, also address counterfactuals as a useful method within comparative methodology for increasing the number of cases virtually.
One additional qualitative type of research bemoaned by King, Keohane, and Verba in Designing Social Inquiry was the “case study.” Against this, John Gerring makes an interesting argument for the case-study approach for comparative methodology—essentially that we would do better to think of the “case study” as complementary to cross-case or “cross-unit” research design. Case studies are more than just a single-N study and can actually be used to generalize across a larger set of units. However, part of the problem of case-study work in general is that the term “case study” can mean a few different things from more ethnographic research to process-tracing research to a study of a single phenomenon.
Therefore, Gerring is very specific in how he defines a case study and I have found it the single most useful definition that I have read since beginning studies in comparative politics. Gerring argues the following:
As a substitute for these flawed definitions, I propose to define the case study as an intensive study of a single unit for the purpose of understanding a larger class of (similar) units.
By “unit” Gerring means something like a nation-state or a revolution or a person and so on and, even though the time boundaries of such a unit can still pose some challenges, this definition of a “unit” is still in line with King, Keohane, and Verba’s conception of a unit. Interestingly, Gerring argues that when a case study is used to investigate causation,
[this] causal relationship hinges on the counterfactual assumption
which is similar to Fearon’s argument above.
Additionally, one of the main benefits of a case study can best be realized when we admit that there will always be “tradeoffs” in one’s research design, regardless of whether is single-N or an N that runs into the thousands.
This tradeoff is important, since as I mentioned in the introduction to this essay, there are strengths and weaknesses to all of the different methods of comparative methodology, so let’s review some of those for the case study versus larger-N studies. Aside from the importance of case studies that represent a “crucial” case in a larger theoretical argument, this tradeoff is often between whether the research question is more exploratory or confirmatory in nature. And of course others have argued the importance of the case study as an exploratory tool for developing new theories and hypotheses across all of social science. A few more tradeoffs beyond exploratory (i.e., generating) versus confirmatory (i.e., testing) have to do with validity (meaning internal versus external validity), with causal insight (meaning specific mechanisms versus end effects), with the scope of research (meaning deep versus broad explanations), with the type of units (meaning more heterogeneity versus more homogeneity), and with a few more tradeoffs, of which I find these to be the most applicable to myself. Both sides of any of these “tradeoffs” are needed in the subfield of comparative politics and I think it would be a shame to suggest that we go without either of them.
Altogether, these two examples—counterfactuals and single case units—are just some of the methodological tools that were left out of Designing Social Inquiry. Yet either way, the publication by King, Keohane, and Verba was one of the most cited causes of angst for many across political science in the latter half of the 1990s and early 2000s. Methodenstreit or a “Tale of Two Cultures” appeared to be a very real phenomenon as dissatisfaction came to a head during the so-called “Perestroika” movement within political science to seek for greater methodological pluralism as well as other issues.
New methodological textbooks sought to answer the question of the validity of “casual inference”—where, and where not, it ought to be applied in the study of politics. From outside of political science (and America) came a plea not just for methodological pluralism but also to “make social science matter“—that is, for us “making political science matter“—by moving beyond just the purely analytic or technical to the prudent (“phronesis“) or appropriate. Did comparative methodology find a way to clean up the “messy center” in the middle of its research traditions?
The Comparative Method as Methodological Pluralism
Mahoney has recently argued that the basic question of how exactly one might combine more qualitative work with quantitative “still need to be resolved.” He suggests that this may have to do more with an underlying philosophical questions, even though they are ultimately felt at the methodological level. From the 2000s onward, the answer to this may be more in the eye the beholder than anything that has been objectively settled. This is how editors of the newest edition of Comparative Politics wish to describe the current state of comparative methodology.
Lichbach and Zuckerman identify three research traditions or “paradigms” as composing comparative methodology of today, each of which has questions and subject matter and preferred methods. They are:
- Rationality, which includes “bargaining, collective action, credible commitment, [and] incomplete information”;
- Culture, which includes “identity, difference, membership, [and] boundaries”;
- And Structure, which includes “regime, capitalism, state, [and] society.”
As far as the comparative method goes, Lichbach would argue that most scholars will normally work within one of these emerging traditions, often utilizing various comparative methodology that best fit their questions as hand, but a few might jump between all of them. This would mean there would be room in comparative politics for methodological textbooks as diverse as the following three books: Comparative Historical Analysis in the Social Sciences, Design Social Inquiry, and Nested Games: Rational Choice in Comparative Politics.
Some, like David Laitin, have called for all three paradigms to come under one methodology. Laitin tells us that for comparative methodology
a new consensus is on the horizon, one that emphasizes a tripartite methodology, including statistics, formalization, and narrative.
Some just wish to combine just two methods, like the historical with ration choice. This has, no doubt, been part of the popular trend across all of political science for “mixed methods” approaches to one’s research.
Others ask us to concentrate on particular types of casual understanding via process tracing and the unearthing of critical junctures. Others still are less concerned about the methodology as long as we keep our theorizing to the “middle-range,” meaning not a theory of everything (like Talcott Parsons or Marxism), yet also as Ziblatt worries not a theory that has no reach beyond the parochial.
The sociologist Marshall probably would have called this middle-range theory “small stepping stones.” Indeed, they certainly would be between
[the smaller] working hypotheses that evolve…during day-to-day research and the all inclusive systematic efforts to develop a unified theory that will explain all [the social behaviors across the globe]
which Robert Merton (1968, 39-72) advocated for social scientific research. Many of us studying in the comparative methodological tradition have been reminded on more than one occasion when constructing a research design to try to find theories that can answer a “puzzle,” which born from a particular case or few cases also has general applicability because it explains why some situation or happening is different from the norm or the expected; and, as recent Southeast Asia-focused comparativists have argued, to construct arguments that can be
stated in terms of general casual factors that could certainly be converted to other regional contexts.
And in fact, I have taken this to heart and agree that, though it sounds somewhat tawdry to use this particular word to describe one’s research, a “puzzle” or an “intellectual puzzle” is a good way to begin thinking about one’s research rather than just a “topic.” Leslie Anderson argues that the researcher should
choose their own data collection methods as best suit their intellectual puzzle, defining that puzzle first and only then choosing the method of data collection.
Additionally, I should want to add that such a “puzzle” probably ought to appeal to others than just myself and outside of the region of the world which I love to study the most.
The idea is that to be an area specialist is still very much needed, but one must also actively seek to engage within the larger community of scholars in comparative politics via our theories, methods, and questions. Without this latter stipulation, an area studies specialist could fall into the trap of utilizing only one of the methods of a more inclusive comparative methodology.
Scholar Example: Daniel Posner
Daniel Posner’s article “The Political Salience of Cultural Difference: Why Chewas and Tumbukas Are Allies in Zambia and Adversaries in Malawi” in the American Political Science Review is a great article with which to illustrate how a mixed-methods approach can be successful today in the 21st Century. Posner’s work displays a confidence in behavioralism, rationality, and area studies, by combing them in a natural experiment, which can be used as an exemplar for what many would probably consider the real value that comparative methodology has to offer to the wider political science community.
His argument, in brief, is that the importance of the cultural cleavage between the Chewa and Tumbuka tribes, which straddle the Zambia-Malawi border, increases in utility for political actors only as the population size of the Chewa and Tumbuka in relation to the overall national population consumes a majority of the national electorate. In Malawi, where the two tribes entail most of the electorate, politicians utilize cultural differences to gain votes. In Zambia, where the two tribes are easily less than half the electorate, politicians seek to down play cultural differences between the tribes in order to construct a base that includes them both equally.
Methodologically, what makes this argument so interesting is the way that Posner is able to unearth an already occurring natural experiment. Essentially, because of the whimsical way the colonial powers carved up the African continent, there are many spots where traditional nations of people were divided into two new groups, becoming smaller ethnics groups of a larger, multiethnic national population. Posner identifies such new groupings as profitable ways to test different theories. In his article, Posner breaks down his argument into four sections.
The first part entitled “A Natural Experiment” he discusses his reasons for seeing potential in such international divisions of traditional tribes as mentioned above. The second section entitled “Chewa-Tumbuka Relations in Zambia and Malawi” delves into the particulars of his research design. Because experiments are uncommon for most political scientists, and even more so in the comparative politics subfield, Posner needed to dedicate three pages of his article, including geographic maps, justifying procedures like his selection of research villages, where they were located from the border, their populations, their contact with other villages across the cultural cleavage, the number and demographics of the citizens asked to participate in the study, the way they went about engaging the villages, and finally the specific survey questions they administered. Methodologically, Posner is also shows an ease with supporting his field survey data with statistical support. Nowhere is the internalized normative commitment to the first two tenants of behavioralism mentioned above better displayed than by the acquiescence to performing an additional logit regression in order to
control for respondents’ tribal affiliations, gender and age, and the number of cultural differences they mentioned in response to the open-ended question about Chewa-Tumbuka differences.
Interestingly, Posner seems to perform something of a literature review of other current explanations in the last section of this research article entitled “Why Chewas and Tumbukas are Allies in Zambia and Adversaries in Malawi.” By waiting to lay out his argumentation against past studies from other traditions (e.g., modernization theory, ascriptive ethnic explanations, constructivist explanations), we are allowed to compare each past study against the empirical results as provided by Posner’s gathering and analysis of data. This makes for a persuasive argument and makes it easier to understand why such an article was accepted for publication in APSR.
Posner adds one additional section to his article before the conclusion, entitled “The Power of Administrative Boundaries.” Methodologically, he performs one more additional test to shore up his argument. Similar to his design above, Posner does an additional survey, but this time instead of using international boundaries, he uses administrative boundaries within Malawi. What is so fascinating here is that his argument holds up again for these lesser population divisions as well. Though almost not needed at this point, this last test adds a rational choice-like argument about the way that politicians go about using cultural cleavage to maximize their chances to win votes in elections, both at the national and the subnational level.
The reason why I wanted to pause momentarily on Poser’s study was because it serves as wonderful example in how there is no such thing as one comparative method but instead a mixing of comparative methodologies. As I think about the design for my own research projects, I find works like Poser’s useful because they help me to think beyond just one method or one case study that I may have otherwise have opted to do.
Future Methods and Theories of Comparative Methodology?
If the above takes us up to the 2000s and 2010s, what about the future? For the sake of originality and a bit of fun let me venture a prediction for the near-future.
Along the horizon, I espy a new ideation slowly emerging that has the potential to be a trend in comparative methodology. This trend is theoretically tied to the importance of causation, yet also somewhat free in its methods. Milja Kurki calls this paradigm “critical realism.” Not to be confused with “critical theory,” which is Marxist in nature, this “critical realism” is an admission that causation is vitally important for social science and hence any variation of comparative methodology. But, that causation does not only have to be something that is pattern-like in nature or have to be observed—that is that there is a real ontological world independent of human observation and that we as social scientists ought not be afraid to explore it and seek causal relation within the unobserved.
Critical realism is also a poke in the eye of a century long flirtation with various forms of positivism that still pervade the study of politics and society, especially in America. In fact, just this question, “Is there a world independent of human observation?”, may not seem like much, but actually how one chooses to answer this question has a profound, if unacknowledged, effect on one’s choices of methodology. One of the core tenants of positivism, running backward from Milton, to Popper, to Dewey, to Comte, and to Hume is that we must assume in “scientific” studies that there is no such thing as a world that is independent of observation.
Suffer me this one quotation by Nietzsche:
In some remote corner of the universe, poured out and glittering in innumerable solar systems, there once was a star on which clever animals invented knowledge. That was the haughtiest and most mendacious minute of ‘world history’—yet only a minute. After nature had drawn a few breaths the star grew cold, and the clever animals had to die.
Indeed, the problem with any brand of positivism that insists only on observation is that it must assume a micro universe based only on human observation. The fun thing about a critical realist approach to comparative methodology is that it would place additional importance on methods that search out those dogs that don’t bark for us.
Interestingly, Kurki has begun the argument for a critical realist approaches in the subfield of international relations. But for comparative politics, I have only read a quick mention of it by Peter A. Hall in his “Beyond the Comparative Method” and a slightly more in-depth recognition of its importance by Philip Gorski with his “Social ‘Mechanism’ and Comparative-Historical Sociology: A Critical Realist Proposal.” I very much agree with Gorski that political-social phenomena on a very large scale like social revolutions or state formation
are better conceived in realist terms [whilst still keeping those] causal models of the social processes that produce certain outcomes
This means for me that I should be allowed to search out causation without the hindrance of also having to defend a general-covering law where none empirically exists.
However, no one within comparative politics, particularly in America, has used it empirically with their work in either journal or book form that I know of. (If you have stumbled upon it, please tell me because I am very curious how such an empirical study might be fleshed out.) I say that this may be the theoretical “new thing” just around the corner for comparative methodology because I am, admittedly somewhat smitten with this meta-theory at the moment.
For those who occasionally worry about the philosophy that underpins one’s methodology, like Mahoney (2011) above, a critical-realist approach offers one of the only “commonsensical” ways to re-ground our methods for causation in a real-world ontology. Below is recent lecture given by Murki that I find endlessly fascinating. Despite it having “international relations” in the title, I would argue that those concepts she refers to, particularly when she speaks about causation, are something that is easily transferable to comparative political and comparative methodology for us here.
What, then, is the comparative method and is it distinct from what others do in political science?
Peter Mair argues that he likes to say that the comparative method is distinct from other realms of political inquiry within political science because the comparative method is, in fact, more of a subfield than it is a one particular method. Thus the comparative method—or more specifically as I have argued here—comparative methodology is defined as much by its substance (“the study of foreign countries or a plurality of countries”) as it is by its various methods (some more quantitative, some more qualitative).
The comparative method of today, then, is a diverse affair of various approaches, methods, and questions. The key for me is to work between these different “paradigms”, “traditions”, and “methods” of comparative methodology so that scholars can test one another’s theories against each other’s empirical findings, whilst also being knowledgeable about the various methods popular within the subfield in order to facilitate easier communication amongst scholars. From Mahoney’s perspective, though he favors the comparative historical approach, what is important is that comparative methodology engages in both “iterated hypothesis testing” as much as it does “hypothesis elaboration.” I should like to espouse a similar approach as Mahoney on this question of the comparative method.
This essay has argued that the comparative method is better thought of as a tradition of methods that make up comparative methodology which as inclusive as it is serious about questions of causation in the real empirical world.
In one sense, comparative methodology is not so distinct from the rest of political science that also attempted to be more inclusive about different methods. In another sense though, because of our penchant for constructing research projects that utilize a level of analysis that promises a small-N, that have emphasized in the past culture or history as thematically more important than survey behavioralism, or that actively seek out questions which incorporate states and societies of more than one country, the comparative method is very much distinct from other areas of political science.
In the contemporary subfield of comparative politics in America, the comparative method really does mean a collection of methods under a more inclusive comparative methodology.
*This post was written and published by T. F. Rhoden; image credit for this goes to Hevesh5. Unless otherwise stated, all posts on this website are under Creative Commons licence.