Selected Published Books and Papers
BOOKS
J. Scott Armstrong (2007), Persuasive Advertising. An evidence-based approach for developing advertisements. Forthcoming from Palgrave Macmillan. Draft copy available upon request.
Principles of Forecasting: A Handbook for Researchers and Practitioners (2001, Kluwer Academic Publishers) - A book designed to summarize all knowledge about forecasting methods; the primary emphasis is on empirical findings, but reliance is also placed on expert opinion.
Long-Range Forecasting, 2nd. Ed. (1985, John Wiley & Sons) - The only comprehensive summary of research on forecasting prior to 1985, it covers judgmental and quantitative methods, and it has stood the test of time. The book is out of print but used copies and reviews are available at amazon.com.
PAPERS (in PDF format)
Applied Statistics
Raymond Hubbard and J. Scott Armstrong, "Why We Don't Really Know What 'Statistical Significance' Means: A Major Educational Failure", Journal of Marketing Education, 28 August (2006), 114-120.
J. Scott Armstrong (1970), "How to Avoid Exploratory Research," Journal of Advertising Research, 10 (4), 27-30 - Stepwise regressions led to false conclusions. Moral: use theory not statistics to select variables.
J. Scott Armstrong and Peer Soelberg, (1968), "On the Interpretation of Factor Analysis," Psychological Bulletin, 70 (5), 361-364 - Theory should precede, not follow the use of factor analysis.
J. Scott Armstrong. (1967), "Derivation of Theory by Means of Factor Analysis, or Tom Swift and His Electric Factor Analysis Machine," The American Statistician (December), 17-21 - Problems in the use of factor analysis for deriving theory are illustrated by means of an example in which the underlying factors are known.
Communications
J. Scott Armstrong (1978), "The Graffiti Solution," The Wharton Magazine, Winter, 12-14 - Want to stop graffiti? Pay people to do it but hire few people. And, you know, Philadelphia has done this.
Education
J. Scott Armstrong (2004), "Improving Learning at Universities: Who is Responsible?" University of Pennsylvania Almanac, Vol. 51, No. 15, December 14, 2004 - Contains evidence-based suggestions for making students responsible for their learning.
J. Scott Armstrong (1998), "Are Student Ratings of Instruction Useful?" American Psychologist, 53 (November), 1223-1224 - Teacher ratings have been shown to be unrelated to traditional learning measures. It seems likely that they are detrimental to the learning of skills. Direct measures of learning should be used, not those based on liking the teacher.
J. Scott Armstrong (1995), "The Devil's Advocate Responds to an MBA Student's Claim that Research Harms Learning," Journal of Marketing, 59 (July), 101-106 - Faculty who are engaged more heavily in research spend less time in teaching, yet they do not receive lower teacher evaluations. Their students seem to learn more. Also, graduates of schools that produce more research earn more in the five years after graduation.
J. Scott Armstrong and Tad Sperry (1994), "Business School Prestige - Research versus Teaching," Interfaces, 24 (2), 13-43 - The reputations of business schools were strongly related to research, even when one considers only the top schools. Teacher evaluations were not related to business school prestige.
J. Scott Armstrong (1994), "Would Mandatory Attendance be Effective for Economics Classes?" Internet publication only - Based on prior empirical studies, mandatory attendance would not help learning - a conclusion that goes beyond economics classes.
J. Scott Armstrong (1990), "The Case for Minimum Teaching Standards," Almanac (University of Pennsylvania), January 16, pp. 9-11 - A review of empirical evidence suggests that an emphasis on teaching will harm learning.
J. Scott Armstrong (1983), "Learner Responsibility in Management Education, or Ventures into Forbidden Research" with comments, Interfaces, 13, 26-38 - Learning is highly dependent on the learner taking responsibility. Grading leads to a loss of responsibility. This paper reports on evidence showing that the use of time-contracts (grades based only on learning time) led to higher student responsibility and increased learning.
J. Scott Armstrong (1980), "Teacher vs. Learner Responsibility in Management Education," Working Paper - Provides empirical evidence showing that learning depends on learner responsibility.
J. Scott Armstrong (1979), "The Natural Learning Project," Journal of Experimental Learning and Simulation, 1, 5-12 - Reports on five learning experiments in business school courses. Self-oriented skill training was more than three times as effective as traditional courses in producing useful behavioral changes.
J. Scott Armstrong (1977), "Designing and Using Experiential Exercises," in Experiential Learning in Marketing Education, M. D. DeLozier, D.M. Lewison, and R. Andress (eds.), Occasional Studies No. 11, March, Division of Research, College of Business Administration, University of South Carolina, pp. 8-17. Describes evidence showing how to design experiential learning.
Forecasting
J. Scott Armstrong, (2006) " How to Make Better Forecasts and Decisions: Avoid Face-to-face Meetings ," (with commentaries) in Foresight: The International Journal of Applied Forecasting , issue 5 (Fall), 3-15
J. Scott Armstrong and Alfred Cuzán "Index Methods for Forecasting: An Application to the American Presidential Elections," Foresight, Issue 3, February 2006.
J. Scott Armstrong (2006), "Findings from Evidence-Based Forecasting: Methods for Reducing Forecast Error," International Journal of Forecasting, 22, 583-598.
J. Scott Armstrong and Robert Fildes (2006), "Making Progress in Forecasting," forthcoming in the the International Journal of Forecasting, 22, 433-441.
Kesten Green and J. Scott Armstrong (2006), "Structured Analogies for Forecasting" - The use of the structured analogies procedure led to a 41% error reduction in forecasting the decisions in eight conflict situations.
J. Scott Armstrong and Kesten C. Green (2006), "Demand Forecasting: Evidence-based Methods," in Strategic Marketing Management: A Business Process Approach, Luiz Moutinho and Geoff Southern (eds.), forthcoming.
J. Scott Armstrong (2005), "The Forecasting Canon: Nine Generalizations to Improve Forecast Accuracy," Foresight: The International Journal of Applied Forecasting, 1 (1), June, 29-35.
Kesten C. Green & J. Scott Armstrong (2005), "The War in Iraq : Should we have expected better forecasts?" in Foresight: The International Journal of Applied Forecasting, 1 (October), 50-52
J. Scott Armstrong, Fred Collopy, and J. Thomas Yokum (2005), "Decomposition by Causal Forces: A Procedure for Forecasting Complex Time Series," International Journal of Forecasting, 21, 25-36. - For nine series in which conditions were met, decomposition of time series by causal forces reduced forecast error by more than half.
J. Scott Armstrong (2004), "Damped Seasonality Factors: Introduction," International Journal of Forecasting, 20, 2004, 523-527 - Introduction to a special section on the value of damping seasonality estimates when uncertainty is encountered.
J. Scott Armstrong (2003), "Review of Predicting Presidential Elections and Other Things," (Ray Fair, Stanford University Press; Stanford, CA 2002), International Journal of Forecasting, 19, 760-761 - Shows how to use econometric models to forecast such practical things as marital happiness, how attendance at classes relates to grades, and how fast you can run the marathon. Unfortunately, Fair's models do not help much with respect to policy changes, the area where econometric models are most advantageous.
J. Scott Armstrong (2002), "Assessing Game Theory, Role Playing, and Unaided Judgment," International Journal of Forecasting, 18 (3), 345-352 - Unaided judgment and game theorists were unable to provide useful forecasts of decisions made in conflicts. In contrast, simulated interactions (a type of role playing) were quite accurate.
Written as a response to: Kesten C. Green, "Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgment," International Journal of Forecasting, 18, 321-344
See also Green's reply to commentators, "Embroiled in a conflict: Who do you call?" International Journal of Forecasting, 28, 389-395 -
J. Scott Armstrong (2002), Review of Batchelor, Roy (2001), "How useful are the forecasts of intergovernmental agencies? The IMF and OECD versus the consensus," (Applied Econometrics, 33, 225-235), International Journal of Forecasting, 18, 482-483
J. Scott Armstrong (2001), Combining Forecasts, from Principles of Forecasting: A Handbook for Researchers and Practitioners, J. Scott Armstrong (ed.): Norwell , MA : Kluwer Academic Publishers, 2001. Reprinted permission of Kluwer/Springer.
J. Scott Armstrong (2001), "Should We Redesign Forecasting Competitions?" International Journal of Forecasting, 17, 542-545 - In the future, competitions should start with hypotheses as to which methods will be most effective under what conditions. In addition, they should allow for the use of domain knowledge.
Monica Adya, Fred Collopy, J. Scott Armstrong and Miles Kennedy (2001), "Automatic Identification of Time Series Features for Rule-Based Forecasting," International Journal of Forecasting, 17, 143-157 - Automatic procedures, which are less expensive and more reliable than judgmental procedures, produced rule-based forecasts with little loss in forecast accuracy.
J. Scott Armstrong and Fred Collopy (2001), "Identification of Asymmetric Prediction Intervals through Causal Forces," Journal of Forecasting, 20, 273-283 - When forecast errors are large, as is common in annual forecasting, errors are asymmetrical in percentage terms. Log transformations can correct for this asymmetry, although asymmetry in the logs occurs for "contrary" time series.
J. Scott Armstrong and Thomas Yokum (2001), "Potential Diffusion of Expert Systems in Forecasting: An Application of Diffusion Theory," Technological Forecasting and Social Change, 67, 95-105; Summary review by Monica Adya - Surveys of researchers, educators, practitioners, and decision makers revealed high perceived risk for the adoption of expert systems.
J. Scott Armstrong, Vicki G. Morwitz and V, Kumar (2000), "Sales Forecasts for Existing Consumer Products and Services: Do Purchase Intentions Contribute to Accuracy?" International Journal of Forecasting, 16 383-397 - Intentions data are useful even when one has historical time series. In the study of consumer durables, intentions led to a 1/3 reduction in forecast errors. This paper was named as one of four outstanding papers published in the International Journal of Forecasting for the period 2000-2001."
Monica Adya, J. Scott Armstrong, Fred Collopy, & Miles Kennedy (2000), "An Application of Rule-based Forecasting for a Situation Lacking Domain Knowledge," International Journal of Forecasting, 16, 477-484 - This paper was part of the M3-competition. A simplified version of RBF performed well without using domain knowledge.
Armstrong, J. Scott (1999), "Sales Forecasting," in The IEBM Encyclopedia of Marketing, Michael J. Baker (Ed.), London, International Thomson Business Press, 278-290.
J. Scott Armstrong (1999), "Forecasting for Environmental Decision-Making," in V.H. Dale and M.E. English, eds., Tools to Aid Environmental Decision Making, New York: Springer-Verlag, pp. 192-225
J. Scott Armstrong and R. J. Brodie (1999), "Forecasting for Marketing," in G. J. Hooley and M. K. Hussey (eds.), Quantitative Methods in Marketing, 2nd ed., London: International Thompson Business press, pp. 92-119 - This review of empirical evidence leads to recommendations on how to improve forecast accuracy in marketing.
J. Scott Armstrong (1998), "Commentaries on 'Generalizing about Univariate Forecasting Methods: Further Empirical Evidence,'" International Journal of Forecasting, 14, 359-366 - Comparative studies such as the M-competitions should fully describe the conditions that describe the data.
J. Scott Armstrong and Fred Collopy (1998), "Integration of Statistical Methods and Judgment for Time Series Forecasting: Principles from Empirical Research," in G. Wright and P. Goodwin (eds.), Forecasting with Judgment. John Wiley & Sons Ltd., 269-293, with review by N. R. Sanders, International Journal of Forecasting, 15 (1999), 345-346 - Presents all feasible ways to combine judgment and statistical forecasts and reviews empirical evidence on each.
J. Scott Armstrong and Robert Fildes (1995), "On the Selection of Error Measures for Comparisons Among Forecasting Methods," Journal of Forecasting, 14, 67-71 - This review of empirical evidence leads to the conclusion that mean square errors are highly unreliable and should not be used in forecasting.
J. Thomas Yokum and J. Scott Armstrong (1995), "Beyond Accuracy: Comparison of Criteria Used to Select Forecasting Methods," International Journal of Forecasting, 11, 591-597 - According to experts, accuracy is always important, yet other criteria, such as ease of use, are nearly as important. The importance of criteria varied depending on the type of forecasting problem.
J. Scott Armstrong and Fred Collopy (1994), "How Serious are Methodological Issues in Surveys: A Reexamination of the Clarence Thomas Polls," Internet publication. - Decisions can be improved if criteria are established before making a decision. Once a decision was made, decision makers were reluctant to change even if their decision violates their own criteria.
Fred Collopy, Monica Adya & J. Scott Armstrong (1994), "Principles for Examining Predictive Validity: The Case of Information Systems Spending Forecasts," Information Systems Research, 5, 170-179 - Demonstrates the use of principles for evaluating forecasting methods. This paper shows how a previously published paper falsely concluded that a diffusion model was useful in forecasting spending on information systems. In fact, a single linear trend extrapolation produced more accurate forecasts than the diffusion model.
J. Scott Armstrong (1994), "The Fertile Field of Meta-Analysis: Cumulative Progress in Agricultural Forecasting," International Journal of Forecasting, 10, 147-149
MacGregor, D. & J. Scott Armstrong (1994), "Judgmental Decomposition: When Does It Work?" International Journal of Forecasting, 10, 495-506 - Decomposition is useful when dealing with large numbers (or very small numbers) and when one knows more about the parts than the whole. Otherwise, it leads to increased errors.
J. Scott Armstrong and Fred Collopy (1993), "Causal Forces: Structuring Knowledge for Time-series Extrapolation," Journal of Forecasting, 12, 103-115 - Domain knowledge, expressed as expectations about trends in time series, led to improved accuracy in time-series forecasts.
J. Scott Armstrong and Fred Collopy (1992), "Error Measures for Generalizing about Forecasting Methods: Empirical Comparisons," International Journal of Forecasting, 8, 69-80 - Proposes the Relative Absolute Error (RAE) and compares it with other error measures for comparisons among time-series methods. It is reliable and easy to interpret. A commentary on this paper was followed by our reply:
Fred Collopy and J. Scott Armstrong (1992), "Expert Opinions about Extrapolation and the Mystery of the Overlooked Discontinuities," International Journal of Forecasting, 8, 575-582 - Traditional time-series extrapolative methods do not perform well when there are discontinuities in the data, according to this survey of forecasting experts.
Fred Collopy and J. Scott Armstrong (1992), "Rule-Based Forecasting: Development and Validation of an Expert Systems Approach to Combining Time Series Extrapolations," Management Science, 38 (10), 1394-1414- Uses prior knowledge about forecasting methods and domain knowledge to formulate rules for time series forecasting.
J. Scott Armstrong (1991), "Prediction of Consumer Behavior by Experts and Novices," Journal of Consumer Research, 18 (September), 251-256 - Academics who are familiar with research on consumer behavior were no more accurate than practitioners or high school students when making forecasts about the outcomes of studies on consumer behavior.
Fred Collopy and J. Scott Armstrong (1989), "Toward Computer-Aided Forecasting Systems: Gathering, Coding, and Validating the Knowledge," in George R. Widmeyer (ed.), DSS-899 Transactions: Ninth International Conference on Decision Support Systems, Institute of Management Science, pp. 103-119 - Describes how to develop rules for forecasting.
Stephen Dakin and J. Scott Armstrong (1989), "Predicting Job Performance: A Comparison of Expert Opinion and Research Findings," International Journal of Forecasting, 5, 187-194 - Despite a wealth of useful research on personnel selection, practitioners rely on invalid procedures.
J. Scott Armstrong (1989), "Combining Forecasts: The End of the Beginning or the Beginning of the End?" International Journal of Forecasting, 5, 585-588
J. Scott Armstrong (1988), "Review of Ravi Batra, The Great Depression of 1990," International Journal of Forecasting, 4, 493-502 - Identifies faulty forecasting procedures in a popular book.
J. Scott Armstrong (1988), "Communication of Research in Forecasting: The Journal," International Journal of Forecasting, 4, 321-324 - Reports on a survey of 201 readers of two forecasting journals. A substantial amount of findings were used by practitioners and researchers.
J. Scott Armstrong (1987), "Forecasting Methods for Conflict Situations," in G. Wright and P. Ayton (eds.), Judgmental Forecasting, pp. 157-176 - For a more recent report, see the-Role Playing chapter in Principles of Forecasting.
J. Scott Armstrong, R. Brodie & S. McIntyre (1987), "Forecasting-Methods for Marketing: Review of Empirical Research," International Journal of-Forecasting, 3, 335-376
J. Scott Armstrong (1986), "Research on Forecasting: A Quarter-Century Review, 1960-1984," Interfaces,-16, 1 (Jan-Feb), 89-109 - This review shows that substantial progress has been made since 1960. That said, much effort is still-being devoted to areas that have little promise, while some highly promising areas are ignored.
J. Scott Armstrong (1986), "Review of Steven J. Rosenstone, Forecasting Presidential Elections,"-International Journal of Forecasting, 2, 248-249. - This book used causal models to forecast U.S.-presidential votes by state, then aggregates across states. It tested well in three elections.
J. Scott Armstrong (1984), "Forecasting by Extrapolation: Conclusions from 25 Years of Research," Interfaces,-14 (Nov.-Dec.), 52-66, with commentaries and reply. Relatively simple extrapolation methods were found to be more accurate than sophisticated methods.
J. Scott Armstrong (1984), "Do Judgmental Researchers Use Their Own Research? A Review of Judgment Under Uncertainty: Heuristics and Biases," Journal of Forecasting, 3 (1984), 235 239.
J. Scott Armstrong (1983), "Relative Accuracy of Judgmental and Extrapolative Methods in-Forecasting Annual Earnings," Journal of Forecasting, 2, 437-447 - Management forecasts of firms' annual earnings were more accurate than those by professional independent analysts, and these were more accurate than statistical extrapolations.
J. Scott Armstrong & Ed Lusk (1983), "Commentary on the Makridakis Time Series Competition-(M-Competition)," with comments by Gardner, Geurts, Lopes, Markland, McLaughlin, Newbold, Pack, and replies by Andersen, Carbone, Fildes, Parzen, Newton, Winkler, & Makridakis, Journal of Forecasting 2, 259-311. - Provides an introduction to the commentaries on the M-competition.
Robert Carbone and J. Scott Armstrong (1982), "Evaluation of Extrapolative Forecasting Methods: Results of a Survey of Academicians and Practitioners," Journal of Forecasting 1, 215-217 - Reports on a survey of academics and practitioners dealing with beliefs about forecasting methods. For example, they favored the RMSE for evaluation.
J. Scott Armstrong (1981), "How Expert Are the Experts?" Inc., December, 15-16 - A simpler version of the paper from the Technology Review (1980).
J. Scott Armstrong (1980), "The Seer-Sucker Theory: The Value of Experts in Forecasting," Technology Review, June/July, 16-24 - Expertise above a minimal level does little to improve the accuracy for forecasts of changes over time.
J. Scott Armstrong (1978a), "Forecasting with Econometric Methods," Journal of Business, 51 (4), 549-564 - This study, using a survey of the world's leading econometricians, found conflicts between the econometricians' procedures and results from empirical studies. The paper was published along with commentaries by econometricians. This was followed by the next paper.
J. Scott Armstrong (1978b), "Econometric Forecasting and the Science Court," Journal of Business, 51 (4), 595-600
J. Scott Armstrong (1975), "Tom Swift and His Electric Regression Analysis Machine: 1973," Psychological Reports, 36, 806 - Illustrates problems with non-theoretical statistical analyses by reporting on a published study that used stepwise regression with 115 variables and 19 observations.
Armstrong, J. S., W. Denniston, Jr. & M. Gordon (1975), "The Use of the Decomposition Principle in Making Judgments," Organizational Behavior and Human Performance , 13, 257-263. Judgmental decomposition improved estimation for difficult problems.
J. Scott Armstrong and Alan C. Shapiro (1974), "Analyzing Quantitative Models," Journal of Marketing, 38, 61-66 - We present a framework for analyzing models, and illustrate its use in analyzing a popular marketing model.
J. Scott Armstrong & M. Grohman (1972), "A Comparative Study of Methods for Long-Range Market Forecasting," Management Science, 19, 211-221 - Econometric methods were more accurate than extrapolation methods.
J. Scott Armstrong (1971), "Exploratory Analysis of Marketing Data: A Reply" [to Douglass C. Crocker, Journal of Marketing Research, 8 (1971) 509-510], Journal of Marketing Research, 8, 511-513
J. Scott Armstrong and T. Overton (1971), "Brief vs. Comprehensive Descriptions in Measuring Intentions to Purchase," Journal of Marketing Research, 8 (February), 114-117 - Brief descriptions of new products produced similar estimates for intention to purchase as elaborate and expensive descriptions.
J. Scott Armstrong (1970), "An Application of Econometric Models to International Marketing," Journal of Marketing Research, 7, 190-198 - Econometric methods produced substantially more accurate long-range forecasts of the photographic markets in 17 countries than did extrapolations.
J. Scott Armstrong and James G. Andress (1970), "Exploratory Analysis of Marketing Data: Trees vs. Regression," Journal of Marketing Research, 7, 487-492 - Tree analysis (using Automatic Interaction Detector) was more accurate than regression in forecasting the sales at 2,717 gas stations. With "A Reply to Cocker," Journal of Marketing Research, 8 (1971), 511-513.
J. Scott Armstrong and John U. Farley (1969), "A Note on the Use of Markov Chains in Forecasting Store Choice," Management Science, 16 (4) - Markov chains proved to be of little value in forecasting customers' choice of a food store.
Implementing Change
J. Scott Armstrong (1983), "Review of Intentional Changes: A Fresh Approach to Helping People Change, by Allen Tough," Academy of Management Review 8, 509-511 - Groups, especially those with an appointed leader, inhibit learning.
J. Scott Armstrong (1982), "Strategies for Implementing Change: An Experiential Approach," Group and Organization Studies, 7 (4), 457-475 - Describes the "Delta" approach to implementing change in organizations. The key is to gain agreement on the design of a small experiment, rather than to try to sell the change itself.
Marketing
J. Scott Armstrong and Kesten C. Green (2005), "Demand Forecasting - Evidence-Based Methods," Working paper.
J. Scott Armstrong (2003), "How to be Less Persuaded or More Persuasive - Review of Age of Propaganda: The Everyday Use and Abuse of Persuasion," by Anthony R. Prakanis and Elliot Aronson, Journal of Marketing, 67 (1), 129-130 - This review illustrates the use of some common but non-obvious attempts to persuade.
J. Scott Armstrong (2003), "Discovery and Communication of Important Marketing Findings: Evidence and Proposals," Journal of Business Research, 56, 69-84, with commentaries by Rossiter and Lehmann, 85-90. Response to commentary published as "The Value of Surprising Findings for Research on Marketing," Journal of Business Research, 2003, 91-92 - A review of empirical evidence shows that current procedures for journal publication inhibit scientific progress. This paper suggests ways to improve the process.
Raymond Hubbard and J. Scott Armstrong (1997), "Publication Bias Against Null Results," Psychological Reports, 80, 337-338 - In some fields, such as psychology, well over 90% of papers reject the null hypotheses. This is unfortunate because many papers with null results provide useful findings.
J. Scott Armstrong (1996), "How Should Firms Select Advertising Agencies? A Review of Randall Rothenberg's Where the Suckers Moon," Journal of Marketing, 60, 131-134 - This paper describes the process used to select an advertising agency for Subaru America. It then describes a structured approach to improve the selection of an agency.
J. Scott Armstrong (1994), "Review of Scott Plous, The Psychology of Judgment and Decision Making," Journal of Marketing, 58 (July)
J. Scott Armstrong and Roderick J. Brodie (1994), "Effects of Portfolio Planning Methods on Decision Making: Experimental Results," International Journal of Research in Marketing, 11, 73-84 - Portfolio planning methods, such as the BCG matrix, harm decision making.
J. Scott Armstrong and Roderick J. Brodie (1994), "Portfolio Planning Methods: Faulty Approach or Faulty Research? A Rejoinder to Making Better Decisions by Wensley," International Journal of Research in Marketing, 11, 91-93
J. Scott Armstrong and Fred Collopy (1994), "The Profitability of Winning," Chief Executive, June, 1994, 61-63 - A discussion of research on the myth of market share. This was aimed at executives.
J. Scott Armstrong and J. Thomas Yokum (1994), "Effectiveness of Monetary Incentives: Mail Surveys to Members of Multinational Professional Groups," Industrial Marketing Management, 23, 133-136 - This review of experimental studies shows that small monetary incentives always improve response rates, even for the situation we examined, professional groups, where their use would seem odd.
J. Scott Armstrong, Nicole Coviello, and Barbara Safranek (1993), "Escalation Bias: Does It Extend to Marketing?" Journal of the Academy of Marketing Science, 21 (2), 247-253 - Escalation bias did not harm decision-making in marketing. Furthermore, prior studies on escalation bias did not demonstrate that it harmed decision making.
J. Scott Armstrong and Randall L. Schultz (1993), "Principles Involving Marketing Policies: An Empirical Assessment," Marketing Letters, 4:3, 253-265 - We examined nine marketing textbooks, published since 1927, to see if they contained useful marketing principles. Four doctoral students found 566 normative statements about pricing, product, place, or promotion in these texts. None of these statements were supported by empirical evidence.
Raymond Hubbard and J. Scott Armstrong (1992), "Are Null Results Becoming an Endangered Species in Marketing?" Marketing Letters, 3:2, 127-136 - Of 692 papers using statistical significance between 1974 and 1989 in the three major marketing journals, fewer than 8% led to a rejections of the null hypothesis. This percentage dropped by 1/2 from the 1970s to the 1980s. This is an unfortunate situation.
J. Scott Armstrong (1990), "Class of Mail Does Affect Response Rates to Mailed Questionnaires: Evidence from Meta-Analysis," Journal of the Market Research Society, 32 (3), 469-472, with a reply by Lee Harvey - U.S. business reply postage should not be used in survey research.
J. Scott Armstrong and Philip D. Hutcherson (1989), "Predicting the Outcome of Marketing Negotiations: Role-Playing Versus Unaided Opinions," International Journal of Research in Marketing, 6, 227-239 - Role playing was much more accurate than unaided judgment in predicting decisions made in negotiations.
J. Scott Armstrong and Edward J. Lusk (1987), "Return Postage in Mail Surveys: A Meta-Analysis," Public Opinion Quarterly, 51 (2), 233-248. Real stamps improve the return rate. Surveys using business reply postage are less effective.
J. Scott Armstrong (1978), "The Manager's Dilemma: Role Conflict in Marketing," in Future Directions for Marketing, G. Fisk, J. Arndt, and G. Gronhaug (eds.), Cambridge, MA: Marketing Science Institute, pp. 78-89.S
J. Scott Armstrong (1978), "Review of Don A. Dillman, Mail and Telephone Surveys," Journal of Business, 54 (4), 622-625. Review of the "Bible" for those involved in survey research. This book was revised by Dillman in 2000 under the title Mail and Internet Surveys.
J. Scott Armstrong and Terry S. Overton (1977), "Estimating Nonresponse Bias in Mail Surveys," Journal of Marketing Research 14, 396-402. Bias occurs on questions that are related to one's interest in a topic. One way to estimate nonresponse bias is to look at trends over waves.
J. Scott Armstrong (1975), "Monetary Incentives in Mail Surveys," Public Opinion Quarterly, 39, 111-116.
This meta-analysis showed that small monetary incentives produced substantially larger response rates than did promised incentives.
Robert W. Nason and J. Scott Armstrong (1972), "Role Conflict: Society's Dilemma with Excellence in Marketing," Wharton Quarterly, 7 (1), 13-16. David B.
David B. Montgomery and J. Scott Armstrong (1970), "Brand Trial after a Credibility Change," Journal of Advertising Research, 10 (5), 26-32. This study analyzed consumer responses to Crest toothpaste after the tooth decay prevention class was supported by the American Dental Association.
Scientific Methods and Peer Review
J. Scott Armstrong (2004), "Does an Academic Research Paper Contain Useful Knowledge? No (p.05)" (Commentary on Mort, et al., "Perceptions of Marketing Journals by Senior Academics in Australia and New Zealand," pp. 51-61), Australian Marketing Journal, 12 (2), 62-63. - Assume that you were handed a paper randomly selected from an academic journal in your field. What is the likelihood that it contains useful knowledge? Our estimates suggest that it would be less than 3%.
J. Scott Armstrong and Ruth A. Pagell (2003), "Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project," Interfaces, 33 (6), 89-111 - Provides data showing that invited papers are twenty times more likely to be important than are those accepted through the traditional peer review system. This paper was followed by commentary and by this reply,
J. Scott Armstrong (2003), "Incentives for Developing and Communicating Principles: A Reply " Interfaces 33 (8), 2003, 109-111.
Also, see a review of this paper by Raymond Hubbard in International Journal of Forecasting, 20 (2004), 740-741.
J. Scott Armstrong, Robert J. Brodie, and Andrew G. Parsons (2001), "Hypotheses in Marketing Science: Literature Review and Publication Audit," Marketing Letters, 12 (2),171-187. Our survey showed that while experts believe that scientific progress is aided by the method of multiple hypotheses, this method is not widely used. Nor, as assessed by our audit of the primary marketing journals, do many papers assess conditions.
J. Scott Armstrong (1998), "Management Science: What Does It Have to Do with Management or Science?" Marketing Bulletin, 9 (May), 1-15. Management scientists typically violate accepted scientific procedures.
J. Scott Armstrong (1998), "Democracy Does Not Make Good Science: On Reforming Review Procedures for Management Science Journals, Interfaces, 28 (4), 88-91 - A Reply to Hugh J. Miser (1998), "Reaction to Armstrong's 'Management Folklore and Management Science,'" Interfaces, 28 (4) 81-93. Reviewers should not be asked to recommend whether or not a paper should be published.
J. Scott Armstrong (1997), "Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation," Science and Engineering Ethics, 3, 63-84. A review of research from 68 empirical studies on peer review, of which 12 were experimental studies. Journal reviewing procedures are ineffective at selecting the most useful papers, they are biased against new and important findings, and they miss many errors.
J. Scott Armstrong (1996), "Management Folklore and Management Science - On Portfolio Planning, Escalation Bias, and Such," Interfaces, 26, No. 4, 28-42. Studies that challenge current management practices are more likely to be rejected for journal publication and less likely to be cited.
J. Scott Armstrong (1995), "Publication of Research on Controversial Topics: The Early Acceptance Procedure," International Journal of Forecasting, 11, Notes: 1-4.
J. Scott Armstrong (1995), "Quality Control Versus Innovation in Research on Marketing," Journal of Marketing Management, 11, 655-660. An emphasis on quality control leads to the rejection of useful papers.
Raymond Hubbard and J. Scott Armstrong (1994), "Replication and Extensions in Marketing - Rarely Published But Quite Contrary," International Journal of Research in Marketing, 11, 233-248. Replications and extensions are rare in marketing as they make up only one percent of the space. In addition, 60% of the replications conflict in some way with the original findings.
J. Scott Armstrong (1992), "Editorial Policies for the Publication of Controversial Findings," International Journal of Forecasting, 8, 543-544. Concluded, based on prior empirical research, that reviewers should not be asked to make judgments on whether a paper should be accepted for publication.
J. Scott Armstrong (1992), Review of T. Hancock, J. Lane, R. Ray, and D. Glennon 1992, "The Ombudsman: Factors Influencing Academic Research Productivity: A Survey of Management Scientists," Interfaces, 22 (5), 26-38. Introduces a study and commentary on the productivity of academics in management science. Prolific publisher spent 1/3 less time on teaching activities - with no apparent effect on teacher ratings.
J. Scott Armstrong and Raymond Hubbard (1991), "Does the Need for Agreement among Reviewers Inhibit the Publication of Controversial Findings?" Behavioral and Brain Sciences, 14, 136-137. Improvements in the reliability of peer review would pose additional barriers for research that produces important new findings.
J. Scott Armstrong (1989), "Readability and Prestige in Scientific Journals," Journal of Information Science, 15, 123-124. A follow-up to Armstrong 1980 paper on "Unintelligible Management Research"
J. Scott Armstrong, Estella Bee Dagum, Robert Fildes, and Spyros Makridakis (1986), "Publishing Standards for Research in Forecasting (An Editorial)," International Journal of Forecasting, 2, 133-137.
J. Scott Armstrong (1983), "The Importance of Objectivity and Falsification in Management Science," Journal of Management, 9, 213-216. A follow-up to Armstrong 1980 paper on "Advocacy as a Scientific Strategy."
J. Scott Armstrong (1983), "Cheating in Management Science," with comments by M. K. Starr and M. J. Mahoney, Interfaces 13 (4), 20-29. Cheating seems to be rare in management science. When it does occur, it is discussed widely, but informally.
J. Scott Armstrong (1982), "Barriers to Scientific Contributions: The Author's Formula," Behavioral and Brain Sciences, 5 (June), 197-199. This paper draws upon empirical studies to produce guidelines on how to publish a paper in an academic journal (e.g., do not study an important problem and do not obtain surprising results).
J. Scott Armstrong (1982), "Is Review by Peers as Fair as It Appears?" Interfaces, 12, 62-74, with comments by Martin Starr, Herbert F. Ayres, Stephen M. Robinson, and Seth Bonder. I conducted a survey of management scientists to see if they could predict the outcome of a study that resubmitted previously published papers to the journals that had published them (but with new titles and authors). My sample was unable to predict the outcome. This was the inaugural paper for the "Ombudsman" column. I stand by all my proposals for paper review except for blind receiving. Research since this paper has shown that blind reviewing (shielding the reviewer) is detrimental.
J. Scott Armstrong (1982), "Research on Scientific Journals: Implications for Editors and Authors," Journal of Forecasting, 1, 83-104. For a more recent review of the empirical evidence on peer review, see the Armstrong 1997 paper from Science and Engineering Ethics.
J. Scott Armstrong (1980), "Advocacy as a Scientific Strategy: The Mitroff Myth," Academy of Management Review, 5, 509-511. Famous scientists often rely on advocacy to advance their theories.
J. Scott Armstrong (1980), "Unintelligible Management Research and Academic Prestige," Interfaces, 10, 2 (April), 80-86. According to a study by U.G. Gupta in Interfaces (1997), 27, No. 2, 85-101), this is the second most frequently cited paper in Interfaces from 1970-1992. Academics who have nothing to say can add to their prestige by writing in a less intelligible manner.
J. Scott Armstrong (1979), "Advocacy and Objectivity in Science," Management Science, 25 (5), 423-428. An audit of papers published in Management Science shows that most papers rely heavily on advocacy. The method of multiple hypotheses was rarely used.
J. Scott Armstrong (1974), "Eclectic Research and Construct Evaluation," in Models of Buyer Behavior: Conceptual, Quantitative, and Empirical, Jagdish N. Sheth (ed.), New York: Harper and Row, pp. 3-14. Argues for splitting a research budget into small projects with much different approaches to the same problem.
Social Responsibility in Management
J. Scott Armstrong, "The Panalba Role Playing Case," adapted from an AMA proceedings paper, updated Nov. 2001. A learning exercise
J. Scott Armstrong (1990), "Review of Paul Blumberg, The Predatory Society: Deception in the American Marketplace" (New York: Oxford University Press, 1989), Journal of Marketing (October), 127-128. The quasi-contract is written as part of the product guarantee. in the event of any harm due to the use of the product, people are then given an option to collect on the policy - or to sue. Guess what's happening?
J. Scott Armstrong (1988), "Review of Peter W. Huber, Liability: The Legal Revolution and Its Consequences." New York: Basic Books, Inc. The quasi-contract was proposed as a way to reduce product liability costs.
J. Scott Armstrong (1980), "Case of the Detrimental Drug: Implications for the Stakeholder Theory of Directorship," Directors and Boards, 4 (4), 22-39.
J. Scott Armstrong (1977), "Social Irresponsibility in Management," Journal of Business Research, 5 (September), 185-213. Subjects who role-played as members of Upjohn's Board of Directors were unwilling to remove a detrimental drug. However, when the boards were composed of stakeholders and when social accounting was used, they did remove the drug.
J. Scott Armstrong (1975), "Review of Stanley Milgram, Obedience To Authority (New York: Harper & Row, Publishers, 1974)," Journal of Marketing 39 (July), 125.
Strategy and Planning
Richard H. Franke, J. Scott Armstrong, and Paul M. Vaclavik (1998), "Competition vs. Profitability," presented at the Strategic Management Society 18th Annual International Conference - Alternative financial criteria and extended times found additional support showing that a competitor-orientation harmed profits.
J. Scott Armstrong (1997), "Why Can't a Game Be More Like a Business? A Review of Co-opetition by Brandenburger and Nalebuff," Journal of Marketing, 61 (April), 92-95. This book offers interesting and creative ideas about the use of game theory in business applications. Unfortunately, it contains no evidence.
J. Scott Armstrong and Fred Collopy (1996), "Competitor Orientation: Effects of Objectives and Information on Managerial Decisions and Profitability," Journal of Marketing Research, 33, 188-199. When market share is an objective, profits suffer, according to a series of laboratory studies and a long-term field study.
J. Scott Armstrong (1996), "Market Share Superstition" (Letter), Sloan Management Review, Fall, 38, 4-5. Discusses another field study concluding that market share should not be used as an objective.
J. Scott Armstrong (1991), "Strategic Planning Improves Manufacturing Performance," Long Range Planning, 24 (4), 127-129. Formal planning was superior in 20 studies and harmful in only 3 (with 5 ties). For manufacturing firms, the score was 9 to 0.
J. Scott Armstrong (1988), "Review of Noel Capon, John G. Farley, and James M. Hulbert, Corporate Strategic Planning (New York: Columbia University Press)," Journal of Marketing, 54 (1990). In practice, formal planning was found to be useful.
J. Scott Armstrong (1989), "On the Effectiveness of Marketing Planning," New Zealand Journal of Business, 11 (October), 11-12.
J. Scott Armstrong (1988), "Review of Alfie Kohn, No Contest: The Case Against Competition," Journal of Marketing, July. This book presents an impressive array of evidence to suggest that a competitor orientation harms performance under most conditions.
J. Scott Armstrong (1986), "The Value of Formal Planning for Strategic Decisions: Reply," Strategic Management Journal, 7, 183-185. This paper presents evidence to refute a challenge to my conclusion that formal planning improves the performance of firms. In effect, researcher bias does not explain the results.
J. Scott Armstrong and David J. Reibstein (1985), "Evidence on the Value of Strategic Planning in Marketing: How Much Planning Should a Marketing Planner Plan?", from Strategic Marketing and Management, (H. Thomas and D. Gardner, eds.), John Wiley & Sons, Ltd. The intent here was to do a meta-analysis on the value of strategic planning in marketing. No evidence was found. This paper describes what evidence is needed.
J. Scott Armstrong (1983), "Strategic Planning and Forecasting Fundamentals," in Kenneth Albert(ed.), The Strategic Management Handbook, New York: McGraw Hill. This paper provides a step-by-step planning procedure, with evidence on why it works.
J. Scott Armstrong (1982), "The Value of Formal Planning for Strategic Decisions: Review of Empirical Research," Strategic Management Journal, 3, 197-211. In this meta-analysis, formal planning was superior in 10 of 15 comparisons from 12 studies. Informal planning was more successful in only 2 studies.