User Perceptions of Decision Support Effectiveness: Two Production Planning Experiments *
Contributor(s)School of Business Administration, University of Michigan, Ann Arbor, MI 48109
Carlson School of Management, University of Minnesota, Minneapolis, MN 55455
Full recordShow full item record
AbstractDecision support systems continue to be very popular in business, despite mixed research evidence as to their effectiveness. We hypothesize that what-if analysis, a prominent feature of most decision support systems, creates an “illusion of control” causing users to overestimate its effectiveness. Two experiments involving a production planning task are reported which examine decision makers' perceptions of the effectiveness of what-if analysis relative to the alternatives of unaided decision making, and quantitative decision rules. Experiment 1 found that almost all subjects believed what-if analysis was superior to unaided decision making, although using what-if analysis had no significant effect on performance. Experiment 2 found that decision makers were indifferent between what-if analysis and a quantitative decision rule which, if used, would have led to significant cost savings. Thus, what-if analysis did create an illusion of control: decision makers perceived performance differences where none existed, and did not detect large differences when they were present. In both experiments, decision makers exhibited difficulty realizing that their positive beliefs about what-if analysis were exaggerated. Such misjudgments could lead people to continue using what-if analysis even when it is not beneficial and to avoid potentially superior decision support technologies.
Davis, Fred D.; Kottemann, Jeffrey E. (1994). "User Perceptions of Decision Support Effectiveness: Two Production Planning Experiments * ." Decision Sciences 25(1): 57-76. <http://hdl.handle.net/2027.42/73794>
Ahituv, N. Assessing the value of information: Problems and approaches. Proceedings of ICIS-89, Boston, December 1989, 315 – 325.
Alavi, M., & Henderson, J. C. An evolutionary strategy for implementing a decision support system. Management Science, 1981, 27, 1 – 9.
Aldag, R. J., & Power, D. J. An empirical assessment of computer-assisted decision analysis. Decision Sciences, 1986, 17, 572 – 588.
Alter, S. L. Decision support systems: Current practice and continuing challenge. Reading, MA: Addison-Wesley, 1980.
Arkes, H. R., Dawes, R. M., & Christensen, C. Factors influencing the use of a decision rule in a probabilistic task. Organizational Behavior and Human Decision Processes, 1986, 37, 93 – 110.
Benbasat, I., & Dexter, A. S. Individual differences in the use of decision support aids. Journal of Accounting Research, 1982, 20, 1 – 11.
Benbasat, I., & Nault, B. R. An evaluation of empirical research in managerial support systems. Decision Support Systems, 1990, 6, 203 – 226.
Benbasat, I., & Schroeder, R. G. An experimental investigation of some MIS design variables. MIS Quarterly, 1977, 1, 37 – 50.
Bowman, E. J. Consistency and optimality in managerial decision making. Management Science, 1963, 9, 310 – 322.
Chakravarti, D., Mitchell, A. A. & Staelin, R. Judgment-based marketing decision models: An experimental investigation of the decision calculus approach. Management Science, 1979, 25, 251 – 262.
Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 1989, 13, 319 – 340.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. User acceptance of computer technology: A comparison of two theoretical models. Management Science, 1989, 35, 982 – 1003.
Dawes, R. M. The robust beauty of improper linear models in decision making. American Psychologist, 1979, 34, 571 – 582.
Dawes, R. M., & Corrigan, B. Linear models in decision making. Psychological Bulletin, 1974, 81, 95 – 106.
Dickson, G. W., Senn, J. A., & Chervany, N. L. Research in management information systems: The Minnesota experiments. Management Science, 1977, 23, 913 – 923.
Dos Santos, B. L., & Bariff, M. L. A study of user interface aids for model-oriented decision support systems. Management Science, 1988, 34, 461 – 468.
Einhorn, H. J. Learning from experience and suboptimal rules in decision making. In T.S. Wallsten ( Ed. ), Cognitive processes in choice and decision behavior. Hillsdale, NJ: Erlbaum, 1980, 66 – 80.
Einhorn, H. J., & Hogarth, R. M. Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 1975, 13, 171 – 192.
Einhorn, H. J., & Hogarth, R. M. Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 1978, 85, 395 – 416.
Einhorn, H. J., & Hogarth, R. M. Behavioral decision theory: Processes of judgment and choice. Annual Review of Psychology, 1981, 32, 53 – 88.
Fripp, J. How effective are models ? International Journal of Management Science (Omega), 1985, 13, 19 – 28.
Fudge, W. K., & Lodish, L. M. Evaluation of the effectiveness of a salesman's planning system by field experimentation. Interfaces, 1977, 8, 97 – 106.
Goslar, M., Green, G., & Hughes, T. Decision support systems: An empirical assessment for decision making. Decision Sciences, 1986, 17, 79 – 91.
Hammond, K. R., Hamm, R. M., Grassia, J., & Pearson, T. Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. IEEE Transactions on Systems, Man & Cybernetics, 1987, SMC-17, 753 – 770.
Hogarth, R. M. Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics. Psychological Bulletin, 1981, 90, 197 – 217.
Hogarth, R. M., & Makridakis, S. The value of decision making in a complex environment: An experimental approach. Management Science, 1981a, 27, 93 – 107.
Hogarth, R. M., & Makridakis, S. Forecasting and planning: An evaluation. Management Science, 1981b, 27, 115 – 138.
Holt, C. C., Modigliani, F., & Muth, J. F. Derivation of a linear decision rule for production and employment. Management Science, 1956, 2, 159 – 177.
Holt, C. C., Modigliani, F., Muth, J. F., & Simon, H. A. Planning production, inventories, and work force. Englewood Cliffs, NJ: Prentice-Hall, 1960.
Johnson, E. J., & Payne, J. W. Effort and accuracy in choice. Management Science, 1985, 31, 395 – 414.
Keen, P. G. W., & Scott Morton, M. S. Decision support systems: An organizational perspective. Reading, MA: Addison-Wesley, 1978.
Kleinmuntz, B. Why we still use our heads instead of formulas: Toward an integrative approach. Psychological Bulletin, 1990, 107, 296 – 310.
Kleinmuntz, D. N. Cognitive heuristics and feedback in a dynamic decision environment. Management Science, 1985, 31, 680 – 702.
Kleinmuntz, D. N., & Thomas, J. B. The value of action and inference in dynamic decision making. Organizational Behavior and Human Decision Processes, 1987, 39, 341 – 364.
Kottemann, J. E., & Remus, W. E. Evidence and principles of functional and dysfunctional DSS. International Journal of Management Science (Omega), 1987, 15, 135 – 144.
Kottemann, J. E., & Remus, W. E. The performance effects of what-if analysis in a production planning task. Proceedings of conference on environments for supporting decision processes, Budapest, 1990.
Langer, E. J. The illusion of control. Journal of Personality and Social Psychology, 1975, 32, 311 – 328.
Langer, E. J. The psychology of control. Beverly Hills, CA: Sage, 1983.
Langer, E. J., & Roth, J. Heads I win, tails it's chance: The illusion of control as a function of the sequence of outcomes in a purely chance task. Journal of Personality and Social Psychology, 1975, 32, 951 – 955.
McIntyre, S. An experimental study of the impact of judgment-based marketing models. Management Science, 1982, 28, 17 – 23.
Meehl, P. E. Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis: University of Minnesota Press, 1954.
Paese, P. W., & Sniezek, J. A. Influences on the appropriateness of confidence in judgment: Practice, effort, information and decision-making. Organizational Behavior and Human Decision Processes, 1991, 48, 100 – 130.
Pentland, B. T. Use and productivity in personal computing: An empirical test. Proceedings of ICIS-89, Boston, December 1989, 211 – 222.
Powell, J. L. An attempt at increasing decision rule use in a judgment task. Organizational Behavior and Human Decision Processes, 1991, 48, 89 – 99.
Remus, W. E. Graduate students as surrogates for managers in experiments on business decision making. Journal of Business Research, 1987, 14, 19 – 25.
Sharda, R., Barr, S. H., & McDonnell, J. C. Decision support system effectiveness: A review and an empirical test. Management Science, 1988, 34, 139 – 159.
Sprague, R. H., & Carlson, E. D. Building effective decision support systems. Englewood Cliffs, NJ: Prentice-Hall, 1982.
Turban, E. Decision support and expert systems. New York: Macmillan, 1988.
Wilson, T. D., & Schooler, J. W. Thinking too much: Introspection can reduce the quality of preferences and decisions. Journal of Personality and Social Psychology, 1991, 60, 181 – 192.
Yates, J. F., & Kulick, R. M. Effort control and judgments. Organizational Behavior and Human Performance, 1977, 20, 54 – 65.