1. In a process of design` the main function of experiments is to generate information (Predictions or Observations) that we can use to evaluate Options for products, activities, or strategies (in general design) or theories (in science). When we compare Goals with Predictions (made in Mental Experiments) and with Observations (made in Physical Experiments) we are doing Mental Quality Checks and Physical Quality Checks, with quality defined by our Goals. In a third type of comparison, which is the logical foundation of science, a Mental-and-Physical Reality Check (Theory Check) lets us evaluate the quality of a Theory by comparing theory-based Predictions with reality-based Observations.
2. But which experiment(s) should you do? During a creative-and-critical process of Experimental Design — summarized below and in a section-summary and with more detail in a longer summary and with a different perspective in A Quick Overview of Scientific Method and Detailed Overview of Scientific Method — you creatively generate Options for Experimental Systems, critically evaluate these Options by asking “which E-Systems will let us make Predictions or Observations that could be useful for our Design Project?”, and decide which E-Systems to use. You design experiments (in Mode 2C) so you can do Mental Experiments (in 2D) or Physical Experiments (2E) to make Predictions or Observations, which are used for Evaluations-of-Options in Quality Checks (2E) and Reality Checks (2E). To improve your experiments with critical thinking, you can think ahead by imagining the tough questions that will be asked by critics — “is the experimental group large enough, and does it accurately represent the whole population?” or “what are the sources of random errors & systematic errors, and what are their effects?” and more — so in critical evaluations the conclusion will be “a well designed experiment.”
3. A creatively divergent process of Designing Experiments occurs by Guided Generation & Free Generation that can be stimulated by Creative Uses of Analysis to help you discover “gaps in knowledge” to fill with information (predictions & observations) from experiments.
4. During a process of design, Mental Experiments are important in several ways: for making Predictions to use (see #1 above) in evaluative Quality Checks & Reality Checks; for quick-and-cheap Designing of Experiments (#2, 3) by generating-and-evaluating a variety of Options for Experimental Systems, ranging from conventional to highly innovative; for exploring systems that would be difficult or expensive to observe with Physical Experiments. In addition to the links above (in #1, 2, 3), in a page about Theory-Based Models you'll find many ideas about designing-and-doing Mental Experiments, including predictions (quantitative or qualitative) using simulation and to serve other functions during a process of design.
EXPERIMENTS let us make OBSERVATIONS In science, information about nature comes from our observations. Consider two types of observation-situations: a) You make observations, for a month, about the moon's appearance, and the times & locations of its rising & setting. b) You observe the growth of young plants in many contexts, by varying many factors: light, temperature, type of seed (lima bean,...), type of soil, amount and frequency of watering, amount and type of fertilizer, treatment of seed (by soaking, cooking,...) before planting, and more. You try different combinations of factors, and for each experiment you make observations both above and below the soil surface, before and after the plant grows through the surface. A) In an uncontrolled observation-situation (like observing the moon) the situation is set up by nature. B) In a controlled observation-situation (as in the experiments with seeds) humans set up the situation, but “what happens” depends on nature. Degrees of Control: In setting up an observation-situation, the degree of human control can range from no control through partial control to total control. For practical reasons, most controlled situations are actually semi-controlled, regarding types & amounts of control. Throughout the range-of-control, observations can be logically compared with predictions to allow reality checks. Therefore, in this website all observation-situations will be called experiments, even though some people use the term experiment for only situations that are at least partially controlled. MORE about Field Studies Observations: We observe using human senses (to see, hear, touch, taste, or smell) plus instruments (watch, ruler, scale, pipet, compass, thermometer, microscope, telescope, spectrometer, chromatograph,...) that help us measure more precisely and observe more widely. We translate raw data (from senses or instruments) into observations that we record using symbolic representations that are verbal (using words), visual (pictures,...), and mathematical (numbers,...). This is (with minor modifications, mainly in the paragraph for "Degrees of Control") from my Introduction to Scientific Method which later includes ideas about:EXPERIMENTAL Design — Creative Generation and Critical Evaluation By defining the term broadly, experiments can include all observation-situations, both controlled experimental situations and uncontrolled field situations that can be called field experiments or natural experiments. When scientists design an experiment (or set of experiments) their general goal is to fill gaps in current knowledge by gathering information about experimental systems or techniques. More specifically, an experiment can be done to “see what will happen” in a new situation or to test the reproducability of observations from previous experiments, to resolve an anomaly, impress a funding agency, or provide support for an argument, as in a crucial experiment [or a crucial question]* that can distinguish between competing theories. ..... New opportunities for experimenting can arise from new events (like an ozone hole) or new discoveries (of old dinosaur bones,...). Scientists may want to test a new sub-theory or explore its application for a variety of systems. A new observation technology may allow new experimental systems. Scientists who are aware and creative, thinking with open-minded imagination, can take advantage of opportunities. Mental Experiments: Scientists often run thought experiments — usually so they can be more efficient (to waste less of their valuable time and resources), but also occasionally for systems that cannot be physically observed — by asking "if we do this, what might happen and what would we learn?" One possibility is a mental-and-physical simulation by "running a model [of the solar system]" and using physical objects to make your predicting more easy and effective. Another type of thought experiment, becoming much more common during the past few decades, are the computer simulations that help scientists do mathematical thinking more easily and effectively. And you can imagine the questions that will be asked later, by yourself and others, during evaluation — "is the sample large enough and does it accurately represent the whole population?" or "what are the effects of systematic errors and random errors?" or... — and then design experiments to answer these questions. * Crucial Experiments and Crucial Questions are analogous; both serve similar functions, by helping us "distinguish between competing theories."Here is a summary, condensed from my PhD dissertation:6. Experimental Design (Generation-and-Evaluation) In my model of Science Process an "experiment" is defined broadly to include both controlled experiments and field studies. Three arrows [in the full diagram for Scientific Method] point toward generate experiment, showing inputs from theory evaluation (which can motivate-and-guide a designing of experiments), gaps in system-knowledge (that can be filled by experimentation, and provide motivation) and "do thought experiments..." (to facilitate the process of designing experiments). The result of experimental design (which combines generating an experiment with evaluating an experiment) is a "real-world experimental system" that can be used for hypothetico-deductive logic [Reality Checks that compare predictions with observations]. Sometimes experiments are done just to see what will happen, but an experiment is often designed to accomplish a specific goal. For example, an experiment (or a cluster of related experiments) can be done to gather information about a system or experimental technique, to resolve anomaly, to provide support for an argument, or to serve as a crucial experiment that can distinguish between competing theories.* To facilitate the collection and interpretation of data for each goal, logical strategies are available. When using these strategies, scientists can think ahead to questions that will be raised during evaluation, regarding issues such as sample size and representativeness, the adequacy of controls, and the effects of random errors and systematic errors. Often, new opportunities for experimenting (and theorizing) emerge from a change in the status quo. For example, opportunities for field studies may arise from new events (such as an ozone hole) or new discoveries (of old dinosaur bones,...). A new theory may stimulate experiments to test and develop the theory, or to explore its application for a variety of systems. Or a new observation technology may allow new types of experimental systems. When an area of science opens up due to any of these changes, opportunities for research are produced. To creatively take advantage of these opportunities requires an open-minded awareness that can imagine a wide variety of possibilities. [ * In situations with a low degree of predictive contrast — because when you ask, for previous experiments, “how much contrast exists between the predictions of competing theories?”, all of the theories make similar predictions — you need a crucial experiment where theory-predictions will differ. ]
These ideas – quoted from my Overview of Scientific Method – are developed more fully, with illustrative examples, in A Detailed Overview of Scientific Method. |