Home + Summaries - Site-Using Tips -   into right frame 

Experience plus Principles

Based on what we know about thinking, learning, and performing, we should expect that a well-designed combination of experience plus principles-for-inquiry (taught using a model of Design Process) will be more educationally effective than experience by itself, to help students improve their thinking skills and their understanding of design-and-science.

We can examine this claim in two stages, first by asking “should we teach principles?” and then (because the answer is “yes” but is trivial) questions that are more important — “WHAT principles should we teach?” and “HOW?” and “with how much emphasis?”

As explained at the end of its page-summary`, this page — which is closely related to pages about "other strategies for teaching & developing instruction to teach Design Process & logical reasons to teach Design Process — will not try to “prove my claims” because we need more evidence, and the logical interpretation of evidence is difficult, and the questions (what, how much, how?) are complex," and effects differ for achieving different educational goals. {a summary of difficulties & complexities}   Instead I'll explain why it seems reasonable to decide, as “a good way to bet”, that "Design Process might be very useful in education, so its possibilities are worth exploring and developing."

Does this claim seem justifiable?  IF “no” (and IF this doesn't seem likely to change based on new evidence), then maybe I should abandon my development of Design Process.  IF “yes”, then hopefully we can work together to develop instruction that combines Design Process with other strategies for effective teaching-and-learning.

In all 3 parts of this page — 1. transfer and 2. metacognition/organization and 3. empirical evidence — it's important to recognize the complexities of analyzing multiple factors using analogical reasoning when we make evaluations of effectiveness that will differ for a variety of educational goals.

1. Transfers of Learning

The page-summary describes one type of evidence:

1) To increase transfers of learning (a major goal of education), the research-based recommendations of How People Learn are to teach knowledge in multiple contexts, in a form that can be easily generalized, and we do both with Design Process.

Appropriate Humility:   With transfer, all evaluations are difficult, and many “conclusions” are controversial and are disputed.   As in most of this page, currently my claims are based on analogical reasoning:  IF transfers of learning increase when we do X and Y, and IF using Design Process lets us do X and Y better, THEN teaching Design Process will promote partial transfers of learning.   The structure of this argument is logically valid, but... we must carefully examine each IF (re: general principles for transfer, and the effects of teaching Design Process) to decide what kinds of THEN-conclusions are justifiable, and with what levels of confidence.

I.O.U. — Later I will learn more, and then write more, maybe in May 2014.  Before then you can read an "executive summary" plus details in the page-summaries (shorter & longer) and full page.

2. Using Metacognition and Organizing Knowledge

And the page-summary describes another type of evidence:

2. Most educators acknowledge the effectiveness of using metacognition and organizing knowledge so we should ask:
• “can either of these, or both, be improved by using Design Process?” and
• (if yes) “in what ways would this improve our educational effectiveness?”

For background knowledge about Metacognition-and-Organization, the full-length page asking "Why should we use Design Process?" has a summary with links to “where you can learn more” that open on the left side, so this link (Metacognition-and-Organization) opens on the right side.  After you've read it and have explored its links — which explain how Design Process can help students develop-and-apply valuable metacognitive Strategies for Learning-and-Performing and why the logical organization of Design Process probably will be educationally beneficial, and more — a link at the end of its "Organization" section will bring you back to here.

The two questions above (••) inspire sub-questions:

Metacognition  —  How is performance affected by metacognition?   The effects will depend on:  the situation and type of performance;  the kinds of learning measured;  the type of metacognition, which should be regulated so it's optimal (so it's not too little, not too much, and of the right types) to achieve maximum benefits with minimum interference.   Or, bringing Design Process into the questions:  based on everything we know about metacognition, how would using Design Process for instruction affect the "type of metacognition" and its regulation, for different types of performance in different situations?

Organization  —  What are the principles for how knowledge-organization (of various types) affects various types of learning, thinking, and performing?  There is strong evidence showing the benefits of organization for conceptual knowledge, where most research has been done;  in what ways, and to what degree, is the evidence also strong for organizing procedural knowledge and its components, such as conditional knowledge?  And if Design Process is used for organizing knowledge (conceptual & procedural), would this improve the understanding and/or skills of students?

This section now splits into sub-sections for Organization and Metacognition.

2A. Research about ORGANIZATION

Another page – The Educational Benefits of Organizing Knowledge – summarizes principles and examines research.

2B. Research about METACOGNITION

During instruction of any type, the main reason for using Design Process is to promote metacognitive reflection.  And "those who study metacognition (educators, psychologists,...) think it can be very useful."  Logically, can we conclude that using Design Process will be useful?  I think this is “a good way to bet” but proving this (beyond a reasonable doubt) is difficult, for reasons explained in the page-introduction and because for metacognition the sub-questions are complex.   {more about metacognition}

This section will describe research that might be useful when we're wondering whether to expect educational benefits from using Design Process, and (if "yes") then “what kinds of benefits, and how beneficial?” for different kinds of instruction.  The descriptions-of-research below supplement the summary-of-benefits above:

Marsha Lovett has been designing ways to teach metacognition at Carnegie Mellon, and doing research to study the results.  She described principles and research-results in a talk for Educause (mp3 + Powerpoint-PDF), and Carol Ormand wrote a talk-summary for Carleton.  The Powerpoint-PDF is a 49-page description of her research about Teaching Metacognition:  introduction (pages 2-10) including importance (5-6) and some failures (10);  Beliefs about Learning (11-21);  using an SRL Cycle of Planning and Goal Setting (22-28) plus Monitoring and Adapting (29-47);  Overall Conclusions (48).

This website examines Beliefs about Learning (as part of a student's Self-Perception), and using an SRL Cycle (Plan-Monitor-Adapt, 24) to design metacognitive Learning Strategies, and using metacognitive Wrappers (34-39).

I.O.U. — In the near future I'll continue searching for research about metacognition (a lot of it exists), and after careful evaluation to decide what is worth sharing with you, I'll share what I find.

Below is the “informal research” of conclusions based on experience.

Reflection — which is described by edutech-wiki as "a metacognitive strategy ... an active exploration of experiences to gain new or greater understanding" — is a way to learn more from experience in metacognitive Strategies for Learning-and-Performing.  Is reflection educationally valuable?  Let's look at what teachers say about one kind of student experience.  The National Service-Learning Clearinghouse says "the process of reflection is a core component of service-learning";  in another page they explain how reflection is "the use of creative and critical thinking skills to help prepare for, succeed in, and learn from the service experience," serving as a "link that ties student experiences in the community to academic learning," that helps students "integrate prior knowledge and experiences with new experiences to develop critical thinking and problem solving skills."  Chaminade University says "reflection is at the heart of service-learning – doing service without reflecting is like eating without digesting."  Virginia Tech's Service-Learning Handbook claims {their quote is not in new version but was in old whose link was broken by V-Tech, which I'll try to find in wayback-archive} that "reflection is the key ingredient for transforming service experiences into learning."  A manual for Facilitating Reflection says "reflection is a vital component of service-learning."

Observations by One Teacher

Based on her observations when teaching Critical Thinking and Scientific Method, Dany Adams reports that:

Because the scientific method is a formalization of critical thinking, it can be used as a simple model that... puts critical thinking at the center of a straightforward, easily implemented, teaching strategy. ...  Explicitly discussing the logic and the thought processes that inform experimental methods [which is one aspect of scientific method and Design Process] works better than hoping students will “get it” if they hear enough experiments described.

A Logical Argument-by-Analogy

Similar to Dany Adams, I make a claim — that principles-for-process do exist, and we should teach them explicitly — in the "yes" part of "no and yes" when we ask "Is there a method?", and with more detail in Principles for Players & Designers:

When we examine hockey and ask “are there principles for skillful playing?” and “should coaches help their players learn these principles?” we say “yes” and “yes”. ...  When asking analogous questions for design — “are there principles for skillful designing?” and “should teachers help their students learn these principles?” — it seems logical to again say “yes” and “yes”.  Although in hockey and design the details are different, the basic concept — that teaching principles is a useful way to improve learning-and-performance — is similar.  In my opinion, we should agree that teaching principles for designing is a worthy educational objective.  We should examine a variety of strategies for HOW teachers can help students learn skills-and-strategies for solving problems;  one option is to supplement Design Activities by teaching Design Process with a combination of Experience plus Principles.

This section is about WHY to promote metacognition by using Design Process.  For HOW, with different kinds of instruction, some options are in Using Design Process to develop metacognitive Thinking Strategies.

3. Research about Instruction

And the page-summary` describes a third type of evidence,

3) Does empirical research show that instruction is more effective (or less effective) when inquiry-experiences are supplemented with inquiry-principles?  in what ways?  how does the effectiveness change when inquiry-principles are taught with instruction that includes a model, instead of no model or a semi-model?  what will change if the model is Design Process?

The page-intro explains why the most important questions are “WHAT principles should we teach?” and “HOW?” and “with how much emphasis?”

Learning about the Nature of Science

Regarding a goal of "helping students understand the nature of design-and-science," NSTA says in September 2011:  "The rhetoric of the Framework calls for and seems to imply that students will understand why the practices are used by scientists, but experience tells us that unless the instruction is explicit, the knowledge of the purpose and reason for the practices will not be understood."  And later, in June 2012, NSTA says: 

The claim that simply doing inquiry (engaging with scientific practices) will result in knowing about inquiry (a claim that knowledge about inquiry and nature of science would develop implicitly) is refuted by 50 years of empirical research.  Much of the earlier work is summarized in Abd-El-Khalick and Lederman (2000)¹ but even more recent work by Khishfe and Abd-El-Khalick (2002)², Howe and Rudge (2005)³, Peters and Kitsantas (2010)⁴, and Yacoubian and BouJaoude (2010)⁵ provide ample evidence to dispute that claim.  Even strong proponents of practice-based approaches to science education such as Sandoval and Morrison (2003)⁶ found that while there are many benefits of such approaches, they do not lead to helping students develop ideas about the nature of science.     { References - 123456 }

Experience plus Reflection-and-Principles

For helping students improve their problem-solving procedural knowledge, I'm wondering how strongly the Instruction Principles below – quoted from LearnLab-PSLC with added italics-emphasis and [comments] – support my claims for the educational effectiveness of experience (in inquiry activities) plus principles (in Design Process), because their research-based conclusions are for situations-and-instruction that are analogous to (but not identical with) using Design Process for inquiry-instruction.*   LearnLab-PSLC says:

    Instruction that combines or helps students' combine learning from examples [experience] and learning of or from rules [principles] tends to be more effective than instruction that includes the same examples and rules but does not help students combine them. [what about instruction that excludes principles? it seems that this should be even less effective than experience+principles without combining?]
    ... Students learn more efficiently and more robustly when more frequent study of worked examples [to illustrate principles of problem-solving strategies] is interleaved with problem solving practice [experience].
    ... Prompting students to self-explain each step of a worked example [to promote reflection] causes higher learning gains than having them study the material without such prompting.
    Explaining [with reflection + principles] how and why incorrect solutions are incorrect will help students [understand what to avoid doing in the future, and why] ...

* These research-based principles (which involve Metacognition because "these ‘two types of evidence’ are overlapping and mutually supportive") seem to support my claims, although the complexities of interpreting research provide reasons for caution.

Predict-Observe-Explain

Similarly, research shows that instruction using a sequence of Predict-Observe-Explain (POE) will help students improve their learning of conceptual knowledge.*   If we use principles of Design Process to help students understand the process of scientific logic they are using with POE, will this help them improve their procedural knowledge?  We have many reasons to think “yes” although, as explained above and below, there also are reasons for caution.   {*references available later}

I.O.U. — As with Metacognition & Organization, soon (maybe in mid-to-late July) I'll be studying research while thinking about these questions, and the rest of this section will be a sharing of what I'm learning.