Design choice speaks to validity –the evaluator’s ability to draw conclusions in terms of the cause and effect or association between the program/intervention and outcomes (internal validity), and to generalize likely outcomes to broader samples/populations (external validity). As Cook and Campbell (1979) assert, there is no single best design approach.
Designs are grouped into three primary categories – experimental, quasi-experimental and non-experimental, with a range of choices within each category. Traditionally, experimental designs have been characterized as the “gold standard,” a decidedly biased representation, as the “best;” when in fact, design choice should be informed by the evaluation questions to be addressed, and the precision needed in outcome estimates, along with practical considerations. Design choices, whether experimental, quasi-experimental or non-experimental all have limitations and practical considerations in terms of their use in evaluation studies.
The course will cover the following design categories, highlighting advantages and disadvantages of each; identifying when best to use specific design types; and will provide case examples of each.
The course is intentionally interactive. Participants will work with case materials, identifying design types; selecting designs that are best aligned with evaluation questions; building a rationale for the strength of evidence design choices yield; and characterizing the pros and cons of design choices. Participants will be sent materials and resources prior to the course.
Recommended Audience:
The course is geared to individuals having familiarity with evaluation or applied research.