This page adapts an academic project for the portfolio. It summarizes the work and its methods; the original document is not published here.
Methods at a glance
| Research task | Approach | Evidence and limits |
|---|---|---|
| Working-group underperformance | Observational case-study plan with workflow coding and record review. | Hypothetical case; no field observations collected. |
| Veterans’ experience of the PACT Act | Survey population and measures for awareness, administrative difficulty, and satisfaction. | Questionnaire proposed, not administered; no outcomes measured. |
| Brookline Early Education Project | Appraisal of a matched comparison and threats to causal inference. | Methodological critique; no original effect estimate. |
| Food, academic performance, and research reporting | Comparison of secondary reporting with the underlying observational study. | Revised the critique after reviewing the study; association does not prove causation. |
What an employer can evaluate
This collection provides examples of research scoping, measurement design, evidence appraisal, and methodological judgment. It does not demonstrate that the proposed survey, organizational observation, or intervention was carried out. The value is in showing how research choices are explained, justified, and bounded.
Overview
Public administrators rarely work with perfectly controlled data or laboratory conditions. More often, they must make sense of incomplete records, organizational behavior, stakeholder experiences, observational evidence, and policy outcomes.
This portfolio project synthesizes a series of graduate research-methods exercises into one applied framework for asking a practical question:
How can public-sector researchers choose an appropriate method, define what they are trying to measure, collect credible evidence, and recognize the limits of the conclusions they draw?
The page is a synthesis of course exercises rather than a claim that one empirical study was conducted across all of these settings.
Case research: diagnosing organizational underperformance
One exercise begins with a hypothetical working group experiencing underperformance and asks how organizational and workflow patterns might be studied.
The proposed design treats the working group as a case studied in its natural setting. Rather than changing the workflow immediately, the approach combines direct observation with existing organizational records such as processing times, completed work, error records, missed deadlines, workflow documentation, organizational charts, and internal reporting.
The observation plan identifies concrete patterns to track, including:
- workflow interruptions;
- task handoffs;
- waiting and delay periods;
- duplicated work;
- communication around tasks;
- approval bottlenecks;
- corrections;
- missed or delayed work.
A preliminary coding scheme includes categories such as Waiting for Approval, Missing Information, Task Interruption, Duplicated Work, System Delay, and Supervisor Meeting.
The exercise also emphasizes triangulation—comparing observational evidence with existing organizational records—and explicitly recognizes researcher subjectivity, uncertainty about the meaning behind observed behavior, and the limited transferability of findings from one organization to another.
Operationalization and survey design
A second exercise applies research design to veterans’ experiences with PACT Act implementation.
The proposed population includes not only veterans who have already filed related claims, but also potentially eligible veterans who may not yet have accessed the policy or may not be aware of available benefits.
The exercise then moves from abstract concepts to measurable constructs. The survey design operationalizes several dimensions of implementation:
- awareness;
- satisfaction;
- administrative difficulty;
- service or benefit outcomes;
- areas for implementation improvement.
The assignment connects those measures to conceptualization, operationalization, content validity, clarity, and neutral wording. The resulting questions are designed to capture both whether a veteran can access a program and how the administrative process is experienced.
Causal inference and quasi-experimental design
Another exercise evaluates the Brookline Early Education Project and asks whether its design qualifies as a true experiment or a quasi-experiment.
The analysis distinguishes random assignment from matching and explains why random assignment matters for internal validity. It identifies the principal BEEP comparison as a posttest-only matched comparison design rather than a true randomized experiment because the intervention and comparison groups were not randomly assigned to participation.
The exercise then identifies threats to causal inference, including:
- selection bias;
- preexisting differences between groups;
- maturation;
- family environment;
- schooling;
- other educational experiences.
Potential improvements include collecting baseline outcome measures, matching on additional relevant characteristics, statistically controlling baseline differences, and preserving blinded observational procedures.
Evidence evaluation and revising conclusions
A paired set of exercises examines reporting about a study of ultra-processed food consumption and academic performance.
The first response reacts skeptically to a secondary article and questions the quality of the evidence as presented. The follow-up then reviews the underlying peer-reviewed study and revises that assessment.
The later analysis identifies the study as quantitative, observational, and cross-sectional, notes the inclusion of several adjustment variables, and concludes that the underlying research design was considerably stronger than the secondary article made it appear.
The criticism therefore shifts from the study itself to the way the secondary source communicated it, particularly the omission of useful statistics and methodological context.
This sequence demonstrates an important research habit: a conclusion should change when better evidence changes the basis for that conclusion.
Measurement in social science
A final strand addresses a recurring challenge in public-sector research: many important concepts cannot be observed directly.
Concepts such as democracy, happiness, identity, and group cohesion require researchers to translate abstract constructs into observable indicators. Different researchers may define and operationalize those constructs differently, which can affect consistency, precision, and interpretation.
That problem connects the entire project. Good research is not simply collecting data; it also requires deciding what counts as evidence, how a concept will be measured, which method fits the question, and how much confidence the resulting evidence can support.
What this project demonstrates
Taken together, the exercises demonstrate the ability to:
- translate practical problems into researchable questions;
- select an appropriate research design;
- define populations and constructs;
- operationalize abstract concepts;
- build survey measures;
- use observation and coding in qualitative research;
- triangulate evidence;
- evaluate causal claims and threats to internal validity;
- distinguish primary research from secondary reporting;
- recognize methodological limitations;
- revise conclusions when stronger evidence becomes available.
Why this work matters
Public administration depends on evidence, but evidence is only useful when the method used to produce it is appropriate to the question.
This project demonstrates applied research reasoning across organizational analysis, implementation research, survey design, quasi-experimental evaluation, and evidence literacy. It complements the portfolio’s policy-writing and governance work by showing the methods used to decide what evidence can—and cannot—support a conclusion.
Portfolio context
This page synthesizes a sequence of graduate research-methods discussion assignments completed at Arizona State University in PAF 501.