RESEARCH & EVALUATION
Overview
RESEARCH & EVALUATION
The Academy for Professional Excellence is involved in a number of research and training evaluation activities. Most of the research is conducted through the Southern Area Consortium of Human Services (SACHS) which conducts regional research as identified by the SACHS Members during the Annual Planning Session. To support this regional research SACHS engages and assists in four distinct research models: Literature Reviews, Secondary Data Collection and Analysis, Outcome Studies, and Consultation on Studies.
Literature Reviews
SACHS will conduct four literature reviews (one per quarter) to support each of the quarterly discussion topics. These literature reviews provide a comprehensive summary of the current research that has been conducted on the following topics. Each literature review also provides a list and a brief summary of the articles that were reviewed to complete the literature review.
Secondary Data Collection and Analysis
As resources allow, SACHS staff will collect and analyze secondary administrative County data and report trends to Directors.
Program Evaluation/Outcome Studies
SACHS will engage in Program Evaluation and Outcome Studies as needed. These evaluations can either be single-county or multi-county. The purpose of these studies is to assess the effectiveness of the program(s) being evaluated.
Consultation on Studies
To help meet the regional research needs of the southern region, SACHS is working this year to develop an infrastructure for improved coordination and consultation with faculty and students from university social work programs (currently California State University San Bernardino and San Diego State University).
Collaboration on Reserach
In addition, concentrated efforts will be made to coordinate research with other Regional Training Academies, CalSWEC, CDSS, the Research Training Network, Southern Counties Research and Data Collection Forum, and federally funded child welfare clearinghouses around the country. Regular updates on any new research developments and/or communications with the above entities will be reported out at quarterly SACHS Meetings.
Training Evaluation
All Academy training projects are actively involved in the evaluation of their training. Training evaluation has a number of purposes: it provides feedback for course improvement, training design, and training structure; it identifies trainee’s knowledge, skills, and values; it provides data for individual accountability; and it identifies facilitators or barriers to achieving program goals. Training evaluation does not establish a direct cause and effect relationship between a training program and agency or client outcomes.
To evaluate the Academy’s training projects, a multi-level evaluation methodology was designed based on Kirkpatrick’s four-level evaluation schema. While Kirkpatrick includes only four levels (Satisfaction/Opinion, Knowledge, Behavior, and Outcomes), the Academy’s training evaluation methodology has added two additional evaluation levels (Tracking and Formative). These six evaluation levels help to build a chain of evidence about training effectiveness.
Stakeholders rightfully want to know that training “works,” meaning that trainees learn useful knowledge, values, and skills that will translate to effective practice and support better client outcomes. However, many variables other than training can affect whether these desirable goals are achieved (e.g., agency polices, caseload size, availability of needed services). While it is impossible to design training that can control these variables, building a chain of evidence establishes a linkage between training and desired outcomes for the participant, the agency, and the client such that a reasonable person would agree that training played a part in the desired outcome.
For information on specific research or training evaluation projects for individual programs, please go to the individual projects webpages.