YPQI process and schedule
The YPQI process for the Richmond region follows a fall to spring cycle. There are seasonal trainings, meetings and tasks that together form the process and cycle of improvement.
The sequence begins each August, and key steps include:
- Basics training
- External assessment
- Improvement planning
The YPQI Assess-Plan-Improve sequence begins in August of each year with the kickoff event and continues throughout the year. Each site designates a staff member to serve as the site lead. The site lead is usually a site manager, supervisor or director, but could be anyone at the site. It is important that this person has sufficient time to coordinate the process and attend all trainings. An OST time organization that operates multiple sites should designate a lead for each site, rather than having one person designated for multiple sites.
Following the kickoff event, which is held as a welcome and introduction for the new sites, the site lead attends the Basics training. Basics is an all-day training that prepares the leads to conduct the program self-assessment. At the training, participants will learn about the continuous quality process, as well as the Program Quality Assessment tool that will be used in the self-assessment process. The PQA evaluates the quality of youth experiences as youth attend workshops and classes, complete group projects, and participate in meetings and regular program activities. Sites can choose to use either the Youth PQA (for grades 4-12) or the School-Age PQA (for grades K-6).
After the site lead completes the Basics training, they then assemble the site team that will conduct the assessment. This team is typically comprised of program staff, but may also include volunteers, youth, board members, parents or even community members.
In addition to the site lead, it is recommended that the team consist of one to three additional members. After the site team members have been selected, the lead will train the other team members on the PQA and create a schedule for the self-assessment. The schedule will give each team member an opportunity to observe an offering and ensure a variety of offerings are observed and scored on the PQA.
After all of the data has been collected, the site lead guides the team in scoring a single, site-wide PQA. These scores are then entered into the online Scores Reporter by the site lead. A brief, one-hour webinar will familiarize new site leads with the online Scores Reporter system.
While the site lead is coordinating the self-assessment with the site team, an external assessor will also come into the site to conduct an assessment. External assessors are fellow site leads, supervisors and managers from OST time organizations within the Richmond region who are also involved in the YPQI process.
To become an external assessor, a participant must attend a two-day training that primarily consists of practice video scoring of the PQA. Participants who successfully achieve acceptable levels of reliability (80% at the item level) through a video reliability check become endorsed assessors and are able to collect research-quality data. After the first year, external assessors attend a one-day refresher training and must successfully achieve acceptable levels of reliability. The external assessor also inputs the scores from the assessment into the online Scores Reporter.
The Assess-Plan-Improve sequence continues with the Planning with Data training. Assessment and evaluation can supply a wealth of valuable data about the quality of a youth program. But assessment just provides the data – a list of numbers and words. Planning with Data is about examining data to make meaningful improvement plans. This training prepares site leads to develop effective program improvement plans and to take this planning process back to their program settings where they will develop an Improvement Plan with their site team.
Once the improvement plan is entered into the online Scores Reporter system, it is now time to implement the final aspect of the Assess-Plan-Improve sequence. Implementing a program improvement plan can be considered the most important step. While it is crucial to assess and plan for improvement, unless change actually happens, the process was in vain. To support this process, various Methods trainings are offered for both the site lead and staff members to attend. These two-and-a-half-hour workshops are aligned with the items on the PQA and are facilitated by other members of the Richmond region who have attended the Youth Work Methods Training of Trainers. The sites then spend the remainder of the spring and into the summer to implement their improvement plans and gear up for another year.