Program Evaluation & Impact Analyst

Description

JOB TITLE: Program Evaluation and Impact Analyst

Department: Family Care

Job Status: Active

FLSA Status: Non-exempt

Reports To: Chief Program Officer

Grade/Level: Administrative/Evaluator

Amount of Travel Required: up to 10% travel time required

Job Type: Full Time, Hourly

Positions Supervised: Designated Interns

Work Schedule: 30-35 hours, In Person, Monday-Friday (with very occasional evening/weekend hours)

Pay & Benefits: $17.00-$23.00 per hour

JOB IS LOCATED IN: Lynchburg Virginia and required in person work

POSITION SUMMARY: The Program Evaluation and Impact Analyst ensures FIVE18 has reliable, meaningful, and accessible program data that equips the Chief Program Officer and program leaders to evaluate effectiveness, make evidence informed decisions, continuously improve services and strengthen a culture in which data and consumer voice are routinely used to understand and increase impact.

The Analyst joins FIVE18 at an important stage in the development of its evaluation infrastructure, stewarding strong existing practices while expanding organization-wide measurement systems, dashboards, standardized outcomes, and reporting processes.

This role is expected to build, maintain, and improved the scalable measurement infrastructure needed to support FIVE18's Evidence-Based Impact value and long term goals. This individual will communicate effectively with colleagues and consumers to create and cultivate a positive impression of FIVE18, striving to present a culture of hospitality and competence.

ESSENTIAL FUNCTIONS

Reasonable Accommodations Statement: To accomplish this job successfully, an individual must be able to perform, with or without reasonable accommodation, each essential function satisfactorily. Reasonable accommodations may be made to help enable qualified individuals with disabilities to perform the essential functions.

Program Evaluation Systems

• Build, maintain, and improve meaningful outputs, outcomes, indicators, evaluation tools, and recurring evaluation processes.

• Maintain continuity of established evaluation systems while identifying thoughtful opportunities for refinement.

• Document measures, definitions, methodology, data sources, reporting processes, and historical trends so evaluation knowledge is organizationally accessible.

2. Data Quality, Analysis, and Interpretation

• Monitor the accuracy, completeness, consistency, and reliability of program data; identify and communicate data-quality concerns, gaps, anomalies, and barriers to useful analysis.

• Analyze quantitative and qualitative data to identify meaningful trends, patterns, gaps, risks, and opportunities rather than simply reporting numbers.

• Apply healthy skepticism, test assumptions, investigate inconsistencies, and distinguish what the evidence supports from what it does not.

• Develop sufficient understanding of FIVE18's programs, consumers, strategy, goals, and evaluation systems to exercise sound judgment about what deserves further investigation, program-level attention, or CPO awareness.

• Determine what to elevate, when to elevate it, and the appropriate level of analysis before doing so, without relying on the CPO to routinely direct what should be analyzed.

3. Program Impact, Learning, and Continuous Quality Improvement

• Facilitate recurring evaluation processes that connect evidence to program learning, improvement, and decision-making.

• Assess the impact of program adaptations and improvement efforts over time.

• Integrate participant feedback, satisfaction data, qualitative information, and consumer voice alongside quantitative measures.

• Maintain a systems-level understanding of who FIVE18 serves, who may not be represented in the data, and what quantitative measures may not fully capture.

4. Organizational Measurement and Reporting

• Build, maintain, and improve dashboards, Right Performance Metrics (RPMs), organizational impact measures, and routine reporting tools.

• Support reliable reporting of reach, engagement, outcomes, satisfaction, consumer characteristics, and other meaningful indicators.

• Increase the efficiency and reliability of data collection, aggregation, analysis, visualization, and reporting, including appropriate use of automation and emerging technology.

• Help build toward standardized, scalable outcome measurement and the infrastructure required to assess progress toward annual and long-term organizational goals.

5. Collaboration and Data Capacity

• Work collaboratively and proactively with Focus Area Leaders to strengthen evaluation practices across programs and increase consistency where appropriate.

• Help staff and leaders understand, interpret, and routinely use data for learning and improvement.

• Communicate complex information clearly and respectfully to people with varying levels of comfort with data, welcoming questions and adapting explanations as needed.

• Partner with Focus Area Leaders around data-quality needs while maintaining clear accountability boundaries: program staff own accurate and timely data entry, and Focus Area Leaders own program-level follow-through and performance.

6. Evidence-Informed Program Development

• Research relevant evidence and promising practices and help establish appropriate measures for new or evolving initiatives.

• Identify gaps between what FIVE18 measures today and what it will need to measure as programs, reach, partnerships, and organizational complexity grow.

7. Regular Impact and Learning Rhythms

• Ongoing - Monitor data and system health, maintain dashboards and evaluation systems, investigate meaningful questions, and proactively collaborate with program leaders.

• Monthly - Lead the Program Impact & Learning Review with the Chief Program Officer, setting the primary analytical agenda and bringing forward the reports, trends, concerns, opportunities, and evaluation questions most deserving of CPO attention.

• Quarterly - Conduct deeper program and organizational analysis examining performance, outcomes, reach, consumer experience, trends, data quality, and continuous improvement efforts.

• Annually - Provide a broader assessment of program and organizational impact, consumer profile, outcomes, satisfaction, multi-year trends, annual goals, and progress toward FIVE18's long-term measurement and impact objectives.

• As Needed - Conduct timely targeted analysis when significant programmatic or organizational questions arise.

8. Organizational Participation and Representation

• Participate in meetings, training, and consultative supervision as appropriate.

• Create and cultivate a positive impression of FIVE18 in all communication modalities and contexts, striving to present a culture of hospitality and competence.

• Produce accurate, timely, consistent, thorough, and concise work.

Non-Essential Functions:

• Other administrative duties as necessary or assigned
• Other hospitality and/or housekeeping duties as necessary or assigned
• Other tasks as needed to support the overall mission and operations of FIVE18

Deliverables:

• Comprehensive Program Evaluation Reports - Conduct detailed program evaluations and produce reports that synthesize quantitative and qualitative findings, identify meaningful trends and areas for improvement, and provide evidence-informed insights and actionable recommendations for program enhancement.
• Regular Data Analysis & Leadership Reporting - Produce timely analysis, reports, dashboards, and other reporting outputs that provide meaningful visibility into program performance, consumers served, outputs, outcomes, and emerging trends. Exercise professional judgment in determining which findings warrant routine reporting, deeper investigation, or leadership attention.
• Monthly Program Impact & Learning Review - Lead the monthly review with the Chief Program Officer, bringing forward the reports, analysis, trends, data-quality concerns, emerging insights, and recommendations most deserving of leadership attention. Set the primary analytical agenda while incorporating specific areas of review requested by the CPO.
• Quarterly Evaluation & CQI Reports - Produce quarterly reports and analysis examining program performance, outcomes, reach, consumer experience, data quality, trends, and continuous improvement efforts. Include relevant comparisons over time and identify findings or recommendations that warrant program or leadership attention.
• Annual Impact & Evaluation Report - Produce an annual synthesis of program and organizational impact, outcomes, consumer characteristics, satisfaction, multi-year trends, annual goal performance, and progress toward longer-term organizational impact objectives. Identify significant findings and evaluation priorities for the coming year.
• Consumer Voice, Survey & Feedback Reporting - Develop and maintain appropriate consumer and stakeholder feedback processes. Produce meaningful summaries and analyses that integrate qualitative feedback, satisfaction, and consumer voice with quantitative program data.
• Impact Assessment Frameworks & Dashboards - Build, maintain, and improve evaluation frameworks, outcome measures, dashboards, RPMs, and related measurement tools that enable ongoing assessment of program and organizational effectiveness.
• Targeted Decision-Support Analysis - Produce timely analyses, research summaries, or briefings in response to significant programmatic or organizational questions, providing evidence and context that support CPO and leadership decision-making.
• Evaluation Documentation & Data Integrity - Maintain accurate and timely documentation of evaluation activities, measures, definitions, methodologies, data sources, calculations, reporting processes, and findings. Maintain a living record of what FIVE18 measures, why it is measured, where the data resides, how it is calculated, how often it is reviewed, and who uses it.

Work Contacts/Employees Supervised:

Communicates with:

• Chief Program Officer through routine meetings and digital communication, including the monthly review, and through timely escalation of significant findings or concerns.
• Focus area leaders (Senior Directors) and program staff through proactive collaboration related to evaluation, data quality, program learning, and reporting.
• Colleagues and appropriate internal or external stakeholders as necessary to support effective evaluation and organizational learning.

Direct Supervision Over:

• Designated Interns, as assigned
  

Decision Making Authority:
This position carries HIGH influence with MODERATE decision-making authority. The Analyst has authority to initiate appropriate analysis, communicate findings, identify concerns and opportunities, and recommend improvements. Final programmatic and organizational decision-making authority remains with the Chief Program Officer and appropriate leadership. The Analyst identifies and communicates data-quality and performance concerns but does not assume responsibility for program-level operational accountability.

Competency Statements

• Strategic Analytical Judgment - Ability to discern which findings, trends, risks, and questions warrant deeper analysis or leadership attention; forms an evidence-based point of view and communicates it with appropriate confidence.

• Human-Centered Systems Thinking - Ability to move between individual, program, and organizational perspectives while maintaining awareness that every data point represents real people and lived experiences.

• Analytical Rigor and Healthy Skepticism - Ability to value precision, validate findings, ask strong questions, recognize inconsistencies, and avoid overstating what evidence can support.

• Relational Humility and Influence - Ability to build credibility through expertise, curiosity, patience, and clear communication rather than positional authority; receives questions and feedback without defensiveness.

• Ownership and Follow-Through - Ability to own recurring rhythms and systems without routine direction, notice when something is not working, and follow analysis through to useful understanding.

• Comfortable Building While Operating - Ability to provide reliable day-to-day execution while simultaneously helping build the systems FIVE18 will need in the future.

• Digital Systems Fluency and Technology Agility - Ability to quickly learn and effectively use technology, with curiosity about tools that can improve data collection, analysis, reporting, visualization, and organizational efficiency.

• Discernment and Prioritization - Ability to distinguish what is merely interesting from what is important and consider what decision, learning, or improvement an analysis can meaningfully inform.

• Communication, Oral and Written - Ability to explain analytical findings clearly and concisely to technical and non-technical audiences.

• Accountability and Accuracy - Ability to accept responsibility for work and perform it accurately and thoroughly.

• Active Listening - Ability to actively attend to, convey, and understand the comments and questions of others.

• Honesty / Integrity - Ability to be truthful, represent original work accurately, and be seen as credible in the workplace.

• Organization and Time Management - Ability to prioritize multiple responsibilities and complete work within appropriate deadlines.

Working Conditions/Physical Effort:

• This position requires minimal physical effort
• Includes operation of general office equipment and driving an automobile
• Primarily office setting/indoors

POSITION QUALIFICATIONS

Knowledge of:

• Applied program evaluation, outcome measurement, data-quality practices, continuous quality improvement, and evidence review
• Quantitative and qualitative analysis and appropriate research ethics
• Dashboard, reporting, survey, database, and data visualization practices
• Human services and trauma-informed approaches to care (preferred upon hire)
• Trauma-informed modalities of care (preferred upon hire)

Job Competencies:

Upon hire, the Analyst must:

• Be committed to diversity
• Demonstrate analytical rigor, curiosity, discernment, empathy, and relational humility
• Work collaboratively and proactively with others
• Be energized by an environment in which some systems are established, others are developing, and new needs will emerge
• Be comfortable asking questions and receiving questions in a high-feedback culture

After hire, the Analyst must:

• Commit to the Mission, Vision, Values, and Evidence-Based Impact culture of FIVE18
• Develop a strong working understanding of FIVE18's programs, consumers, strategy, evaluation systems, and organizational goals

Tools and Equipment Used:

• Computer
• Telephone
• Scanner
• Copier
• Printer
• Vehicle

Education & Experience:

1. Bachelor's degree with a minimum of 3 years of applied analysis, program evaluation, performance measurement, or closely related experience preferred.
2. Associate's degree may be considered with more than 5 years of strong applied analysis and evaluation experience.
3. Nonprofit or human services experience is strongly preferred; relevant experience in healthcare, education, public administration, research, or another mission-driven program environment may also be considered.

Experience Defined: Applied experience conducting program evaluations, analyzing quantitative and qualitative data, assessing outcomes, maintaining data quality, designing or using surveys and feedback tools, developing reports or dashboards, communicating findings to non-technical audiences, and using evidence to support program improvement and decision-making. Providing direct trauma-informed care to children and/or families is a plus but not required.

Technology Skills: Demonstrated proficiency with Google Workspace, particularly advanced spreadsheet analysis in Google Sheets. Experience with databases, case-management or program data systems, dashboards, reporting or visualization platforms, survey tools, and related data technologies. Ability and willingness to quickly learn unfamiliar systems and emerging technologies; prior experience with FIVE18-specific platforms is not required.

Certification & Licenses: none required

Other Requirements:

• Authentic passion for working with a team to collectively effect change in families.
• Ability to maintain appropriate confidentiality and handle consumer and organizational data responsibly.
• Must have a valid driver's license and an acceptable driving record, with safe and reliable transportation for work-related purposes.

PHYSICAL DEMANDS

N (Not Applicable) Activity is not applicable to this position.

O (Occasionally) Position requires this activity up to 33% of the time (0 - 2.5+ hrs/day)

F (Frequently) - Position requires this activity from 33% - 66% of the time (2.5 - 5.5+ hrs/day)

C (Constantly) - Position requires this activity more than 66% of the time (5.5+ hrs/day)

---------------------------------------------------------------------------------------------------------------------

Stand: F Walk: F Sit: F

Manually Manipulate F Reach Outward O Reach Above Shoulder O

Climb O Crawl N Squat or Kneel O

Bend O Grasp O Speak C

Other Physical Requirements:

Vision (Near, Distance, Color, Peripheral, Depth) – Required

Sense of Sound – Yes, majority of work is in relationship and communication

Sense of Smell – No

Sense of Taste – No

Sense of Touch – Yes, work on computers and equipment is necessary for the job.

Ability to wear Personal Protective Equipment (PPE) – Yes (as required)

Sense of Balance – Yes, due to the mobile nature of the job

Job Context and Work Environment:

This position works primarily within their local FIVE18 shared office space in Forest, VA. They may occasionally be able to conduct this position remotely when needed (prior approval required). The position occasionally requires travel to other FIVE18 settings, including offices and camp. The position may also require travel to volunteer or client homes.

While completing office duties, work is in an office environment in a controlled atmosphere. There may be times when work will be done in other settings such as agency office buildings, conference centers, and client homes. Work may occasionally be outdoors, such as when attending community events.

Prepared By: Nicholette Antoniuk Date: August 31, 2026

HR Approval: Brandis Saunders Date: 8/31/2026

Employee Signature: ________________________ Date: _________

The company has reviewed this job description to ensure that essential functions and basic duties have been included. It is intended to provide guidelines for job expectations and the employee's ability to perform the position described. It is not intended to be construed as an exhaustive list of all functions, responsibilities, skills and abilities. Additional functions and requirements may be assigned by supervisors as deemed appropriate. This document does not represent a contract of employment, and the company reserves the right to change this job description and/or assign tasks for the employee to perform, as the company may deem appropriate.

Requirements

JOB IS LOCATED IN: Lynchburg Virginia and required in person work