a.Target population/audience, if applicable: Program Managers, Evaluators, Administrators, Educators, Affiliations
b.Number or percentage of the target population/audience reached, if applicable: N/A
c.Public health issue that this practice addresses: 1.There exists a need for financial and other outcomes data on public health education programs, especially at the supervised practice level. These data can be utilized to justify continuation of, or adjustments to, such programs, particularly in times of fiscal crisis. This is important because interns are able to extend our work in the community and complete projects we could not do otherwise. Examples are included in the following sections of this application.
2.This practice addresses two of the 10 Essential Public Health Services and two Healthy People 2010 objectives as follows: ?Assure a competent public health and personal health care workforce ?Evaluate effectiveness, accessibility, and quality of personal and population-based health services ?Increase the proportion of Federal, Tribal, State, and local public health agencies that gather accurate data on public health expenditures, categorized by public health service (HP2010 developmental objective 23-16) ?Increase the proportion of schools for public health workers that integrate into their curricula specific content to develop competency in the essential public health services (HP2010 developmental objective 23-9)
3.The practice helps meet five required competencies of our accredited dietetic internship, all of which relate to analyzing financial data and maximizing fiscal outcomes.
d.Primary Goal and Objectives of the practice: 1.Goal: Justify program sustainability for an accredited dietetic internship, which provides supervised practice in community/public health nutrition, as well as other public health programs.
2.Objectives:?Utilize consistent methods for capturing cost and benefit data on nutrition programs over a specific time period ?Use program outcome data to justify continuation of or changes to program components and fees as well as showcasing tangible and intangible program benefits to decision-making administrators and community affiliations/partners ?Teach cost/benefit methodology to dietetic interns and require supervised practice of this important skill, thereby producing competent public health dietitians who are able to better evaluate any nutrition program they work in following completion of the internship.
e.When (month and year) the practice was implemented: Pilot project began 8/06 and routine data collection began 8/07
g.Outcomes of Practice:1.Were all of your objectives met? Yes, all objectives were met. We have implemented cost/benefit analysis of the dietetic internship on an annual basis and are requiring dietetic interns to complete a supervised practice experience to acquire competency in cost/benefit methodology. We have also used these outcome data to adjust program tuition and share with our partners the financial benefits of hosting our interns.
2.What specific factors led to the success of this practice? ?Director of Nutrition Services at TCHD completed Master’s thesis on cost/benefit analysis of dietetic internship in 1994, laying groundwork for current practice ?Willingness of staff and dietetic interns to record time/activity data ?Collaboration and sharing of data between nutrition division and administration ?Support of LHD Executive Director to continue regular cost/benefit analysis ?Uncertainty in funding streams means more emphasis on showing a program’s bottom line impact ?Adaptability of the practice of cost/benefit analysis to other public health programs, services, and education
What PH issue does this practice address?1.There exists a need for financial and other outcomes data on public health education programs, especially at the supervised practice level. These data can be utilized to justify continuation of, or adjustments to, such programs, particularly in times of fiscal crisis. This is important because interns are able to extend our work in the community and complete projects we could not do otherwise. Examples are included in the following sections of this application. 2.This practice addresses two of the 10 Essential Public Health Services and two Healthy People 2010 objectives as follows: ?Assure a competent public health and personal health care workforce ?Evaluate effectiveness, accessibility, and quality of personal and population-based health services ?Increase the proportion of Federal, Tribal, State, and local public health agencies that gather accurate data on public health expenditures, categorized by public health service (HP2010 developmental objective 23-16) ?Increase the proportion of schools for public health workers that integrate into their curricula specific content to develop competency in the essential public health services (HP2010 developmental objective 23-9) 3.The practice helps meet five required competencies of our accredited dietetic internship, all of which relate to analyzing financial data and maximizing fiscal outcomes.
Agency Community RolesThe Dietetic Internship Director at the LHD is responsible for completing the cost/benefit analysis on a regular and on-going basis to ensure that the dietetic internship remains financially viable and to determine if any changes in tuition/fees or time commitments of particular staff are needed to reduce the cost burden on the LHD. Additionally, the Dietetic Internship Director provides training and supervised practice experience to all interns on cost/benefit methodology as an important public health administrative skill.
ImplementationTime and activity codes were developed with specific definitions so that interns and LHD staff could record internship activities on their timesheets. Some activities provide an in-kind benefit to the LHD, some result in a cost, and some are cost-neutral. Interns are trained at the beginning of their internship year on the mechanics and importance of accuracy in coding their time and are encouraged to record their time daily as well as asking the Internship Director if they are unsure which activity code to use for a certain task. Intern timesheets are reviewed by the Internship Director for accuracy. LHD staff were also trained on coding some internship activities to a log sheet. As mentioned previously, there was a pilot test of the cost/benefit methodology for one year so that any corrections to data collection procedure could be made. Accounting and payroll staff provided program expense and income data as well as lump sum salary figures once the amount of staff time was determined. In using cost/benefit outcome data for program planning purposes, we looked at the overall cost/benefit ratio as well as the amounts of time that the Internship Director, interns, and other LHD staff coded to internship activities. After the pilot test year, decisions were made to increase program tuition, consider preferential hiring of internship graduates in our WIC clinics, and combine intern training with staff training to use time more effectively and reduce costs.
Cost/benefit outcome data have been presented to the LHD Executive Director, the local Board of Health, and the Dietetic Internship Advisory Committee to demonstrate fiscal responsibility and seek feedback on other ways these data may assist in program planning and make our internship stronger. One of the goals of the TCHD Dietetic Internship program is to produce competent entry level public health dietitians. Costing methodology classes and cost/benefit projects occur annually as part of our dietetic internship. The Internship Director and Director of Nutrition Services meet beforehand to discuss project ideas that will most benefit the LHD while also providing a good learning experience for interns. The internship year of August 1st to July 31st annually was selected as the time period for cost/benefit analysis. Data gathering and analysis is generally completed by the end of the calendar year, following the end of each internship year. Intern cost/benefit projects are completed each year, within the administrative rotation of the internship. Due dates fluctuate based on projects assigned, which differ from year to year based on department needs.
1.Utilize consistent methods for capturing cost and benefit data on nutrition programs over a specific time period.
?Performance measures: Established detailed definitions of how interns code their time and which program related activities should be recorded by staff for the cost/benefit analysis. Pilot tested methodology for data collection in year one, so that it was known that appropriate information was obtained and to reduce chances for needing to change data collection methods later.
?Data Collection: Gathered program expense data from accounting, preparation/planning time data from TCHD staff gathered on spreadsheet log, intern and internship director time and activity data collected on weekly timesheets, salary cost information gathered in lump sum amount from payroll.
?Evaluation results: Data are collected from same sources in same way each year.
?Feedback: Dietetic Internship Director reviews all intern timesheets and nutrition staff time allocated to the internship program, to ensure consistent and proper coding of time according to established definitions. Interns are alerted if they are not coding their time consistently or using the appropriate time and activity codes. By giving regular feedback to staff on how these data are being used, as well as by thanking them for their contribution, they remain willing to assist in the process.
2.Use program outcome data to justify continuation of, or changes to program components and fees as well as showcasing tangible and intangible program benefits to decision-making administrators and community affiliations/partners
?Performance measures: Overall cost/benefit ratio of the internship program can be used as evidence of program viability as well as to determine appropriate tuition/fees and budgeted staff time for the program.
?Data Collection: Data are collected as stated above under objective #1. Internship expenditures are totaled, adding in a factor for indirect costs. LHD staff time is quantified in dollars by multiplying time spent on internship activities by actual hourly salary figures. Factors for fringe benefits and indirect costs are added in to arrive at total personnel costs. In-kind professional services provided by interns (based on specific activities recorded on timesheets) are quantified as benefits using the same method stated above for personnel costs. Internship income in the form of tuition and application fees are tallied and added to in-kind intern benefits along with any other dollar quantifiable benefit, such as saving recruiting/hiring costs when we hire an internship graduate. The total benefit dollars are divided by the total cost dollars to arrive at the cost/benefit figure, which is expressed as a ratio. Other tangible and intangible benefits are also determined, such as numbers of clients seen or increased community visibility.
?Evaluation Results: In the pilot test year for this practice (2006-2007), it was determined that the overall cost/benefit ratio was 1:0.89, meaning that the program only returned a benefit of 89 cents for every dollar spent (total dollar quantifiable benefits of $80,551 and total costs of $90,623). It was determined that an immediate program change in the form of a tuition increase was necessary to make the program at least break even. The following year (2007-2008), the overall cost/benefit ratio increased to 1:1.72 (total dollar quantifiable benefits of $144,585 and total costs of $84,222), partly due to the tuition increase, but also because the LHD was able to hire two of the internship graduates as entry level nutrition professionals, saving significant recruiting, hiring, and training costs. Another way the cost/benefit results were used to decrease costs was with training time. It was determined that LHD staff spent many hours planning and conducting training sessions for dietetic interns each year.
There is a strong stakeholder commitment to continue the practice of annual cost/benefit analysis of the dietetic internship program as well as including training for dietetic interns on cost/benefit methodology as an important public health administrative skill. Both the Executive Director and Director of Nutrition Services at the LHD have encouraged the Dietetic Internship Director to make cost/benefit analysis one of the ways the internship program is evaluated each year. The LHD staff in both the nutrition and administrative divisions has been very helpful in providing data for the cost/benefit analysis of the internship. By giving regular feedback to staff on how these data are being used, as well as by thanking them for their contribution, they remain willing to assist in the process. The contributions of the interns in the form of their cost/benefit projects are also highly valued for the insights they bring to LHD leadership. These projects can assist in decision making regarding resources allocated to nutrition programs and services. For example, a clinic-based program in need of additional funding has benefited from an intern cost/benefit project, which has yielded information useful in securing grants to help sustain the program. In 2010, dietetic interns worked with our clinical preceptors in the community to collect data on costs and benefits of precepting one of our interns. While these data do not affect the bottom line impact of the internship on the LHD, they show our affiliating preceptors how much they give and receive in monetary terms by precepting, which we hope will increase their buy-in to provide practice experience for our interns year after year. While this practice has been very useful to the nutrition division at TCHD, cost/benefit analysis can also be extended to many other projects outside the dietetic internship and nutrition field.
The Executive Director of the LHD has expressed support for expanding this practice to uses in program evaluation (and therefore planning) of other public health services. As described previously, we have planned to continue the cost/benefit analysis of the dietetic internship on an on-going basis as well as including practice experiences with cost/benefit methodology as a regular part of the internship curriculum. Additionally, with new financial management software implemented at TCHD in 2010, we are streamlining the process of staff and intern time and activity documentation. This will make data collection and analysis easier and potentially more accurate, reducing the time necessary to pull together cost/benefit data from different sources. The activities described above in this model practice application are just the first step. Beyond the dietetic internship, we want to use what we have learned about cost/benefit analysis as a springboard to other projects. For example, we would like to utilize cost/benefit analysis on some of our worksite wellness initiatives. It would be useful to find out staff time (and therefore dollars) involved in worksite wellness activities, and if we can quantify some of the benefits gained by such activities. Another way we can use cost/benefit analysis is to look critically at some of the community outreach services we perform, such as health fairs. When cost/benefit analysis is applied to such activities, we will be better able to determine the value of the services and whether we need to increase or decrease staff time allocated to them. We may also be able to see a greater cost/benefit ratio from using newer technologies, such as web conferencing or web-based training for both staff and the public. If these new paradigms can be proven financially worthwhile through cost/benefit analysis, we may have a greater chance of obtaining funding for projects. In summary, the practice of using cost/benefit analysis can be easily taught to other public health professionals and utilized across all divisions as well as in o