PLANNING AND CONDUCTING THE ASSESSMENT

3-1
CHAPTER 3
PLANNING AND CONDUCTING THE ASSESSMENT
3.1 Chapter Summary
This chapter discusses the steps to plan and conduct a health system performance assessment
using this manual. It should be used as a compendium of best practices for managing the
logistical aspects of the assessment to make the best use of the analytical talent and skills set of
the assessment team. After the U.S. Agency for International Development (USAID) Mission
has selected the country where the assessment will take place, it will undertake the activities
described in this chapter. The chapter’s intended user is the person managing the assessment, and
the design of the chapter assumes that that assessment manager and others involved in the
assessment have desk research as well as field-based research experience.
The next chapter (Chapter 4, “Synthesizing Findings and Developing Recommendations”)
focuses on approaches for analyzing the data and assessment findings, and recommends proven
methodologies for developing, validating, and prioritizing interventions.
The following key activities are involved in planning and conducting an assessment; some of
them occur concurrently—
1. Identify the needs and priorities of the USAID Mission.
2. Agree on the scope, time frame, and dates of the assessment.
3. Prepare an assessment budget.
4. Assemble an assessment team and assign responsibilities.
5. Prepare the logistics checklist.
6. Schedule and conduct team planning meetings.
7. Compile and review background materials.
8. Prepare a contact list and interview key informants.
9. Organize a stakeholder workshop.
10. Hold a post-assessment debriefing as needed.
11. Prepare the assessment report.
Several templates and samples of documents are provided as annexes to this chapter; others are
available in the CD that accompanies this manual. Annexes for Chapter 3—
• Annex 3A. Template for Assessment Scope of Work
• Annex 3B. Sample Assessment Budget Templates (included electronically on CD only)
• Annex 3C. Sample Local Consultant Scope of Work
• Annex 3D. Sample Logistical and Task Checklist
• Annex 3E. Sample Team Planning Meeting Agenda
Health Systems Assessment Approach: A How-To Manual
3-2
• Annex 3F. Sample List of Background Documents—Desktop Review for Azerbaijan
Assessment
• Annex 3G. Sample In-Country Interview Schedule
• Annex 3H. Sample Contact List
• Annex 3I. Sample Stakeholder Workshop Agenda
• Annex 3J. Suggested Outline for Final Assessment Report
• Annex 3K. Outline of Assessment Report from Pilot Test in Angola—Angola Health
System Assessment (full report included electronically on CD that accompanies this
manual or available for download at www.healthsystems2020.org)
• Annex 3L. Outline of Assessment Report from Pilot Test in Benin—Benin Health
System Assessment (full report included electronically on CD that accompanies this
manual or available for download at www.healthsystems2020.org)
3.2 Activity 1: Identify the Needs and Priorities of the USAID Mission
Once you have decided to proceed with a rapid health system assessment, you will need to
address a few key points before moving forward—
• Review the purpose of this assessment tool and what kind of information it can provide
to the Mission (see Chapter 1, “Health Systems Strengthening: An Introduction” and
Chapter 2, “Overview of the Approach”). Make sure that the information from the
assessment will match the type of information needed by the Mission and its programs.
• Identify any special needs the Mission may have (e.g., specific areas of interest) and
determine if this assessment tool can meet that need. The tool is designed to assess
overall health system performance. It does not focus on specific health programs or
conditions, such as Integrated Management of Childhood Illness or tuberculosis. If
program-specific information is needed, the assessment organizers and the Mission
should agree on an approach to address these needs. Strategic priorities may also require
special consideration.
• Identify recent or upcoming in-country studies or activities that may be useful to the
assessment. The Mission and other organizations working in the country may be aware of
recent studies, health sector (or subsector) assessments, or other publications that may be
useful in planning and preparing for this assessment. This research will also help to
identify overlaps between the proposed assessment and any recent or future activities incountry.
• Determine which modules would be most relevant. Given the priorities and needs of
the USAID Mission, this assessment could cover all the technical modules or only a
Chapter 3. Planning and Conducting the Assessment
3-3
subset of them. Note that the core module is mandatory and must be completed regardless
of the overall scope.
• Define structure and scope of the final assessment report. Annex 3J provides a
suggested outline for the final assessment report. Discuss this report with the USAID
Mission and reach an agreement for the overall structure and scope of the report.
(Outlines of the assessment reports prepared as part of the Angola and Benin pilot tests
are included as Annexes 3K and 3L and serve as examples.)
3.3 Activity 2: Agree on the Scope, Time Frame, and Dates of the Assessment
The final scope of your work will be influenced by the following considerations.
• The overall level of effort is based on the number of modules to be applied. An estimated
two person-weeks per module will be required. This estimate is based on one week for
preparatory work and report writing plus one week for fieldwork for each module. It does
not include travel time.
• The time required will also be influenced by the number of people on the assessment
team. For example, if all seven modules will be implemented, the team could be set up
with two people who do three or four modules each, three people who do two or three
modules each, or four people who do one or two modules each. The expertise of the team
members, the ready availability of data, and type of final report requested will also
influence the time requirements. Time for translation of materials may also need to be
considered.
• The time and number of team members required will be also depend on where the
assessment will take place. Although the assessment primarily focuses on data that can be
collected at a national (central) level, you may find that conducting all or some of the
assessment at a subnational level is appropriate, particularly in decentralized systems or
in cases where information and systems must be verified at a provincial or district level.
If a provincial- or district-level visit is to be conducted, you will need to consider the
following issues.
o Site selection. Work with USAID Mission and possibly a local bilateral or
nongovernmental organization (NGO) project to identify possible locations or sites.
o Budgetary implications. Most likely, you will have travel costs associated with the
trip, and they need to be integrated into the budget. Furthermore, the level of effort
for assessing regional or lower level agencies and stakeholders, in addition to central
agencies, will imply a larger budget for interviews.
o Time implications. If the in-country travel will take several days, you may need to
extend the time of the assessment or send only part of the assessment team.
Health Systems Assessment Approach: A How-To Manual
3-4
o Technical considerations. You may need to develop field questionnaires based on the
modules to help team members ask the appropriate questions for the subnational
level.
• Any specific information needs agreed to with the Mission will need to be considered.
• Estimate the time frame in which all assessment activities will be conducted. The
availability of selected team members, holidays, and other events will determine the exact
dates of the assessment. The assessment activities are not limited to the fieldwork, but
also include time for organizational and logistic preparation, team member preparation,
and post-fieldwork.
Draft the scope of work for the assessment early in the process to help inform potential
assessment team members of their role and tasks. A template for developing the assessment
scope of work is presented in Annex 3A.
3.4 Activity 3: Prepare an Assessment Budget
You will need to prepare the budget early in the planning process. An Excel® template, which is
presented in Annex 3B (and is available on the CD that accompanies this manual), can be used to
draft the budget. It should be updated as additional information becomes available, such as
personnel daily rates and the cost of interpreters and translators, if needed. Some key
considerations for the budget are listed below.
• Team member time
o Planning time—technical lead and administrative or logistics support
o Team member time—preparatory, fieldwork, and report preparation
• Travel costs (as needed)
o Airfare
o Per diem
o Visa costs
o Telecommunications costs (phone and Internet access)
• Contracted services (as needed)
o Local consultant
o Translator(s)
o Driver(s) and car(s)
Chapter 3. Planning and Conducting the Assessment
3-5
o Conference room facilities for the stakeholder workshop (room charge, food costs,
and equipment rental)
• Other
o Photocopies for reference materials, reports, and other documents
o Postage (mailing of documents before visit, if needed)
3.5 Activity 4: Assemble an Assessment Team and Assign Responsibilities
The selection of the assessment team is a critical step in planning the assessment. Team members
may include the Mission Population, Health and Nutrition (PHN) officer, other staff, staff from
other USAID offices, in-country consultants, and external consultants. You will likely assemble
the team and assign roles and responsibilities accordingly.
The roles, qualifications, responsibilities, and estimated level of effort of each assessment
participant are described below.
3.5.1 The Assessment Coordinator’s Responsibilities
The assessment coordinator is the point person responsible for the organization and logistics of
the assessment. This person may be the Mission PHN officer or another designated person who
will work in collaboration with assessment team members, the USAID Mission, and any local
consultants. The assessment coordinator could be a member of the assessment team or could
function in a purely organizational role.
The coordinator should have experience in organizing data collection efforts and managing
consultants, and should have strong research and interpersonal skills. The person would ideally
have some familiarity with the country’s health sector, contacts with stakeholders, and advanced
command of the language of the assessment country as well as English.
A local consultant may be hired to assist with the local coordination activities if the assessment
coordinator is not based in the country. Responsibilities would need to be divided accordingly. If
a local consultant is hired to take on part of the coordination activities, a local consultant scope
of work will be necessary. A sample scope of work is supplied in Annex 3C.
Key responsibilities of the assessment coordinator are divided into preparatory work and support
to the team during fieldwork.
3.5.1.1 The Assessment Coordinator’s Preparatory Work
In advance of fieldwork, the assessment coordinator will need to do the following.
1. Prepare scopes of work, background documents, and the like.
Health Systems Assessment Approach: A How-To Manual
3-6
2. Assist in selecting the assessment team.
3. Prepare the assessment logistics checklist and budget.
4. Manage logistical preparations, including the following—
a. Interface with USAID regarding logistics for the team.
b. Assist with invitations and arrangements for the stakeholder workshop.
c. Prepare the schedule of work for the team members (each team member will have
independent and team or group meetings), including scheduling and confirming
appointments. Provide guidance on appropriate informants in the health sector.
d. Obtain quotes for mobile phone rental for the team.
e. Plan travel.
5. Organize team meetings.
6. Work with the assessment team to obtain reports and other data sources required in
advance and extract specified information.
7. Hire a local consultant (if needed).
8. Hire local translator(s) to work with the team (if needed).
9. Hire a car and driver to provide transportation for the team during the visit, including
pick-up and drop-off at the airport.
10. Provide guidance on general work protocols for the team, including regular daily working
hours (start, lunch, end), holidays, introductions, and language.
11. Establish protocols for interview note-taking, sharing notes among team members, and
report preparation templates or formats before the trip begins.
3.5.1.2 The Assessment Coordinator’s Support of the Team during Fieldwork
During the fieldwork, the assessment coordinator will need to do the following.
1. Meet with team at the start of field activities and participate in team meetings.
2. Assist the team as needed during the initial briefing meeting with USAID.
3. Assist the team to collect data as needed.
4. Interpret or translate as needed.
5. Help prepare for and participate in the stakeholder workshop.
Chapter 3. Planning and Conducting the Assessment
3-7
a. Confirm conference room arrangements (including availability of overhead digital
projector, flipchart paper, markers, notepads, and pens among others).
b. Arrange for photocopies as requested by the team.
6. Contribute to Country Health Systems Assessment Report as needed.
7. Travel to one or two provincial areas (as required).
The expected level of effort for the coordinator is a minimum of five days of preparatory work,
plus time to support the team as needed during the fieldwork. Although the preparatory work is
estimated at five days, this work would be done over a two-month period to allow time for the
various planning steps to be taken. If the assessment coordinator is also a team member, the level
of effort would need to be revised accordingly. Similarly, the level of effort may need to be
revised if delays occur.
3.5.2 The Assessment Team Leader’s Responsibilities
The assessment team leader is responsible for the overall management of team activities in the
field and for the timely completion of the assessment. The team leader will do the following—
1. Lead the team and its activities; clarify the scope and timeline with the assessment
coordinator, the team, and country counterparts.
2. Liaise with the assessment coordinator (and local consultant if needed) and the Mission
on scheduling interviews, site visits, and logistics.
3. Coordinate with the assessment coordinator and the Mission to prepare for and conduct
the stakeholder workshop.
4. Plan for daily activities during fieldwork with other team members.
5. Facilitate daily team meetings.
6. Deliver final assessment report to the USAID Mission. The team leader is likely to be the
lead author of the assessment report, although one of the other team members can take on
this role. Either way, the team leader will be responsible for finalizing the report and
delivering it to the USAID Mission.
3.5.3 The Assessment Team Members’ Responsibilities
Assessment team members should have a health-system background; knowledge of at least one
of the areas of study (e.g., health financing, pharmaceuticals, human resources, health
information systems); and preferably have the ability to speak, write, and read in the language of
the assessment country to facilitate document review and interviews. Having these language
skills will also reduce costs associated with interpretation and translation services. The
assessment team will be responsible for the following tasks listed below.
Health Systems Assessment Approach: A How-To Manual
3-8
3.5.3.1 The Team Members’ Preparatory Work
In advance of fieldwork, team members will need to do the following.
1. Read through this manual.
2. Participate in team planning meetings and discussions.
3. Work through the modules they are assigned.
4. Prepare lists of documents needed and potential interviewees to submit to the assessment
coordinator, based on the modules they are assigned.
5. Review, analyze, and understand Component 1 data for all the modules, which are
derived from established databases.
6. Review background documents and prepare the desk study (Component 2) to the degree
possible; at least some parts of each module can be completed with a desk study and the
information verified during fieldwork. Note that the core module, particularly, should be
completed as much as possible at this stage since it provides valuable background
information for the entire team.
7. Identify information gaps, based on preparatory work, that are to be filled during
fieldwork.
3.5.3.2 The Team Members’ Fieldwork
The assessment tool was designed to be implemented in-country over one person-week per
module. Note that this estimate does not include travel time and assumes that sufficient
preparatory work is completed as described above. The level of effort may be revised based on
the number of modules each team member is responsible for, the level of experience of the team
members, and the like. Key fieldwork tasks required of the team are the following.
1. Meet with team at the start of fieldwork and participate in regular team meetings.
2. Collect data on assigned module(s) through document review and interviews.
3. Prepare preliminary analyses in cooperation with team members. Draft relevant sections
for the Country Health Systems Assessment Report, including recommended potential
activity areas and interventions.
4. Prepare for and conduct a stakeholder workshop.
5. Travel to rural areas or regional and district level locations, as required.
Chapter 3. Planning and Conducting the Assessment
3-9
3.5.3.3 Report Preparation
Post-fieldwork activities will vary depending on the reporting needs of the Mission. Key postfieldwork
tasks are the following.
1. Finalize the Country Health Systems Assessment Report, including recommendations,
based on input from the stakeholder workshop and mission staff.
2. Participate in follow-up meetings, as needed.
3.6 Activity 5: Prepare the Logistics Checklist
A sample checklist of tasks and logistical steps is presented in Annex 3D. You will need to make
travel arrangements for team members not based in the country. In addition, depending on the
country and the interests of the USAID Mission, you may need to plan for trips to areas outside
of the central capital city. A local consultant may be particularly helpful in making these
arrangements.
3.7 Activity 6: Schedule and Conduct Team Planning Meetings
Before the assessment, schedule a meeting for the team to review the purpose of the assessment,
review the manual, and assign responsibilities. A second team meeting may be scheduled after
the preparatory work has been completed and before fieldwork. The focus of this meeting should
be the review of remaining information gaps and scheduling the fieldwork. At a minimum all
team members and the coordinator should be present and participate. (This meeting may be
conducted by conference call.) A sample team planning meeting agenda is presented in Annex
3E.
In addition, during the fieldwork, regular daily team meetings led by the team leader are
recommended.
3.8 Activity 7: Compile and Review Background Materials
Compile background information on the country, and in particular any general health documents,
early in the assessment process. Each module should have identified specific documents and
types of documents from which relevant information may be obtained. A sample list of
background documents that was prepared for Azerbaijan is included in Annex 3F.
The assessment coordinator should facilitate the collection of the documents and distribution to
the team members. Hard copies of key documents can be compiled in a binder or electronically
on a CD and shared at the first team meeting. Encourage team members to keep a list of all
documents consulted and provide the list as part of the assessment report.
Health Systems Assessment Approach: A How-To Manual
3-10
Information that is pulled from any document must be properly cited, so that the source of the
information can be checked later if questions are raised about it. Simple tools may be used to
manage the information that is gleaned from documents. For example, information pulled from
documents may be entered into a database to facilitate sorting by topic. Software programs that
can assist with this include Reference Manager® and MS Access.®
3.9 Activity 8: Prepare a Contact List and Interview Key Informants
Before fieldwork begins, you will need to
consult with the USAID Mission, the
assessment coordinator, and the team members
to identify key informants. Other donors and
stakeholders may be queried about potential key
informants in advance of the fieldwork, and
country reports can also provide a lot of names
of people to follow-up with. The generic titles
of likely key informants are listed in the
individual modules. When selecting the specific
individuals to be interviewed, specify the topics
and types of information that will be discussed
during the interview to make sure that the most
appropriate person will be selected.
The local consultant can assist with scheduling
the interviews. A sample interview schedule is
presented in Annex 3G. A contact list of team
members, Mission contacts, and interviewees
should also be prepared and maintained
throughout the assessment. A contact list
template is presented in Annex 3H. One of the
team members should be assigned the
responsibility of maintaining this list.
This tool assumes that the assessment team
members have some relevant field-based
research experience. Nonetheless, you may want
to remind them of good information gathering
and interviewing practices. Box 3.1 provides
some basic tips for conducting a successful
interview.
This tool does not include questionnaires to be
applied during interviews. Rather, the questions
that need to be addressed to obtain data for the
indicators are simply listed by topic without any
Box 3.1
Interview Tips
Insist on getting copies of documents
and texts. Whenever a respondent refers to
a study, policy, law, or other document, ask
for a copy, or at least a citation for the
document. If needed, get an independent
translation. Having your own copy will allow
for independent evaluation of the content of
the document and serve to confirm the
informant’s interpretation of the contents.
Use consistent questions with flexible
follow-up across all the sources
interviewed. Interviews must be designed to
get consistent information. Start with a list of
questions, and try to cover all of them in the
interview. In particular, when both the
provider and patient are being interviewed,
be sure to cover the same topics with each.
Seek information from multiple
perspectives. For many reasons, different
parties may perceive the same situation in
different ways. An informant may be a great
distance from the reality on the ground.
Some informants may not be exposed to
what is actually happening, or may only feel
comfortable speaking to the ideal, or the way
things should be.
Document interview notes promptly.
Document your interview notes every night. If
your team splits up to interview different
informants, you can share your experiences
through the notes. The notes then become
an important resource as the team prepares
the final report.
Source: Ravenholt and others (2005).
Chapter 3. Planning and Conducting the Assessment
3-11
particular order with respect to the most likely respondent. In advance of the field visit, and as
part of the documentation review, team members should draft country and site-specific (e.g.,
central versus regional) interview guides according to most likely respondent. Careful
preparation will help avoid duplication of questions to the same individual and will also ensure
that the sequence of the questions asked will be logical.
3.10 Activity 9: Organize a Stakeholder Workshop
The stakeholder workshop will be the final fieldwork activity for the assessment team. The
stakeholder workshop is intended to be a forum in which stakeholders can—
• Review, discuss, and validate team’s major findings
• Provide input on their priorities, based on strengths and weaknesses discussed
• Provide input on the team’s recommendations
• Identify how they will or can be involved in follow-up activities, how to move forward,
or how to provide feedback and recommendations on major options presented by the
team
Organizing the workshop is the responsibility of the team leader in coordination with the USAID
Mission. Key activities include the following.
• Identify invitees, set the agenda, and confirm dates.
• Send invitations.
• Reserve a location, such as a hotel conference room, and plan for coffee breaks (best
done in advance or immediately upon arrival in-country).
• Reserve audiovisual equipment and procure other supplies such as flipcharts and markers
(also best done in advance or immediately upon arrival in-country).
• Prepare presentations and handouts for the workshop.
• Meet with USAID before the workshop to review draft findings and agenda.
An example of a stakeholder workshop agenda is included in Annex 3I.
3.11 Activity 10: Hold a Post-Assessment Debriefing as Needed
In addition to the stakeholder workshop, the Mission may request a debriefing meeting after
fieldwork is completed. This meeting may also be requested by USAID Washington depending
on the availability of team members.
Health Systems Assessment Approach: A How-To Manual
3-12
3.12 Activity 11: Prepare the Assessment Report
Assessment team members should start drafting their findings early, during preparatory and
fieldwork. The assessment findings, recommendations, and discussion in the stakeholder
workshop should be documented in the final report. Each module chapter contains guidance on
summarizing findings. Annex 3J includes a suggested outline for the final report. Annexes 3K
and 3L include outlines of the assessment reports prepared for the Angola and Benin pilot tests
and serve as examples. Team members will need to agree on a timeline and approach for
finalizing and disseminating the report, in consultation with the USAID Mission.
Reference
Ravenholt, Betty, Rich Feeley, Denise Averbug, and Barbara O’Hanlon. 2005. Navigating
Uncharted Waters: A Guide to the Legal and Regulatory Environment for Family Planning
Services in the Private Sector. Bethesda, MD: Private Sector Partnerships-One Project, Abt
Associates Inc.
Chapter 3. Planning and Conducting the Assessment
3-13
Annex 3A. Template for the Assessment of Scope of Work
SCOPE OF WORK
Health Systems Assessment Approach [Country]
Background
USAID’s Office of Health, Infectious Diseases and Nutrition (HIDN) has developed a Health
Systems Assessment Approach as part of its global Mainstreaming Health Systems
Strengthening Initiative. The Approach is meant to serve the following purposes:
• Allow Population, Health and Nutrition (PHN) officers from USAID (with the assistance
of experts/consultants if necessary) to conduct an assessment of a country’s health
system. This includes diagnosing the relative strengths and weaknesses of the health
system, prioritizing key weakness areas, and identifying potential solutions. This may be
particularly relevant during early phases of program development.
• Inform PHN officers about the basic elements and functions of health systems.
• Improve the capacity of bilateral projects to achieve USAID’s health impact objectives
through increased use of health systems interventions.
• Help health systems officials at USAID to conceptualize key issues, increase the use of
health systems interventions in technical programs, and to improve the role of the Health
Systems Division to support these programs.
The assessment tool covers the following components of the health system—governance; health
financing; human resources and health facilities; pharmaceutical supply system; and health
information systems. More details on the assessment methodology and topical areas are in
Annexes 1 and 2.
The technical team will be composed of three team members plus a local consultant. The team will
include a Team Leader from [organization] and other team members from [organizations].
The team will be assisted by the USAID PHN officer in [country]. The assessment team will review
documents and conduct interviews to gather specific information on the health system in [country].
The visit will conclude with a brief workshop with USAID representatives and other key
stakeholders.
Overall Scope of Work
Assessment:
• Systematically assess strengths and weaknesses of the health system using health system
assessment tool. The tool is designed to provide a broad assessment of the performance of the
health sector. Topics to be covered are governance, health financing, human resources and
health facilities, pharmaceuticals, and health information systems (see Annex 1).
• Provide general recommendations on potential activity areas for health system strengthening
following from the strengths and weaknesses identified in the assessment.
Health Systems Assessment Approach: A How-To Manual
3-14
• Conduct a stakeholder workshop at the end of the assessment visit to build consensus on what
the key health system priority areas are.
• Liaise with the new PHN bilateral program in [country] to share findings and information.
• The assessment will not evaluate disease- or program-specific areas as the tool is not designed
to do so. However, given the Mission’s interest in tuberculosis and family planning issues, the
team may be able to provide information as available that may be relevant to those two areas.
Tasks of Assessment Team Members
The allocation of tasks among team members will be discussed at team planning meetings.
Prior to team arrival (LOE: expected 5 days)
1. Participate in team planning meetings and discussions.
2. Review assigned module(s) and discuss any questions with module authors.
3. Review background data (Component 1 data will be compiled by [organization]).
4. Prepare a draft donor map based on a review of available documents.
5. Prepare lists of documents needed and potential interviewees (entire team). The lists will be
provided to the local consultant who will compile the documents and facilitate translation as
needed.
6. Review background documents and prepare the desk study (Component 2) to the degree
possible. This activity will be supported by the local consultant who will work to obtain
reports and other data sources required in advance and extract specified information.
During team visit (LOE: expected 15 days)
1. Meet with team upon arrival and participate in team planning meeting.
2. Collect Component 2 data through document review and interviews.
3. Assist in mapping current interventions/reforms to address weaknesses identified in
assessment.
4. Prepare preliminary analyses in cooperation with team members. Draft relevant sections for
the Country Health Systems Assessment Report, including recommended potential activity
areas and interventions.
5. Prepare and conduct stakeholder workshop.
6. Liaise with USAID PHN officer as needed to prepare for the stakeholder workshop and other
activities.
7. Liaise with new health bilateral program personnel to share and discuss findings.
8. Provide input as part of the pilot test with regard to approach/methodology, indicators,
timeline, level of effort, and format.
9. Travel to one rural area, to be determined, may be required. It is expected to be a brief trip.
10. Work will be conducted in [language], and will be assisted by translators as needed.
The team will work under the overall direction of the Team Leader. All team members will
contribute to day-to-day problem solving, solutions to issues of data availability, technical
questions, etc. This may require daily team meetings and other updates while in [country].
Chapter 3. Planning and Conducting the Assessment
3-15
Post in-country visit (LOE: expected 5 days)
1. Review any final comments received from the Mission and local counterparts.
2. Make corrections and adjustments to report for finalization.
Outputs
1. Stakeholder workshop report
2. Country Health Systems Assessment Report (draft outline will be provided)
The deliverables will be prepared in English but may be translated into relevant local language if
requested.
Annex 1. Outline of the Health Systems Assessment Approach
Introduction
This chapter provides the motivation for and the purpose of the approach. It also describes the
layout of the product (the manual and the CD). This section will draw from the framework paper
previously presented to USAID (“Health systems assessment approach: draft framework”).
Table of Contents
Chapter 1: Health Systems Strengthening: An Introduction
This is a background chapter explaining health systems and discussing their key functions. This
chapter serves as an informational piece for those less familiar with health systems. The chapter
builds on the paper written for the Child Survival Technical Resource Materials (TRM) on
Health Systems Strengthening. The chapter also provides a reference list for additional papers on
health systems.
Chapter 2: Overview of the Approach
This chapter describes the framework for the approach, listing and explaining the structure of the
technical modules and their components. The approach draws from the framework paper
previously presented to USAID (“Health systems assessment approach: draft framework”). An
annex provides a list of all the indicators and qualitative questions in each module, grouped by
topical area.
Health Systems Assessment Approach: A How-To Manual
3-16
Chapter 3: Planning the Assessment
This chapter provides guidelines for planning the assessment process, including—
• Identifying needs and priorities of the USAID Mission—this is so that the assessment can
appropriately focus on the right issues and help provide recommendations to the Mission.
• Time frame/schedule for the planning process and the in-country assessment
• Budgeting for the assessment
• Guidelines on how to select the assessment team (e.g., types of consultants to be
recruited, how many)
• Terms of reference (TOR)for staff and consultants for assessment team
• Agenda for assessment team planning meetings
• Types of documents to be reviewed before beginning the assessment and during the
assessment phase
• Types of stakeholder interviews to schedule
• Identifying districts/provinces to visit outside the central capital area
• Organizing the stakeholder workshop, including purpose of the workshop, suggested
agenda for the workshop, and a template for presenting findings
• Logistics checklist for planning the assessment, including the stakeholder workshop
• Overview of the assessment report that should be prepared using this approach; annexes
provide a suggested outline and outlines from two completed reports from prior
assessments as samples
Chapter 4: Synthesizing Findings and Developing Recommendations
This chapter includes guidelines on how to process, analyze, and interpret the findings from each
module, with particular attention to synthesizing these findings across all modules. Focus is on
how to identify key strengths and weaknesses of the health system, and how to identify root
causes of problems to be addressed. Guidelines are also provided for how to develop
recommendations for the Mission and how to link the recommendations to the USAID Mission’s
overall goals and priorities, including (to the extent possible) those of its bilateral projects. It will
address strategic objective (SO)-specific goals as well those related to the fragile state
framework.
Chapter 5: Core Module
This is the background/foundation module and will be required to be completed by all users. In
particular, if any users are planning to work through only a subset of the technical modules
(Chapters 6–11), they would need to complete this core module to understand the basic
background information about the country and its health systems.
Component 1: This includes basic demographic, health, and socioeconomic indicators for the
country. Data for the indicators is provided in an electronic format on the CD provided with this
manual (data file titled “Component 1 data”). Data for regional and income peer country
comparisons are also provided in the data file.
Chapter 3. Planning and Conducting the Assessment
3-17
Component 2: This will not be solely based on indicators as in the case of the other technical
modules (see below for Chapters 6–11). This section focuses on developing some basic
understanding and profiles of a country’s health system. Topics covered include:
• Political and macroeconomic environment: Provides guidance on how to describe the
political structure of the country.
• Business environment and investment climate: Provides sources of information and
guidance on how to analyze the factors that affect private investment and enterprise
growth and to identify the barriers to sustaining and expanding the private sector.
• Top causes of mortality and morbidity: These data are to be collected in-country and
could help guide any disease-specific recommendations to the USAID Mission. In
addition to the top causes of morbidity and mortality, prevalence rates for HIV/AIDS and
malaria will be collected, if important in the country context. Note that the health systems
assessment approach does not have a disease specific focus, but a user may have to
address this in developing recommendations for the USAID Mission.
• Structure of the main government and private organizations involved in the health care
system: This includes a template for developing a Ministry of Health (MOH)
organizational chart to help support the assessment process.
• Decentralization: This includes indicators to understand the level of decentralization in
the country—this will be important for determining the type of assessment that should be
conducted.
• Service delivery organization: This section provides an overview of the structure of
service delivery, including types of health facilities in the country, and of the engagement
of the private sector, including proportion of services and facilities in the private sector
and involvement of NGOs and the commercial sector.
• Donor mapping: This includes a template for mapping donor activities in the health
sector—this will be important for understanding the level of activities in the country, as
well as to identify gaps.
• Donor coordination: This includes indicators for assessing the level of donor coordination
and the related strengths and weaknesses.
Chapters 6–11: Topical Chapters—the Technical Modules
Chapter 6: Governance module addresses the information assessment capacity of the health
system, policy formulation and planning, social participation and health system responsiveness,
accountability, and regulation.
Chapter 7: Health financing module covers sources of financial resources; the pooling and
allocation of health funds, including government budget allocation and health insurance; and the
process of purchasing and proving payments.
Chapter 8: Service delivery module examines service delivery outputs and outcomes; the
availability, access, utilization, and organization of service delivery; quality assurance of
healthcare; and community participation in service delivery.
Health Systems Assessment Approach: A How-To Manual
3-18
Chapter 9: Human resources module covers systematic workforce planning, HR policies and
regulation, performance management, training/education and incentives.
Chapter 10: Pharmaceuticals management module evaluates the health system’s pharmaceutical
policy, laws, regulations; selection of pharmaceuticals; procurement, storage, and distribution;
appropriate use and availability of pharmaceuticals; access to quality pharmaceutical products
and services; and financing mechanisms for pharmaceuticals.
Chapter 11: Health information systems (HIS) module reviews the current operational HIS
components; the resources, policies and regulations supporting the HIS; data availability,
collection, and quality; and, analysis and use of health information for health systems
management and policy-making.
These chapters include technical modules, each with a set of indicators for conducting a health
system assessment. The key elements of each module are—
• System profile: This section provides guidelines for developing a basic profile of the
health system aspect assessed in each module. It includes templates for doing this, such
as mapping tools, flowcharts, etc.
• Component 1: This component includes indicators for which data are easily available
from international datasets. Data for Component 1 indicators is provided in an electronic
format on the CD provided with this manual. Specific attention will be given to including
regional or other peer country comparisons wherever feasible. Charts indicating possible
ways of presenting the data will also be included in an annex.
• Component 2: This component presents the indicators grouped by subtopic within each
module. Each indicator will be linked to one of five performance criteria: equity,
efficiency, access, quality, and sustainability. Users will have to conduct a combination
of desk review of documents and stakeholder interviews to collect data for these
indicators. Detailed descriptions of each indicator will be included (a template and
guidelines have been provided to chapter authors).
• Assessment process: Each chapter provides module-specific guidelines on the process for
working through each module, synthesizing findings and preparing recommendations for
interventions. These guidelines are meant to complement Chapter 4.
Chapter 3. Planning and Conducting the Assessment
3-19
Annex 3B. Sample Assessment Budget Templates
Note: Additional lines and items can be added to this template as needed. This template is available in MS Excel format on the accompanying CD.
Line Item Rate Unit Quantity Total
(Rate x Quantity)
Labor (add lines for as many people as needed)
Name Title $ /day # days $
Name Title $ /day # days $
Name Title $ /day # days $
Name Title $ /day # days $
Name Title $ /day # days $
Subtotal US labor $ Subtotal
Travel
Travel – airfare Destination $ /trip # fares at that rate $
Travel – airfare Destination $ /trip # fares at that rate $
Travel – airfare Destination $ /trip # fares at that rate $
Per diem Destination $ /days # days $
Per diem Destination $ /days # days $
Per diem Destination $ /days # days $
Other costs—local travel Destination $ /trip # $
Other costs—visa $ /trip # $
Other costs—misc. $ /trip # $
Subtotal travel $ Subtotal
Subcontracts/Outside services
Conference room Stakeholder workshop $ /day # days $
Coffee service Stakeholder workshop $ /person # people $
Audiovisual equipment Stakeholder workshop $ /day # days $
Driver and car $ /day # days $
Translators $ /day # days $
Subtotal Subcontracts $ Subtotal
Other costs
Postage $ $
Communications $ $
Other $ $
Subtotal Other $ Subtotal
Total Assessment Budget $ (Sum of Subtotals)
Health Systems Assessment Approach: A How-To Manual
3-20
Annex 3C. Sample Local Consultant Scope of Work
SCOPE OF WORK
Local Short Term Consultant
Health Systems Assessment Approach [Country]
Draft Month, Day, Year
Background
USAID’s Office of Health, Infectious Diseases and Nutrition (HIDN) seeks to develop a Health
Systems Assessment Approach as part of its global Mainstreaming Health Systems
Strengthening Initiative. The approach is meant to serve the following purpose:
• Allow Population, Health and Nutrition (PHN) officers from USAID to conduct an
assessment of a country’s health system, possibly during early phases of program
development (with the assistance of experts/consultants if necessary). This includes
diagnosing the relative strengths and weaknesses of the health system, prioritizing key
weakness areas, and identifying potential solutions.
• Inform PHN officers about the basic elements and functions of health systems.
• Help improve the capacity of bilateral projects to achieve USAID’s health impact
objectives through increased use of health systems interventions.
• Help health systems officials at USAID to conceptualize key issues, increase the use of
health systems interventions in technical program interventions, and to improve the role
of the Health Systems Division.
The assessment tool covers the following components of the health system:
stewardship/governance; health financing; human resources and health facilities;
pharmaceuticals; private sector engagement; and health information systems. More detail is on
the assessment methodology is in Annexes 1 and 2.
The technical team will be comprised of:
1. Team Leader (organization)
2. Technical specialist (organization)
3. Technical specialist (organization)
4. Local short-term consultant (contracted through [organization])
The team will be assisted by the USAID PHN officer in [country]. As part of the pilot test the
assessment team will review documents and conduct interviews to gather specific information on
the health system in [country]. The visit will conclude with a brief workshop with USAID
representatives and other key participants.
Objective of the Technical Assistance (Local consultant)
The local, short-term consultant will work with the technical team to identify relevant sources of
data for the assessment, obtain data and documents, and assist in document review. Further, the
consultant will assist the team with coordinating the program of visits, facilitating access to key
Chapter 3. Planning and Conducting the Assessment
3-21
informants (setting up interviews and meetings), participating in the data collection activities,
providing translation, and assuring that local technical and logistic needs are met in a timely and
effective way. The local consultant will be expected to help the team members who will speak
English and [language] to interact with counterparts in [language].
Expected Specific Tasks
[insert dates]
Prior to team arrival (LOE: minimum 5 days)
1. Work with technical team to obtain reports and other data sources required in advance
and extract specified information. This will assist the team with collecting data for Level
2 and 3 of the assessment (see Annex 1). Lists of the types of documents needed will be
provided closer to the team visit.
2. Manage logistical preparations:
a. Interface with USAID regarding logistics for the team.
b. Obtain quotes for mobile phone rental for team.
c. Assist with invitations and arrangements for a workshop to be held on the last
day of the visit.
d. In consultation with [organization], prepare the schedule of work for the team
members (each team member will have independent meetings and team or group
meetings), including scheduling and confirming appointments. Provide guidance
on appropriate informants in the health sector.
e. Provide other logistical support as needed.
3. Coordinate with and/or hire local translator(s) to work with the team to translate from
[language] to English. The number of translators will depend on team requirements
Translators would
f. Accompany team members on interviews to provide interpretation services
g. Review and translate documents are required
4. Provide guidance on general work protocols for the team, including regular daily working
hours (start, lunch, end), holidays, introductions, language, etc.
5. Hire car and driver to provide transportation for the team during the two-week visit,
including pick-up and drop-off at the airport.
During team visit (LOE: expected 15 days)
1. Meet with team upon arrival and participate in team planning meeting.
2. Assist team as needed during initial briefing meeting with USAID.
3. Assist team to collect Level 2 and Level 3 data (see description of the tool in Annex 1)
4. Interpret/translate as needed in [language]. Work with other translators as needed.
5. Contribute to preparations, and participate in the stakeholder workshop. Confirm
conference room arrangements (including availability of overhead digital projector, flip
chart paper, markers, notepads and pens, among others). Arrange for photocopies as
requested by the team.
6. Provide input on the pilot test process.
7. Draft relevant sections for the Country Health Systems Assessment Report, including
recommended solutions.
Health Systems Assessment Approach: A How-To Manual
3-22
8. Travel to one rural area, to be determined, may be required. It is expected to be a brief
trip.
A more specific list of tasks with dates will be provided when the dates of the visit are
confirmed. The Team will work under the overall direction of the Team Leader. All team
members will contribute to day-to-day problem solving, solutions to issues of data availability,
technical questions, etc.
Consultant Profile
The following background and experience are required.
• Familiarity with the health sector as a health professional in medicine, public health,
health financing/economics, or health services administration
• Experience in evaluation and/or health systems research, preferably at national level
• Excellent quantitative and qualitative skills
• Experience working in health sector in [country]
• Advanced command of [language] and advanced reading, writing, and speaking skills in
English
• Ability to work in teams
• Helpful to have familiarity and contacts in the ministry of health, private sector, and/or
donor community
Outputs
The reports will be prepared in English. Reporting deadlines will be specified when the
assessment schedule is finalized.
Contact Information

Attachments:
• Annex 1: Brief description of the assessment tool
Chapter 3. Planning and Conducting the Assessment
3-23
Annex 1. Outline of the Health Systems Assessment Approach
Introduction
This chapter provides the motivation for and the purpose of the approach. It also describes the
layout of the product (the manual and the CD). This section will draw from the framework paper
previously presented to USAID (“Health systems assessment approach: draft framework”).
Table of Contents
Chapter 1: Health Systems Strengthening: An Introduction
This is a background chapter explaining health systems and discussing their key functions. This
chapter serves as an informational piece for those less familiar with health systems. The chapter
builds on the paper written for the Child Survival Technical Resource Materials (TRM) on
Health Systems Strengthening. The chapter also provides a reference list for additional papers on
health systems.
Chapter 2: Overview of the Approach
This chapter describes the framework for the approach, listing and explaining the structure of the
technical modules and their components. The approach draws from the framework paper
previously presented to USAID (“Health systems assessment approach: draft framework”). An
annex provides a list of all the indicators and qualitative questions in each module, grouped by
topical area.
Chapter 3: Planning the Assessment
This chapter provides guidelines for planning the assessment process, including—
• Identifying needs and priorities of the USAID Mission so that the assessment can focus
appropriately on the right issues and help provide recommendations to the Mission
• Time frame/schedule for the planning process and the in-country assessment
• Budgeting for the assessment
• Guidelines on how to select the assessment team (e.g., types of consultants to be
recruited, how many)
• TOR for staff and consultants for assessment team
• Agenda for assessment team planning meetings
• Types of documents to be reviewed before beginning the assessment and during the
assessment phase
• Types of stakeholder interviews to schedule
• Identifying districts/provinces to visit outside the central capital area
Health Systems Assessment Approach: A How-To Manual
3-24
• Organizing the stakeholder workshop, including purpose of the workshop, suggested
agenda for the workshop, and a template for presenting findings
• Logistics checklist for planning the assessment, including the stakeholder workshop
• Overview of the assessment report that should be prepared using this approach; annexes
provide a suggested outline as well as outlines from two completed reports from prior
assessments as samples.
Chapter 4: Synthesizing Findings and Developing Recommendations
This chapter includes guidelines on how to process, analyze, and interpret the findings from each
module, with particular attention to synthesizing these findings across all modules. Focus is on
how to identify key strengths and weaknesses of the health system, and how to identify root
causes of problems to be addressed. Guidelines are also provided for how to develop
recommendations for the mission and how to link the recommendations to the USAID mission’s
overall goals and priorities, including (to the extent possible) those of its bilateral projects. It will
address SO-specific goals as well those related to the fragile state framework.
Chapter 5: Core Module
This is the background/foundational module and will be required to be completed by all users. In
particular, if any users are planning to work through only a subset of the technical modules
(Chapters 6–11), they would need to complete this core module to understand the basic
background information about the country and its health systems.
Component 1: This includes basic demographic, health, and socio-economic indicators for the
country. Data for the indicators is provided in an electronic format on the CD provided with this
manual (data file titled “Component 1 data”). Data for regional and income peer country
comparisons are also provided in the data file.
Component 2: This will not be solely based on indicators as in the case of the other technical
modules (see below for Chapters 6–11). This section focuses on developing some basic
understanding and profiles of a country’s health system. Topics covered include:
• Political and macroeconomic environment: Provides guidance on how to describe the
political structure of the country.
• Business environment and investment climate: Provides sources of information and
guidance on how to analyze the factors that affect private investment and enterprise
growth, and to identify the barriers to sustaining and expanding the private sector.
• Top causes of mortality and morbidity: These data are to be collected in-country and
could help guide any disease-specific recommendations to the USAID Mission. In
addition to the top causes of morbidity and mortality, prevalence rates for HIV/AIDS and
malaria will be collected, if important in the country context. Note that the health systems
assessment approach does not have a disease specific focus, but a user may have to
address this in developing recommendations for the USAID Mission.
• Structure of the main government and private organizations involved in the health care
system: This includes a template for developing a MOH organizational chart to help
support the assessment process.
Chapter 3. Planning and Conducting the Assessment
3-25
• Decentralization: This includes indicators to understand the level of decentralization in
the country—this will be important for determining the type of assessment that should be
conducted.
• Service delivery organization: This section provides an overview of the structure of
service delivery, including types of health facilities in the country, and of the engagement
of the private sector, including proportion of services and facilities in the private sector
and involvement of NGOs and the commercial sector.
• Donor mapping: This includes a template for mapping donor activities in the health
sector -this will be important for understanding the level of activities in the country, as
well as to identify gaps.
• Donor coordination: This includes indicators for assessing the level of donor coordination
and the related strengths and weaknesses.
Chapters 6–11: Topical Chapters – the Technical Modules
Chapter 6: Governance module addresses the information assessment capacity of the health
system, policy formulation and planning, social participation and health system responsiveness,
accountability, and regulation.
Chapter 7: Health financing module covers sources of financial resources; the pooling and
allocation of health funds including government budget allocation and health insurance; and the
process of purchasing and proving payments.
Chapter 8: Service delivery module examines service delivery outputs and outcomes; the
availability, access, utilization, and organization of service delivery; quality assurance of
healthcare; and community participation in service delivery.
Chapter 9: Human resources module covers systematic workforce planning, HR policies and
regulation, performance management, training/education, and incentives.
Chapter 10: Pharmaceuticals management module evaluates the health system’s pharmaceutical
policy, laws, regulations; selection of pharmaceuticals; procurement, storage, and distribution;
appropriate use and availability of pharmaceuticals; access to quality pharmaceutical products
and services; and financing mechanisms for pharmaceuticals.
Chapter 11: Health information systems module reviews the current operational HIS
components; the resources, policies and regulations supporting the HIS; data availability,
collection, and quality; and, analysis and use of health information for health systems
management and policy-making.
These chapters include technical modules, each with a set of indicators for conducting a health
system assessment. The key elements of each module are—
Health Systems Assessment Approach: A How-To Manual
3-26
• System profile: This section provides guidelines for developing a basic profile of the
health system aspect assessed in each module. It includes templates for doing this, such
as mapping tools, flowcharts.
• Component 1: This component includes indicators for which data are easily available
from international datasets. Data for component 1 indicators is provided in an electronic
format on the CD provided with this manual (data file titled “Component 1 data”).
Specific attention will be given to including regional or other peer country comparisons
wherever feasible. Charts indicating possible ways of presenting the data will also be
included in an Annex.
• Component 2: This component presents the indicators grouped by subtopic within each
module. Each indicator will be linked to one of five performance criteria: equity,
efficiency, access, quality and sustainability. Users will have to conduct a combination of
desk review of documents and stakeholder interviews to collect data for these indicators.
Detailed descriptions of each indicator will be included (a template and guidelines have
been provided to chapter authors).
• Assessment process: Each chapter provides module-specific guidelines on the process for
working through each module, synthesizing findings and preparing recommendations for
interventions. These guidelines are meant to complement Chapter 4.
Chapter 3. Planning and Conducting the Assessment
3-27
Annex 3D. Sample Logistical and Task Checklist
Indicate who will be responsible for
completing the task, the expected
due date, and when it was completed USAID/DC
USAID/
Mission
Coordinator
Local
Consultant
Team
Lead
Team
Members
Date
Due
Date
Completed
Preparatory Work
General coordination
Identify scope of assessment and how
many modules will be completed
Identify team composition
Set dates for the assessment—consider
relevant holidays and events
Schedule meeting with USAID Mission
regarding intent and timing of
assessment
Prepare scopes of work (team and local
consultant, as needed)
Schedule and participate in team
planning meeting(s) and discussions
Determine if in-country travel will be
required
Module prep work
Prepare briefing binder for first team
meeting with country information,
background materials, and other
assessment information
Assign modules to team members
Health Systems Assessment Approach: A How-To Manual
3-28
Indicate who will be responsible for
completing the task, the expected
due date, and when it was completed USAID/DC
USAID/
Mission
Coordinator
Local
Consultant
Team
Lead
Team
Members
Date
Due
Date
Completed
Team members review assigned
module(s) and prepare lists of
documents needed and potential
interviewees
The assessment coordinator compiles
needed documents and facilitate
translation as needed
Compile Component 1 data (provide on
CD)
Complete Core Module
Review background document and
initiate Component 2 (desk study)
Request organizational charts for
central level MOH and relevant
departments; each team member
should identify departments relevant to
their module and provide the
information to the assessment
coordinator
Logistics/other preparations
Contract local consultant, if needed;
assign responsibilities
Prepare contact list
Prepare interview schedule
Make travel arrangements
Chapter 3. Planning and Conducting the Assessment
3-29
Indicate who will be responsible for
completing the task, the expected
due date, and when it was completed USAID/DC
USAID/
Mission
Coordinator
Local
Consultant
Team
Lead
Team
Members
Date
Due
Date
Completed
Identify local travel options—select
location and date
Identify participants for stakeholder
workshop; set time and date and send
invitations; reserve room; work with
Mission to coordinate and set agenda
Hire translators
Hire drivers
Materials for travel: memory sticks, flip
charts, markers, name tags, paper,
portable printer
Field work
Week 1
Meet with team and participate in team
planning meeting
Confirm or re-schedule interviews
Daily: Team members review data
collected and identify gaps; identify
additional interviews required, if any,
and schedule with consultant; document
names/titles of all people interviewed.
Collect additional information needed to
complete Component 2 through
document review and interviews
Health Systems Assessment Approach: A How-To Manual
3-30
Indicate who will be responsible for
completing the task, the expected
due date, and when it was completed USAID/DC
USAID/
Mission
Coordinator
Local
Consultant
Team
Lead
Team
Members
Date
Due
Date
Completed
Using SWOT analysis and root cause
methodologies (in Chapter 4), map
possible interventions/reforms to
address weaknesses identified in
assessment.
Prepare preliminary analyses and draft
relevant sections for the Country Health
Systems Assessment Report, including
recommended potential activity areas
and interventions
Week 2
Daily: Team members review data
collected and identify gaps
Work on draft report
Schedule and conduct follow-up
interviews as needed
Liaise with USAID PHN officer as
needed to prepare for the stakeholder
workshop and other activities
Prepare and conduct stakeholder
workshop
Request feedback from a designated
reviewer on draft report
Ongoing
Chapter 3. Planning and Conducting the Assessment
3-31
Indicate who will be responsible for
completing the task, the expected
due date, and when it was completed USAID/DC
USAID/
Mission
Coordinator
Local
Consultant
Team
Lead
Team
Members
Date
Due
Date
Completed
Liaise with any in-country program
personnel to share and discuss findings
Travel to one provincial area may be
required
Post-field work
Finalize relevant sections for the
Country Health Systems Assessment
Report, including recommendations,
based on input from the stakeholder
workshop and mission staff
Schedule/conduct any requested
debriefing meetings
Distribute report in some form--print /CD
version
Health Systems Assessment Approach: A How-To Manual
3-32
Annex 3E. Sample Team Planning Meeting Agenda
Angola Team Planning Meeting
July 18, 2005
Objectives
• Clarify roles/responsibilities, including assignment of modules
• Agree on schedule/SOW while in Angola
• Agree on role of team leader
• Discuss how to work together
Opening, introductions, overview of day; guidelines for working together
What are you looking forward to in-country:
• Completed a good job
• The team has identified strengths and weaknesses for the Mission
• Testing the new tool in the country
Expectations:
• Help team feel more comfortable with the process and workload
• Have a plan for next two weeks before Angola trip
• Get clarity about specific—report, workshop, day-to-day schedule
• Sorting out R&R, making it useful—who, how, methods
• Define my role, know what to include in report
• How people in PHRplus/HQ can help team
Guidelines for working together:
• Stay focused on topic
• Keep time—assign time checker
• Seek closure today; while in-country be comfortable with not having definitive answers
to all questions
• Develop action points/to-do list
Update on current status of activity
Roles and responsibilities for preparation of report
Chapter 3. Planning and Conducting the Assessment
3-33
Draft Report Writing Assignments:
Chapter Author(s) Page Length Due Date
1. Executive summary
2. Background
3. Overview of country’s
health system
4. Methodology
5. Strengths and weaknesses
of the health system
5.1. Stewardship
5.2. Health financing
5.3. Human resources and
health facilities
5.4. Private sector
engagement
5.5. Pharmaceuticals and
supplies
5.6. Health information
systems
6. Summary
7. Options for USAID
8. Conclusions/executive
summary/next steps
Health Systems Assessment Approach: A How-To Manual
3-34
Annex 3F. Sample List of Background Documents—Desktop Review for
Azerbaijan Assessment
AZERBAIJAN DOCUMENT LINKS (2000-2005)
USAID/U.S. Government
USAID Country Profile: Azerbaijan
http://www.usaid.gov/locations/europe_eurasia/countries/az/azerbaijan.pdf
USAID/Caucacus/Azerbaijan PHC Assessment (2005)
http://pdf.dec.org/pdf_docs/PNADC991.pdf
USAID Azerbaijan Annual Report (2005)
http://pdf.dec.org/pdf_docs/PDACD919.pdf
USAID Azerbaijan Health Statistical Report (2004)
http://pdf.dec.org/pdf_docs/PNADC004.pdf
State Department Background Notes, Azerbaijan (October 2005)
http://www.state.gov/r/pa/ei/bgn/2909.htm
World Bank
World Bank (WB) Health Sector Assessment (2005)
http://wwwwds.
worldbank.org/servlet/WDS_IBank_Servlet?stype=AllWords&all=31468&ptype=sSrch&pc
ont=results&sortby=D&sortcat=D&x=10&y=5
International Monetary Fund (IMF)/WB Azerbaijan Country Report on Millenium Development
Goals (2003)
http://www1.worldbank.org/prem/poverty/strategies/cpapers/cr04322.pdf
IMF/WB Assessment of Poverty Reduction Strategies (2004)
http://www.imf.org/external/pubs/ft/scr/2004/cr04323.pdf
WB Country Procurement Assessment Report (2003)
http://wwwwds.
worldbank.org/servlet/WDSContentServer/WDSP/IB/2003/09/30/000112742_20030930122
244/Rendered/PDF/267780AZ.pdf
WHO/UN System
EURO/WHO report: Health Care Systems in Transition Azerbaijan (2004) by John Holley
http://www.euro.who.int/Document/E84991.pdf
UNICEF Multiple Indicator Cluster Survey Azerbaijan Assessment report and data tables (2000)
http://www.childinfo.org/MICS2/newreports/azerbaijan/azerbaijan.htm
Chapter 3. Planning and Conducting the Assessment
3-35
UN Economic Commission for Europe Azerbaijan Environmental Performance Review (2003)
http://www.unece.org/env/epr/studies/azerbaijan/welcome.htm
UNICEF Micronutrient Deficiency briefing paper (no date)
http://www.micronutrient.org/VMD/CountryFiles/AzerbaijanDAR.pdf
UNICEF Child Protection Systems in Azerbaijan Report (2005)
http://www.unicef.org/azerbaijan/AZ_ChildProtection_map_report.doc
Azerbaijan Government/NGO/Background
Azerbaijan Development Gateway (no date)
http://www.gateway.az/eng/webdir/health.shtml
Azerbaijan MOH portal (information on programs, donors, health statistics, etc.—no date)
http://www.mednet.az/
State Statistical Committee of the Azerbaijan Republic
http://www.azstat.org/indexen.php
Other
DevTech Gender Assessment (2004)
http://www.usaid.gov/our_work/cross-cutting_programs/wid/pubs/ga_azerbaijan.pdf
AIHA/Virginia Commonwealth University Azerbaijan Project Summary (2004)
http://www.aiha.com/index.jsp?sid=1&id=966&pid=10
Asian Development Bank National Immunization Program Financing Assessment (2002)
http://www.adb.org/Documents/Books/Natl_Immunization/AZE/azerbaijan.pdf
Asian Development Bank Azerbaijan Country Strategy and Program Update 2004-2006 (2003)
http://www.adb.org/Documents/CSPs/AZE/2003/CSP_AZE_2003.pdf
UMCOR Azerbaijan health program webpage (no date)
http://gbgm-umc.org/umcor/ngo/azerbaijan/
Transparency International Country Corruption Assessment: Public Opinion Survey (2004)
http://www.transparency-az.org/files/25.pdf
Organization for Security and Co-operation in Europe Assessment on Freedom in the Media
(2005)
http://www1.osce.org/documents/rfm/2005/07/15783_en.pdf
Country Analytic Work website—Search for Azerbaijan documents
http://www.countryanalyticwork.net/Caw/CawDocLib.nsf/vewAsiaPacific?SearchView&Query
=FIELD%20Country%20CONTAINS%20%20"Azerbaijan"&Country=Azerbaijan&DocType=
NULL&SearchOrder=4&SearchMax=5000&Start=1&Count=20
Health Systems Assessment Approach: A How-To Manual
3-36
Annex 3G. Sample In-Country Interview Schedule
Health Systems Assessment team: Preliminary TDY schedule in Angola, August 2005
Sat Sun Mon Tues Wed Thurs Fri
6
• Team meeting
with local
consultant—
1:00 pm–5:00
pm
• To review—
Interview
schedule,
documents
collected,
USAID
meeting, getting
information
from local
consultant,
guidance for
team as visitors
7
Team meeting with
Write-ups
• — lunch 1:00
– 5:00
• To review—
Technical
discussion on
health systems
strengthening
(presentation)
8
• Meeting with
USAID:
planning
Interviews
Team check-in
Write-ups
9
• Send
invitations for
stakeholders
workshop
Interviews
Team check-in
Write-ups
10
Interviews
Team check-in
Write-ups
11
Interviews
Team check-in
Write-ups
12
• Meeting with
USAID—
Unanswered
questions,
guidance
• Potential
province visit
Interviews
Team check-in
Write-ups
13
• Final drafts of
Ch. 4 by 1:00
• 1-5:00 Team
meeting:
analysis options
for Mission
14
• Optional team
meeting
• Write up
options: send to
PHRplus to
review (Sun pm
or Mon am)
15
• Potential
province visit
• Finish
interviews
• Reflect on how
tool has worked
(Mon or Tues)
• Afternoon free
16
• AM: PHRplus
feedback on
report to team
• Design
stakeholder
workshop
• 6:00 pm
conference call
with PHRplus
17
• Briefing for
USAID on
options/stakeholder
workshop,
review of draft
report
18
• Prepare for
stakeholder
meeting
19
• Stakeholder
meeting
20
• Write-up results of workshop
• Reflect on how tool has worked
• Send latest draft of report to Mission
before departure
Chapter 3. Planning and Conducting the Assessment
3-37
Annex 3H. Sample Contact List
Contact Name
Title
Organization
Module/Area
For Discussion Meeting date Email Phone Location of office
Health Systems Assessment Approach: A How-To Manual
3-38
Annex 3I. Sample Stakeholder Workshop Agenda
Stakeholder Workshop Agenda
Health Systems Assessment: Angola
Stakeholder Workshop
Date: Friday, August 19, 2005 8:30 – 13:00
Venue: Hotel Tropico, Luanda
Purpose: gather stakeholders that seem critical to the success of the options on the table /
impacted by the results; get their buy-in; get their feedback and reactions on findings and
recommendations.
Objectives:
By the end of the day participants will have:
• Reviewed and discussed team’s major findings
• Provided input on their priorities, based on strengths and weaknesses discussed
• Provided input into recommendations and identify how they will/can be involved in
implementing concrete options; how to move forward OR provided feedback and
recommendations on major options presented by team
Participants: (maximum 30 people)
• USAID, MOH, Donors, private sector, NGOs
Preliminary Workshop Agenda
Time Topic Responsible Materials
8:30 Coffee/registration Registration sheet
9:00 Welcome USAID/MOH
9:30 Introductions and expectations, overview of objectives and
agenda, guidelines for working together
Handout of agenda and
objectives
Guidelines (preprepared)
10:00 Overview of methodology, results and recommendations
• Highlight key findings
• Present suggested recommendations
• Q&A/discussion
Presentation(s)
Handouts of slides,
write-up of options
10:45 Coffee break
11:00 Small group discussion: go over recommendations and
discuss applicability and feasibility in Angola
Questions for discussion
11:45 Reports from small groups – 10 min each per group
12:30 Summarize Team
12:45 Closing comments USAID
1:00 Workshop evaluation. Adjourn for Lunch Evaluation form
Small Group Discussion Questions
Looking at the strategies listed on the four last slides:
1. Which would be the three principal strategies that you would recommend?
Chapter 3. Planning and Conducting the Assessment
3-39
2. How could your organization collaborate with USAID in these areas?
3. What would be your advice to USAID as it begins to work on strengthening the health
system?
Workshop Handouts:
• Sign-in registration
• List of participants and contact information
• PowerPoint presentation handouts
• Write-up of options or strategies – 1 page in Portuguese
• Arrange for LCD projector and flipcharts
• Evaluation from
• Guidelines for small group discussions
• Objectives and agenda
• Paper/pens, workshop name and dates
Health Systems Assessment Approach: A How-To Manual
3-40
Annex 3J. Suggested Outline for Final Assessment Report
Executive Summary
1. Background (2-3 pages)
Context—why was the assessment carried out and with what purpose?
2. Overview of health system (3-5 pages)
Core module should be used to prepare this chapter.
Basic description of the nature of the health system, focusing on—
2.1 Macroeconomic and political context
2.2 Epidemiological profile including key health indicators and causes of top mortality
and morbidity
2.3 Bureaucratic structures in-country (including decentralization)
2.4 Structure of health service delivery system
2.5 Health financing – profile and structure
2.6 Donor activities and gaps
2.7 Key stakeholders in the health system (including some discussion of the role of the
private sector, whether there is any social health insurance, etc.)
2.8 Business environment and investment climate, particularly as it affects for private
health care
No more than a couple of paragraphs on each of the subjects above (5 pages total)—to be
drafted in advance of trip. Where possible, differences across provinces/regions should be
highlighted.
3. Methodology (2 pages)
3.1 Framework for the health systems assessment approach
3.2 Description of tool and how it was used
4. Strengths and weaknesses of the health system (5-10 pages for each module)
4.1 Governance
4.2 Health financing
4.3 Service delivery
4.4 Human resources
4.5 Pharmaceutical management
4.6 Health information systems
4.7 Summary of findings (5–10 pages)
See Chapter 4
Chapter 3. Planning and Conducting the Assessment
3-41
Recommendations
4.8 Priority interventions based on the assessment
Drawing upon Chapter 4 this subsection should propose interventions that USAID might
consider supporting to address health system weaknesses. For each recommendation,
should discuss the relative time and cost involved.
4.9 Stakeholder views on the priority intervention areas
This should be based upon the workshop discussions and interviews with donors,
government, and other stakeholders, and should give some broad view of (1) what is
already being done by other stakeholders and how USAID might complement or
supplement their activities, and (2) what type of interventions there is political support
for. Also, what can local stakeholders take responsibility for or assist with? What are next
steps or potential action plan?
5. Conclusions
This section should identify key issues that were identified as part of the assessment,
responses to them by USAID and counterparts. It should also summarize the next steps that
can be expected as discussed by the various stakeholders.
Annex A
Contact list
Annex B
List of documents consulted
Annex C
Stakeholder workshop agenda
Annex D
Stakeholder workshop presentation
Health Systems Assessment Approach: A How-To Manual
3-42
Annex 3K. Outline of Assessment Report from Pilot Test in Angola
Angola Health System Assessment (full report included electronically on CD that accompanies
this manual or available for download at www.healthsystems2020.org)
ACRONYMS
ACKNOWLEDGMENTS
1. BACKGROUND
2. COUNTRY OVERVIEW
2.1 General
2.2 Health
2.2.1 Health Status
2.2.2 Health System
3. METHODOLOGY
3.1 Framework for the Health Systems Assessment Approach
3.2 Description of Assessment Tools
3.3 Pre-assessment Desk Research
3.4 In-country Key Interviews
4. STRENGTHS AND WEAKNESSES OF THE ANGOLAN HEALTH SYSTEM
4.1 Governance
4.1.1 Background
4.1.2 Health Information Capacity
4.1.3 Regulation
4.1.4 Policy Formation and Planning
4.1.5 External Participation and Partnerships
4.1.6 Accountability
4.2 Health Financing
4.2.1 Overview
4.2.2 Resource Flows
4.2.3 The Budgetary Process
4.2.4 Out-of-pocket Expenditures
4.3 Human Resources and Health Facilities
4.3.1 Policies, Plans, and Regulations
4.3.2 Number and Distribution of Health Facilities and Human Resources
4.3.3 Other Aspects of Health Service Delivery
4.4 The Role of the Private Sector
4.4.1 General Environment
4.4.2 Legal Framework and Regulation
4.4.3 Private Health Providers
4.4.4 Public–Private Partnerships
Chapter 3. Planning and Conducting the Assessment
3-43
4.5 Pharmaceutical Sector
4.5.1 Overview
4.5.2 Drug Procurement at the PHC Level: The National Essential Drug Program
4.5.3 Drug Procurement at the Hospital Level
4.6 Health Information Systems
4.6.1 Health Information Resources, Policies, and Regulations
4.6.2 Data Availability and Quality
4.6.3 Data Analysis
4.6.4 Use of Information for Management
4.7 Summary of Findings
4.7.1 Strengths
4.7.2 Weaknesses
4.7.3 Opportunities
4.7.4 Threats
5. RECOMMENDATIONS
5.1 Health Financing
5.2 Essential Drugs
5.3 Service Delivery
5.4 Public–Private Partnerships
5.5 Health Information
ANNEX A: DOCUMENTS CONSULTED
ANNEX B: CONTACT LIST
ANNEX C: GROUP DISCUSSION WITH NGOS
ANNEX D. STAKEHOLDER WORKSHOP AGENDA
ANNEX E. HANDOUTS FOR OVERVIEW OF HEALTH SYSTEMS STRENGTHENING
ANNEX F. DONOR HEALTH PROGRAMS IN ANGOLA
ANNEX G. MAP OF MOH STRATEGY AND DONOR INPUTS (OTHER THAN USAID)
FOR HEALTH SYSTEM STRENGTHENING
ANNEX H. COMPARATIVE INDICATORS FOR ANGOLA AND SSA
ANNEX I. 2005 CONTENTS OF THE THREE TYPES OF DRUG KITS PROVIDED UNDER
THE NATIONAL ESSENTIAL DRUG PROGRAM (NEDP)
ANNEX J. STAKEHOLDER WORKSHOP PRESENTATION
Health Systems Assessment Approach: A How-To Manual
3-44
Annex 3L. Outline of Assessment Report from Pilot Test in Benin
Benin Health System Assessment—Rapid Assessment of the Health System in Benin, April 2006
(full report included electronically on CD that accompanies this manual or available for
download at www.healthsystems2020.org)
ACRONYMS
ACKNOWLEDGMENTS
SECTION 1: INTRODUCTION
Methodology: The Health Systems Assessment Approach
Preassessment Activities
In-Country Assessment
Challenges
SECTION 2: BACKGROUND
Overview
Political and Macroeconomic Environment
Major Causes of Morbidity and Mortality
SECTION 3: OVERVIEW OF THE HEALTH SYSTEM IN BENIN
Structure of Health Care System
Decentralization and Organization of Service Delivery
SECTION 4: SUMMARY OF THE ASSESSMENT FINDINGS
Stewardship
Health Financing
Health Service Delivery
Human Resources
Pharmaceutical Management
Health Information Systems
Private Sector Engagement
Summary of Strengths and Weaknesses of Benin’s Health System
SECTION 5: PRIORITY INTERVENTIONS FOR CONSIDERATION AND ACTION
Possible Options for Strengthening Health System Governance
Possible Options for Improving Incentives for Health System Performance and Management
of Human Resources for Health
Possible Options for Improving Health Financing
SECTION 6: OPTIONS FOR USAID
Improving Financial Protection in Health
Improving Information
Fostering Greater Public-Private Integration
Chapter 3. Planning and Conducting the Assessment
3-45
ANNEX 1. SUMMARY OF PROPOSED INTERVENTIONS BY ASSESSMENT MODULE
Stewardship
Health Financing
Health Service Delivery
Human Resources Management
Pharmaceutical Management
Health Information Systems
Private Sector Engagement
ANNEX 2. IN-COUNTRY ASSESSMENT SCHEDULE
ANNEX 3. CONTACTS
Central Level
Department of Mono/Couffo
Department of Zou/Collines
ANNEX 4. SOURCES
Background
Overview of Health System
Stewardship
Health Financing
Health Service Delivery
Human Resources
Pharmaceutical Management
Private Sector Engagement
Health Information Systems
Health Systems Assessment Approach: A How-To Manual
3-46