Evaluation Manual
www.ifmsa.org
1
Evaluation Manual
IFMSA Contributors: Saniya Sahasrabudhe
Program Support Assistant 2017/18
Pablo Estrella Porter
SCOME Director 2017/18
Hana Lučev
Vice-President for Activities 2017/18
Matteo Cavagnacchi
SCOME General Assistant 2017/18
Alaa Abu Sufyan
SCOME Regional Assistant for Africa 2017/18
Layout
Mustafa Hushyar (IFMSA-Kurdistan) The cover was designed using resources from freepik
Publisher
The International Federation of Medical Students’ Associations (IFMSA) is a non-profit, non-governmental organization representing associations of medical students worldwide. IFMSA was founded in 1951 and currently maintains 140 National Member Organizations from over 129 countries across six continents, representing a network of 1.3 million medical students. IFMSA envisions a world in which medical students unite for global health and are equipped with the knowledge, skills and values to take on health leadership roles locally and globally, so to shape a sustainable and healthy future. IFMSA is recognized as a nongovernmental organization within the United Nations’ system and the World Health Organization; and works in collaboration with the World Medical Association.
International Federation of Medical Students’ Associations (IFMSA) International Secretariat: c/o IMCC, Norre Allé 14, 2200 Kobenhavn N., Denmark
Email: gs@ifmsa.org Homepage: www.ifmsa.org
This is an IFMSA Publication
Notice
© 2020 - Only portions of this publication may be reproduced for non political and non profit purposes, provided mentioning the source.
All reasonable precautions have been taken by the IFMSA to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material herein lies with the reader.
Disclaimer This publication contains the collective views of different contributors, the opinions expressed in this publication are those of the authors and do not necessarily reflect the position of IFMSA.
Contact Us
vpprc@ifmsa.org
2
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the IFMSA in preference to others of a similar nature that are not mentioned.
Some of the photos and graphics used in this publication are the property of their respective authors. We have taken every consideration not to violate their rights.
Index
www.ifmsa.org
1. Basic Concepts and Terms 2. Activity/ Project Cycle 3. Why evaluate your Activity? 4. When to start planning the evaluation? 5. Tools for Activity evaluation f. Baseline Assessment b. Backwards Mapping c. Evaluation Methodology □□ Qualitative ■■ Asking People (individual/ group discussions) o Structured Interviews ●● Open-Ended Questions. (Qualitative value) ●● Close-Ended Questions (Quantitative value) o Semi-structured interviews o Questionnaires o Evaluation stories ■■ Physical Methods o Photographic Records o Direct measurements o Participant Observation □□ Quantitative ■■ Progress and calculation of the data from/ by using the indicators ■■ Grouping of data ■■ Statistical modalities. o Mean o Standard Deviation d. SWOT Analysis e. TWOS Matrix f. Stakeholder Mapping 6. What to do after evaluation?
www.ifmsa.org
Pg. 4 Pg. 7 Pg. 11 Pg. 12 Pg. 13
Pg. 30
3
Evaluation Manual
“
“
IF YOU CAN MEASURE IT, YOU CAN MANAGE IT!
Basic concepts and terms Activity: In IFMSA an activity is defined as A unique set of planned interventions designed and implemented to achieve certain specific objectives – within a given budget and a specified period of time. This definition corresponds to the definition of a project.
●● Activity evaluation: “Rigorous analysis of completed or ongoing activities
that determine or support management accountability, effectiveness, and efficiency.” □□ It means that different audiences may assign a different value to the same information. □□ An activity evaluation can be used for evaluating the efficiency of the activity management structure, human resources, technical resources, financial resources etc. ■■ Generally speaking, the efficiency of an activity relates to how productive/ fruitful the implementation of the activity was. In simple terms it refers to resourcefulness (good use of the resources, including workforce, time, etc) of the project and the efficacy of the implementation systems. ■■ Effectiveness:- The extent to which the project or activity achieves the intended objectives (outputs). □□ Summative (done at the end of the activity) □□ Formative (done before or during the activity)
4
●●Activity monitoring Monitoring refers to collecting data and observing trends during the life of the activity further leading to Activity Evaluation and/or Impact Assessment.
●●Activity assessment Assessment is defined as a methodical way of acquiring, reviewing and using information about an activity or its participants, to make improvements where necessary. □□ Outcomes Measures the change in behavior, resources or the acquirement of new skills and knowledge. Used in relation to the goals and objectives of the project. Outcomes are usually considered in terms of their expected timeframe: ■■ Short–term (or immediate) ■■ Intermediate ■■ Long-term □□ Impact Refers to the measures of change that result from the activity/intervention being completed, such as responses to surveys, requests for further information, or number of products taken up (e.g. number of trainer participants; changes in the pre and post evaluation for knowledge-based interventions; changes in the attitude of people participating in workshop, etc.). Indicators of success and the monitoring of the progress helps immensely in a) measuring the impact of the activity and b) increasing the efficiency of the activity.
www.ifmsa.org
5
Evaluation Manual â—?â—?Difference between evaluation and assessment Whenever you are making an activity, it is important to know when you are assessing or evaluating it. Both can be used at different times of the activity cycle.
Assessment
Evaluation
Definition
The process of collecting, reviewing and using data, to improve the current performance of the activity.
The act of assigning value and comparing on the basis of a set of standards.
Purpose
Diagnostic: it aims to identify areas of improvement.
Judgemental: it aims at providing an overall grade/conclusion.
Orientation
Process Oriented.
Product Oriented.
What does it do?
Provides feedback on performance and areas of improvement.
Determines the extent to which objectives are achieved.
Modified from Surbhi, S. 2016.1
1
Surbhi, S. (2016). Difference Between Assessment and Evaluation. Key differences.
6
“
“
LIFE IS ACTIVITY MANAGEMENT
Activity/ Project Cycle:
Activity Planning (Activity Management/ Activity Design) → Activity Implementation → Activity Monitoring → Activity Evaluation → Revisiting the Activity Design (Backwards mapping)
Activity management is the most important part of the activity planning stage in the development of an activity. It is also known as activity design. In order to have a well structured activity plan, and the evaluation according to the goals we are planning to reach, it’s important to fulfill all the steps in the activity management.
www.ifmsa.org
7
Evaluation Manual Recap → Activity design and activity management basics ●● Vision/ Goal:- long-term desired change, describes why the activity is being done. ●● Objectives (SMART):- How the activity is being done? The long-term goal is broken down into SMART objectives so that the methodology of implementing the activity and division of the resources are further concrete. Specific – target a specific area for improvement. Measurable – quantify -or at least suggest- an indicator of progress. Assignable – specify who will do it. (Achievable - it is realistic that the objectives will be fulfilled) Realistic – state what results can realistically be achieved, giving the available resources. Time-related – specify when the result(s) can/will be achieved. *sometimes there are differences in specific words in SMART acronym, we can use either assignable or achievable. For example, if your goal is to improve the knowledge on mental health in the universities, a SMART objective can be to deliver X lectures on the most prevalent mental health issues during one year to university students in your city. ●● Output:- What the activity achieves or brings about? The outputs are the direct products of the activity that help achieve the immediate outcome. Outputs need to be precise and verifiable. Sometimes, we mistake outputs with outcomes, because both present results from the activity. For example, if you have an activity working on peer education, an output would be that you’ve trained X people during Y number of classes. The outcome would be the change in their behaviour/attitude/knowledge on the given topic. ●● Indicators Indicators provide the basis for monitoring and evaluation of your activity. They indicate the extent to which the objectives are achieved or are ways to measure the progress of the objectives.
8
They need to be concrete, which means they have to be a percentage, a number (quantitative) and not just words indicating progress/ regress like “more, less, increase, decrease” etc. (qualitative) Common examples of indicators include: participation rates, change in attitudes, individual behaviors, incidence and prevalence of a disease etc. The indicators you select should answer your evaluation questions and help you determine whether or not your objectives have been achieved. That means that all your indicators should be related to one or more objectives. Ideally each objective should have at least one indicator. As we mentioned the example of mental health lectures, in this case the indicator would be: 80% of students attending the lectures have an increase of knowledge on mental health topic done during the lecture. ■■ Indicators should be: □□ Specific Provides a clear description of what you want to measure, i.e. “In-school adolescents aged 13–18 who test positive for Chlamydia” Instead of “youth who have an STD” □□ Observable Focuses on an action or change, i.e. “The proportion of school-clinic staff who can list two risk factors for Chlamydia” Instead of “the proportion of school-clinic staff who can identify the risk factors for Chlamydia” □□ Measurable Quantifies change and generally reported in numerical terms, such as counts, percentages, proportions or ratios (2) ■■ Indicators can be: □□ Raw Numbers (number of new members). □□ Progress (percentage of how much of an objective is complete). □□ Change (percentage of increase in knowledge). The lagging indicator means that it occurred, it is an outcome (percentage of increase in members). A leading indicator tells whether you are
www.ifmsa.org
9
Evaluation Manual approaching or going to hit that increase, and it also tells you whether you are on track or not, so it can serve as a monitoring tool.
■■ Indicators can be broadly classified into two categories in relation to the specific objectives of the activity: □□ GENERAL: Number of events, Number of participants, Number of people screened, Number of beneficiaries reached, Number of the cities the event has been conducted in etc □□ SPECIFIC: Specific to measure the progress of a particular objective
To summarize: Indicators provide useful information that can measure processes and outcomes. They should be related to the evaluation questions. They should be feasible in terms of data availability and timely data collection, and are adequate to capture the information you need. You may need to develop more than one indicator, but avoid creating too many indicators because they can detract from the evaluation goals.
If you want to know more in detail about the activity management step by step process, please refer to our Activity Management toolkit: https://issuu.com/ifmsa/docs/from_idea_to_impact_-_manual_on_act
10
Why evaluate your activity?
Benefits of evaluation: Evaluation projects and actions are undertaken for a number of reasons: ●● to gather data on the effectiveness and efficiency of the intervention/project/ activity ●● to be accountable to those who are funding the activity ●● as a debrief to the group on the completion of an activity ●● to develop the skills and understanding of people involved in a project by enabling them to reflect on, and learn from their experiences ●● to provide information for planning future activities ●● to determine the effectiveness and efficiency of an activity; ie: whether the impact achieved is worth the time and resources invested. ●● to justify and promote a management action to the wider community ●● to create a historical record of management success over time ●● to see if the work is contributing to the objectives ●● to determine whether the project objectives have been met.
www.ifmsa.org
11
Evaluation Manual
When to start planning the evaluation?
The best time to start developing your evaluation plan is when you are building your activity. Planning and evaluation go hand-in-hand together. In order to have the evaluation done according to the goals of the activity, it should be implemented every step of the way when planning the activity. The results are then reflected on after the activity and used for further improvement and sharing with stakeholders.
12
Tools for Activity Evaluation
1. Baseline Assessment 2. Backwards Mapping 3. Timeline and Resources 4. Evaluation Methodology 5. SWOT Analysis 6. TOWS matrix 7. Stakeholder Analysis
www.ifmsa.org
13
Evaluation Manual 1. Baseline Assessment Simply put, a baseline assessment is an analysis of where you are now, your competitive advantages and disadvantages, what resources or sources do you have at your disposal. Anything that helps you find out your current state is a baseline assessment tool. Baseline assessment should be conducted before the activity implementation phase. This is a very helpful process that is important for prioritization and defining the focus of your actions. It also helps you to utilize your resources in order to dictate the framework of your organization and decide on your next move. When talking about activities, baseline assessment is a useful tool as a part of a research of the topic you’re working on. Not all the topics that are worked on in IFMSA are relevant for all the countries, regions etc, so in order to understand why you are doing this activity, it’s useful to have a baseline assessment of the problem you are tackling. For example, if you are starting to work on the topic of STIs in your community, it would be good to check how big of a problem it is, how much work has already been done and what kind of interventions are already done on your target group.
14
2. Backwards mapping (revising the activity design) Backwards mapping is a tool we use in order to break down the process of activity on smaller steps and try to see what are the changes that will need to be made in order to reach the final envisioned goal.
Baseline assessment (1) ↓
Defining the problem statement (2) ↓
Activity planning (3) ↓
Implementation + monitoring (4) ↓
Evaluation (5) ↓
Reflection on the results and necessary improvements (6) ↓
Go back to implementation! (7)
Backwards mapping is a tool which is connected to the Theory of Change, a theory upon which IFMSA Programs were made, and which reflects how a long term measurable change can be achieved throughout the process if we plan our activities well. Theory of Change is a model that assumes that if all the outcomes are achieved at each step of the process, they will ultimately lead to achieving the final goal. This is why the theory is based on the assumptions how each of the steps will impact the target group and lead to the change.
www.ifmsa.org
15
Evaluation Manual The outcomes are arranged based on their timeline in short term, intermediate and long term outcomes, based on the steps that need to be fulfilled before the other can take place. For example, if our ultimate goal is to provide equal access to healthcare to immigrants in our country, we would need to start with making an assessment (1) to know what is the current situation of immigrants at that moment. With that information, we will clearly set our problem statement (2) with evidence based information, and start the activity planning (3), including setting the objectives, the strategy/methodology, the search of main stakeholders, etc. The next step will be to implement the project (4), where we will create a set of lectures/workshops for medical students in which the lack of access to healthcare will be discussed. During these workshops, we will ask participants to fill an input form, where we will be able to monitor (4) the project, and improve it if necessary for the following lectures/workshops. When the project is over, we will make a final evaluation (5) to our participants, to know the impact we had with the intervention. Finally with those results, we will reflect (6) and analyze what was done right, what can be improved, and make changes for similar events in the future (7).
16
Review (step-by-step) activity management and activity evaluation map. Context
PROBLEM
Problem Statement ↓
↓ Activity Effectiveness
ACTIVITY
Goal
↓
↓ Each goal should have multiple SMART objectives
Activity Efficiency
SHORT TERM OUTCOMES
Each objective has an outcome.
↓
The outcome can be short-term, intermediate, or long-term.
INTERMEDIATE OUTCOMES ↓ LONG TERM OUTCOMES (Problem effectively solved with help of the activity / goal achieved) (each step of the process has different outcomes that we check based on the assumptions that we have)
↓ Each outcome needs to have one or more indicators. (indicators need to be specific, measurable and observable) (the outcomes need to have defined target population)
www.ifmsa.org
17
Evaluation Manual 3. Timeline mapping It is a process of arranging important outputs (events, grants, actions, achievements and other milestones) of the activity in a chronological order and observing them in temporal relationship to one another. The outputs are based on the outcomes we are trying to achieve and we are also taking into account the key contextual factors (social, economic, political, cultural events and trends). Timeline mapping helps in backwards mapping, better utilisation of human and material resources, helps identify how external factors influence the progress of the activity and puts a group’s progress and challenges into the frame of a broader context. A basic timeline map is typically a graphic design showing a long bar labelled with dates/activity schedule/deadlines and the to-do output of every objective of the activity. A Gantt chart is a form of timeline used in activity management. Gantt Chart can be modified to simultaneously show progress (linear bar shaded in proportion to the degree of the output completion). Further each liner bar can be connected to another to represent the dependency of the tasks.
18
4. Evaluation methodology Evaluation can be generally divided into 2 different types: qualitative and quantitative. Qualitative evaluation is mostly relying on questions or interviews which give us more information about the process/activity/results; while the quantitative evaluation is focusing on numbers, concrete results and measurable change which is or isn’t achieved. Below you can find some examples of both types of the evaluation methodologies. Sometimes it’s hard to draw a line between qualitative and quantitative analysis. You can always combine multiple methodologies in order to achieve your goal. Choosing one or another is mostly based on the objectives that you’re set up on your activity and the indicators which show the success of it. For example, if your activity is based on knowledge evaluation, which is something that can be evaluated with the pre and post evaluation, it’s logical that you would use one of the quantitative types of the evaluation. On the other hand, if your objective would be that the experience for the people participating has been good or that their comments or suggestions would be one of the outcomes of the activity, then the qualitative evaluation would be suitable.
OUTLINE □□ Qualitative ■■ Asking People (individual/ group discussions) o Structured Interviews ●● Open-Ended Questions. (Qualitative value) ●● Close-Ended Questions (Quantitative value) o Semi-structured interviews o Questionnaires o Evaluation stories
www.ifmsa.org
19
Evaluation Manual ■■ Physical Methods o Photographic Records o Direct measurements o Participant Observation □□ Quantitative ■■ Progress and calculation of the data from the indicators ■■ Grouping of data ■■ Statistical modalities. o Mean o Standard Deviation
DETAILS □□ Qualitative ■■ Asking People (individual/ group discussions) o Structured Interviews Interview questions must be selected carefully, as there is no room to change the questions once the process has commenced. ●● Open-Ended Questions. (Qualitative value) Open ended questions are often used to gather expert review, or understand personal experience or opinion in detail. For example the knowledge on an individual over a certain topic or for example the degree of satisfaction on a service or a product. These questions are of a little significance when one is doing conclusive large-scale research requiring a numerical output. ●● Close-Ended Questions (Quantitative value) You would want to use these questions when you know for sure that you have all the options possible included, for example if you
20
wanted to see how many people ordered food at a restaurant that only serves 3 meals (Pancakes - Spaghetti - Steak). Another example is when you have a positive or negative result, for example if you want to know how many students passed their Biology exams you’d have two options (Passed - Did not pass). Close-ended questions usually produce quantitative data for a conclusive result and are best when used in large groups, demographic studies or prevalence; while open-ended questions are used for more qualitative data and smaller groups and for more genuine opinions. o Semi-structured interviews These are informally guided. Some questions are predetermined and new questions are developed from the discussion. Questions are mainly open, providing an opportunity for the respondent to provide an opinion. Semi-structured interviews are used to understand an interviewee’s experiences and impressions. o Questionnaires Questionnaires can be conducted in person, by telephone, or by mail. They are used to quickly obtain information from a wide variety of people. o Evaluation stories Evaluation stories are based on collecting and reviewing stories of significant change associated with the activity or action being evaluated. Stories are collected from those most directly involved in the project. Example questions before the project: What is the proposed project area like at the moment? How could the project area be improved? Example questions after the project: What was the project area like before the project was undertaken? How do you believe it has changed?
www.ifmsa.org
21
Evaluation Manual ■■ Physical Methods o Photographic Records:- Photographs of the activities, outcomes, observations etc. o Direct measurements:- If the outcome is any particular product and has measurable dimensions/ measurements. o Participant Observation:- Detailed observation of the activity and outcomes by the participants/ volunteers. □□ Quantitative ■■ Progress and calculation of the data by using indicators o Fixing the benchmark of performance: Means setting a reference to compare with from the best competitors or the ideal results you were expecting. o Measurement of performance: It is done through many methods like financial statements, impact assessment, data analysis, etc. o Analyzing variance: when comparing the actual performance and the standard performance (the benchmark) you will get a degree of variance which you have to analyze and see what went wrong. If you are over-performing or underperforming, why are you under-performing in a certain area, and if the degree of variance is large or small. ■■ Grouping of data Aggregation of individual observations of each question (open ended or close ended collected through pre and post event event surveys etc) into groups, so that the frequency distribution of these groups serves as a convenient means of summarizing the outcomes or analysing the data ■■ Statistical modalities. 1. Mean The arithmetic mean, more commonly known as “the average,” is the sum of a list of numbers divided by the number of items on the list. The mean is used in determining the overall trend of a data set or providing a rapid snapshot of your data. Another advantage of the mean is that it’s very easy and quick to calculate.
22
Pitfall: Taken alone, the mean is a dangerous tool. Iin a data set with a high number of outliers or a skewed distribution, the mean simply doesn’t provide the accuracy you need for a nuanced decision. 2. Standard Deviation The standard deviation, often represented with the Greek letter sigma, is the measure of a spread of data around the mean. A high standard deviation signifies that data is spread more widely from the mean, where a low standard deviation signals that more data align with the mean. The standard deviation is useful for quickly determining dispersion of data points. Pitfall: Just like the mean, the standard deviation is deceptive if taken alone. For example, if the data have a very strange pattern such as a non-normal curve or a large amount of outliers, then the standard deviation won’t give you all the information you need.
www.ifmsa.org
23
Evaluation Manual 5. SWOT Analysis SWOT Analysis simply stands for (Strengths - Weaknesses - Opportunities Threats). It is a simple but effective way in order to perform an analysis of your current status, frame up a picture of where you are now and prepare for your next moves. SWOT analysis is plotted in a Table.
Positive
Negative
Internal
Strengths
Weaknesses
External
Opportunities
Threats
Each side presents a perspective, the upper rows (Strengths and Weaknesses) presents your internal aspect and the lower rows (Opportunities and Threats) presents you external aspect. You should know that you can impact and have a direct influence on your internal aspects, but when it comes to your external ones you can only have some influence but not a direct impact. Data sources are the sources you use to fill your SWOT analysis with information about the 4 categories (strengths - weaknesses - opportunities - threats) in order to find out what are you dealing with from each category. The data sources themselves can also be categorized into: a) Internal (Executive Boards, Officials, Members, your own Key Performance Indicators). b) External (Mega Trends*, Industry or field of work, Communities, Other organizations). *Mega Trends: They are global, sustained and macro-economic forces of development that impact economy, business, society, cultures and personal lives, thereby defining our future world and its increasing pace of change. Phenomena that affect our lives like women empowerment, virtual reality, wireless intelligence, renewable power, veganism, artificial intelligence, There should always be more weaknesses than threats as a general rule of thumb.
24
6. TOWS Matrix By doing a SWOT analysis you have identified internal and external factors that influence your activity in advantageous and disadvantageous ways. But how do you move on from here? How do you utilize your new knowledge? Answer is TOWS The TOWS Matrix is a relatively simple tool for generating strategic options. By using it, take the best advantage of the opportunities open to you, minimize the impact of weaknesses and protect against external threats. Used after detailed analysis of your threats, opportunities, strengths and weaknesses, it helps you consider how to use the external environment to your strategic advantage, and so identify some of the strategic options available for you. Threats, Opportunities (in the environment) Weaknesses, Strengths (of the activity) should be analyzed in the order. Steps of TOWS analysis 1. Identification of the activity profile (goals, objectives, problems, theory of change, needs/ resources etc) 2. Identification of the external factors like political, economic, social, legal, technical, cultural, ethical 3. Anticipate possible changes of environment and its influence on organization (scenario analysis, trends, perspectives) 4. Audit and assess internal resources of the organization to identify weaknesses and strengths 5. Arrange elements in the TOWS matrix 6. Analyse interaction matrix between TOWS factors, and indicating best alternative strategies 7. Test internal consistency for every strategy (main or alternate) 8. Preparation of contingency plans in case of failure of the undertaken actions.
www.ifmsa.org
25
Evaluation Manual
An activity should avoid threats, An activity should exploit opportunities, An activity should strengthen its weaknesses, An activity should base important activities on its strengths.
( Strength + Opportunity) SO-Actions: Maximise the efficiency of these actions and get maximum effectiveness/impact. (Weakness + Opportunities) WO–Actions: As the weakness is internal, either improve the framework or partner/ collaborate with some external/ partner who is strong in these aspects. ( Strength + Threat) ST–Actions: These are the risk management actions, where the organization finds a way to use its internal strengths to eliminate threats. The threat is external and cannot be controlled. (Internal Weakness + External Threat) WT–Actions: These are more monitoring actions, in which the organization makes sure that its internal weaknesses do not create any external threats to them or the activity. Again it is important to state that since TOWS is based on SWOT, it is context specific. Whenever the SWOT changes, so does the TOWS.
26
Strengths
Weaknesses
(S)
(W)
Opportunities
Threats
(O)
(T)
(A step forward of SWOT is TOWS)
(Internal Factors and Origins) (External Factors and Origins)
Opportunities (O)
Threats (T)
Weakness (W)
Strengths (S)
W-O
S-O
As the weakness is internal, either improve the framework or partner/ collaborate with some external/ partner who is strong in these aspects
Maximise the efficiency of these actions and get maximum effectiveness/impact
W-T
S -T
These are more monitoring actions, in which the organization makes sure that its internal weaknesses do not create any external threats to them or the activity.
These are the risk management actions, where the organization finds a way to use its internal strengths to eliminate threats. The threat is external and cannot be controlled.
www.ifmsa.org
27
Evaluation Manual 7. Stakeholder Mapping Stakeholder mapping is a process that you perform in order to identify possible partners and opportunities and determine how to approach them. Before getting into the topic of stakeholders mapping let us first talk about what is a stakeholder? A “Stakeholder” is a person or an organization that either has an interest in your organization and its work or has a certain degree of influence on or is influenced by your flow of work, profit, processes or decisions. Due to the different nature of stakeholders, different interests and different benefits gained from the organization, you’ll find out sometimes that there is what is called a “stakeholder conflict” which means that a decision that you make or an action that you take as an organization may sometimes be supported by a certain stakeholder and opposed or not supported by another. So as an organization, the trick is that you have to draw the interest of people or organizations that support and believe in your cause and try to satisfy them while not creating opposers who’d want to stop your action. Now- Stakeholder Mapping is a simple process that depends on two simple concepts: a) Stakeholders vary in their degree of interest towards your organization. b) Stakeholders vary in their degree of power over your organization.
High level of Stakeholder power
High level of Stakeholder interest
Low level of Stakeholder Interest
Key players, take notice of them and engage them directly. You should usually spend most of your time and efforts working with them
Keep them satisfied
(High interest - Low power Stakeholders)
(High interest - High power Stakeholders) Low level of Stakeholder power
28
Communicate regularly with them
Communicate with them when necessary
(Low interest - High power Stakeholders)
(Low interest - Low power Stakeholders)
As you can see stakeholders are plotted or categorized into 4 different categories by cross-matching. And you deal with each one depending on their respective category as demonstrated by the table. ●● High interest - High power Stakeholders. ●● High interest - low power Stakeholders. ●● Low interest - High power Stakeholders. ●● Low interest - Low power Stakeholders. This is how you perform a stakeholder analysis, and the map that results can be used as a tool of assessment in order to know when, how and why to approach and deal with each stakeholder.
www.ifmsa.org
29
Evaluation Manual
What to do after the evaluation? Remember when we were discussing the SMART objectives? There is also a more complex version of it - SMARTER. The SMART letters are the same, but with addition of Evaluate and Reflect. Since this manual is all about evaluation, after it comes the reflection on the results you’ve achieved. As mentioned in the Activity Circle, after you’ve gathered your evaluation results, now is time to focus on what to do with them. Here are some options: ●● Reflect on the results in order to improve your activity, whether in its methodology, reach or objectives. ●● Present the data to your stakeholders (this is why it’s important to do the stakeholder mapping - to see who would be interested in your results) ●● Use the data for the advocating on the issue. If you have a good data, you can always achieve more. Think big and try to imagine what would your activity do if implemented on a larger scale.
There are a couple of more important things that we would like to mention: ●● Don’t forget about the ethical implications of your activity. Think about the consequences that it can cause and how to prevent them. ●● Don’t forget about the privacy of the data you are collecting. Especially in Europe with GDPR (General Data Protection Regulation), all the personal information collected needs to have specific permissions in order to collect them. Get informed about them!
If you’d like to get inspired by more examples than the ones mentioned in this manual, check the IFMSA Activities Database at https://ifmsa.org/activities/. To finalize, we have gathered some further reading materials from which we’ve extracted our materials, check them in the annex.
30
Further reading and resources 1. Evaluation Planning Checklist 2. https://www.cdc.gov/std/Program/pupestd/Developing%20Evaluation%20Indicators. pdf 3. http://devinfolive.info/impact_evaluation/img/downloads/Theory_of_Change_ENG. pdf 4. https://ceopedia.org/index.php/TOWS_analysis 5. http://www.businessdictionary.com/definition/evaluation.html 6. Activity management manual 7. Strategy - Europe 8. https://drive.google.com/open?id=0B5JqiqOUqcG6OUFkQnFjR0liaUE 9. https: //www.bigskyassociates.com/blog/bid/356764/5-Most-Important-Methods-For-Statistical-Data-Analysis 10. Gantt Chart image source:- https://www.free-power-point-templates.com/articles/ free-gantt-chart-excel-template/ 11. https://brungerblog.wordpress.com/2016/03/20/tows-matrix-for-marketing-brainstorming/ 12. https://www.pm4ngos.org/download/project-management-for-development-professionals-guide-pmd-pro-guide-english/ 13. https://ec.europa.eu/europeaid/sites/devco/f iles/methodology-aid-delivery-methods-project-cycle-management-200403_en_2.pdf
www.ifmsa.org
31
Evaluation Manual
Afghanistan (RMSA
Dominica)
Kuwait (KuMSA)
Republic of Tatarstan
Afghanistan)
Dominican Republic (ODEM)
Kyrgyz Republic (AMSA-KG)
(TaMSA)
Albania (ACMS Albania)
Ecuador (AEMPPI)
Latvia (LaMSA)
Rwanda (MEDSAR)
Algerwia (Le Souk)
Egypt (IFMSA-Egypt)
Lebanon (LeMSIC)
Senegal (FNESS)
Argentina (IFMSA-Argentina)
El Salvador (IFMSA-El
Lithuania (LiMSA)
Serbia (IFMSA-Serbia)
Armenia (AMSP)
Salvador)
Luxembourg (ALEM)
Sierra Leone (SLEMSA)
Aruba (IFMSA-Aruba)
Estonia (EstMSA)
Malawi (MSA)
Singapore (SiMSA)
Australia (AMSA)
Ethiopia (EMSA)
Malaysia (SMMAMS)
Slovakia (SloMSA)
Austria (AMSA)
Finland (FiMSIC)
Mali (APS)
Slovenia (SloMSIC)
Azerbaijan (AzerMDS)
France (ANEMF)
Malta (MMSA)
South Africa (IFMSA-SA)
Bangladesh (BMSS)
Gambia (GaMSA)
Mauritania (AFMM)
Spain (IFMSA-Spain)
Belgium (BeMSA)
Georgia (GMSA)
Mexico (AMMEF-Mexico)
Sudan (MedSIN)
Bolivia (IFMSA-Bolivia)
Germany (bvmd)
Montenegro (MoMSIC)
Sweden (IFMSA-Sweden)
Bosnia & Herzegovina
Ghana (FGMSA)
Morocco (IFMSA-Morocco)
Switzerland (swimsa)
(BoHeMSA)
Greece (HelMSIC)
Namibia (AMSNA)
Syrian Arab Republic (SMSA)
Bosnia & Herzegovina –
Grenada (IFMSA-Grenada)
Nepal (NMSS)
Taiwan - China (FMS)
Republic of Srpska (SaMSIC)
Guatemala (IFMSA-
The Netherlands
Tajikistan (TJMSA)
Brazil (DENEM)
Guatemala)
(IFMSA NL)
Thailand (IFMSA-Thailand)
Brazil (IFMSA-Brazil)
Guinea (AEM)
Niger (AESS)
Tanzania (TaMSA)
Bulgaria (AMSB)
Haiti (AHEM)
Nigeria (NiMSA)
Togo (AEMP)
Burkina Faso (AEM)
Honduras (IFMSA-Honduras)
Northern Cyprus, Cyprus
Trinidad and Tobago
Burundi (ABEM)
Hungary (HuMSIRC)
(MSANC)
(TTMSA)
Cameroon (CAMSA)
Iceland (IMSA)
Norway (NMSA)
Tunisia (Associa-Med)
Canada (CFMS)
India (MSAI)
Oman (MedSCo)
Turkey (TurkMSIC)
Canada – Québec
Indonesia
Palestine (PMSA)
Turkey – Northern Cyprus
(IFMSA-Québec)
(CIMSA Indonesia)
Pakistan (IFMSA-Pakistan)
(MSANC)
Catalonia - Spain (AECS)
Iran (IMSA)
Panama (IFMSA-Panama)
Uganda (FUMSA)
Chile (IFMSA-Chile)
Iraq (IFMSA-Iraq)
Paraguay (IFMSA-Paraguay)
Ukraine (UMSA)
China (IFMSA-China)
Iraq – Kurdistan (IFMSA-
Peru (IFMSA-Peru)
United Arab Emirates
China – Hong Kong
Kurdistan)
Peru (APEMH)
(EMSS)
(AMSAHK)
Ireland (AMSI)
Philippines (AMSA-
Colombia (ASCEMCOL)
Israel (FIMS)
Philippines)
Costa Rica (ACEM)
Italy (SISM)
Poland (IFMSA-Poland)
Croatia (CroMSIC)
Ivory Coast (NOHSS)
Portugal (ANEM)
Cyprus (CyMSA)
Jamaica (JAMSA)
Qatar (QMSA)
Czech Republic
Japan (IFMSA-Japan)
Republic of Moldova (ASRM)
(IFMSA-CZ)
Jordan (IFMSA-Jo)
Republic of North
Democratic Republic of the
Kazakhstan (KazMSA)
Macedonia (MMSA)
Congo (MSA-DRC)
Kenya (MSAKE)
Romania (FASMR)
Denmark (IMCC)
Korea (KMSA)
Russian Federation (HCCM)
Dominica (IFMSA
Kosovo - Serbia (KOMS)
Russian Federation –
Commonwealth of
www.ifmsa.org
32
United Kingdom of Great Britain and Northern Ireland (SfGH) United States of America (AMSA-USA) Uruguay (IFMSA-Uruguay) Uzbekistan (Phenomenon) Venezuela (FEVESOCEM) Yemen (NAMS) Zambia (ZaMSA) Zimbabwe (ZIMSA)
medical students worldwide