Needs Assessments in the Field of SSG/R

1. Introduction

Welcome to the fourth module of the OSCE Training on Security Sector Governance and Reform (SSG/R). After completing both e-learning and in-class parts of this module, you should be able to:

• Explain benefits of SSG/R needs assessments
• Describe the various steps necessary to prepare and conduct an SSG/R needs assessment
• Develop a plan (terms of reference) for conducting an SSG/R needs assessment

The module is based on the fourth chapter of the Guidelines on SSG/R for OSCE Staff (2nd edition), which is required reading for this course.

2. Needs Assessments in the Field of SSG/R

Needs assessments are part of the strategic planning, which is used to determine priorities, to strengthen programme and project design, to improve decision-making, and subsequently to enhance monitoring and evaluation. They are an integral element of the identification phase of the project cycle. Needs assessments help OSCE programme/project managers to understand a variety of factors – notably, the national priorities of reform processes; the context in which the programme/project will take place; the different stakeholders, their interests, expectations and challenges; and the strategic options to address challenges. A thorough assessment of these elements will facilitate sustainable and results-oriented approaches to SSG/R support.

Needs assessments have different functions and provide important added value to different processes such as:

• Supporting national reform processes
• Facilitating more effective OSCE programming by contributing to:

o  Stronger engagement with national stakeholders
o   The identification of entry points for relevant support
o   The identification of cross-dimensional approaches to SSG/R
o   The early identification of risks and challenges
o   Stronger co-operation with other international actors

• Providing baseline data for subsequent planning, monitoring and evaluation efforts Likewise, there are different types of needs assessments.

Likewise, there are different types of needs assessments.

For additional information, see section 1 in Chapter 4 of the Guidelines.

3. Planning for Needs Assessment - 1

It is important to engage with national stakeholders during the planning phase in order to ensure that they can help in shaping the assessment to their needs. Engaging national stakeholders in this phase also increases their commitment to implement the recommendations arising from the assessment. A key output of the planning phase is the development of terms of reference (ToR) and agreement on them with national stakeholders. It is good practice to highlight in the ToR the need for a comprehensive, inclusive, gender-sensitive and people-centred approach to the conduct of the assessment, which will consider the needs of different groups of women, men, girls and boys. In the area of SSG/R, such an approach includes, for example, tailoring the methodology to include consultations not only with formal (state) security sector actors but also with representatives of civil society and non-state security actors.

The planning phase consists of number of steps that are briefly described below. For additional information, see section 2 in Chapter 4 of the Guidelines.

Defining the Objectives and Scope: OSCE staff should start by defining the overall objectives and scope of the assessment, as this will directly affect the timeline and the resources needed. Decisions are required as to whether the assessment will be broad (i.e., sector-wide) or limited to one or multiple subsectors. The objectives should be clear to all stakeholders involved.

Defining the Methodology: Different methodologies are used to conduct needs assessments in the field of SSG/R. The choice of methodology will depend on the purpose, the financial and human resources available and the access to data. Ideally, a mixed methodology should be applied, even if there are often time constraints and insufficient financial and human resources that hinder the assessment of all aspects. However, even when compromises need to be found, it is necessary to have a clear understanding of the challenges and opportunities offered by the methodology selected.

4. Planning for Needs Assessment - 2

Defining the Methods and Recognizing Challenges in Data Collection: The ToR should set out the initial sources and methods for data collection, ideally consisting of a combination of public and internal data sources, qualitative and quantitative data collection methods and analysis methods relevant to the sector. Data collection, whether desk-based or field research, can be challenging, mainly because of the accessibility and reliability of data. To build trust with respondents, clear assurances are required on how information will be stored and about how the findings of the needs assessment will be disseminated. To increase the reliability and effectiveness of the needs assessment, data sources need to be triangulated through both quantitative and qualitative data collection methods.

Setting Benchmarks and Developing Guiding Questions: Benchmarks can be used to identify the priorities to be examined during a needs assessment. While there is no consistent set of internationally recognized benchmarks for SSG/R, there is general agreement that the security sector should be both effective and accountable. Therefore, some of the key SSG/R benchmarks are related to effectiveness (e.g., adequate service delivery) and accountability (e.g., internal accountability and external oversight, respect for human rights and gender equality, etc.). In some areas of SSG/R, specific international standards may exist that can be used to set benchmarks. To complement the benchmarks set and to specify in greater detail the issues that the assessment will focus on, the ToR should always include (preliminary) guiding questions for the assessment. The guiding questions need to be derived from the assessment’s scope. They can be formulated generically but should be specific enough to articulate what should be examined.

Establishing Roles and Responsibilities: Clear roles and responsibilities need to be assigned, not only when the needs assessment is jointly conducted with other international actors or beneficiary driven but also when the OSCE is in the lead. Roles and responsibilities should be assigned in four areas: (1) leading the assessment, (2) contributing to the assessment (human and financial resources), (3) collecting data and analysing it and (4) disseminating the results of the assessment.

5. Planning for Needs Assessment - 3

Engaging with National Stakeholders from the Outset: When planning a needs assessment, defining co-ordination modalities among the different partners is key. Even when an assessment is OSCE-driven, efforts should be made to involve national authorities and non-governmental actors, including civil society, in order to support national ownership. In OSCE-driven assessments, it is thus important to explain and consult regarding the purpose of the assessment with national stakeholders, to request their practical support and to keep them informed of the results. The OSCE should also discuss with national representatives the findings of the assessment prior to their publication and agree an appropriate format to disseminate them.

Managing Expectations: As SSG/R can be a sensitive topic for individuals and institutions, some national stakeholders may object to the assessment and not wish to provide information. A number of approaches can be used to mitigate this challenge. First, OSCE staff should take the time to explain the purpose of the needs assessment, also highlighting the benefits that national stakeholders may derive from it. Second, OSCE staff should determine the modalities for disseminating and following up on the results of the assessment; it may be necessary to agree with national stakeholders whether the results will be made public or remain internal or to confirm that they will not be publicly released without prior approval. However, OSCE staff should take care not to raise expectations, including hopes among national stakeholders that the assessment will result in immediate funding for reforms.

Integrating Gender into SSG/R Needs Assessments: Gender is an important factor in any needs assessment and should, therefore, be mainstreamed throughout the entire process. Gender mainstreaming is much more than mere consultations with women’s groups; it involves understanding gender as an integral part of the assessment, rather than an external factor. Gender aspects should be included within the scope of the assessment, either as a separate objective or as a key aspect of other set objectives. Gender is also important in the selection of the assessment team and involves not only selecting women and men as experts but also ensuring that all team members have the skills to integrate gender perspectives or at least understand the importance of doing so.

6. Conducting the Needs Assessment: Preparatory Analysis

The preparatory analysis for a needs assessment is conducted through desk research to systematically gather available primary and secondary data. The analysis maps the broad political, economic, security, institutional and legal context in which the security sector operates. It is also used to identify gaps in data and the questions that will need to be answered during the in-country assessment in order to fill those gaps. A comprehensive preparatory analysis thus helps to plan a purposeful in-country assessment and will consequently reduce the amount of resources required.

The preparatory analysis consists of several steps that are briefly described below. For additional information, see section 3.1 in Chapter 4 of the Guidelines.

Designing the assessment questions: Based on the guiding questions specified in the ToR, the assessment team will develop more detailed questions to help in collecting and analysing data. These assessment questions should be used during both the desk-research analysis of data and the in-country collection of data. For examples of assessment questions, see Info Box 2 in Chapter 4 of the Guidelines.

Reviewing relevant sources of information: Information from multiple sources and institutions should be reviewed to ensure a broad understanding of the political context, the legal framework, previously identified gaps and challenges, evaluation findings and lessons learned, and the roles and responsibilities of security institutions. Gender-related aspects must also be considered, such as the security risks and needs of women, men, boys and girls and their access to security services, as well as what hinders gender equality in the security sector. A specific gender analysis of the security sector, including on the representation of women in senior positions and their roles in decision-making, can help to identify gender-related gaps and needs. For examples of relevant sources of information, see Info Box 3 in Chapter 4 of the Guidelines.

Reflecting on the regional dimension: During the desk research, appropriate consideration also needs to be given to the regional dimension, as it is an important element of situation analysis. The regional dimension may include regional conflict dynamics, sources of cross-border tensions and regional co-operation dynamics. Regional challenges may require specific actions in SSG/R, such as strengthening international co-operation. Desk research should therefore seek to answer the following questions: Are there ongoing initiatives or existing commitments at the regional or international level that may provide entry points for national reform efforts? Have other states in the region recently undergone similar reform processes from which lessons may be learned? Would a regional approach to SSG/R support be valuable?

Stakeholder mapping and identifying priorities for the in-country assessment: Data collected during the preparatory phase should be analysed to identify priorities for the subsequent in-country assessment. Priorities may be identified on the basis of:

• Gaps in information: The desk-based research may highlight areas where information is lacking. Such areas should be given special attention during the in-country assessment so that interviews can help to complete the picture.
• Potential priorities for reform: Potential gaps and priority needs for SSG/R may be identified during the research. Assumptions about needs should be tested through interviews during the in-country assessment to ensure they hold true.

On the basis of the preliminary gaps and priorities identified during the desk-based preparatory analysis, the key stakeholders can be mapped. Based on the stakeholder mapping, decisions can be taken on specific stakeholders who should be interviewed in order to fill specific gaps, in addition to interviewing a broad representative sample of interlocutors.

7. Conducting the Needs Assessment: In-Country Assessment

The next step of the assessment process is to collect primary data in the country in question. The in-country assessment will be based on the preparatory analysis and tailored to fill the information gaps identified and/or to test the validity of the assumptions made about priority needs. The in-country assessment should include meetings with national and international stakeholders, who represent a variety of areas relevant to the assessment, including national authorities (beneficiaries), civil society organizations and other non-state actors (academia, think tanks, research groups), international actors active in the sector. In the OSCE context, the most common methods of primary data collection in the area of SSG/R include semi-structured interviews and focus groups. Occasionally, processes and procedures may also be observed.

Inclusive approach to stakeholder engagement and interviews: Assessments in the area of SSG/R must be inclusive when selecting interlocutors and should seek to engage both those providing SSG/R assistance and those benefiting from such support. As the perceptions and security needs within and outside the capital may differ significantly, efforts should also be made to meet with relevant representatives from regional and rural areas. When organizing meetings, representatives of the following categories of interlocutors should be included: security sector officials, civil society and other non-state actors, oversight actors, international actors, and OSCE field staff. For additional information, see section 3.2.1 of Chapter 4 of the Guidelines.

Primary data collection: Primary data should be collected on the basis of the questions developed during the preparatory phase. The purpose is to collect data that is not available in the secondary and primary sources collected through the desk research. In the OSCE context, this will be usually done through semi-structured interviews and/or focus groups. Several important points should be kept in mind when using these methods. First, the same set of questions need to be asked of all respondents representing the same groups so that the data collected can be compared and triangulated. Second, to capture all information, thorough notes are required. Respondents should always be made aware that all information they provide will be treated as confidential and anonymous. If possible, the interviewer should be accompanied by a dedicated note taker. Finally, when conducting interviews with persons in vulnerable situations, the do-no-harm principle must always be strictly followed. For additional information, see section 3.2.2 of Chapter 4 of the Guidelines.

8. Conducting the Needs Assessment: Analysing and Validating the Results

Once sufficient information has been gathered by the assessment team, it must be compiled and analysed to identify key findings and recommendations. The guiding questions developed at the start of the exercise can be used to structure the analysis and the options for the way forward. Following that structure, the data can be analysed as follows:

• Understanding the context (the context should include a situation analysis and a stakeholder analysis);
• Considering the potential for national ownership of reforms;
• Understanding which security needs of the population are being met and which are not and why (this is the core of the assessment and the problem analysis);
• Identifying options for the way forward.

For key questions on each of these areas, see section 3.3 of Chapter 4 in the Guidelines.

After analysing the results, they should be validated. The purpose of validation is to categorize the preliminary findings and recommendations so that they can be presented to the beneficiaries and partners for feedback and confirmation. Feedback can be sought in various ways, including validation workshops, focus group discussions, follow-up interviews with selected respondents from the in-country assessment, or written questionnaires. If the needs assessment is meant to provide recommendations for OSCE programmatic activities, OSCE staff should be involved in the validation exercise. Feedback from the validation exercise should be incorporated into the final needs assessment report and recommendations.

9. Following Up on the Needs Assessment

Disseminating the Findings and Recommendations: With whom the findings should be shared and how they will be disseminated should have been determined during the planning phase. This planning should also have included how feedback will be incorporated and who will be responsible for implementing the recommendations. In most cases, the findings will be delivered in written form as a report to the main stakeholders. If the results are to be shared more widely, the findings should first be shared with the stakeholders involved to obtain their feedback and to agree on how to address the recommendations made.

Feeding the Findings into Planning, Monitoring and Evaluation: The results of the needs assessment should be analysed and fed into OSCE project and programme planning. In particular, the findings should inform the development of projects and programmes that the OSCE will prioritize for support, as well as the short-, mid- and long-term results, to which OSCE activities should contribute. In addition to the analysis of the data and the main problems identified, these findings should contribute to an understanding of relevant cause-and-effect relationships – for example, by means of a Theory of Change – and identify how the activities and outputs of the projects will lead to addressing the core problem identified or what results would be required towards the mid-term outcome and objective.

When developing UB and ExB projects and programmes that address the findings of the needs assessment, some of the data gathered should be included in the UB programme document and project logframe and the M&E frameworks: (1) to identify and measure project indicators for objectives, outcomes and outputs, (2) to identify risks and assumptions and (3) to generate baselines against which to measure progress and possible targets. Moreover, if findings from the needs assessment are relevant to other OSCE dimensions, and these were not involved from the outset, staff should ensure that these are appropriately shared with their colleagues so that they can also feed into their project and programme planning.

All these points will be discussed in the next module.

10. Wrap-up Questions

Get to the questions