A LEARNING ORGANISATION  or How Can Evaluation Help Us Learn?

Round table for organisations working with youth
facilitated by: Barbora Komberec Novosadová, Jan Hněvkovský and Jakub Vrobel

This article is a brief summary of the roundtable attended on 9 February 2022 by professional evaluators and representatives of leading Czech organisations and institutions working with youth.

The roundtable was held online and it was organised under the auspices of the Czech Evaluation Society.

The aim of the roundtable was to introduce the participants to the concept of a learning organisation. The facilitators worked with the following questions:

What is organizational learning and what are the characteristics of a learning organization?
What does this mean in the practice of a non-profit organization?
What tools can we use to develop learning in our organisations?
How can we use evaluation to learn?
How does effective learning help us to have a greater impact?

1/ INTRODUCTION (brief theoretical background outlined by the facilitators – summarized below):

Why do we to learn?

In terms of the VUCA world concept , learning is absolutely essential if we want to succeed.

It also means creating a space in which the organization sets up so-called feedback loops, which are of two types:

Level 1: Are we doing things right? That is, “How well are we delivering on our organisation’s strategy?”
Level 2: Are we doing the right things? That is, “How relevant our organisation’s work is?”

If we are only engaged in the 1^st feedback loop, we may be solving very complex problems and creating more and more defined + rigorous methodologies and rules to manage them, but we never ask whether our action is even meaningful.

In contrast, strengthening the 2^nd loop means using only a few simple principles and constant feedback (similarly as within complex ecosystems, e.g., in an anthill).

What exactly is a “well-functioning organization?”

It needs to be looked at not only from a technical perspective but also from the perspective of relations, while the latter is absolutely key to delivering well-functioning services. We are used to using tools such as logframes and tracking outcomes in general, but the viewpoint of relations view is sometimes missing.

What is essential:

well-set processes;
well-functioning interpersonal relationships – without them there are no results.

A learning organisation is one that creates, preserves and transfers its knowledge in order to improve its performance. Evaluation and monitoring are one of the resources that we need to achieve this.

There is a distinction between formative evaluation (during the process; providing feedback that shapes us and helps us to improve; answers the question “are we doing things right?”) and summative evaluation (final, what has been achieved; answers the question “are we doing the right things?”). But even the summative evaluation should help us as a basis for deciding how to move forward.

An example of good practice is that an evaluation of what has happened precedes a decision on the way forward for the organisation + this forms the basis for the NGO’s theory of change (it can also serve as a basis for the donor).

Preserving and transferring knowledge to make it more effective – including its common components (digital or physical); but often lacking the use of this information in practice, to be used further (due to other, more urgent tasks). However, if time and resources are invested in working with these documents, or working with a team (e. g. shadowing), such an investment pays back many times over.

However, elements of the theory of change can be automatically applied to the functioning of the whole organisation – to the whole annual cycle.

How can we make learning an entertaining and regular part of our work?

The ideal is to start with small steps and partial tools that would help us deal with everyday situations and become a normal part of our practice.

EXAMPLES:

COLLABORATE, LEARN AND ADAPT FRAMEWORK used by USAID

Source: The USAID Collaborating, Learning, and Adapting (CLA) Framework, https://usaidlearninglab.org/qrg/understanding-cla-0

PRE-MORTEM

“Pre-mortem analysis report” – a tool that helps to identify the risks of a project before we start; it also teaches how to work with error; once we put the risks together, various evaluation questions arise that we can then use further.

Source: Pre-Mortem, translated and adapted from the original by the Alberta Foundation by Impact Academy, available from: https://knihovna.impactacademy.cz/wp-content/uploads/2020/03/Pre-mortem_Handout_prazdny.pdf

CHRONICLE OF TRAINING

So-called “Learning Logs”; there are several types. This tool gradually helps to make the most of success and failure in the organisation and to build organisational memory.

It also provides a framework for how to talk about a given success or failure.

Source: translated and adapted from the original by Impact Academy, available from: https://knihovna.impactacademy.cz/wp-content/uploads/2020/03/Kronika-uceni_Handout_prazdny.pdf

STILL FRAMEWORK

A tool to help individuals, teams and organisations reflect on events and identify the “lessons learned” they bring; besides, to describe the change that has happened as a result of the events.

Author: Barbora Komberec Novosadová, 2021

A question we can ask ourselves at the very beginning might be: “What do we need to be able to learn something?”

2/ In the second part of the meeting, a facilitated discussion was held in two groups of participants, based on the questions that the facilitators collected from the participants during the first exercise. Below we have written a summary of the suggestions, notes and comments arising from the group work and the subsequent group discussion.

HOW TO BEGIN?
(How to involve colleagues? What do we need to create the safe space to learn? What tools do we need? How do we make time for that?)

The organisation (its people) needs to desire to become a learning organisation
We need to abandon the idea that evaluation is something extra
The key is to say what we want and what we need
We need to ensure support from the management, key personnel across the organization + not only them but also other people in the organization (participatory and non-hierarchical approaches)
We need to prepare a plan to integrate evaluation and learning into the day-to-day running of the organisation
We need to speak the same language – showing management that the learning process produces results that are important to them
It is good to create a set of easily available tools and continuously update them
A good start is also to identify what a learning organization is for us (i. e. the one that is able to continuously evaluate one’s own actions, learn from them; that does not persist in what is not functional + to be able to define what it is)
It is good to set a goal to work towards
Do not use the word “evaluation” 🙂

WHAT MIGHT BE THE FIRST STEPS?

Experiencing the philosophy and methodology of Art of Hosting (or other participatory methods)
Regular feedback on benefits (not just money, but also time spent) – in diaries, during/after meetings
Experiencing “internship” in an organisation where “it works well”
Trying the developmental evaluation method: what did we do and why? What is the context? What now?

WHAT MIGHT FOLLOW?

Setting up a functional mechanism for institutional memory (e.g., mentoring)
Applying evaluation immediately after the event, for example after a meeting with volunteers; create space for this
Continuous evaluation one’s actions and learn from them
Not persisting in dysfunctional processes
Recognizing the real target
Naming the change mechanism continuously – “we make change based on this and that” (being aware of learning loops)
Inserting a learning element into the design of the new project itself
Clear dedication of the time for education – both personal and formal
Trying to use the same tool for a while so that people get used to it
Incorporate learning tools into the regular agenda
Framing issues in a positive way, emphasizing what works (reinforcing feedback)
Guided visualization – what things should look like, where the principles of the learning organization should be manifested; also find out what is shared and in what the ideas differ
Ask persistently the question “what would help you to make that thing better?”
Work on intrinsic motivation, create added value

CHALLENGE: HOW TO MAKE TIME FOR LEARNING AND EVALUATION?

POSSIBLE SOLUTIONS:

Integrate tools into the regular agenda
Incorporate a quick reflection at the end of each meeting: review what worked, what didn’t work + include it into what to do differently next time + use the “why” questions, e. g. “what would help us to make things better”, to provide deeper answers; for more introverted colleagues, scaling may help
Give higher priority to activities leading to learning
It is important to ensure that evaluation means add value to people – ideally immediately (it brings intrinsic motivation)

CHALLENGE: HOW TO GET THE SUPPORT OF COLLEAGUES AND MANAGEMENT?

POSSIBLE SOLUTIONS:

As above – speaking the same language (showing management that the learning process produces results that are important to them)
Take advantage of situations where this approach helps to “pull the thorn out of the heel” and show how it helps
Only one specific department/project can start – then it is easier to promote elsewhere
Letting the team experience the impact of their own activities; trying out different sub-tools and if successful, pointing this out and building on it
Use the OECD’s list of steps to implement the principles of the learning organisation in education as a guide (see below in resources)

THE CHALLENGE: HOW TO SUPPORT A MECHANISM FOR INSTITUTIONAL MEMORY?

POSSIBLE SOLUTIONS:

Mentoring and shadowing new employees
Give a more consistent structure to the records of important issues
Doing things simply and trying to streamline processes
Functional platform for searching by certain parameters, e. g. in final reports
Organizations that do large events usually compile a debrief document that describes the processes and what was important (what worked and what did not) – but briefly
Work with a platform that does not use emails, but all communication can be found there (the essentials) – e. g. Podio

WHAT TO AVOID?

Don’t try to break through the wall with your head 😊 (but go about things smartly)

SOURCES:

ARGOTE, Linda. Organizational Learning: Creating, Retaining and Transferring Knowledge. 2^nd edition.Berlin: Springer Science & Business Media, 2012.
KOOLS, M. and L. Stoll (2016), “What Makes a School a Learning Organisation?”, OECD Education Working Paper, No 137, OECD Publishing, Paris. https://oecdedutoday.com/how-to-transform-schools-into-learning-organisations.
LALOUX, Frédéric. Budoucnost organizací: průvodce budováním organizací v 21. století na základě evoluce lidského uvažování. Translated by Viktor Jurek. 2^nd edition. Prague: PeopleComm, 2020.
PINK, Daniel H. Pohon = Drive: překvapivá pravda o tom, co nás motivuje!. Translated by Veronika Poláková. 2^nd edition. Olomouc: ANAG, 2017.
SENGE, Peter M. Pátá disciplína: teorie a praxe učící se organizace. Translated by Irena Grusová. 1^st edition. Prague: Management Press, 2016.