Outcomes Measurement and Evaluation Capacity Building Project
About the Project
The Federation is undertaking a project designed to build the evaluation capacity of selected Victorian CLCs and create a shared outcomes measurement framework for the entire Victorian CLC sector. The project is generously funded through Victoria Legal Aid’s Innovation and Transformation Grants Program.
The project has two key objectives:
- To collaboratively build a sector-wide outcomes measurement framework that is endorsed by the Victorian CLC Sector as representative of the key outcomes it aims to achieve through its activities and service delivery.
- To work with seven Victorian CLCs and the Federation to build their evaluation capacity so that they can better monitor and evaluate the impact of their work.
Sector-wide Outcomes Measurement Framework
The Federation, through its Members, has identified that the development of a sector-wide Outcomes Measurement Framework will assist the Victorian CLC sector to demonstrate, articulate and measure the outcomes it achieves through its activities and service delivery. This will strengthen the ability of CLCs and the Federation to advocate on behalf of the sector and clients.
The Framework will also be accompanied by an M&E toolkit for the sector that will have tools, templates and instructions for data analysis and reporting.
Phase 1 Draft Framework Released
The Phase 1 Draft of the Sector-wide Outcomes Measurement Framework has been released and is available here.
We are providing this to share with the sector the High Level Outcomes identified through consultations to date.
With the assistance of Lirata Consulting, we are currently developing the full draft Framework. This will include intermediate outcomes, indicators and data collection tips that will assist CLCs to measure the outcomes in the Framework. These will draw on input gathered from the sector to date as well as additional consultation where required.
The full Draft Framework will be released by mid-2017 and we will be holding further consultation later in the year to get further feedback from the Sector.
Consultation Process for Framework Development
The Framework is being developed through a collaborative and sector-driven process. This means that the content of the Framework will be determined through consultation with staff from Victorian CLCs.
We held the first two Roundtable consultation sessions on 29 July 2016 in Werribee and 4 August 2016 in the CBD with over 50 representatives from 33 CLCs attending and sharing their insights about the common outcomes that CLCs work towards.
See the Consultation Plan for further details about the consultation process.
Evaluation Capacity Building in CLCs
Evaluation capacity building (ECB) means engaging in a process to increase individual and organisational skills, knowledge and motivation to conduct and effectively use evaluation.
There are a range of strategies that can be used to build evaluation capacity and the best combination of strategies will depend on an organisation’s objectives and context. In this project we are building CLC capacity through a combination of:
- Training in monitoring and evaluation basics for CLC staff;
- Expert support to CLCs to develop theories of change, M & E frameworks and M&E Plans;
- Expert support and mentoring to CLC staff when implementing their M & E frameworks;
- Peer support, through the creation of an Evaluation Practitioners Group;
- The creation of an M&E toolkit and support to CLCs in trialling tools
Some of the resources produced from the training and other activities will be shared with the sector through the Federation’s intranet.
The project consultants, Lirata Consulting, have produced this ECB Model which further explains ECB in the context of this project.
Project Consultants – Lirata Consulting
The project is being delivered in partnership with Lirata Consulting who are leading the development of the Sector-wide Outcomes Measurement Framework and working with the CLCs to build their staff’s evaluation capacity and their organisational M & E systems. The project consultants are Mark Planigale, Nadu Dove and Elaine Hendrick. You can find out more about Lirata at their website http://www.lirata.com/
Below is a list of and links to publicly available reports on outcomes measurement and evaluation in legal assistance services. Sharing the resources on this website does not necessarily mean the Federation endorses them.
Reports on Outcomes Measurement and Evaluation in Legal Assistance Services in Australia
Curran, L. (2007) Making the Legal System More Responsive to Community: A Report on the Impact of Victorian Community Legal Centre (CLC) Law Reform Initiatives, Australian National University – College of Law, ACT.
Curran, L. (2012) A Literature Review: examining the literature on how to measure the ‘successful outcomes’: quality, effectiveness and efficiency of Legal Assistance Services, Commonwealth Attorney General’s Department
Curran, L. (2012) We can see there’s a light at the end of the tunnel now: Demonstrating and ensuring quality service to clients, Legal Aid Commission, ACT
Curran, L. (2013) Encouraging Good Practice in Measuring Effectiveness in the Legal Services Sector, Legal Workshop, ANU College of Law, ACT.
Diguisto, E. (2012) Effectiveness of public legal assistance services, Justice issues paper 16, Law and Justice Foundation of NSW, Sydney.
Storer, B.E., Stubbs, J, & Lux. C. (2012) Economic Cost Benefit Analysis of Community Legal Centres, report prepared for the National Association of Community Legal Centres Inc by Judith Stubbs and Associates.
Rich, N. (2009) Reclaiming Community Legal Centres: Maximising our potential so we can help our clients reach theirs, Victoria Law Foundation Community Legal Centre Fellowship 2007-2008 Final Report
Allen Consulting Group (2012) Review of the National Partnership Agreement on Legal Assistance Services Working Paper One: Final evaluation framework, prepared for the Australian Government Attorney-General’s Department
Allen Consulting Group (2013) Review of the National Partnership Agreement on Legal Assistance Services Working Paper Two: Evaluation of Legal Assistance Services, prepared for the Australian Government Attorney-General’s Department
Wilczynski, A, Karras, M & Forell, S 2014, The outcomes of community legal education: a systematic review, Justice issues paper 18, Law and Justice Foundation of NSW, Sydney.
Forrel, S & Gray, A (2009), Outreach legal services to people with complex needs: what works? Justice issues paper 12, Law and Justice Foundation of NSW, Sydney
If you know of other reports you think we should include send them to Rohan Thwaites, Outcomes Project Manager, firstname.lastname@example.org
Evaluation and impact reports from CLCs
Women’s Legal Service Victoria (2015) Impact Report 2014-2015.
Justice Connect. (2015) Impact Report 2014-15. Melbourne: Justice Connect. Report prepared by Emma Pritchard.
Not for Profit Law (2015) Evaluation of the Not-For-Profit-Law Information Hub’s Launchpad Resources. This evaluation looks at the reach and effectiveness of the Launchpad Resources, which are online resources providing legal information to start-up not-for-profits. The evaluation uses a range of methods, including the use of control groups in a focus group setting to help determine the impact that providing guidance to a user has on their ability to navigate online legal resources.
Have an evaluation report from your CLC that you want to share? Send it to Rohan Thwaites, Outcomes Project Manager email@example.com
Resources on Monitoring and Evaluation
There are a wide range of monitoring and evaluation resources online. Some of our favourites are:
The Evaluation Toolbox: http://evaluationtoolbox.net.au/
The Evaluation Exchange: http://hfrp.org/evaluation/the-evaluation-exchange
Australasian Evaluation Society: http://www.aes.asn.au/resources/useful-links-to-evaluation-resources/other-useful-websites.html
See also The Change Toolkit on this website, which has further information and resources for evaluating advocacy in CLCs.