Chapter 9 - part 2
Completing the cycle: evaluation

 

The plan

• Specify exactly what you are evaluating. Set priorities against what you need to achieve and the time and resources available. Obtain agreement from the PCT/ practice or pharmacy team on the nature and scope of the evaluation.
  
• Describe the expected impact of the programme or activity and who will be affected.
• Define the criteria of success – these might relate to the what, how or outcome (see
  above).
• Identify the information required to demonstrate what the team achieved; e.g. observation, audit, prospective recording by staff or survey of patients receiving self care support.
• Determine what everyone involved in the evaluation will do by what time (see Tool 7).

If you are considering the quality of self care support with which a patient is being provided you should question whether it:

• was available – to all patients on an impartial basis (whatever their gender, age, social class, ethnicity, religion, postcode of their residence)

 

Table 9.1: Criteria for evaluation by PCTs of the extent to which self care is promoted and supported

Aspect of self care	What is it about?	How did it work?	What was achieved?
Employment of staff	Good practice resources available to practices, material available to add to job descriptions describing role around self care	Log shows all practice enquiries re help in describing self care role are dealt with satisfactorily	Responsibility for promoting and supporting self care is included in job descriptions and reviewed at annual appraisals for key personnel in all practices, community pharmacies and the PCT
Training	PCT-wide training needs analysis template includes the field of self care in relation to general practice teams and pharmacists	Training needs re promotion of self care identified on PCTwide basis; relevant courses provided or signposted	PCT maps competence gained; case examples of staff who gain new or delegated skills are communicated and acclaimed by senior PCT staff
Strategy	Strategy for promoting and supporting self care exists, with operational plan and essential resources; strategy is a local adaptation of national policies and guidance	Senior and relevant PCT staff actively involved: i.e. senior manager in primary care, clinical governance lead, training lead; scoping exercise undertaken to map good practice; gaps identified	Evaluation of implementation of operational plan shows key indicators have been met, e.g. number of staff trained in self care, patients provided with, or signposted to resources and take-up of specific help
Standards	PCT has policy and resources to promote standards in self care: including patient safety, patient satisfaction, accessibility and accuracy of materials	Monitoring process to check that self care is being promoted, supported and adopted appropriately	PCT collation of practices’ self-assessment of standards of self care show standards met 100% of the time for patient safety; 90% of the time for patient satisfaction and type of self care support resources
Patient satisfaction	Method of assessing PCT-wide patient satisfaction includes enquiry relevant to promotion and adoption of self care	Patient satisfaction assessed in respect of opportunities for convenient self care linked to general practices and community pharmacies	Changes made to supporting self care, in response to patient feedback from satisfaction survey(s)

 

Table 9.2: Criteria for evaluation by general practices or community pharmacies of the extent to which self care is promoted and supported across the team

Aspect of self care	What is it about?	How did it work?	What was achieved?
Employment of staff	Good practice exists e.g. material describing role with self care for new staff and updated job descriptions	Responsibility for promoting and supporting self care is reviewed at annual appraisals for key personnel in the practice or pharmacy	Good signposting recorded in pharmacy contract; appraisals reveal staff commitment to promoting and supporting patients’ self care
Training	Learning needs analysis template or in-practice event exists or version that is part of PCT-wide initiative; includes field of self care in relation to the practice or pharmacy team	Training needs re promotion of self care identified throughout the practice or pharmacy; relevant course provided inhouse or signposted	Practice/pharmacy maps competence gained
Skill mix	Protocol or plan for teamworking and learning in general practice/pharmacy about self care	Audits to assess if self care is being supported via teamworking protocols	Case studies or problem reviewed by practice/ pharmacy shows team is working in multidisciplinary way taking on new skills in promoting and supporting self care
Standards	Practice/pharmacy has protocol to monitor standards in self care: including patient safety, patient satisfaction, accessibility and accuracy of materials	Monitoring process to check that self care is being supported and adopted appropriately	Practice/pharmacy’s selfassessment of standards of self care show standards are met 100% of the time for patient safety; 90% of the time for patient satisfaction and type of self care support resources
Patient satisfaction	Method of assessing patient satisfaction includes enquiry relevant to promotion and adoption of self care e.g. OTC medicines	Patient satisfaction assessed in respect of opportunities for convenient self care linked to e.g. OTC medicines	Changes made to supporting self care, in response to patient feedback from satisfaction survey(s) e.g. OTC medicines

 

• was appropriate
  
• was safe
  
• was acceptable to the patient (e.g. type of treatment, frequency, fits with their religious or ethical beliefs)
  
• was delivered at the right time in the right way
• achieved the desired effects or outcomes
  
• achieved the desired outcomes with minimum effort, expense and waste.
  

Some examples of how you might focus on the essential components of self care support are summarised in Tables 9.1 (for a PCT) and 9.2 (for a general practice or pharmacy team). Each table has five aspects of the service relating to self care on the vertical axis, with what it is about, how it worked and what was achieved on the horizontal axis. In this example, you could generate up to 15 different aspects of the service to evaluate. Focus on several of these 15 aspects to look at the quality of self care support from various angles.

For example, if you were carrying out an evaluation of the promotion of self care you provide for people with asthma in a general practice, you might look at:

• whether patients from hard to reach groups, such as young people who receive self care advice on prevention of attacks, put advice about self care into practice and relieve worsening symptoms
• the process of communicating self care well with patients
• the extent to which a GP, nurse or pharmacist stick to the protocol to emphasise self care at successive consultations.

For each of these components you would organise a separate but linked audit.

 

 

References

1 Pawson R and Tilley N. Realistic Evaluation. London: Sage; 2000.

2 Chambers R and Wakley G. Making Clinical Governance Work for You. Oxford: Radcliffe Medical Press; 2000.

 

part 1

This document is intended as a guide only.
© Crown Copyright September 2006.
The text in this document may be reproduced free of charge in any format or medium providing that it is reproduced accurately and not used in a misleading context or used for commercial gain. The material must be acknowledged as Crown Copyright and the title of the document specified. Where others have been referenced on specific documents and tools, the organisation must be acknowledged in full and the title of the document specified. All links to external websites have been provided for your information and the Working in Partnership Programme take no responsibility for any material or content accessed on any such websites