David Cox

At the beginning of my fellowship...

At the beginning of the fellowship period there was a clear plan for the project that would be the core of my Digital Fellowship.

The project was to be a collaboration between Health Education England, Great Ormond Street Hospital for Children and UCL Knowledge Lab / UCL EDUCATE ventures: ‘Research and analysis of barriers and facilitators to optimal clinician use of electronic health records systems (EHRS); strategy and design of educational tools and evaluation frameworks to support clinician learning and use of the EHRS’

The planned project objectives were:

1. Data-driven / evidence-based report on ‘effectiveness’ of current training
– Define good practice with EHRS, decompose good practice into constituent parts (derive and develop from Pontefract & Wilson paper)
– Establish quantifiable metrics of good practice with EHRS
– Assess whether training confers benefit / learning with quantifiable metrics of improvement
– Also consider the effect of repeated training (i.e. 2 training sessions of the same training) on user performance.

2. Strategise and design educational resources to support professionals using the EHRS
– Proof of concept educational resources that can be tested and evaluated for effect
– Demonstration of the evolution of design principles for education and edtech
– Design and evaluate the use of an edtech evaluation framework for healthcare education.

3. Promote the importance of modernised, personalised education of healthcare technology use through delivering a project with impact, and through widespread stakeholder engagement.

Project methodologies were identified relatively early into the project plan:
(6 methodologies to be used – 3 subjective / 3 objective modes for data collection)

1. FOI request – Assessing the NHS landscape of current or planned EHRS use – scale of NHS use and choice of technologies, and education and training for EHRS use (Objective)
2. Standardised questionnaires – based on theories of acceptance model and questionnaire used by Tubaishai et al. (Subjective)
3. Workshops using theory of change and logic model constructs (Subjective)
4. Structured interviews (Subjective)
5. Lab simulations of EHRS use (Objective)
6. Interrogation and data analysis of ‘Signal’ – software underpinning the Epic EHRS (Objective)

1. Freedom of information request and internet search – send to NHS trusts and other organisations including HDRUK, NHS Digital, Health Education England. Information requests regarding choice of technology r.e. EHRS, main drivers for move to EHRS and education and training planned to achieve these objectives. Planned education and training of the wider workforce.

2. Standardised questionnaires – constructed and interpreted to determine the key contributing factors of the ‘Acceptance model’ of the technology. Paper-based, to be used both opportunistically, but also at times of mandatory training to capture high-volume, diverse clinical workforce.

Aims: Assess professional- and personal-demographic related attitudes on the EHRS as a tool. Questions relate to perceived usefulness and perceived ease of use.

Depending on answers relating to negative perceptions on perceived usefulness vs perceived ease of use we will derive information as to whether efforts need to be focussed on improving training and user interfaces if challenges lie with perceived ease of use vs. modified education and communications strategy if perceived usefulness (and therefore sub-optimal clinician behaviours regarding the tool) is the barrier to optimal use.

3. Focus groups / workshops – aiming to generate insights into what users want from their education and training re EHRS use, and what modifications they would make to training / educational resource provision.

Aims: Based on Logic models and the Theory of Change construct. Facilitated workshops with diverse clinical workforce groups aiming to generate user-centred views of how practitioners evaluate the current training programme / education received, and project of the impacts of desirable changes to the current training / available educational resources.

Aim to conduct 3-4 separate workshops, each constructing at least 1 workable logic model. With the granular data collected from participant data (opinions) aim to conduct theme and semantic analysis (using open source APIs) and use network analysis by graph theory to ascertain common and divergent opinions between different clinical workforce groups.

The aim is to be able to draw conclusions on how clinicians would like to see education and training for EHRS use change, and whether these opinions vary by clinician role, clinical setting (e.g. intensive care vs in-patient vs out-patient) or education / training status.

4. Structured Interviews – analyse alignment between intention of education / training programmes for EHRS use from the perspectives of board/strategic level, the curators and providers of training, and the clinical workforce being trained.

Aim: To determine, is training delivering what is intended? And, is this what the clinical workforce perceives as important for their role or function? Conduct approx. 12 structured interviews ~ 45 mins each to generate granular data on intentions and perceived impact of the training. Use network analysis and graph theory as per focus groups / workshops to provide pipeline of consistency when interpreting data from different sources. ?use digital platforms such as Smapley and Dovetailapp.com to capture the user journey ad manage qualitative information from different sources.

5. Lab simulations – Perform direct observations of user training (naïve users during induction, experienced users getting repeat training as part of GOSH THRIVE programme for better EHRS use), and of clinicians using the tool for clinical practice.

Aim to see common contextual facilitators / barriers to optimal EHRS use, and the common behaviours and practical methodologies seen in clinical practice to overcome barriers or that potentiate ‘misuse’. Consider how altered education and training, and new educational resources and methodologies may support ‘good practice’.

6. Interrogation of the ‘Signal’ platform that underlies Epic EHRS at Great Ormond Street Hospital. ‘Signal’ records with high levels of detail (down to individual keystrokes and time-stamps) individual and group usage of the Epic EHRS.

Aim: To determine the consistency of use of the EHRS in the GOSH clinical workforce, and if hypotheses of ‘good practice’ with EHRS is borne out by real-world recordings of high-volume use. Interrogate Signal to look at the consistency of use across the clinical workforce at GOSH – aiming to assess which clinicians / groups of clinicians demonstrate usage consistent with the factors determined as surrogates of ‘good practice’. Also interpret where general usage could be improved and whether this can be altered through a change to current education and training provision.

During my fellowship...

This Fellowship year has not panned out as expected, with the core project being particularly affected by the COVID-19 pandemic. The hosting trust postponed institutional support for the project work (including publishing and appointing to job adverts for a supporting academic expert research associate and research nurse; not allowing external access to the clinical workforce to protect staff time and mitigate transmission of COVID-19 infection – which affected many of the planned project methodologies).

However, I have taken many positives from my fellowship year – and have been willing and encouraged to throw myself into, and learn from, a multitude of opportunities.

I have used Digital Fellowship to achieve some of my pre-planned project objectives – particularly those applicable to wider understanding around the state of EHRS deployment in the NHS – whilst contributing to Health Education England’s CCOVID-19 educational resource development, and also leading projects designing new digital tools to support both neonatal research and to evolve education. I have also taken part in supporting engagement in digital healthcare through being part of the HETT Steering committee and engaging with the Revolutionizing Healthcare series (van der Schaar labs).

1. Detailing and strategising EHRS use in the NHS

a. Article submitted to Future Healthcare Journal ‘EPRs in the NHS, current use and strategies for optimal use’. (Pending decision on publication)
b. Strategic paper submitted to HEE TEL team on managing education and training for digital healthcare tools using EHRs/EPRs as an example
c. Project infrastructure outline (listing methodologies to be employed and project plan) for collaboration with GOSH r.e. EPIC EPR use – for when project can recommence

2. COVID-19 educational resource development

I have been part of the HEE team (HEE’s COVID-19 Content Review Group) that has reviewed and classified external digital educational material on COVID-19 (from central systems such as the WHO, Public Health England , health and well-being charities and free-to-access industry resources) and designed a platform that presents the resources as an easy to search online directory format (akin to a streaming service). My role was in digital resource review, developing strategy for classification and in helping design the platform for ease of user interaction. We used HEE North’s WordPress website – https://www.telnorth.nhs.uk/ – and designed and added the COVID-19 Directory of Resources to it – https://www.telnorth.nhs.uk/directory-of-resources/

I have subsequently been redeployed from neonates into supporting adult critical care in the pandemic and have written a blog explaining how I utilised the resources to support my redeployment.

3. Design of new digital tools

a. Having identified a problem with neonatal research data collection (a misalignment between interval data collection of therapies and continuous monitoring of cardiovascular physiology) I worked with an external company, Prophesion, to design and implement research software which tracks neonatal intensive care therapies and outputs the therapies in the same temporal resolution as the continuous non-invasive cardiovascular monitoring. This will facilitate data science and machine learning to produce predicative analytics of cardiovascular function in neonates. This software is now governanced and in use at Imperial College Healthcare NHS Trust for research purposes in the regional tertiary NICU.

b. I am the project lead for a HEE-UCL IXN collaborative project, building a Clinical Learning Portal. This project is using a clinical lead to drive a collaboration between academia and industry to build a digital tool for the NHS. The aim is to build a digital tool to collate and annotate educational resources – turning individual resources into digitally personalised educational resources and pathways.

4. Strategy documents for HEE’s TEL programme on ‘Healthcare 4.0’ and personalised learning pathways

I am leading the writing of short strategy documents with HEE’s TEL team to guide how education for digital healthcare and digitally-supported education. These will be used internally to support strategy and commissioning of work.

5. Supporting wider engagement with digital healthcare in the NHS

I am on the HETT Steering Committee and have supported, and will continue to support, the Revolutionizing Healthcare series (both ongoing)

6. Establishing an educational innovation hub at Imperial College London Centre for Paediatrics and Child Health.

This new centre is looking to revamp how education underpins learning and the translation of research into clinical practice. We are actively applying (and I am joint education lead) for grants to support the creation of an education innovation hub – that will design and evaluate new learning pathways and resources, based around digitally-enabled learning methodologies.

Lessons learned:

Lessons learned from the fellowship:

1. The importance of buy-in.

The NHS exists as a complex network of relationships. Any project of improvement or change is likely to affect more people (both within and outside the organisation) than anticipated. Spending time at the outset of any project identifying the key stakeholders, ensuring their understanding and support, and what they expect from any project, is necessary to ensure that you (and your team) get the support required for the project to be successful. It is also worth understanding the factors the key stakeholders foresee as being potentially problematic to successful completion and implementation of the project so you can start to build strategies to mitigate these risks.

2. The need to marry long-term strategic planning with an agile project plan.

If COVID-19 has taught anything it is that external factors can have a major influence on even the best laid plans. However, the year has taught me the need for a clear overall strategy that generates understanding through all the stakeholders and allows changes to a plan to be mitigated without threat to the overall strategy.
Setting strategy and project priorities whilst running an ‘agile’ project plan, provides the greatest opportunity to rapidly identify and deal with unforeseen disruptions to the project.

3. Learning from adversity

Transformation in any area of healthcare can be challenging. Much of my learning from this year has come from when unforeseen challenges have arisen; challenges such as:

• needing to seek higher-level approval and support for a project that I conceived had already had the necessary approvals
• the project as conceived being postponed due to COVID-19 and the challenge of face-to-face staff interaction and evaluation
• the challenge of establishing governance collaborations between NHS, academic and industry partnerships.

Each challenge has provided opportunities for personal learning which I would/could not have found outside of such a fellowship and led to reflection that will positively affect any digital project I undertake in the future.

4. The need for a positive mindset and self-belief

There is much room for improvement in our current health service and a great need for ‘leaders’ to help manifest this change. Whilst it can feel difficult and intimidating as an individual to engage and improve large systems, identifying and being positive about the value you bring as an individual – even if not accepted initially – is key to being able to change the environment you work in for the better.