Introduction
The process under consideration is that relating to the current purchase of a new Computed Tomography (CT) scanner, by a Hospital Trust, as part of its capital equipment rolling replacement programme.
The Trust currently has 4 scanners across 2 hospital sites for both emergencies and planned investigations.
During the Covid-19 pandemic the Trust has used CT scanning as its preferred imaging modality when diagnosing the disease in patients, this has been a novel usage which has worked well.
The Trust is aware that technology has moved on since it last purchased its existing scanners and it is keen to review its options.
The Trust must adhere to NHS Procurement regulations which governs decision-making in this area. However, the Trust has flexibility in terms of the make and model of the CT scanner it procures.
The main decision in this paper centres around the type of CT scanner to be purchased, addressing functional and operational requirements which will be used to inform the business case. This decision cannot be made in isolation and the various stages required to facilitate and inform this process are shown at Figure 1.
| Figure 1 – Current Process Schematic |
The Current Process
This section examines the process steps referenced in the introduction. It considers issues relating to the current process and how they might occur.
Agreeing to Proceed
The Radiology Clinical Lead has concerns about the CT scanning service in the Trust which she relays to the Diagnostic Services Manager (DSM). The concerns centre around the age of a machine in the Radiology Department on the principal site. The current guidelines for a CT scanner are that it should not exceed 7 years and this machine has been in service for 8 years. The lead clinician puts her case to the manager who agrees that there is a requirement and gives formal approval to establish a project.
The clinical lead has identified that there have been service disruptions but although downtime is recorded the reason is not specified and could be caused by issues other than machine failure, for example unavailability of a radiologist. The clinician feels that downtime is mostly caused by machine failure but this could be availability bias (Tversky & Kahneman, 1973) as this was the most recent reason for service disruption. This has little impact on the decision to proceed but it has influenced the clinical lead’s view of the reliability of the machine.
Mobilisation of the Project
Initially, the DSM must negotiate with the Projects Director to secure a skilled Project Manager and also to agree the time that the PM will be able to devote to the project. The negotiation to secure the resource demonstrated some optimism bias on behalf on the DSM in terms of the time commitment required.
Next, the PM must assemble the Project Team and decide on its composition. It is found that the same people within Diagnostic Services are used on technology Project Teams within the Trust. They are people that are familiar with each other and the ways of working of the organisation. They work ‘comfortably’ together and rarely confront each other exhibiting a high degree of confirmation bias (Wason, 1960). They are generally people that are wellestablished within the department having worked there for many years and as such are prone to status quo bias (Samuelson & Zeckhauser, 1988).
The radiology department has Siemens CT scanners and the majority of its other diagnostic devices are Siemens devices which the Project Team are happy with and gives a significant level of anchoring bias towards Siemens as a supplier. There is a risk that this might conflict with the view of the clinical lead.
There is a further risk that the Project Team do not effectively represent the views of the whole of the radiology department and only a subset of it.
Exploring the Market
As the Trust has not procured a CT scanner for over 5 years the clinicians are keen to survey the market. The Trust solely operates Siemens CT scanners but the Project Team have agreed to also review machines from Philips Healthcare, GE Healthcare and Toshiba Medical Systems Corporation.
Presentations were provided by the suppliers to the Project Team which were used to inform the decision-making process. This activity is prone to attribute framing (Levin, et al., 1998) with each supplier highlighting the positive features of their scanner whilst avoiding to address any less-desirable aspects. Each supplier will be looking to position itself as the preferred choice.
The author attended the supplier demonstrations and subsequent to studying ‘framing’ has been able to reflect on its use. Whilst all organisations played to their strengths and adopted a business-like approach, the existing supplier took a more personable stance. They capitalised on their existing relationship and subtly alluded to investment in their technology priming sunk-cost bias. Kahneman and Tversky’s (1979) prospect theory describes the impact of the effect of losses in comparison to gains and the Project Team appeared to be influenced by the negative aspects of moving to another supplier.
Selecting the Preferred CT Scanner
Once the Project Team was familiar with the available technology, they felt they were able to take a more informed view relating to their functional requirements. The Project Team identified that technical decisions need to be based on the following categories:
- Performance
- Patient Safety
- Usability
- Support Services
Sub-categories were agreed for each category and were designated as either ‘Essential’ or ‘Desirable’ with no further qualifications specified.
There is a general consensus within the radiology profession that the more slices the CT scanner is capable of performing the better the image. Additionally, the rotation speed reduces motion blur and in instances where motion may be an issue this can be important. (see Figure 2)
| Figure 2: Components of a CT Scanner (adapted from (Quora, 2021)) |
In order to determine the best solution for number of slices and rotational speed the team had to assess patient data. The team decided to review data from their internal systems relating to numbers of CT scans carried out per week, the body part scanned and the age of the patient as these would most heavily impact on the number of slices and rotational speed.
A high proportion of children (children are prone to move about) and cardiac patients (hearts beat and require greater image definition) would result in a requirement for a high rotation speed and a larger slicing capability. The team focused on identifying the most valuable data to meet their needs. This was a good decision and aligns with the view of Haksoz, et al., (2018) that often having less data can simplify decision-making.
The Project Team decided to use two years of data, 2018 and 2019, to base their patient volumes and patient mix on. It was felt that 2020 was an unrepresentative year due to Covid19.
The Information Department were asked to retrieve data for patients who had scans in these years using age and READ codes (a medical coding system which identifies the type of procedure) as key fields. The data was retrieved from the Patient Administration System (PAS) and the Radiology Information System (RIS). The data was manipulated, via MS Excel, by the Information Department using an approach identical to that of Davenport and Prusak (1998) as shown in Table 1.
| Information Activity | Description |
| Contextualised | Data referred to CT Scanners |
| Categorised | By age and READ code |
| Calculated | Aggregation on a weekly basis |
| Corrected | Some anomalies were identified which resulted in data being amended in the PAS and RIS applications |
| Condensed | Summary information was provided with age data separated into under 18s and over 18s |
| Table 1- Data Manipulation Carried Out by the Information Department |
The decision to only analyse data from 2018 and 2019 introduced bias. A larger data sample across more years would smooth any anomalies and provide more representative information.
There is a risk in using a descriptive rather than predictive data mining technique that the decision will be more reactive than predictive and that it will not properly predict the future capacity requirement.
The decision to exclude data from 2020 gives rise to concerns about uncertainty relating to this Covid-19 in the future and its impact on the CT Scanning Service. Using Millikens’s (1987) classification system the uncertainty is best described as being epistemic in nature and exhibiting the following:
• High state uncertainty – It is hard for the Project Team to assess how Covid-19 will progress in the next few years. Although current analysis provided by researchers suggests that the current vaccines will be able to cope with new variants the validity of
this information cannot be fully assessed by the Project Team resulting in epistemic uncertainty.
- High effect uncertainty- if the virus mutates and the current vaccines prove to be ineffective there will be an increased demand for CT scans.
- Medium response uncertainty – It is anticipated that lessons-learned will inform how the department will respond in the future.
The Project Team have recently evaluated the information they have accumulated and identified a preferred option which will be presented to the Capital Planning Group within the Trust for consideration as part of the business case for the procurement. They are keen to procure a lower specification Siemens CT scanner.
Opportunities for Improvement in the Decision-making Process
This section discusses how the process might be improved by identifying some tools and techniques that could reduce bias, more accurately compare options and solve the dilemma of whether to procure a low or higher specification machine.
Mitigating Biases
The outcome of the process was unsurprising. The clinicians on the team were all familiar with Siemens diagnostic equipment. Maintaining the status quo provides a smooth transition at installation in relation to training and also provides the ability to move radiographers between scanners. However, the very strong anchoring bias linked with the confirmation bias meant that other suppliers were not fairly assessed.
It would be prudent in the future to ‘shake-up’ the Project Team. This could be by using a mix of people on the team that are not familiar with each other or introducing the role of devil’s advocate to challenge the decisions made.
The Project Manager and Project Team were unaware of the concept of biases and did not know to look for them. Training in this area would be beneficial and a checklist of common biases which could be used as an aide-memoire by the PM could be useful.
The Information Management Team would benefit from training on the implications of data bias.
Enhancing Analysis
Whilst the initial decision to proceed with the project was sound it was approached from the point of view of procuring a CT scanner to provide an identical service to that which was currently being delivered.
The focus of the investigation was on the requirements of the new machine and the capabilities of the existing machines were not considered. The functional attributes of the existing machines should be fully documented and a gap analysis performed which would highlight any existing issues that would need to be addressed in the new machine. It would also be useful at this point to look at data relating to service disruptions relating to CT scanners and isolate those that are machine related to obtain a true measure of machine availability and reasons for malfunctions so that these failures are not replicated in the new machine.
A more rigorous approach is required to requirements analysis. The Simple Multi-Attribute Rating Technique (SMART) (Edwards, 1971) would formalise this activity. This first step would require the team to identify their main requirements, perhaps 3 or 4 for each of their categories of ‘Performance’, ‘Patient Safety’, ‘Usability’ and ‘Support Services’. The attributes would need to be assessed for each of the suppliers’ CT scanners under consideration. The attributes will either be natural where a direct measure can be obtained or judgemental requiring a subjective measure.
An example for ‘Performance’ attributes is shown at Figure 3.
| Alternatives | Attributes | ||||
| Number of Slices | CT Rotation Speed | System Cycle Time | |||
| Siemens Model … | |||||
| Philips Model … | |||||
| GE Model … | |||||
| Toshiba Model … | |||||
| Figure 3 – Initial Stage of SMART Analysis |
The next stage would require the attributes to be weighted. This could be carried out by the Project Team but there would be benefits to broaden this review to ensure that a more representative view is obtained.
By weighting the required attributes of CT scanners and then normalising them a more accurate comparison can be made between makes and models. An overall multi-attribute utility score can be derived for each option relating to its functionality, providing an order of preference.
Although, the above opens up thinking by considering the impact of children and cardiac patients it does not identify opportunities for service development and as such The Project Team might be satisficing (Simon, 1956) by accepting a solution that was ‘good enough’ for their purposes. Using SWOT analysis in conjunction with SMART would open up thinking and might have resulted in the Project Team recognising that there is some potential to use CT scanning to replace cardiac angiograms which are invasive in nature (Lewis, et al., 2016). Although a scanner that can provide this facility would be more expensive it would mean that post-procedure care is not required which consumes resources such as nursing time, wound dressing, beds with their associated costs.
One of the greatest uncertainties when making the selection relates to future projections of patient numbers and patient mix. In terms of patient numbers, it is essential that theequipment chosen can handle the patient workflow. It is difficult to be specific about the numbers of patients that may require an emergency CT scan which could be as a result of a road traffic accident or currently Covid-19. It is also difficult to assess the number of routine scans that might be required. The risk of not being able to deliver the CT scanning service should be quantified, addressed and managed by the Project Team.
As there is a strong correlation between specification of the machine (low or high) and the numbers of children and cardiac patients using the CT scanner it would be most useful to analyse this requirement in further detail.
The decision tree process would be appropriate for this analysis with probabilities H1 to H4 and L1 to L4 applied to the uncertainties as shown in Figure 4.
Figure 4: Decision Tree Analysis
The utility of each outcome (U1 to U8) is specified by the Project Team with a higher value being given to value for money outcomes and lower values that would result in a procurement that is either:
- over specified and hence incurring extra cost
- has potential for repeat scans to be carried out due to poor image quality
- results in missed opportunities to provide non-invasive cardiac reviews to patients
The probabilities and utilities can be combined to calculate the subjective expected utility values for the outcomes thus aiding whether to procure a low or high specification machine.
The approach should be repeated with differing values of H1 to H4, L1 to L4 and U1 to U8 to carry out sensitivity analysis to give an indication of the robustness of the solution.
Whilst this approach is not free of biases and involves a high degree of subjectivity it does force the consideration of possible outcomes of a decision using a format which is readily understood.
It is not felt that this decision-making process could be automated but it might benefit from the introducing of a business intelligence application which could sit above the existing PAS and RIS systems facilitating the combination, aggregation and presentation of the information.
The approaches above would require resource and commitment from the Trust to provide training relating to bias and the use of tools and techniques described earlier. There is a risk that even if the Trust decided to invest in these areas that there would be resistance from staff who are not predisposed to change (Lumbers, 2018).
Conclusion
While the main decision-making would appear to be centred around the make and model of a new CT scanner, it has been interesting to look at how decisions made in the previous steps of the process can ultimately influence thinking in this area.
The Project Team feel that they have taken a normative approach adopting System 2 thinking (Kahneman, 2012) but this review would suggest that the decision-making has been subject to bounded rationality (Kahneman, 2003). The Trust needs to be conversant with different types of bias, challenge it and be aware of how to minimise risk associated with it.
The team identified the main machine parameters (CT rotation speed and number of slices) that would be key to their selection success and how these linked to patient data (age and type of procedure) thereby simplifying the decision-making process. This is a good approach and one that should be encouraged in future reviews.
It is suggested that in future procurements the Trust should use a combination of SMART and SWOT analysis to ensure that options are fairly assessed and opportunities are not missed, in this case angiograms being delivered via CT scanning. This approach would also identify strengths and weaknesses delivered by the current suite of CT scanners and identify any existing gaps in current service provision which need to be addressed as requirements.
The use of decision trees would have facilitated the decision of whether to go for a lower or a higher specification of machine and it is a tool that would facilitate decision-making in the future in other clinical areas.
There is a technological opportunity to interface a business intelligence application with the PAS and RIS system but this may be costly and would not be deemed to be essential in the context of this procurement.
It is felt that with minor changes the selection process could be improved to provide a more comprehensive review of requirements and a more equitable assessment of machines from different suppliers also considering future opportunities for service delivery.
References
Davenport, T. & Prusak, L., 1998. Working knowledge: How organisations manage what they know.. s.l.:Harvard Business Press.
Kahneman, D., 2003. A perspective on judgement and choice: Mapping bounded rationality. American Psychologist, 58(9), pp. 697-720.
Kahneman, D., 2012. Thinking Fast and Slow. London: Penguin.
Kahneman, D. & Tversky, A., 1979. Prospect theory: An analysis of decision under risk. Econometrica, 47(2), pp. 263-291.
Levin, I. P., l, S. S. & G. G. J., 1998. All Frames are Not Created Equal: A Typology and Critical Analysis of Framing Effects.. Organizational Behavior and Human Decision Processes, Volume 76, pp. 149-188.
Lewis, M., Pascoal, A., Keevil, S. & Lewis, C., 2016. Selecting a CT scanner for cardiac imaging: the heart of the matter. British Journal of Radiology, Volume 89.
Lumbers, M., 2018. Approaches to leadership and managing change in the NHS. British Journal of Nursing, 27(10), pp. 554-558.
Quora, 2021. How does a CT scanner work?. [Online]
Available at: https://www.quora.com/How-does-a-CT-scan-work-Does-it-mean-a-group- ofx-rays-What-is-the-use-of-a-CT-scan-Where-is-it-used [Accessed 26 February 2021].
Samuelson, W. & Zeckhauser, R., 1988. Status quo bias in decision making.. Journal of Risk and Uncertainty, Volume 1, pp. 7-59.
Simon, H., 1956. Rational Choice and the Structure of the Environment. Pyschological Review, 63(2), pp. 129-138.
Tversky, A. & Kahneman, D., 1973. Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), pp. 207-232.
Wason, P., 1960. On the failure to eliminate hypotheses in a conceptual task.. Quarterly Journal of Experimental Psychology, Volume 12, pp. 129-140.
