The processes analysed in this report are centred around rolling stock bogie frames which have reached their end of life expectancy with an asset replacement project established to ensure safety and serviceability through to retirement of the rolling stock fleet in four years. This is a follow on from a previously unsustainable project to repair the existing frames.
Since 2012 this fleet has seen growing numbers and severity of fatigue cracking damage in its bogie frames. To combat this, the asset owner and its operator established a project to monitor the assets in-service and replace those which exceeded established damage limitations. This experienced good initial success however was put into jeopardy by a defect discovered during in-service monitoring on the first three sets of new frames.
Decision making within this project is a collaborative process between owner and operator with both groups’ priorities, risks and constraints monitored through an asset specific working group which is represented by various stakeholders and decision makers (Table 1).
| Asset Owner | Asset Operator |
| Rolling Stock Project Manager (DM) | Rolling Stock General Manager (DM) |
| Senior Rolling Stock Engineer (SH) | Rolling Stock Head of Technical Services (SH) |
| Rolling Stock Project Engineer (SH) | Business Development Manager (SH) |
| Head of Fleet (SH) | Commercial Manager (SH) |
| Table 1 – Management Process Stakeholders (SH) & Decision Makers (DM) |
Decision making processes were emergent, generally contributed to by all stakeholders and predominantly system 2 thinking (Frederick, 2005) with responsibility for final decisions being joint agreement through negotiation between the Project Manager from the asset owner and General Manager from the asset operator.
Both groups manage the safety, financial and operational service risks which are important to outline (Table 2) and are considered throughout. These risks were common to all decisions being made by the working group and were their driving force to mitigate against alongside the separate scope of uncertainty associated with each decision.
| Description I Impacts R1 Unable to supply enough serviceable frames – Stopped vehicles for use – Service delivery requirements not met R2 Excessively damaged frames operating in service Penalty clauses for the owner in operator’sagreement R3 Manufacturing facility unable to meet demand for frames – Vehicle failures in service Danger to passengers R4 Funding for project not being extended to fully support program |
Table 2 – Asset Management Risks
This report will focus on the working group’s emergent process to close the arising defect with the new frames being introduced into the fleet. They required a mixture of quick, decisive operational choices, and those which took on an analytical approach at a tactical level, collecting evidence and data required to inform a commitment to a single alternative.
Presented with a technical defect in newly manufactured bogie frames which required an operational level decision to be made by the project as to the immediate impacts of the defect and whether or not it warranted further analysis.
New frames were being monitored in-service to ensure they were performing as expected and the first frames to receive this inspection presented small defect indications via non-destructive testing (NDT). A choice whether the project needed to investigate this was needed immediately but carried with it a mixture of risks and uncertainties which would lead the project to either continue ‘at risk’ or move forward with an investigation.
Taking account of the overriding risks associated with the project, this decision was one driven by heuristics (World Minds, 2011). The risks associated with the project held a commitment to safety for passengers and service availability for the operator. A defect within an asset of any kind threatens both of these priorities and on that basis should warrant immediate investigation. However, there were conflicting pressures to this.
The investigation would cost money and the project needed to ensure the funding was available to facilitate the additional work which had not been scoped for. Second, if this defect was found to be fundamentally detrimental to the new frames it would void a replacement strategy that had been strongly advocated for and put the asset owner back in a position of financial certainty, this failing would bring back repairing old frames alongside significant cost and reputational damage.
| Figure 1 – Decision 1 Process |
The decision was made to investigate based on the groups priorities to mitigate safety and availability risks and being content to absorb the additional cost of investigation. A survey of a broad range of frames across in varying conditions was quickly mobilised.
Inspections were carried out on a range of frames; new, old, repaired, non-repaired etc to understand the prevalence of the defect in the new frames versus those which have been in service since the vehicles were manufactured. This showed the defect was sporadically present in all examples of frame condition, even those which were new and not fitted to a vehicle yet. With this information the working group would move into the second decision making phase of investigation.
With agreement this defect warranted action, and a survey completed to understand how widespread the defect was between existing and new assets, the second phase was to decide between two options.
- Stop production and investigate
- Maintain production whilst conducting a parallel investigation
These both had advantages and disadvantages of uncertainty, risk and known impacts of specific action. The risk associated with stopping was that production would need twelve weeks to remobilise prior to restarting. This would impart significant availability risk to the network and was an unavoidable cost of stoppages.
However, a stop in production would allow the investigation to conclude and rectify the defect without committing to the cost of frames produced with defects that may prove to be unsafe or unusable. Given the time needed to investigate, this could expose the asset fleet to a significant amount of safety, availability and financial risk which may not have been recoverable. This risk was reduced in part by the survey results showing the defects prevalence throughout the fleet therefore giving preliminary indications the defect was inherent and not new frame specific.
The project also had confidence with the manufacturer owing to their consistent, high quality products previously with no outstanding non-conformance as well as having an independent onsite observer who had not raised any significant quality issues in the three months leading up to the defect incident. This provided confidence that the defect would not be down to poor workmanship. However, did not mitigate the risk of continuing production of a bogie frame which had an inherent defect within its design which may affect the safety of the fleet.
| Figure 2 – Decision 2 Process |
Given the information the group had available, and the known risks and impacts of stopping production the decision was made to continue manufacturing whilst conducting the investigation in parallel. This gave the project the certainty of being able to supply new frames into the asset pool whilst the cause of the defect was found. There was still risk associated in this decision, but on balance the known prevalence of the defect provided confidence that it was a low safety risk as it was found in frames which were 10+ years old and the risk of having to use fatigue damaged frames in service for longer by not having supply of new frames for an extended period of time was seen as a greater risk to the core values of safety and service availability.
Following the decision to continue manufacturing and using new frames, a root cause investigation was undertaken in multiple stages. This was to assist the working group to remove uncertainties (Table 3) and was intent on using this additional information to determine whether the frame design needed to be changed or not.
| D3-U1 | Is there a manufacturing quality issue? |
| D3-U2 | Is it an inherent design fault? |
| D3-U3 | Does the defect present a safety issue to frames already manufactured? |
| D3-U4 | Was this defect a contributor to the wider fatigue damage seen in existing frames? |
| D3-U5 | Could it be resolved with a change in design or change in process? |
| D3-U6 | Considering the length of service required for the new frames, would a design or process change be worth the financial investment given the knowledge we had on existing frames and their extended service length? |
| Table 3 – Decision 3 Uncertainties |
The quality control records of all newly manufactured frames were reviewed to ensure no out-of-limits defects were present or repair work within the welding had been carried out during manufacturing. Alongside this, an assessment was carried out on the joint design and welding process itself to understand the method of construction and if it was promoting defects in the frame. With information from both of these alongside the historical knowledge of the assets and known service requirements of the new frames, the following answers were derived.
| Result | |
| D3-U1 | No. Zero issues found with workmanship or process quality control |
| D3-U2 | Yes. Defect was found on all frame conditions. |
| D3-U3 | Unlikely. Given the defect was apparent on all frame conditions, some of which had been in service for 10+ years without failure, experience showed that the defect did not pose an immediate safety threat. |
| D3-U4 | Yes. But only over very extended periods of time (10+ Years) and not consistently across all frames. |
| D3-U5 | Partially. The joint design could be modified to reduce the likelihood of the defect appearing but not remove it with certainty. |
| D3-U6 | No. The old frames which showed damage propagating from the defect area had been in service for 10+ years before failing therefore did not present a short or medium term safety threat. The new frame asset lifecycle only required 4 years. |
With the investigations carried out and uncertainties reduced or removed the working group became divided. The asset owner was satisfied with the results of the investigation and content that the key issue of safety was not under threat given the historic knowledge and the future service requirements of the new assets alongside the in-service monitoring of frames which was to continue regardless of outcome. Therefore, the asset owner was happy to recommend no design change was required. The operator did not completely share this opinion and was insistent on more in depth analysis of root cause to attempt to remove all uncertainty from the decision.
The contention between parties was primarily whether the outcome utility of further investigation would bring the group the additional information that would change what the existing information indicated was the right choice for the scenario.
| Figure 3 – Decision Process 3 |
At an impasse, additional analysis was carried out. This confirmed ‘no change’ choice alongside the continuance of in-service monitoring. The defect issue was now closed, however, debate on the value of the final analysis continued.
As the choice was made to not change the design, the impacts of delaying this decision were somewhat masked. However, if the initial choice had been to modify the design and then further confirmation was still sought, this delay had potential to expose the fleet to its primary risk of safety, with potentially more fundamentally defective frames in service. If the design change choice, either way, was made at decision three the impact of a potential change would have been minimised and the number of frames which could have posed more risk would have been reduced.
These decisions were typified by emergent processes full of uncertainty and how best to mitigate enough to ensure the right decisions were made, but equally balanced with ensuring the working group could show due diligence in their rationale. Neither of these criteria are incorrect paths to pursue and in this situation were highly relevant as safety was a main driver. However, a key element that was not considered within either was time. Although the working group made the right choices, upon reflection it was remiss of the group to not consider time as an important component to good decision making.
The operational choice made in ‘Process 1’ needed to happen quickly and was a good example of the group recognising the information they had was sufficient to choose a best course of action. Further down the process though they potentially lost sight of the information they had at their disposal and sought more data to validate decisions, which in hindsight could have been made quicker with the information already to hand.
By choosing to continue manufacturing whilst investigating, the group put itself at risk of being committed to the cost of frames being built whilst investigating if the analysis showed there was fundamental issues with the design which needed changing. This was a calculated risk the group were content to hold, however, delaying the final design change decision with further analysis exposed the project to financial commitment beyond what they were comfortable to absorb and increased the operational safety risks if the decision had been different. If a third criteria of time had been proposed, this may have changed the behaviour of the group and the outcome of the impasse at this final stage, bringing the issue to a close sooner with a decision either way with minimal deviation from accepted risk.
Impacts of Uncertainty on Process
Both aleatory and epistemic (Fox & Ulkumen, 2011) uncertainties were apparent, however, the methods used to mitigate these uncertainties did not consider these types. The prevailing method was to seek more information and data to remove uncertainty from decisions, but this incurred significant cost to the project and delayed the decisions being made which could have exposed the project to financial and safety risks.
By better understanding the concepts of uncertainty the working group may have been in a better position to make rational and justified decisions based on the information they already had available, without attempting to seek total removal of uncertainty through information which seemed apparent during the later stages of the process. Data & Information
The utilisation of data was deemed necessary to remove uncertainty and justify the groups course of action. This was required in some instances, for example collating data on the prevalence of the defect provided the information to its origins but understanding that data was never likely to remove all uncertainty is a criticism of the group’s methodology. This persistence of data and information collection was not only costly but caused timescales for decisions to take significantly longer than expected. The final delay to decision four was approximately two months.
Understanding the impacts of that delay alongside experience using a more heuristic (World Minds, 2011), system one approach was not ignored by the asset owner, but in this case, was disregarded in favour of data by the group. This awareness was certainly positive, but knowledge of how they can be used to inform rational decision making was missing and caused a reliance on data and information.
The recommendations to improve the process followed through this case study rest on three areas:
- Understanding the types of decisions to be made and establishing process where appropriate
When faced with a situation to be solved through collective agreement and decision making it is important to understand the types of decisions being made and understanding the criteria to assess these against, the intended utility and thereby agreeing a process to achieve these.
In this report the group made good use of fast information collection and heuristics to make an operational choice about the immediate threat of the bogie frame defects. However, they lost sight of time when considering decision’s further along the emergent process. Through this, they inadvertently exposed themselves to more risk than needed.
Agreeing an appropriate process, relevant to decision type, which they would all be content to make decisions against would have proven valuable in the latter parts of this case study in saving time, money and minimising risk.
- Recognising the types of uncertainty to be mitigated and utilising relevant methods to do so
Without a defined process for navigating a knowingly uncertain series of decisions, the group tended to play it safe and attempt to remove as much uncertainty as possible with information. Understanding that this may not always be a valuable approach to decision making may have allowed the group to speed up the decisions made and utilise the agreed process and criteria.
Being aware of the various types of uncertainty and their characteristics would likely have changed the groups approach in the latter stages and may have led them away from the further analysis and created certainty through logical reasoning. This would have maintained risk at a level that the group was content with and avoided the inadvertent exposure that the additional analysis and time caused.
- Techniques to utilise when data or information may not be readily available
The critique of the final decision making process highlighted the lack of value in the final set of information sought by the group, and within in this the decision could have been made earlier in the process without further data.
If the group better understood types of uncertainty, and the usefulness of system one (Frederick, 2005) influenced decision making they could have utilised experience and judgement to better rationalise the information gathered, speeding up the final process and removing additional risk. This could be achieved by realising the importance of certain information (World Minds, 2011) and the superfluousness of others (Figure 4).
In this case the information of specific importance should have been the initial survey, review of quality records and the life cycle knowledge of the defect. This showed that identical designs had lasted 10+ years in service before fatigue damage appeared and with the new assets only being needed to last 4 years, this could have been enough information to rationally decide a design change was not of value to the project or the asset group and therefore should continue as is.
Overall, the decisions made throughout these processes were the right choices. They were made through seeking data to form information (Thierauf, 1999) and provide rationale for decision making. This traditionally normative (Beach & Connolly, 2005), emergent process certainly achieved what it set out to, however there are areas where improvements could have been made which would have reduced time, cost and risk throughout.
Ensuring an understanding of the decision and uncertainty types surrounding the choices would have enabled the group to also understand the importance of process to achieve an optimal decision outcome and the usefulness of balancing system one and two thinking in decision making.
Beach L. R. and Connolly, T. 2005. The psychology of decision making: People in organisations. Thousand Oaks, California: Sage, pp.47-63.
Fox, C. R., and Ulkumen, G. 2011. Distinguishing two dimensions of uncertainty. In: Brun,W, Keren, G, Kirkeb0en, G. & Montgomery, H. Perspectives on thinking, judging, and decision making, (eds) Universitetsforlaget, Oslo
Frederick, S. 2005. Cognitive reflection and decision making. Journal of Economic Perspectives. 19, pp. 25-42.
Thierauf, R. J. 1999. Knowledge management systems for business. Greenwood Publishing Group.
van Groenendaal, W. J. 1989. Towards a workable definition of Decision Support Systems. Journal of applied systems analysis, 16, pp.99-112.
World Minds. 2011. Gerd Gigerenzer: Heuristics that make us smart. [Online]. [Accessed 22 February 2021]. Available from: https://www.youtube.com/watch?v=PIsNt_7sah4
Ciampa, D. 2018. Why new managers should be wary of quick wins. Harvard Business Review. 5th June 2018.
Hardman, D. and Harries, C. 2002. How rational are we?. The Psychologist. 15(2), pp 76-79.
