Benchmarking Outsourced IT and OT Services

Overview

To improve information technology’s effectiveness in fulfilling business needs at competitive cost, many industrial organizations now apply processes from the IT Infrastructure Library (ITIL) to benchmark their own IT performance against KPIs. As more and more companies outsource selected IT activities to be able to deal more efficiently with varying workloads and domain areas, they also need to be benchmarking outsourced IT and OT services for performance.

More recently, the need has also arisen for companies to be able to benchmark the performance of outsourced services for activities related to operational technology-related manufacturing operations management (MOM).

This ARC Advisory Group Insight reports about this emerging trend based on user and vendor interviews and provides recommendations for managing the performance of outsourced IT and OT services.

Cost and Quality of IT Operations

During the last decade, industry has gradually improved the cost and quality of IT functions through improved practices.  Similar to other domains such as project management or asset management, available standards provide best practices for IT.  This includes guidance for setting strategy, planning, operating, and improving IT functions. ITIL, the worldwide de-facto-standard for service management, provides broad and publicly available professional documentation on planning, delivering, and supporting IT services to meet corporate requirements effectively and efficiently. 

The first step is for companies to benchmark their performance against internal or external benchmarks and maturity models.  Typically, while companies make progress toward improving their own IT activities against these benchmarks, they will also increasingly outsource non-core IT services that are necessary, but do not add significant value or provide them with a competitive advantage.  IT functions that are typically outsourced include providing standard desktop computing, networking, and (more recently) cloud infrastructure with associated service desks.

While non-core, to help ensure overall IT effectiveness and efficiency, these outsourced services are also benchmarked. More recently, outsourced specialized support and services for manufacturing operations management are also included in maturity assessments and benchmarks against ITIL.

Outsourced Services for Operations IT

Software for operations used to be a patchwork of legacy and in-house applications.  Manufacturing operations management (MOM) has increased in scope and maturity.  Many off-the-shelf suites now provide platform infrastructure such as business process modeling and connectivity, as well as a wide range of functionalities across production, quality, inventory, and maintenance.  As a result, applications built on those suites are more frequently rolled out at a large scale to cover this wide range of functions.  This makes the cost and performance of these applications become more visible to the organization.  An emerging trend is to include outsourced IT services such as hotline and project services or migration services associated with those applications in the overall performance assessment.

A recent trend that ARC has observed is for global industrial companies with multi-site MOM installations to engage in a dialog with the service units of the software vendors.  While, in the past, performance was mainly measured by response and resolution times; to a great extent this dialog can help transform reactive support and bug fixing into proactive activities.  These can range from improving documentation to increasing stability and reliability for maintenance and major releases alike.

The implication here is that the performance is measured by different KPIs.  ITIL processes are the common language in those discussions and KPIs can be agreed upon process by process.  ARC finds that outsourced IT services mainly concern the ITIL service operations and service planning processes.

The clear descriptions that ITIL provides in its standards force users to define their activities precisely, for example, to distinguish incident management from problem management.  Whereas the former concerns restoring service after the fact, the second involves root cause analysis and preventive action.  The standard is worth reading, since the processes were evidently derived from excellent IT operations and describes those processes in useful detail. The standard also covers IT deployment management, release management, and change management to help users avoid issues related to lack of coordination and promotes proactive measures to improve performance long term.

Security management is challenging for vendors, as different companies require different standards related to regional acceptance, thereby limiting the degree to which vendors can standardize it.  Generally, the evolving ISO 27000 series is becoming increasingly accepted in international relationships. We find that even the commitment to become ISO 27000 compliant can be sufficient to develop a relationship of trust.

Performance Assessment and Maturity Models

Maturity modeling for IT processes started at the Software Engineering Institute of Carnegie Mellon University.  The Institute’s Capability Maturity Model guidelines for improving the software process has become a global reference and its principles have been applied to many domains.   While ITIL provides a good starting point to define KPIs for IT service management, the use of a maturity model allows users to define degrees of excellence based on homogeneity of KPI scores throughout an organization.  Maturity models can also help businesses adapt and improve the processes over time as internal and external requirements change.

The IPW Stadia Model provides a maturity model to assess the performance of IT processes that is consistent with ITIL. The model has several dimensions, including the processes characteristics, customer value, and capability.  Table 2, provides a simplified representation of such a maturity model. 

IT organizations and IT governance are responsible for defining KPIs, measuring performance, and the manner in which performance levels are associated with maturity levels.  For outsourced IT services, these definitions need to discussed and agreed upon between the two parties.

IT/OT Convergence

With increasing convergence between IT and OT in industrial organizations, ARC believes this emerging trend will become more important in the future, with practices increasingly adopted by smaller players.  Questions remain however.  For example, will these practices remain focused on IT for operations, or will they also penetrate the actual automation domain due to IT/OT convergence and the emergence of smart manufacturing and Industrial IoT?

ARC believes that governance of automation should receive the same importance as IT governance, and the effectiveness and impact of outsourced automation services should be at the same level as outsourced IT services. Performance management should be applied across IT and OT, from automation via operations IT, to classical enterprise IT.

Recommendations

Based on ARC research and analysis, we recommend the following actions for owner-operators and other technology users:

• Apply ITIL processes to IT- and OT-related activities and set up KPIs and maturity models to measure the effectiveness and efficiency of these processes.
• For outsourced operations IT, and other outsourced OT, agree on performance measurement and management with the vendor
• Make sure the complexity and the effort associated with performance measurement and management is proportional to the scope.  Keep it as simple as possible

• ARC clients can request our lists of KPIs for each of the service operations and service planning processes

If you would like to buy this report or obtain information about how to become a client, please Contact Us  

Keywords: ITIL, Outsourcing, Maturity, Benchmark, Quality, Cost, Effectiveness, Manufacturing Operations Management (MOM), ARC Advisory Group.