Industry Case Study

Quality Metrics By Industry


In the airline and hotel industries, quality is often measured in terms of various customer service parameters (e.g. punctuality, food quality), along with measuring quality against competitors via occupancy penetration rates, and conducting importance-performance analysis. In the telecommunications industry, the quality of service is measured based on a set of international standards that cover things such as support, security, and technological performance; this industry also measures the quality of service experienced by their user base, based on whether user expectations have been met from the perspective of the consumers themselves. In the ecommerce and retail industries, quality is measured based on consumer satisfaction, third-party ratings of store and employee factors (mystery shoppers), as well as tracking the perfect order rate. Lastly, the financial industry uses a wide range of quality metrics depending on the type of service. Some of these include the use of the SYSTRA-SQ quality scale, various metrics that analyze the success of transactions, and KPIs relative to assets under management (withdrawal and growth). These findings have been explained in detail below.

Travel Industry Quality Metrics: Airlines & Hotels

1: Service Quality

  • The travel industry, including airlines and hotels, is largely one of providing services, as opposed to goods. As such, the quality of service is crucial in determining consumer satisfaction and profitability. For instance, academic studies regarding quality measurements in the airline industry are clear that “a high-quality service delivered to passengers makes it possible to identify the competitive position of airline companies by understanding passengers’ perceptions of choice factors. […] It is commonly believed that higher service quality can lead to a customer’s higher overall satisfaction and subsequently to positive behavioral intentions.”
  • In general, service quality is defined as “the degree of discrepancy between customers’ normative expectations about a service and their perceptions of [the actual] service performance.” When applied to travel industries, such as the airline industry, service quality equates to a series of interactions between guests and company representatives (i.e. customer service personnel), wherein the customer service personnel are focused on influencing the perceptions of guests and the image of the brands they represent.
  • The SERVQUAL model is used in the travel industry (and other service-related industries) to measure service quality, by capturing consumer expectations and perceptions across a range of service quality dimensions. These dimensions include physical/tangible, responsiveness, empathy, reliability, and assurance.
  • Academic literature that speaks to the topic of airline service quality measurements largely focus on the following service quality parameters: airline employees, baggage handling, punctuality, convenient flight schedules, seat comfort, in-flight service, food quality/service, in-flight entertainment service, airline safety, reliability of service, handling of customer complaints and abnormal conditions, and frequent flyer programs. These parameters logically align with the dimensions of the SERVQUAL model.
  • In the hotel industry, physical/tangible dimensions would include things like the modernity and aesthetic appeal of facilities, as well as the appearance of hotel employees; assurance/responsiveness dimensions would include things like employee promptness to attending to guest requests and transparency about when services are performed (e.g. maid services, continental breakfast); empathy would include things like personal attention given to guests, and understanding the specific needs of the guests; reliability would include things like doing things right the first time, following-through on promises, and minimizing the rate of error.

2: Market Penetration Index (MPI) – AKA: Occupancy Penetration Index

  • Market penetration indexing is a key performance indicator of the hotel industry. Often referred to as the ‘occupancy penetration index’, this metric measures the hotel against its competitors. More specifically, “this tool helps the hotel see its position and performance in proportion to the competitors and market in general.”
  • MPI is calculated by dividing the hotel’s ‘occupancy over a given period’ by the ‘total number of rooms available in the local market’ and multiplying this result by 100.
  • MPI is usually calculated at a local level (i.e. the ‘total number of rooms available’ figure is typically estimated at the local level, as doing so is the most relevant to analyzing competitors, given that hotels are mostly competing with each other for occupancy within specific geographic locations.

3: Importance-Performance Analysis (IPA)

  • Like SERVQUAL, the Importance-Performance Analysis (IPA) is a main tool used to measure and analyze quality and customer satisfaction in service industries, including the hotel and tourism industries.
  • According to academic literature, “IPA is a procedure that shows the relative importance of various attributes, and the performance of the firm, product or destination under study in providing these attributes. Its use has important marketing and management implications for decision makers, and one of the major benefits of using IPA is the identification of areas for service quality improvements. Results are displayed graphically on a two-dimensional grid, and by a simple visual analysis of this matrix, policy-makers can identify areas where the resources and programs need to be concentrated.” In other words, this analysis provides decision makers with a clear and visual understanding of where the company’s quality level stands in regard to meeting customer expectations, and shows them which areas they are performing well in, which areas need help, and which areas are more or less important. This information helps decision makers understand how to better allocate resources to improve and maintain service quality.
  • An IPA can be conducted once customer reviews/feedback has been obtained. At least one academic study lays out a step-by-step method for how to conduct an IPA using online reviews. The methodology for conducting the analysis is as follows: 1) harvest data from online reviews, 2) use the data to estimate the performance of each attribute, and 3) plot the performance for the attribute on the IPA grid. The grid itself is made up of four quadrants, with each quadrant representing how important the attribute is to the service experience and how satisfied the consumer was with the attribute according to their review feedback. An example of the grid is shown below:

Importance Performance Analysis Chart

Communications Industry Quality Metrics: Telecom & Wireless

1: Quality of Service (QoS)

  • In the telecommunications industry, the term Quality of Service (QoS) is used to label a “set of parameters related to the performance of traffic on telecommunication networks.” The ‘International Telecommunication Union Telecommunication Standardization Sector’ (ITU-T) defines QoS in this way: “The collective effect of service performance which determines the degree of service user satisfaction.”
  • Concepts such as service support performance, service operability performance, serverability performance, and service security performance all fall under the umbrella of QoS.
  • According to insights published by the Bandung Institute of Technology, telecommunication service providers that offer a better QoS gain a wealth of benefits from doing so, including higher customer retention rates and competitive advantages.

2: Service Level (G0S)

  • According to the ITU-T, service level (G0S) in the telecommunications industry refers to “a number of traffic engineering variables that provide a measure of adequacy of a group of resources under specified conditions.”
  • According to insights published by the Bandung Institute of Technology, the following is a list of G0S parameters that are considered to be of top importance when it comes to internet traffic: latency, jitter, and packet loss. A definition of each is detailed below.
  • Latency: A computation of end-to-end delay in regard to transferring packets.
  • Jitter: There are two types of jitter; time jitter and frequency jitter. Time jitter is defined as “undesirable deviations from end-to-end delay.” Frequency jitter is “a scale of frequency; the inverse of time jitter.”
  • Packet Loss: This is a variable that is arrived at through very complicated calculations that involve things like arrival distribution, the distribution of the service on the server, quantity of servers, buffer capacity, and queuing disciplines.
  • As the three parameters above are merely an example of those specifically used to measure the adequacy of the internet service based on the amount of traffic, there are various other G0S parameters that measure the adequacy of other elements of telecommunication.

3: Quality of Service Experienced by the User (QoSE)

  • QoSE is defined as the point of view of the user in regard to their experience using the service. The parameters that fall under this umbrella are generally described in non-technical ways.
  • Telecom providers measure their QoSE is by seeking and analyzing feedback from customers via survey or other methods of contact. This measure of quality is important to telecommunications companies because it reflects the users’ actual experiences combined with insights about what level of technical quality they expected from the service and whether those expectations have been met from the users’ perspective.
  • Aside from technological factors, QoSE analysis takes into account other factors that affect the customer, such as the contracting/onboarding experience, quality of technical support, and the general rapport between the customer and the company.

Commercial Industry: Ecommerce & Retail

1: Consumer Satisfaction (Service Quality)

  • While it’s clear that each of the industries covered in this report so far naturally focuses on service quality, it is also evident that each industry defines ‘service quality’ slightly differently and in a way that is specific to their own industry. The ecommerce industry is no exception. In this regard, ecommerce service quality has been defined in academia as “the consumers’ overall evaluation and judgment of the excellence and e-service quality offerings in the virtual marketplace.” These evaluations are conducted by companies by means of gathering consumer feedback.
  • Likewise, the parameters measured in regard to service quality are unique to the given industry. For ecommerce, the typical service quality elements taken into account are as follows: quality of service provided, customer service, management of processes, ease of use, information quality, website design, responsiveness, security, convenience, reliability, assurance, usefulness, receptiveness, website navigation, website/information layout, content, real-time support, and ease of search.
  • At least one academic study has concluded that “the two main areas of consumer satisfaction assessment in e-commerce are: website features and consumer service quality.”

2: Store and Employee Evaluation (Mystery Shoppers)

  • In the brick-and-mortar retail environment, quality is often measured through the use of evaluations by mystery shoppers. The Mystery Shopping Providers Association (MSPA) was founded 10 years ago and is a trade organization that currently has over 150 members (i.e. mystery shopping providers) . The MSPA states that mystery shopping industry has a market size of $600 million in the U.S. alone. Although the brick-and-mortar retail market is shrinking overall, mystery shopping is also applied as a quality measurement tool in ecommerce.
  • According to the Bureau of Labor Statistics, “Posing as regular customers, most mystery shoppers work as independent contractors for marketing research companies. These companies provide guidelines for each visit, based on the type of assignment. […] Sometimes, mystery shoppers take photos or videos to document their experience. They usually shop at a physical location, but they may do other assignments over the phone or online. […] Most visits take between 15 minutes and an hour. After the visit, mystery shoppers evaluate their experience. This might include explaining how they were treated or what they observed. Shoppers then submit the evaluation along with supplemental material, such as receipts, to receive payment. Whether they actually buy, and keep, specific items or services varies by assignment.”
  • When employing the use of mystery shoppers, retail companies are seeking to evaluate the following quality metrics: store hygiene and cleanliness, store ambiance, employee attitude and behavior, employee knowledge level, employee outfit and cleanliness, general evaluation of the shopping experience, and determine their Net Promoter Score.

3: Perfect Order Rate

  • ‘Perfect Order Rate’ is a retail industry KPI that “measures the amount of orders that are fulfilled and shipped without error or incident.” Errors and incidents include things like inaccuracies, damages, delays, and lost orders.
  • Measuring Perfect Order Rate is important when it comes to evaluating the quality of an ecommerce business, because a low perfect order rate means the supply chain is inefficient, and customer satisfaction and brand reputation is negatively impacted as a result. On the contrary, maintaining a high perfect order rate gives the brand an air of reliability among consumers and equates to cost savings for the brand as a result of less returns/money lost to damaged products, etc.
  • According to insights published by the Department of Management and Social Sciences at the National Institute of Technology in Hamirpur, some parameters analyzed in reference to perfect order fulfillment are as follows: number of orders delivered in full, delivery on customer commits date, accurate documentation, and perfect condition of the product. A paper published by the Institute defines these parameters in the following ways: “An order is considered perfect if the products ordered are the products provided and the quantities ordered match the quantities provided. […] A delivery is measured as perfect if the location, specified customer entity and delivery time ordered is met upon receipt. […] Documentation supporting the order line is considered perfect if it is all accurate, complete and on time. […] The product is considered ideal if the product is delivered with specification, with the correct configuration, with no damage and is accepted by the customer.”

Finance Industry Quality Metrics: Financial Services, Investment, & Banking

1: Retail Banking Service Quality (Retail Bank SQ) and the SYSTRA-SQ

  • “There have been a number of empirical studies of retail bank SQ. Most, but not all, of these have measured SQ by replicating or adapting the SERVQUAL model.” Regardless of the measurement model applied, there are a number of key dimensions that can be generally considered when attempting to analyze bank service quality objectively. These dimensions include customer service quality, technical quality, functional quality, and economic quality.
  • Economic quality is defined as “the value that a customer receives in a relationship, and comprises elements like profitability and productivity.” Some academic research has concluded that customers are not adept at identifying technical service quality problems and that they are most likely to notice functional and economic quality problems (although these findings may logically be overshadowed within the context of online banking).
  • SYSTRA-SQ is a quality scale invented specifically for the retail banking industry. This scale was developed as a result of in-depth academic research. The research focused on collecting qualitative data from consumers and analyzing it to isolate key themes which were then transformed into scale items. The SYSTRA-SQ scale consists of 21-items across four-factors of measurement specific to retail banking service quality. The four factors are s such: service SQ, behavioral SQ, machine SQ, and service transactional accuracy. This scale enhances the traditionally used service quality metrics/dimensions (i.e. customer service quality, technical quality, functional quality, and economic quality) by acknowledging that “customers are able to experience and evaluate organizational and transactional dimensions of service performance.” This scale was designed specifically for use in the U.K. retail banking sector, but it may still have practical applications in other markets as well.

2: Digital Financial Services Quality of Service (QoS)

  • There is no particular set of QoS parameters that can be applied to all digital financial service applications. Despite this, many countries regulate and require standards to be met by such companies operating within those markets. The ITU-T has laid forth a set of suggestions that it recommends regulators consider. These recommendations relate to end-to-end QoS KPI definitions for aspects of digital financial services.
  • Much of the recommended QoS KPIs revolve around the success of transactions. These KPIs include money transfer completion rate, money transfer completion time, money transfer false positive rate, money transfer false negative rate, money transfer failed transaction resolution rate, money transfer account stabilization success rate, money transfer account stabilization time, money transfer loss rate, and money transfer duplication rate.
  • The ITU-T also notes the importance of telecommunication QoS in relation to digital financial service QoS, as the quality of digital financial services relies most heavily on the quality of the telecommunications being used for the services’ operations.

3: Quality KPIs in Investment Banking: Assets Under Management

  • In regard to the investment sector, there are a number of quality KPIs that the industry relies upon, according to a benchmarking report published by Opsdog. These metrics encompass assets under management (AUM) and specifically include metrics such as AUM withdrawal rate and AUM growth rate. In general, assets under management is a key metric used to identify the leading investment banks and firms. This is a logical metric to analyze when attempting to determine quality objectively because AUM is defined as “the total market value of the investments that a person or entity manages on behalf of clients” (i.e the total amount of money clients have entrusted to the bank). According to Investopedia, “investors often consider higher investment inflows and higher AUM comparisons as a positive indicator of quality and management experience.”
  • AUM Withdrawl Rate: The AUM withdrawal rate metric is broken down in terms of departing clients and existing clients. The AUM withdrawal rate of departing clients is found by calculating the “difference between the total dollar amount of assets managed by the firm and the dollar amount of assets withdrawn by clients who are leaving the firms business, divided by the total dollar amount of assets managed by the firm over the same period of time, as a percentage.” The AUM withdrawal rate of existing clients is found by calculating “the difference between the dollar amount of assets managed by the firm and the dollar amount of assets withdrawn by clients who maintain one or more open accounts with the company, divided by the total dollar amount of assets managed by the firm over the same period of time, as a percentage.”
  • AUM Growth Rate: AUM growth rate is found by calculating the “sum of the dollar amount of assets managed by the firm and the dollar amount of assets recently added (through new clients, a merger or acquisition, etc.), divided by the total amount of assets managed by the firm over the same period of time, as a percentage.” There are also specific calculations that can be performed to measure AUM growth due to new clients specifically and AUM growth rate due to
Glenn is the Lead Operations Research Analyst at Simple Manifestation with experience in research, statistical data analysis and interview techniques. A holder of degree in Economics. A true specialist in quantitative and qualitative research.

The Performance of the Luxury Market in 2021

Previous article

U.S. Waste Management Trends and Key Players

Next article

You may also like


Leave a reply

Your email address will not be published.