To Care, or Not to Care: Using XmR Charts to Differentiate Signals from Noise in Metrics
In a data-driven organization, tracking metrics is essential for making good business decisions. When you only have a few key metrics to keep an eye on, the review process is usually pretty simple. But as the company grows, so do the number and complexity of the metrics, making things a bit more challenging.
Have you ever faced a situation where a spike in a particular week's metric made you question its significance? Or when a sudden decrease left you wondering whether it's something to worry about? It's not always easy to tell the difference between a real trend and just random noise
Not a Medium member yet? Use this Friend link to continue read the article.
That's where XmR charts (Individuals and Moving Range charts) come in. These charts are incredibly useful for spotting patterns and variations, helping data analysts and stakeholders quickly figure out which changes are actually worth digging into.
What is an XmR Chart?
The XmR chart, short for "Individuals (X) and Moving Range (mR), is a powerful tool used in statistical process control (SPC). Beside XmR, there are several other control chart, each suited to different types of data. XmR chart is ideal for continuous data, whereas for discrete data, the p-chart, u-chart, and the c-chart are more appropriate [1].
Below is a complete example of XmR chart that we will walk through creating from scratch. As the name suggest, the chart consists of two main components.
First is Individuals (X) Chart. This chart shows actual data points over time with the average (mean), upper and lower control limits. These control limits are calculated based on the natural variability of the process, helping us to identify when a data point falls outside of what is considered normal variation.
The second component, Moving Range (mR) Chart, tracks the range between consecutive data points, monitoring the variability of the process over time. By examining the differences between successive points, the mR chart can reveal shifts in the stability of the process, indicating potential issues that may require further investigation.
Let's create an XmR Chart!
Imagine you're a data analyst at an online platform company, such as an e-commerce site. During a weekly business review meeting, one of the business owners mentions that she's noticed a surge of complaints on social media about how slowly customer service is responding to user issues on the platform.
To validate these concerns, you decide to plot the daily average response time over the past 90 days. The result looks like this:
Disclaimer: The dataset used in this article is fully generated by Python code as well as all the step from scratch. You can find the full code in the bottom section of this article.
Although we can spot a notable increase and a spike by eye-balling the chart, we still lack a mathematical basis to determine whether this is truly a problem or just natural variability in response time.
To make a clear distinction between actual signals and random noise, we'll create an XmR chart.
Step 1: Calculate Moving Ranges
The first step in creating an XmR chart is to calculate magnitudes of the differences between successive values of the data [1]. From the data we have collected earlier, we have the first day's average response time is 21.25 minutes and the second day is 31.71 minutes. Hence, the difference between the first and second day is 10.46 minutes.
Since we only require the magnitude of the difference, we can have the absolute value by dropping any minus sign.
day value mr
0 1 21.251173 -
1 2 31.713402 10.462229
2 3 35.765179 4.051777
3 4 28.737820 7.027359
4 5 34.906604 6.168784Now, we have two plots: one for the actual data (Individuals/X) and another for the moving range (mR) of the daily average response time.
Step 2: Find the mean of actual data and moving ranges
The next step is to get the average (mean) for each of the plot. For X-chart we denote it as mean(X) and for mR-chart we denote it as mean(mR) .
Nothing is really special here, we can calculate the mean as usual and getting the result where mean(X) = 32.02 and mean(mR) = 5.91. This means that, on average, the daily response time is about 32.04 minutes, and the day-to-day change is approximately 7.17 minutes.
We'll hold onto these values for the next step, but before moving on, let's plot them as central line (CL) for both X-chart and mR-chart.
Step 3: Calculate control limit
The final step, and the one that makes the XmR chart unique, is calculating the control limits. Earlier, I mentioned that these limits help us determine when the data falls within normal variation. But how do we calculate these lines?
As explained by Cedric Chin [2], the control limits estimate three standard deviations around the mean line. However, instead of using the traditional standard deviation formula, we use the moving range method. This approach is less sensitive to large values, especially when the data isn't homogenous.
The estimated value of standard deviation (or sequential deviation) can be calculated by dividing the mean(mR) with a constant 1.128. That's why we need to calculate mR and also mean(mR) before getting the control limits. I won't dive into the detailed derivation of this constant, but if you're curious, Kenith Grey [3] has an excellent article that explains and simulates how this magic constant is derived.
Since we will multiply the sequential deviation by three, you can find in many sources the formula of control limit is simplified by using constant 2.66 which is derived from 3 divided by 1.128. Hence, the formula of control limits are:
- UCL = mean(X) + (2.66 * mean(mR))
- LCL = mean(X) – (2.66 * mean(mR))
In our case, UCL (Upper Control Limit) = 32.02 + (2.66 5.91) = 47.75 minutes and LCL (Lower Control Limit) = 32.042— (2.66 5.91) = 16.29 minutes.
The interpretation here is that any daily average response time that falls between the LCL and UCL is considered normal and likely just random noise.
While we have upper and lower control limit for X-chart, for mR-chart we only need the upper limit which can be calculated by multiplying the mean(mR) with constant 3.268. In our case UCL for mR chart is 19.32.
Finally, we can plot these control lines and visualize the complete XmR chart as shown below
What does the chart tell us?
Now that we've successfully generated the complete XmR chart, let's return to the initial problem of customer complaints about slow response times.
The XmR chart clearly shows that there are indeed some days where the daily average response time exceeds the upper control limit (UCL). These data points are considered abnormal.
- First Spike (Day 11): The chart reveals a significant spike on Day 11, where the daily average response time is well above the UCL. The mR chart also confirms that this change is unusual, with two consecutive data points exceeding the UCL. This could indicate a specific event, such as a system outage, that caused a notable delay in response times. Further investigation with the team would be needed to confirm what happened on this day.
- Consistent Issue (Days 75, 77, and 79): Another abnormal pattern appears on Days 75, 77, and 79, where the data points are above the UCL. What's different here is that the days surrounding this period (from Day 70 to Day 80) also show higher-than-usual values, close to the UCL. This suggests a sustained issue, possibly due to increased platform traffic that overwhelmed the customer support team, leading to slower response times.
This scenario illustrates how the XmR chart can guide us in identifying the exact points of concern in the data, helping us focus our investigation. Without the XmR chart, we might resort to guessing where the problem occurred, leading to less effective discussions and solutions.
Some notes
While the XmR chart is indeed a simple yet powerful tool, there are important considerations to keep in mind when using it. In this section, I'll share insights from literature and my personal experience.
Successive values need to be logically comparable
One of the important principle that need to be considered when creating XmR chart, as explained by Wheeler in [4], is "successive values need to be logically comparable". In the referenced manuscript, Wheeler explain the case when this rule is violated the XmR chart will miss the signal that we should get.
Returning to our case study: Let's say after analyzing the XmR chart and identifying periods where response times fell outside normal variance, stakeholders decided to increase the number of agents and improve the system. The expectation was that these changes would lead to a decrease in the daily average response time.
Thirty days after the project began, we need to assess whether we are on track to meet the initial expectations. To evaluate progress, plot the daily average response time for a total of 90 days; 60 days before the initiative and 30 days after.
Since the daily averages before and after day 60 are not directly comparable due to changes in the number of agents and system improvements, it is important to calculate the mean and control limits using data only up to day 60. This approach ensures that the control limits reflect the conditions prior to the initiative, providing a valid baseline for comparison.
We observe that, when calculating the mean of the X-chart using data up to day 60, the mean is 29.54, with the lower control limit (LCL) at 15.30 and the upper control limit (UCL) at 43.79.
With these values, we can see that most of the daily averages after the project began fall below the central line and approach the LCL. This trend aligns with our expectations, indicating that the project is moving in the right direction.
Conditional simplicity
In my regular use of XmR charts, I often incorporate them as a complementary tool for weekly business reviews (WBR). Alongside preparing WBR data, I write narratives for various metrics. A key aspect that stakeholders are interested in is the week-over-week (WoW) comparison.
After presenting the latest week's actual numbers and WoW (or month-over-month, MoM) comparisons, I use the XmR chart to determine whether changes are part of normal business variation or if they warrant further discussion. This approach helps prevent overreactions to normal fluctuations in metrics.
Given the range of metrics discussed in WBRs, the XmR chart helps filter which metrics require deeper investigation. To keep the presentation straightforward and ensure the message is clear, I prefer to show only the X-chart component of the XmR chart, using the mR-chart only for calculations. This method prevents overwhelming attendees with excessive information and keeps the review process simple and focused.
While I agree with Wheeler [4] on the importance of the mR-chart for detailed analysis (also as I demonstrate on the scenario above), I find that, depending on the presentation's objectives, focusing on the X-chart alone can effectively communicate the necessary insights without overwhelming non-technical stakeholders.
Conclusion
In this article, we have demonstrated how XmR charts can be a valuable tool for analyzing and interpreting data, particularly in the context of process control and business reviews. We walked through the steps of creating XmR charts, from calculating moving ranges and means to establishing control limits, and illustrated their practical application in monitoring changes and avoiding overreactions to normal variations.
By using XmR charts in weekly business reviews, you can effectively differentiate between significant trends and random fluctuations, allowing for more informed decision-making. We also discussed the importance of simplicity in presentations, highlighting how focusing on the X-chart can keep your audience engaged without overwhelming them with too much information.
Ultimately, while the complete XmR chart provides a comprehensive view for in-depth analysis, presenting a simplified version ensures clarity and effectiveness in communicating key insights to stakeholders. Balancing thorough analysis with clear, actionable insights is essential for driving continuous improvement and making data-driven decisions.
Follow me in Medium for more insights into data and analytics, or on LinkedIn.
