Quality Advisor

How do you interpret an R&R study?

After performing an R&R study, which can be done using software such as GAGEpack, there are a number of ways to interpret the results. Frequently, since R&R is done in response to a customer requirement, the customer will indicate how to interpret the results. The most common here is the AIAG (Automotive Industry Action Group) standards, which are based on the R&R percentage given under study results. These results may be calculated as a percent of study variation, percent of specification, or percent of process variation.

For percent of study, the process variation is based on the spread of the parts (P) determined by . This is considered a range and using the /d2 relationship, a sigma for the process is estimated. This is then used to calculate the percentages.

A second method is to use the spread of the specs (USL – LSL). Now this must be compared to the estimate of the measurement error (R&R). However, one needs to multiply the sigma of the measurement by 5.15 (old method) or by 6.0 (new method) to compare the total measurement spread with the spec spread. (An alternative method is to divide the spec range by the respective numbers given above.)

The third method uses the information from an chart on the process and characteristic being studied. In this case, enter the , the , and the sample size. This is used to estimate the process spread.

Ideally the measurement error (R&R%) is less than 10% of whatever method is used (process spread or spec spread). It is usable in some cases when the R&R percentage is between 10 and 30%. More than 30% suggests that one should not be using it. [page 60 of MSA Manual 2nd edition or page 77 of MSA Manual 3rd edition]

If the number of distinct categories is 5 or more, it can be considered a capable measurement system. Wheeler and Lyday use a concept closely aligned with distinct categories called discrimination ratio, for which greater than four is satisfactory. The differences, in a nutshell, are that the distinct categories is a truncated number (no rounding or fraction used) and the discrimination Ratio assumes that appraiser variation has been reduced to zero and carries the fractional part as well.

Variables step-by-step interpretation

Are there 100 data values (observations)?

Yes   No

Quality Advisor

Frequently-asked questions about Measurement Systems Analysis

  1. Can repeatability be less than reproducibility if three appraisers are used?
  2. On an R&R study, if I use the specifications, the study is good. If I use the study parameters, the gage fails.
  3. R&R percentages don’t tell you how well repeatability is centered. Is it possible to apply Cpk principles to the estimated deviation to determine margin to the spec limits?
  4. What are Process X-double bar and Process R-bar used for?
  5. What is the difference between tolerance and process variation
  6. What kind of set-up would you recommend for testing one instrument?
  7. What are the objectives of an uncertainty study

Can repeatability be less than reproducibility if three appraisers are used?

Repeatability and reproducibility are relatively independent. The desired reproducibility is zero and this would essentially mean that the appraisers are measuring the same average value. That is, one appraiser is not measuring parts on the average as larger than the other two appraisers are.

On an R&R study, if I use the specifications, the study is good. If I use the study parameters, the gage fails.

In AIAG’s Measurement Systems Analysis manual, the standard approach for R&R percentages is to use percent of study variation. This means that the parts selected for the study are used to estimate the product’s variation (more correctly the product’s process variation). This makes the selection of those parts critical to the analysis. You want to know if your measurement system can detect changes in the process and tell if the process goes out of control.

The prior methodology asked a different question: Can I tell whether a product is good or bad. This is an inspection mentality that the automotive manufacturers want to move away from. In this case, the percentages are based on dividing by the specification range.

My general suggestion is to use the percent of study variation. However, given the push to improve Cp and Cpk, we have reduced the variation in our processes considerably. In some cases, we have Cpk’s > 2. In most of these cases, the R&R percentage will be poor because of the reduced variation in the processes. We can improve our measurement system (many times at great cost and sometimes we cannot improve it), but it will not result in a better product. So a suggestion that I frequently make is to specify some Cpk level above which you use R&R percent of tolerance and below which you use R&R percent of study.

R&R percentages don’t tell you how well repeatability is centered. Is it possible to apply Cpk principles to the estimated deviation to determine margin to the spec limits?

When doing an R&R study, you are checking to see if the variation in the measurement is too large to measure a product dimension in reference to either process variation or specification spread. There is no attempt in the R&R study to determine bias. This is done through a linearity or bias study. Without a true “measurement” for each part used in the study, centering cannot be determined and it can vary based on measurement size. Cpk is targeted to how well you are meeting a single value (normally the nominal or target value).

What are Process X-double bar and Process R-bar used for?

There are two ways to do R&R: One is to do the percentages with respect to tolerances (spec spread). The other is to do the percentages with respect to process spread (5.15 sigma). There are two ways to estimate process spread. The first is to do it from the parts used in the study. The second way bases process spread on control chart information from the process. Process and Process are used in the second method. GAGEpack is able to do R&R both of these ways.

What is the difference between tolerance and process variation?

The major difference between the two approaches relates to the purpose for doing the study. In the past, most companies were concerned only with getting good parts. Being able to sort parts (good from bad) was the requirement for the gage. Now most companies require their suppliers to have controlled, capable processes. If you are going to control a process, you need to be able to detect changes in the process (out-of-control conditions). Therefore, the recommended approach uses process variation based on the parts used to do the study. GAGEpack, gage management software, allows you to also estimate the process variation based on a control chart for the process measurement being studied.

The push to improve processes by improving the capability ratios is a complicating factor. This reduces the variability of the process. As a result, many gages do not pass the R&R % required. Often, the alternative comes down to purchasing a “better” measurement system (often a very expensive gage). In some cases, better systems do not currently exist. In any event, there is some question as to the value of improving the measurement system. In cases where Cpk > 2 (or whatever number you like), the customer is not likely to notice any improvement and the cost of producing (measuring in particular) increases. (Side Note: a frequently used major criteria for selecting improvement project is: Will the customer notice the improvement? Will it make a difference to the customer?) In this case, one might want to consider a policy of using tolerances for situations where Cpk is greater than some value, with the stated objective of applying their improvement efforts to situations where the process is not capable and the potential for improvement is greater.

What kind of set-up would you recommend for testing one instrument?

It measures percent acidity. There are six analysts who use the instrument to test acids whose strengths range from 50% to nearly 100%. I could have each analyst run the same of several samples in triplicate. I could fix the sample weights + or – 0.1 gram.

Answer:

It depends on what you are trying to accomplish. To some extent, R&R studies may not be appropriate. One objective would be to see how the gage accuracy varies over the relevant range (50% to 100%). You can do this with a linearity study. The major difference is that the parts used for the study are different products with values that range throughout the gage range, e.g., 55%, 65%, 75%, 85%, & 95%. You need reference values (true values) for each product. These can come from certified samples (like using gage blocks) or from using a “super” gage with better known accuracy. Since you are using different products, the traditional R&R percentages do not mean anything. However, if you wanted to test if the appraisers are measuring things differently, you could use more than one operator and check the operator bias and the operator uncertainty results.

If you want to run a traditional R&R, I would suggest picking a concentration where most of the tests are run. You can use all six operators in the same study. Given the previous results, I would suggest this as well, providings it does not overcomplicate the running of the tests. Here the samples (5-10) need to come from one product, but have values typical of the total range of the product variation. Two or three replications should work fine. This would allow you to check the gages’ ability to discriminate among the values for the product and to evaluate the performance of the operators. Linearity could not be checked for the gage, however.

What are the objectives of an uncertainty study?

The objective of an uncertainty study is to put bounds on a measurement (x plus/minus some interval). Generally a 95% confidence interval and the t-distribution is used since there are not a lot of observations taken. To do the study use one gage, one part, one part characteristic, and one operator. The characteristic is measured on the part a number of times-10, 15, 25, etc. A standard deviation is calculated (that should be all measurement error), and the appropriate t value obtained and multiplied times the standard deviation. This becomes the plus/minus interval estimate about the measured value (uncertainty).

If you would like a bias estimate using this same data and have a reference value for the part characteristic used, you may place it in the reference block and the program will take the mean of the measurements and subtract the reference value to give an average bias value.

If you would like the bias expressed as a percent of specification or percent of process variation, you enter the USL, LSL and/or six sigmas for the spread of the process (generally obtained from an X-Bar and Range chart on the process), and the respective percents will be calculated.

I am not aware of any guide lines that have been given for these values. When you do a calibration, there is a plus/minus interval that is acceptable. This might be one source for determining acceptable values for both uncertainty and bias values. When you do an R&R, there are guidelines for the percent R&R. This may provide input into development of acceptable percentages.

Start implementing MSA today. Download a free trial of GAGEpack.

Quality Advisor

How do you conduct an R&R study?

The objective of a simple R&R study, which can be done using software such as GAGEpack, is to determine whether the measurement system in use can adequately distinguish between or among units. There are a number of factors that affect the ability of a measurement system to discriminate among the units it measures. These factors may be categorized as those that typically contribute to any process variation: machine (the gage), operator (appraiser), method (method of measurement followed), material (units being measured), and environment.

In conducting a simple study, an attempt is made to minimize or hold most of these factors constant. (Note: the study should replicate as much as possible the conditions under which the process measurement is occurring. This essentially means that if production operators are making the measurements, the study should use production operators under the same conditions that exist when they are “normally” measuring the output. The study should not be conducted using quality technicians in a lab.) Therefore:

  1. One gage is used. (No inter-gage variation)
  2. One method of measurement is employed by all of the appraisers involved. (No variation due to the appraisers using different methods.)
  3. The same dimension on each part is measured each time. A further assumption is that each part is measured in the same place to eliminate the possibility of within-part variation. (No variation introduced due to different characteristics or because the measurements have been taken at different locations on the part.)
  4. The study is conducted under the same type of conditions that exist when the parts are normally measured. (No variation introduced by the location or environmental concerns of where the measurements are taken.)

The two items that are specifically varied are the number of appraisers and the number of units measured. The selection of these two factors is critical to meeting the objective of an R&R study.

The first one addressed is the number of appraisers. The appraisers should be selected from those individuals who are currently measuring the output. If there is only one appraiser, then only one should be used in the study. If there are two or three, use two or three. If there are more than 4 or 5, use two or three “typical” appraisers.

The second is the number of parts, generally between 5 and 10. The major factor to consider in selecting the parts is that they need to represent the total variation for the characteristic being studied for the process producing them. This means that going down to the line and pulling 5 to 10 consecutive parts or going to a bin and selecting 5 to 10 parts, is a not acceptable. The selection of the parts must be spread out over time so that the parts represent the “typical” variation in the process.

The final parameter is the number of replications, generally 2 or 3. A single replication will not allow for separate estimates for equipment variation and appraiser variation. Too many replications complicate the study without adding much value to the analysis. To be statistically sound, take measurements randomly (see note below). This confounds the factor of time, different in these repeated measurements, from appraisers and parts. As a practical matter, it often comes down to making sure that the appraiser does not know which part is being measured (randomizing the order of measurement) and no appraiser completes two replications before the others finish one replication. For example:

Appraiser 1 measures all five parts once.
Appraiser 2 measures all five parts once.
Appraiser 3 measures all five parts once.
Appraiser 2 measures all five parts a second time.
Appraiser 3 measures all five parts a second time.
Appraiser 1 measures all five parts a second time.

Note: Do not always measure parts in the order of 1-2-3-4-5-6-7-8-9-10

This results in something like Table 1 below once the data are organized.

Appraiser Part 1 Part 2 Part 3 Part 4 Part 5
John 2.112 2.112 2.071 2.101 2.114
John 2.114 2.106 2.073 2.097 2.130
Mary 2.112 2.117 2.084 2.098 2.110
Mary 2.112 2.107 2.082 2.099 2.108
Gerry 2.108 2.116 2.102 2.110 2.108
Gerry 2.110 2.124 2.086 2.104 2.093

 

If you enter this type of information in GAGEpack as an R&R event, GAGEpack will calculate R&R percentages.

Variables step-by-step interpretation

The process is not in control. Investigate the cause(s) of the variation and determine if effects are good or bad. For more information, select:

Declare the system stable or unstable and respond to the information on the control chart.

Variables step-by-step interpretation

Are there runs on the range or sigma chart?

Yes   No

Variables step-by-step interpretation

The process is not in control. Investigate the cause(s) of the variation and determine if effects are good or bad. For more information, select:

Declare the system stable or unstable and respond to the information on the control chart.

Variables step-by-step interpretation

You should create a range or sigma chart. These charts let you check for piece-to-piece variability.

Variables step-by-step interpretation

Are there points that are out of control on the range or sigma chart?

Yes   No