Frequently Asked Questions

Without hard data, the judge or jury will base much of its decision on the credibility of witnesses, rather than easier-to-assess facts. And the opposition may likely find an equally credible expert to present a very different opinion.

Surveys allow an expert to base his or her opinion on the kind of easy-to-understand information that newspapers and television use every day. They enable experts to bring real world information into the courtroom and can force the opposition to expend considerable resources to understand the technical details of the survey and collect its own data.

Legal surveys are conducted using the same methods as other market research surveys. Survey data can be collected by phone, at a research facility, at a shopping mall or over the web. Factors influencing the choice of which method to use include:

• The population you are trying to reach
• Whether there is a need to conduct the survey in person (respondents may need to see and touch products related to the survey questions)
• Geographical requirements
• Cost effectiveness

When hiring a survey expert, the most important criteria to consider (in no particular order) are testifying skill, credentials, expertise and availability. Look for a survey expert who can provide an admissible opinion under the Daubert-Joiner-Kumho gate-keeping tests and remain cool when testifying under oath despite intensive questioning. The expert should have prior court and deposition experience as well as strong academic and/or survey practitioner credentials. Ideally, survey research experience should cover both survey development and survey rebuttal. It is also helpful for an expert to understand economic issues and the process for calculating damages.

Last but not least, an expert must be available and able to meet the demands of the case in a timely and thorough manner. Experts often work in partnership with a market research firm, and it is important to ensure that this firm is familiar with legal processes, documentation and communications, and can provide guidance to attorneys when needed.

A survey project for litigation can take anywhere from three weeks to a few months to complete. Expect to receive a firm time estimate from the survey expert or survey research firm up-front. This estimate is typically based on factors that affect the time it takes to design, execute and analyze a survey. Examples of these factors include:

The type of exploratory research that may be needed to better understand

• The subject matter
• The language consumers use to describe their experience
• The characteristics that define the population of interest
• The best ways to reach the population of interest

Recruitment of respondents

• Ease of finding people who meet the qualifications for participation
• Level of cooperation (how busy are the qualified people and how interested are they in taking the survey)
• Survey panels, which pre-recruit people who are willing to participate in surveys, can help shorten the recruiting cycle

The method of executing the survey
Time requirements can vary depending on whether data is collected over the phone, at a research facility, at a shopping mall or over the web. For example:

• Shopping malls may be limited by the number of eligible people they can recruit in a given day
• Many people can take a web survey simultaneously, making this a relatively quick method
• Telephone methods may depend on whether people are at home or in their office when called

Many people incorrectly assume that the number of interviews required for statistical significance is a function of the size of the population of interest—the larger the population, the larger the sample needed. In reality, relatively small samples, carefully selected from large populations, can yield precise estimates of how the total universe of potential respondents would respond. For example, political surveys of less than 1,000 respondents can predict the behavior of one hundred million voters with an accuracy of plus or minus 3%.

While the “confidence interval” (the “plus or minus 3%” quoted above) does get smaller and the precision of the answer improves as the sample size increases, appropriately chosen samples can provide statistically significant results with far fewer respondents. Aside from statistical significance, however, is the intangible factor of having a sample size large enough to appear credible to a judge or jury. This desire for “face validity” often leads to samples of 200 or 300 respondents. Another factor leading to larger sample sizes is the particular analytical techniques used, as some may require larger samples.

Survey researchers and experts are often challenged to define a population of interest that will not be attacked as being either too narrow or too broad. While legal strategy can guide survey experts in determining a population of interest, there are often a number of subtle choices that experts make that lead to the ultimate definition of the respondent population.

A good strategy for guiding these choices is to consider the allegation(s) in light of two groups of people: those who may have been harmed and those who may potentially be harmed. It is then helpful to list as many dimensions as possible characterizing people within this spectrum (for example, how they are exposed to a product, how they use a product, how they get information, etc.) This exercise is beneficial because it exposes potentially subtle yet important differences between segments, and it then enables the expert to create specific screening questions to target and capture the relevant population.

To a great extent, survey costs are driven by the ease of reaching a respondent population, the method of conducting a survey and the sample size. Since these factors are quite variable, typical surveys can range between $30,000 and $100,000. Factors that can contribute to higher costs include the need to expedite a schedule or conversely, needing to spend a lot of time during the planning stage to address complex issues.

While the cost for some surveys can be reliably estimated, others require additional information in order to develop an accurate estimate. To lessen this uncertainty and reduce costs, survey experts can conduct exploratory research to better understand the respondent population and the potential issues in recruiting them. Another good way to estimate and manage costs is to conduct a small-scale pilot research study and then decide about further investment. Your survey expert can advise you as to the best approach for your particular study.