214-244-4608

What does the Manual For Complex Litigation say about Sampling/Opinion Surveys?

What Does The Manual For Complex Litigation Say About Sampling/Opinion Surveys?

§11.493 Sampling/Opinion Surveys

Statistical methods can often estimate, to specified levels of accuracy, the characteristics of a “population” or “universe” of events, transactions, attitudes, or opinions by observing those characteristics in a relatively small segment, or sample, of the population.

Acceptable sampling techniques, in lieu of discovery and presentation of voluminous data from the entire population, can save substantial time and expense, and in some cases provide the only practicable means to collect and present relevant data. In one case, for example, a statistical expert profiled the compensatory damage claims of the class members to assist the jury in fixing the amount of punitive damages.

The choice of appropriate sampling methods will depend on the objective. There is a difference between sampling to generate data about a population so the data will be verified or declared true and sampling, like polling, to measure opinions, attitudes, and actions by a population. In the case of the former, the reliability and validity of estimates about the population derived from sampling are critical.

The sampling methods used must conform to generally recognized statistical standards. Relevant factors include whether

  • the population was properly chosen and defined;
  • the sample chosen was representative of that population;
  • the data gathered were accurately reported; and
  • the data were analyzed in accordance with accepted statistical principles.

Laying the foundation for such evidence will ordinarily involve expert testimony and, along with disclosure of the underlying data and documentation, should be taken up by the court well in advance of trial. Even if the court finds deficiencies in the proponent’s showing, the court may receive the evidence subject to argument going to its weight and probative value.

By contrast, questioning a sample of individuals by opinion polls or surveys about such matters as their observations, actions, attitudes, beliefs, or motivations provides evidence of public perceptions. The four factors listed above are relevant to assessing the admissibility of a survey, but need to be applied in light of the particular purpose for which the survey is offered. In addition, in assessing the validity of a survey, the judge should take into account the following factors:

  • whether the questions asked were clear and not leading;
  • whether the survey was conducted by qualified persons following proper interview procedures; and
  • whether the process was conducted so as to ensure objectivity (e.g., determine if the survey was conducted in anticipation of litigation and by persons connected with the parties or counsel or by persons aware of its purpose in the litigation).

Parties who propose to offer sampling or survey evidence may want to consider whether to disclose details of the proposed sampling or survey methods to the opposing parties before the work is done (including the specific questions that will be asked, the introductory statements or instructions that will be given, and other controls to be used in the interrogation process). Objections can then be raised promptly and corrective measures taken before the survey is completed. A meeting of the parties’ experts can expedite the resolution of problems affecting admissibility.

Parties sometimes object that an opinion survey, although conducted according to generally accepted statistical methods, involves impermissible hearsay. When the purpose of a survey is to show what people believe—but not the truth of what they believe—the results are not hearsay. In the rare situation where an opinion survey involves inadmissible hearsay, Federal Rule of Evidence 703 nevertheless allows experts to express opinions based on the results of the survey.


Rhonda Harper, Trademark Infringement Survey Expert, Expert Witness

Rhonda HarperRetained by 100+ law firms since 2005, Ms. Harper is courtroom proven. She has been engaged to provide 65+ surveys, 80+ reports, 30+ rebuttals, 45+ depositions, and serve in 20+ trials. She has provided services to both Plaintiffs (60%) and Defendants (40%) across trademark and trade dress, packaging, merchandising, defamation, licensing, breach of contract, advertising, and commercial reasonableness. She has provided services in virtually every Circuit as well as JAMS and TTAB.

Ms. Harper's 30+ year career includes serving as a Fortune 100 Chief Marketing Officer. Having authored two books, she is a former Adjunct MBA Marketing Professor.

Ms. Harper is routinely retained to formulate expert surveys, conduct rebuttal critiques, or construct rebuttal surveys to show the potential difference in results with properly designed and executed surveys. She has extensive experience and a deep understanding of survey design, sampling, question construction, data analysis, and methodological pitfalls that introduce bias or systematic error.

Professional Memberships