By Craig A. Woods and Linda X. Shi, partners in the Chicago, IL office of Mayer Brown LLP.

The admissibility of expert testimony is governed by Federal Rule of Evidence 702 and Daubert v. Merrell Dow Pharmaceuticals.¹  Chief Judge Nancy Rosenstengel of the U.S. District Court for the Southern District of Illinois recently applied Rule 702 and Daubert in In re Paraquat Products Liability Litigation, 2024 WL 1659687 (S.D. Ill. Apr. 17, 2024), to exclude the general-causation opinions of the plaintiffs’ expert witness, finding numerous “methodological red flags” with the expert’s analysis.  On the same day, the court granted summary judgment to the defendants in four cases in which plaintiffs relied on that expert, concluding that the plaintiffs lacked admissible expert testimony establishing causation.²

Background

In re Paraquat Products Liability Litigation is a multidistrict litigation (“MDL”) in which over 5,000 individual plaintiffs allege that they developed Parkinson’s disease because of their exposure to an herbicide called paraquat dichloride (“paraquat”).  Paraquat is used to control weeds in farming operations and other settings around the country.  Defendants are Syngenta Crop Protection, LLC and Syngenta AG, which currently manufacture and distribute paraquat for use in the United States, and Chevron U.S.A., Inc., which manufactured and distributed paraquat until 1986.

Defendants filed a Daubert motion seeking to exclude the testimony of Plaintiffs’ expert witness, Dr. Martin Wells, a statistician and epidemiologist at Cornell University who opined that occupational exposure to paraquat can cause Parkinson’s disease.  Dr. Wells was Plaintiffs’ only expert witness on the question of general causation.

Dr. Wells conducted a meta-analysis of seven epidemiological studies that measured a potential association between paraquat and Parkinson’s disease.  Based on this meta-analysis, Dr. Wells found that there was a “near tripling of PD occurrence in [study] participants occupationally exposed to paraquat.”³  Dr. Wells then conducted a weight of the evidence/Braford Hill analysis and concluded that (i) the available epidemiological evidence supports a causal relationship between paraquat and Parkinson’s disease; and (ii) the plaintiffs who had been selected for trial were at near triple the risk of developing Parkinson’s disease.

In a 97-page opinion, Judge Rosenstengel granted the defendants’ motion to exclude the testimony of Dr. Wells.  In reaching that opinion, the court applied the amended version of Rule 702, which took effect on December 1, 2023.  The Advisory Committee on the Rules of Evidence explained that the amended version of Rule 702 emphasizes that “the proponent bears the burden of demonstrating compliance with Rule 702 by a preponderance of the evidence” and that “each expert opinion must stay within the bounds of what can be concluded from a reliable application of the expert’s basis and methodology.”⁴  The Advisory Committee further explained that “expert testimony may not be admitted unless the proponent demonstrates to the court that it is more likely than not that the proffered testimony meets the admissibility requirements set forth in [Rule 702]”⁵ and noted that some courts had incorrectly held that the basis of an expert’s opinion and the expert’s application of his or her methodology were questions of weight, not admissibility.  “Mindful of its role as the witness stand’s ‘vigorous gatekeeper,’” the court noted that it “will closely scrutinize the reliability of proffered expert testimony before permitting an expert to share her opinion with the jury.”⁶

The parties’ Daubert briefing—and the court’s opinion—focused on the reliability of Dr. Wells’ testimony.  As the court explained, Dr. Wells’ “proffered opinion required several methodological contortions and outright violations of the scientific standards he professed to apply.”⁷ These methodological deficiencies rendered Dr. Wells’ proffered opinions not “sufficiently reliable to be presented to a jury.”⁸

Dr. Wells’ Definition of “Occupational” Exposure Was “Impossible to Discern”

First, the court took issue with Dr. Wells’ definition of “occupational” exposure to paraquat.  The definition is important because Dr. Wells’ opinion was that “occupational” paraquat exposure is causally linked to Parkinson’s disease; therefore, an exposure that does not qualify as “occupational” was not within the scope of Dr. Wells’ opinion.  Further, Dr. Wells’ definition of “occupational” exposure affected which studies he included in/excluded from, and consequently the results of, his meta-analysis, on which he relied to opine that “occupational paraquat exposure was associated with a ‘near tripling of PD occurrence.’”⁹

Dr. Wells “redefined ‘occupational’ exposure no less than three times”—which evolved from being related to a person’s workplace, to focusing on the “risk of dermal exposure,” to “direct contact.”¹⁰  Dr. Wells’ “nebulous” definition of occupational exposure would leave to the court (and if he were to testify, the jury) “to figure out the precise contours of his opinion.”¹¹  The court concluded that the meaning of Dr. Wells’ opinion is “impossible to discern”—which was “the first strike against the reliability of Dr. Wells’ expert opinions under Rule 702.”¹²

Flaws in Dr. Wells’ Meta-Analysis

Second, the court identified a litany of flaws with Dr. Wells’ meta-analysis.  One of the initial steps in a meta-analysis involves the search for relevant studies that are then further analyzed for potential inclusion in the analysis, but Dr. Wells did not provide a search narrative that would allow other researchers to validate his process.  “As a result,” the court concluded, “any attempt to replicate his search for relevant literature would require a degree of clairvoyance that this Court does not possess.”¹³

Another “methodological red flag” in Dr. Wells’ meta-analysis was that he offered a “holistic” approach to inclusion/exclusion criteria that purportedly governed a study’s eligibility for his analysis—which was not articulated until the submission of his rebuttal report.  Even then, his “holistic” approach “was neither reduced to writing” nor “offer[ed] any discernible objective criteria that would allow others to replicate Dr. Wells’ eligibility determinations.”¹⁴The court noted that Dr. Wells’ failure to define his eligibility criteria in advance suggested results-driven analysis, or cherry-picking, and that “[t]his type of post hoc methodology is the very antithesis of a systematic review, which relies on predefined eligibility criteria to ensure transparency and scientific objectivity.”¹⁵

An example of Dr. Wells’ results-driven analysis was his inclusion of an article in his meta-analysis that did not meet his own eligibility requirements.  Dr. Wells in his rebuttal report listed five criteria that a study had to meet to be eligible for his meta-analysis; one of those five criteria was that the study was of “occupational” exposure.  Dr. Wells conceded that the one study in his meta-analysis that “almost singlehandedly” generated his odds ratio¹⁶did not meet his initial criteria for constituting an occupational exposure study because the study included residential exposure as well. ¹⁷ To avoid excluding the study, Dr. Wells revised his definition of “occupational” exposure to include “potential dermal contact” and “direct contact.”¹⁸  The court found that, without the study at issue, the meta-odds ratio of 2.8 would have been much lower, given the results of the remaining six studies.¹⁹  The court concluded that Dr. Wells’ changing definition of “occupational” exposure “violated the basic rules of meta-analysis, which mandate that researchers ‘develop a protocol for the review before commencement and adhere to the protocol regardless of the results of the review.’”²⁰

Dr. Wells admitted that there was no algorithm another epidemiologist could use to replicate his inclusion/exclusion criteria, which the court found was a “foundational deficiency in the application of Dr. Wells’ chosen methodology” because “[c]learly defined, objective eligibility criteria are precisely what lend scientific authority to a meta-analysis.”²¹  Instead, Dr. Wells described his approach as unwritten and “holistic,” which meant that “Dr. Wells was free to select the studies he wanted to meta-analyze, and then justify his selections based on the results they provided.”²²

Dr. Wells then used five quality factors to determine which eligible studies to include in his meta-analysis, which the court found to “suffer[] from a similar lack of systematicity.”²³ Of the 36 studies that Dr. Wells initially considered for his meta-analysis, eight case-control studies were eligible for inclusion.  Dr. Wells then qualitatively judged the eight studies based on five criteria.  Only one of the eight eligible studies failed Dr. Wells’ qualitative evaluation and was excluded from his meta-analysis; the one excluded study concluded that there was “no evidence for association” between paraquat exposure and Parkinson’s disease, and Dr. Wells classified that study as “lower quality” on each of his quality metrics. ²⁴ The court determined that Dr. Wells’ “qualitative indictment of [the excluded study] does not withstand scrutiny when viewed in the context of Dr. Wells’ broader qualitative evaluation of the eight eligible studies.”²⁵  In addition, the court found that Dr. Wells inconsistently applied certain quality criteria “in a thinly veiled attempt to ensure the inclusion of the studies that made it into his meta-analysis.”²⁶  The court was “troubled by the blatancy with which Dr. Wells graded favorable studies as ‘higher quality’ using one evaluation method and his concomitant imposition of a more stringent standard on an unfavorable study to grade it as ‘lower quality.’”²⁷

Based on these methodological flaws, the court found that any testimony based on Dr. Wells’ meta-analysis and the resulting odds ratio reflecting a “near tripling” of the occurrence of Parkinson’s disease in people who are occupationally exposed to paraquat is inadmissible under Rule 702.²⁸

Flaws in Dr. Wells’ Weight of the Evidence/Bradford Hill Analysis

Third, the court also found issues with Dr. Wells’ weight-of-the-evidence/Bradford Hill analysis.  Dr. Wells conducted a weight of the evidence review using the Bradford Hill framework to determine whether the association was attributable to a cause-and-effect relationship between occupational paraquat exposure and Parkinson’s disease.  Once the epidemiologic evidence suggests that a true association between an agent and a disease exists, to determine whether the association is causal researchers evaluate the available scientific evidence against a set of causal criteria, often referencing nine “Bradford Hill” factors, which were named after a British researcher who investigated the role of smoking in causing lung cancer.  The court noted that “Dr. Wells’ weight of the evidence/Bradford Hill analysis is a textbook example of the type of standardless presentation of evidence that courts have cautioned against.”²⁹ The “most obvious” methodological defect in Dr. Wells’ weight of the evidence/Bradford Hill analysis was the “absence of any discernible weighting methodology.”³⁰  Dr. Wells did not offer any explanation of the relative weight or importance assigned to each of the six Bradford Hill factors that he analyzed, rendering his general-causation opinion “virtually non-falsifiable.”³¹  Under the Daubert analysis, the expert’s conclusions must be testable, or “falsifiable.”³²  A proposition is “falsifiable” if it is “capable of being proved false,” or “defeasible.”³³

As Chief Judge Rosenstengel explained, falsifiability is “one of the most basic requirements of the scientific method” because “conclusions that are not falsifiable aren’t worth much to either science or the judiciary.”³⁴In addition, Dr. Wells “leaves it to the Court (and by extension, the jury) to identify evidence in his two reports that supports his conclusions.”³⁵Dr. Wells’ analysis also suffered from “extensive selection bias” in his discussion of several Bradford Hill factors, notably those concerning a dose-response relationship and strength of association.³⁶  As the court found, “Dr. Wells was plainly interested in finding ‘positive results’ that supported a dose-response relationship,” while disregarding evidence to the contrary.³⁷  Similarly, with respect to the strength of association factor, Dr. Wells admitted that he excluded studies finding small or no associations between paraquat and Parkinson’s disease—which the court found was indicative of selection bias.³⁸

The court concluded that, “[a]t its core, Dr. Wells’ weight of the evidence/Bradford Hill analysis is a selective presentation of supportive evidence that fails to meaningfully account for data points that refute his conclusions.” ³⁹Consequently, “Dr. Wells’ outcome-driven Bradford Hill analysis compels the exclusion of his general causation opinion.”⁴⁰

Isolation from the Scientific Community

While the majority of the court’s analysis and opinion focused on Dr. Wells’ methodological soundness, the court also addressed whether Dr. Wells’ opinion that occupational exposure to paraquat can cause Parkinson’s disease stands alone in the scientific community.  When asked at the Daubert hearing whether any peer-reviewed publication had found a causal relationship between paraquat exposure and Parkinson’s disease, Plaintiffs’ counsel cited a single article, which was an advocacy piece, not a scientific analysis of the causal relationship between paraquat exposure and Parkinson’s disease.  In addition, Plaintiffs’ counsel was not aware of any scientist outside of the In re Paraquat litigation who had ever conducted a Bradford Hill analysis and concluded that paraquat exposure was causally related to Parkinson’s disease.  Citing a recent review of published articles that found “[n]o author of any published review stated that it has been established that exposure to paraquat causes Parkinson’s disease,” the court noted that “Dr. Wells’ causation theory has not been adopted or independently validated in any peer-reviewed scientific analysis outside of this litigation.”⁴¹  While the lack of peer-reviewed support for Dr. Wells’ general-causation opinion was not dispositive of its reliability under Rule 702, the court concluded that a lack of peer-reviewed support further undermined the reliability of Dr. Wells’ study.

Exclusion of Dr. Wells’ Plaintiff-Specific Opinions

The court further determined that Dr. Wells’ plaintiff-specific opinions should be excluded because they were not severable from his meta-analysis and weight-of-the-evidence /Bradford Hill analysis.  Without any testimony regarding a positive association between occupational paraquat exposure and Parkinson’s disease, or any testimony regarding a causal relationship between occupational paraquat exposure and Parkinson’s disease, the court concluded that there was no testimony that Dr. Wells could give with respect to the four plaintiffs who had been selected for trial and therefore excluded Dr. Wells’ plaintiff-specific opinions under Rule 702.

The In re Paraquat opinion is an exemplar of how a court should do serious gatekeeping under Rule 702 and Daubert.  Particularly in light of the 2023 amendments to Rule 702, courts must carefully evaluate the bases of an expert’s opinion and the expert’s application of his or her methodology as questions of admissibility—and not admit unreliable expert opinions on the premise that it is up to juries to weigh competing scientific arguments.  This case is a textbook example of how courts’ review of admissibility questions under Rule 702 should focus on the particular methodology used by the expert and the application of that methodology to the facts of the case.  Here, the court found that the expert did not objectively and consistently apply a scientific methodology for assessing general causation to the scientific evidence in the case.  As a result, the expert’s ultimate conclusion was not scientifically based and was therefore inadmissible.⁴²

Notes

1. 509 U.S. 579 (1993).
2.   In re Paraquat Prod. Liab. Litig., 2024 WL 1655500, at *2 (S.D. Ill. Apr. 17, 2024).
3. Id. at *2 (alteration in original).
4. Fed. R. Evid. 702 advisory committee’s note to 2023 amendments.
5. Id.
6. In re Paraquat Prod. Liab. Litig., 2024 WL 1655500, at *4 n.9.
7. Id. at *15.
8. Id.
9. Id. at *27.
10. Id. at *23-24.
11. Id. at *24
12. Id. at *25.
13. Id. at *26.
14. Id.
15. Id.
16. An odds ratio is used to measure the strength of association between an exposure to the agent and the disease in question.  An odds ratio is calculated by “dividing the odds of a disease of interest occurring in an exposed person by the odds of the disease occurring in an unexposed person.”  Id. at *9.
17. Id. at *27.
18. Id.
19. Id.
20. Id.
21. Id. at *28
22. Id.
23. Id. at *29.
24. Id.
25. Id.
26. Id.
27. Id. at *30.
28. Id. at *33.
29. Id. at *35
30. Id.
31. Id.
32. Daubert, 509 U.S. at 593.
33. See United States v. Mitchell, 365 F.3d 215, 235 (3d Cir. 2004) (internal citation omitted).  “Proving a statement false typically requires demonstrating a counterexample empirically—for instance, the hypothesis ‘all crows are black’ is falsifiable (because an albino crow could be found tomorrow), but a clairvoyant’s statement that he receives messages from dead relatives is not (because there is no way for the departed to deny this).”  Id.
34. Id.
35. In re Paraquat Prod. Liab. Litig., 2024 WL 1655500, at *4 *36.
36. Id. at *36.
37. Id. at *37.
38. Id. at *39.
39. Id. at *40.
40. Id. at *36.
41. Id. at *41 (emphasis in original).
42. Plaintiffs have filed a notice of appeal from the final judgments against the four plaintiffs whose cases were dismissed on summary judgment.  Presumably, the exclusion of Dr. Wells’ testimony will be a key issue in the appeal.  Plaintiffs’ opening appellate brief is currently due on June 26, 2024.