A Court Performs TAR Document Review — Winfield v. City of New York — eDiscovery Case Law Update

Summary:

Winfield vs. City of New York, 2017 U.S. Dist. LEXIS 194413 (S.D.N.Y.) is the latest case that discusses the use and the requirements of Technology Assisted Review (“TAR”) before a Court. This case presents some important insights: from whether a Court can direct parties to use TAR and whether disclosure of non-responsive TAR training documents are required. In Winfield, after in camera review of non-responsive training documents, the Court found that Defendant’s TAR process was not defective as a whole, but still directed the producing party, Defendant City of New York to produce a random sample of 300 non-responsive and non-privileged documents.

eDiscovery in Dispute:

From an overall data set of 50 custodians, the Defendant applied search terms resulting in over 100,000 documents for manual review. Plaintiffs prevailed on their argument that Defendant’s terms were too narrow. As a result, the Defendant expanded their search terms and returned an additional 90,000 documents.

The Defendant represented that the review of the original 100,000 documents cost of over $350,000 and the additional 90,000 documents would cost an additional $248,000. As a result, the parties agreed to use TAR for review.

After TAR review, the Defendant produced responsive documents from the 90,000-document subset. It is the production from this set of data that is in dispute. Plaintiffs argued that the Defendant failed to train the TAR system properly by labeling responsive documents as non-responsive. As a result, the production failed to include all relevant documents.

Rationale of the Court

While the Court found some discrepancies in the Defendant’s training, the Court stopped short of identifying anything that called the whole process into question. Instead, after an in camera review of seed set documents, the Court found that defendant’s erred in labeling a subset of 20 documents incorrectly in training the TAR system. As a result, the Court ordered the Defendant to produce a small random sample of non-responsive training documents for transparency.

Notably, the Court found that the Plaintiffs presented sufficient evidence to justify their request for greater transparency in the form of samples of responsive documents that Defendants intended to withhold as non-responsive; however, the Court rejected Plaintiffs’ assertions that the TAR process as a whole was defective. Increased transparency was deemed reasonable in light of the volume of documents collected from the custodians, the low responsiveness rate of documents pulled for review by the TAR software, and the examples that Plaintiffs presented, which suggest there may have been some human error in categorization that may have led to gaps in the City’s production.

Takeaways

Did the Court direct the use of TAR?

Is this the first case where a Court directs the use of TAR? The Court states, “given the volume of documents collected, this Court directed the City to complete linear review as to certain custodians and begin using Technology Assisted Review,,,to hasten the identification, review, and production of documents responsive to Plaintiffs’ document requests.” Perhaps the Court suggested the use of TAR and both parties agreed. This is most likely since, the Court cites Hyles v. New York City, where Judge Andrew Peck found that the producing party is better equipped than the court to identify and utilize the best process for producing their own ESI consistent with their obligations under the Federal Rules of Civil Procedure. See Hyles, 2016 WL 4077114, at *3 (citing Principle 6 of the Sedona Conference). It will be interesting to see whether a Court is willing to direct parties to use TAR when either or both parties disagree with its use.

Precedence for In Camera Review in TAR?

It is interesting here that the Court appears to be very hands-on with the training of the TAR system. The Court required the City to provide in camera submissions where it concluded that Defendant “appropriately trained and utilized its TAR system.” The Defendant’s seed set included over 7,200 documents that were reviewed by the City’s document review team and marked as responsive or non-responsive in order to train the system.

In many cases, Courts ask parties to cooperate and hesitate to be as involved in these disputes. Does in camera review at this level of detail set a precedent for more in camera review of documents in cases using TAR?

QC Your Productions:

While Winfield is interesting in jurisprudence about TAR, the Court may not have been so involved but for the Defendant’s technical error in its production. The error provided support for Plaintiff’s allegations of over-designating documents as not responsive in their seed set. Here, Defendant appeared to be withholding non-responsive family member documents by producing “slip sheets” instead of a TIFF. The error here was the production of the actual extracted text for two documents that were “slip-sheeted” as withheld and non-responsive. This error allowed Plaintiffs to read the text contents that the Defendant wanted withheld as non-responsive. However, these two documents were indeed responsive and led to an in camera review of additional Defendant documents.

As a result, the Court found that Defendant incorrectly categorized at least five “slip-sheeted” documents as non-responsive during the electronic review process and correctly categorized 15 other documents as responsive during the electronic review process, but later labeled them as non-responsive.

Where is the line?

While the Court did not find that the accuracy and reliability of the City’s TAR process was deficient as a whole, what is the threshold? The Court cited judge Peck in Moore and Hyles: “In any ESI review, ‘the Federal Rules of Civil Procedure do not require perfection.’” Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012). Instead, the proper inquiry is whether the “search results are reasonable and proportional.” Hyles v. New York City, №10-cv-3119 (AT) (AJP), 2016 WL 4077114, at *3 (S.D.N.Y. Aug. 1, 2016) (citing Fed. R. Civ. P 26(g)(1)(B). Here, 20 total documents were found to be coded incorrectly. How many incorrectly coded documents would cross the line? And these were only found due to a production error and the careful attention of Plaintiffs’ counsel.