Predictive Analytics Cheat Sheet

Transcription

1 Predictive Analytics The use of advanced technology to help legal teams separate datasets by relevancy or issue in order to prioritize documents for expedited review. Often referred to as Technology Assisted Review (TAR) or Predictive Coding (PC). ALGORITHM A specified series of computations executed to accomplish a goal. AMBIGUOUS DOCUMENTS Documents for which the system cannot achieve a sufficiently clear relevance determination. These documents must be reviewed by attorneys. BULK CODING / BULK TAGGING The process of coding all members of a group of documents based on the review of only a few members of the group. CONCEPT-BASED PREDICTIVE ANALYTICS System analyzes the meaning and context of words used within a set of documents and translates that information into mathematical models. Once a model has been built, a find more like these algorithm is applied to the document population to identify documents that are similar in conceptual content. Concept-based Predictive Analytics is most effective when trying to find documents that closely resemble each other. CATEGORIZATION The process by which documents are grouped into specific categories in order to identify relationships. Categorization is performed with human interaction. CLUSTERING also known as THEMES System automatically organizes a document population into smaller subset groups based on conceptual content. These groupings are created and organized purely by the algorithm s classification without human interaction so clustering is most effective when the reviewer has little knowledge of the data content. CONCEPT SEARCH Using an internal language model, an analytics system uncovers and identifies document relationships within and across datasets. Concept search is used to find documents beyond those that would be returned by a simple keyword search and/ or Boolean search.

2 NEAR-DUPLICATE DETECTION The process of comparing electronic documents within a document population based on text content (not metadata) and then using that information to identify similar or duplicate versions of those documents across additional datasets. THREADING Analyzes a set of s based on text content and then groups s from the same conversation string. By identifying the most inclusive as a single point of review, prior versions can be set aside. CONFIDENCE LEVEL A measure indicating the overall reliability of sample-based estimates. It is the probability that a population parameter will fall between two set values. This measure can take any number of probabilities, with the most common being 95% or 99%. For example, 95% confidence means that if one were to draw 100 independent random samples of the same size, and compute the confidence interval from each sample, about 95 of the 100 confidence intervals would contain the true value. CONFUSION MATRIX A table that allows visualization and evaluation of the performance of the algorithm(s). FALSE NEGATIVE A relevant document that is incorrectly identified as non-relevant. FALSE POSITIVE A non-relevant document that is incorrectly identified as relevant. TRUE NEGATIVE A non-relevant document that is correctly identified as such. TRUE POSITIVE A relevant document that is correctly identified as such. DATASET A collection of documents specific to a case/matter.

3 F-MEASURE A balance between recall and precision. A higher F-Measure typically indicates higher precision and recall, while a lower F-Measure suggests lower precision and recall. Currently, there is no industry standard for an appropriate F-Measure, and it is up to the parties involved to define, depending on the particular needs of the case. MACHINE LEARNING The use of computer algorithms to organize or classify documents by analyzing their content and features. ACTIVE LEARNING System strategically chooses a document (often based on uniqueness) for which a reviewer makes a relevance decision. The system learns from these determinations and chooses the next set of exemplars to maximize its learning (ex. predictive coding). SUPERVISED LEARNING System uses subject matter experts coding decisions on a training set of documents in order to tag and rank the remaining documents in the collection based on similarity to the training dataset (ex. categorization). UNSUPERVISED LEARNING Documents are automatically organized, grouped and labeled by the system without any human interaction (ex. clustering). NON-TEXT DOCUMENTS Files (such as photos, poor-quality scans or electronic documents with security restrictions) that are not able to be considered by Predictive Analytics systems because the advanced technology is based solely on the text content of documents. POTENTIALLY PRIVILEGED DOCUMENTS These documents must be reviewed by legal teams in order to confirm that the content is indeed privileged. PRECISION A measure of exactness (actual relevant documents retrieved/total number of documents retrieved); what percent of a given dataset is relevant?

4 PREDICTIVE CODING A Predictive Analytics process involving the use of an active learning algorithm to distinguish relevant from non-relevant documents, based on subject matter experts coding decisions on a training set of documents. QUALITY CONTROL Methods to validate and ensure that reasonable results are being achieved during a review effort, especially when advanced technology is being utilized. CHECKLIST A record of the tasks performed which helps to mitigate the risk of error. DOCUMENT SEEDING Presenting the system with documents subject matter experts have already deemed relevant in order to better train algorithms within Predictive Analytics systems. OVERTURN CORRECTION A workflow utilized by legal teams to reverse a Predictive Analytics system s incorrect document classifications. TRACKING Linking documents back to the source media on which they were collected as well as to specific workflows. This produces a traceable record of data collections, processing, review and productions in order to provide chain of custody documentation. RECALL A measure of completeness (actual relevant documents retrieved/total actual relevant documents); what percent of the relevant documents were retrieved by the algorithm? RELEVANT DOCUMENT A document with content that pertains to the subject matter outlined in a production request. Not all relevant documents are responsive, but all responsive documents are relevant. RELEVANCE Denotes how closely a document pertains to the matter at hand.

5 RESPONSIVE DOCUMENT A document that actually meets the information needs of a party s production request. All responsive documents are relevant, but not all relevant documents are responsive. RESPONSIVENESS Denotes how well a document meets the information need of an opposing party. SAMPLING as used in ediscovery The process of selecting a representative part of a dataset for the purpose of identifying search terms and determining relevance. STABILIZATION Occurs when a Predictive Analytics system has learned all it can in order to predict relevance. SUBJECT MATTER EXPERT (SME) An individual who is familiar with the case information and issues and can make a determination as to the relevance of a particular document. SUPPORT VECTOR-BASED PREDICTIVE ANALYTICS also known as PREDICTIVE CODING Supervised learning models with an algorithm that analyzes data to recognize content patterns and are used for classifying by regression analysis. TRAINING DOCUMENTS Documents that help Predictive Analytics applications learn how legal teams would handle a specific document. They are distinguished between relevant and non-relevant in order to give the technology the required inputs to make future classifications. VALIDATION SAMPLES Confirm the performance of Predictive Analytics algorithms.

6 About D4 D4 is a national provider of electronic discovery, computer forensics, information security and management, and deposition services to law firms and corporations, and has been instrumental in helping customers realize up to a 70% cost reduction over previous ediscovery solutions. At D4, we focus on technology and process to streamline the discovery lifecycle in the most defensible, practical and cost-effective manner possible. We believe that ediscovery doesn t have to break the bank and we make that belief a reality for clients every day. Founded in 1997 in Upstate New York, D4 has grown to a national presence. With over 160 employees, D4 has offices in Buffalo, Chicago, Detroit, Grand Rapids, Lincoln, New York City, Omaha, Orlando, Phoenix, Rochester, San Francisco, San Diego and Tampa. D4 s state-of-theart Tier 3 data center and operations in Rochester are complemented by electronic discovery, litigation support and paper document services in other offices across the country. D4 has been recognized by Inc. Magazine as one of the fastest-growing private companies in the US, and is a four-time Inc. 500/5000 honoree. There s a reason why hundreds of AMLAW 200 firms and Fortune 1000 companies choose D4. Our unprecedented customer service, coupled with our industry experts and best-of-breed technology, is why the D4 way is the better way. marketing@d4discovery.com