Analyses and Displays Associated with Outliers or Shifts from Normal to Abnormal

50 Pages·2015·1.53 MB·English

Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

PhUSE

PhUSE Computational Science Development of
Standard Scripts for Analysis and Programming
Working Group

Analysis and Display White Papers Project Team

Analyses and Displays Associated with Outliers or
Shifts from Normal to Abnormal: Focus on Vital
Signs, Electrocardiogram, and Laboratory Analyte
Measurements in Phase 2-4 Clinical Trials and
Integrated Summary Documents

[Version 1.0] – [2015-09-10] 1
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

Table of Contents
1. Disclaimer .................................................................................................................................................... 3
2. Notice of Current Edition .............................................................................................................................. 3
3. Additions and/or Revisions........................................................................................................................... 4
4. Overview: Purpose ....................................................................................................................................... 5
5. Scope ........................................................................................................................................................... 5
6. Definitions .................................................................................................................................................... 6
7. Problem Statement ...................................................................................................................................... 6
8. Background .................................................................................................................................................. 6
9. Considerations ............................................................................................................................................. 7
10. Recommendations ..................................................................................................................................... 7
10.1. General Recommendation ................................................................................................................. 7
10.2. All Measurement Types ..................................................................................................................... 7
10.3. Laboratory Analyte Measurements .................................................................................................. 11
10.4. ECG Quantitative Measurements .................................................................................................... 13
10.5. Vital Sign Measurements ................................................................................................................. 14
11. Tables and Figures for Individual Studies ................................................................................................ 14
11.1. Recommended Displays .................................................................................................................. 14
11.2. Discussion ........................................................................................................................................ 24
12. Tables and Figures for Integrated Summaries ........................................................................................ 25
12.1. Recommended Displays .................................................................................................................. 25
12.2. Discussion ........................................................................................................................................ 31
13. Example SAP Language .......................................................................................................................... 32
13.1. Individual Study ................................................................................................................................ 32
13.2. Integrated Summary......................................................................................................................... 34
14. Acknowledgements .................................................................................................................................. 37
15. Project Leader Contact Information ......................................................................................................... 37
16. References ............................................................................................................................................... 38
17. Appendix: Figures and Tables ................................................................................................................. 39
List of Tables and Figures
Figure 11.1. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
Low Value: Individual Study .............................................................................................................. 16
Figure 11.2. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
High Value: Individual Study ............................................................................................................. 18
Figure 11.3. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
Low Value with Change Criteria: Individual Study ............................................................................ 20
Figure 11.4. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
High Value with Change Criteria: Individual Study ........................................................................... 22
Table 11.1. Treatment-Emergent Abnormal Summary for Qualitative Safety Measures:
Individual Study ................................................................................................................................. 24
Figure 12.1. Scatterplot and Shift Summary for Quantitative Safety Measures for Low Value:
Integrated Database .......................................................................................................................... 27

[Version 1.0] – [2015-09-10] 2
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

Figure 12.2. Scatterplot and Shift Summary for Quantitative Safety Measures for High Value:
Integrated Database .......................................................................................................................... 29
Table 12.1. Treatment-Emergent Abnormal Summary for Qualitative Safety Measures:
Integrated Database .......................................................................................................................... 31
Table 13.1. Selected Categorical Limits for ECG Data .................................................................................. 33
Table 13.2. Categorical Criteria for Abnormal Treatment-Emergent Blood Pressure and Pulse
Measurement and Categorical Criteria for Weight and Temperature Changes for Adults ............... 34
Figure 17.1. Summary for Quantitative Safety Measures: Individual Study .................................................. 39
Figure 17.2. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
Low Value: Individual Study .............................................................................................................. 40
Figure 17.3. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
High Value: Individual Study ............................................................................................................. 41
Figure 17.4. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
Low Value with Change Criteria: Individual Study ............................................................................ 42
Figure 17.5. Scatterplot and Shift Summary for Quantitative Safety Measures Assessing
High Value with Change Criteria: Individual Study ........................................................................... 43
Figure 17.6. Summary of Common Treatment-Emergent Abnormal for Quantitative Safety
Measures: Individual Study ............................................................................................................... 44
Figure 17.7. Scatterplot and Shift Summary for Quantitative Safety Measures: Integrated Database ......... 45
Figure 17.8. Scatterplot and Shift Summary for Quantitative Safety Measures for Low Value:
Integrated Database .......................................................................................................................... 46
Figure 17.9. Scatterplot and Shift Summary for Quantitative Safety Measures for High Value:
Integrated Database .......................................................................................................................... 47
Table 17.1. Shift Table Analyses ................................................................................................................... 48
Table 17.2. Shift from Normal/High to Low and from Normal/Low to High for Laboratory Measures ........... 49
Table 17.3. Shift from Normal/High to Low and from Normal/Low to High: Integrated Database ................. 50

1. Disclaimer
The opinions expressed in this document are those of the authors and do not necessarily
represent the opinions of PhUSE, the members’ respective companies or organizations, or
regulatory authorities. The content in this document should not be interpreted as a data standard
and/or information required by regulatory authorities.
2. Notice of Current Edition
This edition of the “Analyses and Displays Associated with Outliers or Shifts from Normal to
Abnormal: Focus on Vital Signs, Electrocardiogram, and Laboratory Analyte Measurements in
Phase 2-4 Clinical Trials and Integrated Summary Documents” is the 1st edition.

[Version 1.0] – [2015-09-10] 3
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

3. Additions and/or Revisions
Date Author Version Changes
2015-09-10 See Section 14 v1.0 First edition

[Version 1.0] – [2015-09-10] 4
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

4. Overview: Purpose
The purpose of this white paper is to provide advice for displaying, summarizing, and/or analyzing
measures of outliers or shifts in tables, figures, and listings (TFLs), with a focus on vital signs,
electrocardiogram (ECG) quantitative measurements, and laboratory analyte measurements in
Phase 2-4 clinical trials and integrated submission documents. This white paper also provides
advice on data collection if a particular recommended display requires data to be collected in a
certain manner that may differ from current practice. The intent is to begin the process of
developing industry standards with respect to analysis and reporting for measurements that are
common across clinical trials, and even therapeutic areas. In particular, this white paper provides
recommendations for key TFLs for measures of outliers or shifts for a common set of safety
measurements. Separate white papers address other types of data or analytical approaches (e.g.,
central tendency).
The development of standard TFLs and associated analyses will lead to improved standardization
from collection through data storage. (You need to know how you want to analyze and report
results before finalizing how to collect and store data.) The development of standard TFLs will
also lead to improved product lifecycle management by ensuring that reviewers receive the
desired analyses for the consistent and efficient evaluation of patient safety and drug
effectiveness. Although having standard TFLs is an ultimate goal, this white paper reflects
recommendations only and should not be interpreted as “required” by any regulatory agency.
5. Scope
The scope of this white paper is to provide advice when developing the analysis plan for Phase 2-
4 clinical trials and integrated summary documents (or other documents in which measures of
outliers or shifts are of interest).
Although the focus of this white paper pertains to specific safety measurements (vital signs, ECG
quantitative measurements, and laboratory analyte measurements), some content may apply to
other measurements (e.g., different safety measurements and efficacy assessments). Similarly,
although the focus of this white paper pertains to Phase 2-4 clinical trials, some of the content
may apply to Phase 1 clinical trials or other types of clinical research (e.g., observational studies).
Detailed specifications for TFLs or dataset development are considered out of scope for this
version of this white paper. However, the hope is that specifications and code (utilizing Study Data
Tabulation Model [SDTM] and Analysis Data Model [ADaM] data structures) will be developed
that are consistent with the concepts outlined in this white paper and placed in the publicly
available Pharmaceuticals Users Software Exchange (PhUSE) Standard Scripts Repository.

[Version 1.0] – [2015-09-10] 5
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

6. Definitions
ADaM = Analysis Data Model; ALT = alanine aminotransferase; AST = aspartate
aminotransferase; BMI = body mass index; CBER = Center for Biologics Evaluation and
Research; CDASH = Clinical Data Acquisition Standards Harmonization; CDER = Center for Drug
Evaluation and Research; CS = Computational Science; ECG = electrocardiogram; LLN = lower
limit of normal; PhUSE = Pharmaceuticals Users Software Exchange; SDTM = Study Data
Tabulation Model; TFLs = tables, figures, and listings; ULN = upper limit of normal
7. Problem Statement
Industry standards have evolved over time for data collection (Clinical Data Acquisition Standards
Harmonization [CDASH]), observed data (SDTM), and analysis datasets (ADaM). However,
standards have not been developed for analyses and reports. Lack of standardization leads to
inefficiency in operation (time, cost), unreliable quality, and the creation of displays that may not
be of optimal use to the reviewers.
8. Background
Industry standards have evolved over time for data collection (CDASH), observed data (SDTM),
and analysis datasets (ADaM). There is now recognition that the next step would be to develop
standard TFLs for common measurements across clinical trials and across therapeutic areas.
Some could argue that the industry should have started with creating standard TFLs prior to
creating standards for collection and data storage (consistent with end-in-mind philosophy);
however, having industry standards for data collection and the analysis of datasets provides a
good basis for creating standard TFLs.
The beginning of the effort leading to this white paper came from the initiation of the FDA/PhUSE
Computational Science Collaboration, a yearly conference and ongoing working groups to support
addressing computational needs of the industry. The FDA identified key priorities and teamed up
with the PhUSE to tackle various challenges using collaboration, crowd sourcing, and innovation
(Rosario LA, 2012). The FDA and PhUSE created several Computational Science (CS) working
groups to address several of these challenges. The working group, titled “Development of
Standard Scripts for Analysis and Programming,” has led the development of this white paper,
along with the development of a platform for storing shared code.
There are several existing guidance documents (see bulleted list below) that contain suggested
TFLs for common measurements, such as vital signs, ECG quantitative measurements, and
laboratory analyte measurements. However, many of these documents are now relatively
outdated and generally lack sufficient detail to be used as support for the entire standardization
effort. Nevertheless, these documents were used as a starting point in the development of this
white paper. The documents include the following:
• ICH E3: Structure and Content of Clinical Study Reports
• Guideline for Industry: Structure and Content of Clinical Study Reports
• Guidance for Industry: Premarketing Risk Assessment
[Version 1.0] – [2015-09-10] 6
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

• Reviewer Guidance: Conducting a Clinical Safety Review of a New Product Application
and Preparing a Report on the Review
• ICH M4E: The Common Technical Document for the Registration of Pharmaceuticals for
Human Use: Efficacy
• ICH E14: The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrhythmic
Potential For Non-Antiarrhythmic Drugs
• Guidance for Industry: ICH E14 Clinical Evaluation of QT/QTc. Interval Prolongation and
Proarrhythmic Potential for Non-Antiarrhythmic Drugs
The Reviewer Guidance is considered a key document. As discussed in the guidance, there is
generally an expectation that analyses of outliers or shifts are conducted for vital signs, ECG
quantitative measurements, and laboratory analyte measurements. The guidance recognizes
value to both analyses of central tendency and analyses of outliers or shifts from within reference
limits to outside reference limits (below lower reference limit or above upper reference limit). We
assume both will be conducted for safety signal detection. This white paper covers the outliers or
shifts portion with the expectation that an additional TFL or TFLs will also be created with a focus
on central tendency (see the CS white paper pertaining to central tendency).
9. Considerations
Members of the Analysis and Display White Papers Project Team reviewed regulatory guidance
and shared ideas and lessons learned from their experience. Draft white papers were developed
and posted in the PhUSE wiki environment for public comment.
Most contributors and reviewers of this white paper are industry statisticians, with input from non-
industry statisticians (e.g., FDA and academia) and industry and non-industry clinicians.
Additional input (e.g., from other regulatory agencies) for future versions of this white paper would
be beneficial.
10. Recommendations
10.1. General Recommendation
This section contains some general considerations for the plans of analyses and displays
associated with outliers or shifts from normal to abnormal for laboratory analyte measurements,
vital signs, and ECG quantitative measurements. Section 10.2 discusses general considerations
for all three safety domains. Section 10.3 discusses considerations specific to laboratory analyte
measurements. Section 10.4 discusses considerations specific to ECGs quantitative
measurements. Section 10.5 discusses considerations specific to the vital signs.
10.2. All Measurement Types
P-values and Confidence Intervals
There has been an ongoing debate about the value for (or lack of value for) the inclusion of p-
values and/or confidence intervals in safety assessments (Crowe BJ, 2009). This white paper
[Version 1.0] – [2015-09-10] 7
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

does not attempt to resolve this debate. As noted in the Reviewer Guidance, p-values or
confidence intervals can provide some evidence of the strength of the findings, but unless the
trials are designed for the hypothesis testing of safety endpoints, these should be thought of as
descriptive. Throughout this white paper, p-values and measures of spread are included in
several places. Where these are included, they should not be considered as hypothesis testing. If
a company or compound team decides that these are not helpful as a tool for reviewing the data,
they can be excluded from the display. Although certain statistical methods are recommended in
this white paper for p-values and confidence intervals (for teams that choose to include them),
alternative methods can be considered.
Some teams may find p-values and/or confidence intervals useful to facilitate focus but have
concerns that a lack of statistical significance provides unwarranted dismissal of a potential signal.
Conversely, there are concerns that there could be over-interpretation of p-values due to
multiplicity issues, adding potential concern for too many outcomes. Similarly, there are concerns
that the lower- or upper-bound of confidence intervals will be over-interpreted. (A percentage can
be as high as x causing undue alarm.) It is important for the users of these TFLs to be educated
on these issues.
Importance of Visual Displays
Communicating information effectively and efficiently is crucial in detecting safety signals and
enabling decision making. Current practice, which focuses on tables and listings, has not always
enabled us to communicate information effectively because tables and listings may be long and
repetitive, making it difficult to see trends. Graphics, on the other hand, can provide a more
effective presentation of complex data, increasing the likelihood of detecting key safety signals
and improving the ability to make clinical decisions. They can also facilitate the identification of
unexpected values.
Standardized presentation of visual information is encouraged. The FDA/Industry/Academia
Safety Graphics Working Group was initiated in 2008 and was formed to develop a wiki and to
improve best practice for safety graphics. It has recommendations for the effective use of graphics
for three key safety areas: adverse events, ECGs, and laboratory analytes. The working group
focused on static graphs, and their recommendations were considered while developing this white
paper. In addition, there has also been advancement in interactive visual capabilities. The
interactive capabilities are beneficial but are considered out of scope for this version of the white
paper.
Conservativeness
The focus of this white paper pertains to clinical trials in which there are comparator data. As
such, the concept of “being conservative” is different than when assessing a safety signal within
an individual subject or a single arm. A seemingly conservative approach may end up not being
conservative in the end. For example, for studies that collect safety data during an off-drug follow-
up period, one might consider it conservative to include the adverse events reported in the follow-
up period. However, this approach may result in smaller odds ratios than including only the
exposed period in the analysis. Another example occurs when choosing cutoffs for shift/outlier
analyses. A conservative approach for defining outcomes, from a single-arm perspective, is one
[Version 1.0] – [2015-09-10] 8
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

that would lead to a higher number of patients reaching a threshold. However, a conservative
approach for defining outcomes may actually make it more difficult to identify safety signals with
respect to comparing treatment with a comparator (see Section 7.1.7.3.2 in the Reviewer
Guidance; U.S. Department of Health and Human Services, 2005b). Thus, some of the
approaches recommended in this white paper may appear less conservative than alternatives, but
the intent is to propose methodology that can identify meaningful safety signals for a treatment
relative to a comparator group.
Measurements After Stopping Study Medication
Measurements collected after stopping medications under study (e.g., treatment under study and
comparators) are common for various reasons. In some cases, follow-up phases are included to
monitor patients for a period of time after study medication is stopped. In addition, study designs
where keeping patients in a study (for the entire planned length of time) after deciding to stop
medication early are becoming more popular as newer methods are developed for handling
missing data. In these cases, patients can be off study medication for an extended period of time.
Measurements post study medication can also arise not by design. For example, a subject can
decide to stop study medication at any time, and then later attend the planned visit where the
planned measurements are obtained. There is currently no standard approach on how to handle
safety assessments post study medication. Some guidance contains advice on how long to collect
safety measurements post study medication (e.g., 30 days post or x half-lives). Any advice or
decisions related to the collection of safety measurements post study medication should not be
confused with how to include such data in displays and/or analyses. It is extremely important to
document within the database for analysis the best estimate of the last date study treatment was
taken, as well as the dates on which all numerical safety data were collected, so that an accurate
determination can be made of time of data collection relative to the last dose of medication.
We recommend that the TFLs in this white paper generally exclude measurements taken during a
follow-up phase. Separate TFLs can be created for the follow-up phase and/or the treatment and
follow-up phases combined. We also recommend that the TFLs in this white paper exclude
measurements taken after the visit, which is considered the “study medication discontinuation”
visit. In the study designs that keep patients in a study for the entire planned length of time even
after stopping medication, separate TFLs can be created for the “off-medication” time and/or the
treatment and off-medication times combined. This enables the researcher to distinguish between
drug-related safety signals versus safety signals that could be more related to discontinuing a
drug (e.g., return of disease symptoms, introduction of a concomitant medication, and/or
discontinuation or withdrawal effects of the drug) or due to subsequent therapy.
We assume it is important to distinguish among these. Generally, at least some TFLs that include
data from follow-up phases and/or off-medication time will be required, but not usually as many as
are done during treatment and not necessarily in the same format as provided in this white paper.
For some compounds (e.g., compounds with a long half-life compared to the duration of the study,
compounds used for a short time, such as antibiotics), a more complete set of TFLs including
such data may be required. The ease of interpretation from such TFLs will vary depending on the
compound, disease, and design aspects, such as the half-life of the compound, the likelihood of
[Version 1.0] – [2015-09-10] 9
Project: Analysis and Display White Papers Project Team Working Group: Standard Scripts
Title: Analyses and Displays Associated with Outliers or Shifts for Analysis and Programming

taking alternative therapy, or the allowed concomitant medications during the observation period.
For the case where a subject decides to stop study medication at any time and then later attends
the planned visit to obtain the planned measurements, we recommend that the measures taken at
the study medication discontinuation visit be included. Although some patients may be off
medication, the time is generally short in these situations. For this example, the inclusion of such
measurements may more accurately reflect the safety profile of a compound versus their
exclusion. In study designs with a long period of time between visits, an alternative approach may
be warranted.
Measurements at a Discontinuation Visit
When creating displays or conducting analyses over time, how to handle data collected at
discontinuation visits should be specified. Because a subject’s discontinuation visit is not always
aligned with planned timing, it is not obvious whether to include these measurements in displays
or analyses over time. Such measurements are “planned” per protocol but not consistent with the
planned timing. We generally recommend including measures taken at the discontinuation visit
toward the next timepoint. For example, if a patient discontinues medication and the study
between Visits 6 and 7 and then goes to the office for their discontinuation visit, we recommend
that the measurements taken at the discontinuation visit are grouped with Visit 7. The inclusion of
such measurements may more accurately reflect trends over time for the compound than their
exclusion. In study designs with a long period of time between visits, an alternative approach may
be warranted.
Measurements Collected in Reflex Manner
In study designs, it is possible to have some measurements collected only when another
measurement meets a certain criteria (i.e., collected in a reflex manner). For example, sometimes
a peripheral smear is only performed when certain complete blood count analytes meet a
specified threshold. How to handle such measurements should be specified in analysis planning,
which requires an understanding of collection practices. Generally, measurements collected in a
reflex manner would be used for individual patient management and possibly for individual patient
listings or individual case descriptions (e.g., as included in patient narratives). Summaries of such
measurements within or between treatment groups tend to be uninterpretable because you
cannot generally assume normality among those who did not have the measurement, and a
summary among those meeting the criteria for receiving the measurement (sometimes a small
denominator) tends to not be helpful for signal detection purposes.
Screening Measurements versus Special Topics
The focus of this white paper pertains to measurements as part of normal safety screening. For
many compounds, some measurements are relevant to addressing a priori special topics of
interest. In these cases, it is possible that additional TFLs and/or different TFLs are warranted.
TFLs designed for special topics are out of scope for this white paper. In addition, it is possible
that additional TFLs are warranted when a safety signal is identified using the TFLs
recommended in this white paper and/or the TFLs that focus on central tendency (separate white
paper). Additional TFLs that would be considered post-hoc for further investigation are considered
out of scope.
[Version 1.0] – [2015-09-10] 10