Risk-Based TMF Management: Considerations to Enhance Quality, Compliance and Efficiency

In recent years, clinical research has become increasingly complex and expensive with longer drug development processes that result in the need for innovative solutions. To reduce the impact, regulatory authorities recommend organizations employ risk-based approaches focused on patient safety and clinical data integrity. The successful implementation of the principles of Quality by Design (QbD) in protocol development and risk-based monitoring demonstrably increase efficiency.

Risk-based methodologies have been used in Trial Master File (TMF) management and oversight, with some organizations adopting risk-based approaches to record Quality Control (QC) aligned with ALCOA+ principles. Despite these implementations, Regulatory authorities often find that the implementation of the principles is inadequate and that the required quality and completeness of the TMF is not achieved as a result.

Risk-based approaches inherently preclude a “one size fits all” solution. The CDISC TMF Reference Model provides some assistance by identifying “core” and “recommended” TMF records to be collected during the course of clinical trials; however, many organizations modify their TMF indices to meet their specific needs. Risk is multifactorial and contingent on many variables, among them

• trial design / complexity;
• technology maturity / complexity; and
• operating model.

Each organization has the responsibility to construct its risk-based approaches based on its own understanding of risk, its appetite for risk, its ability to justify its approach, and any associated limiting factors such as available organizational resources.

This white paper offers a practical framework to help organizations adopt risk-based approaches to Trial Master File (TMF) management. Designed to inform and guide adaptable, organization-specific practices, the paper:

• combines sources of relevant regulations, guidance, laws, and industry standards
• proposes practical methods for applying risk-based strategies to the TMF;
• offers key considerations for evaluating TMF-related risks; and
• includes a TMF Risk Management Tool to aid in implementation.

It is not intended to prescribe a single formula for TMF risk management, but rather to provide a flexible framework that organizations can adapt to their needs.

Provided risk-based approaches are implemented correctly, the risks associated with complex clinical trials are manageable and the required expenditure foreseeable. Careful reading and implementation of the measures proposed in this paper should increase the likelihood of success in managing TMF risk effectively.

2. Background

The TMF comprises all records in all repositories that collectively permit the evaluation of a clinical trial and serve as evidence of compliance with the protocol, legislation, and regulatory / ethical guidance.

Adopting a risk-based approach requires that some TMF elements are subject to more scrutiny than others. Accordingly, baseline TMF controls should be increased or decreased relative to the potential threats identified in relation to impacts on

• patient rights, well-being, safety or dignity;
• regulatory expectations for data integrity;
• “essentiality” as stipulated in ICH GCP E6 (R3);
• the evidential value and quality of records; and
• the completeness of the TMF.

Successful risk management activities based on robust compliance assessments will establish the correct correlation between risk and criticality, lead to appropriate resource planning, and increase the chance of timely detection of issues or errors in TMF records.

3. A Risk-Based Approach

3.1. The ICH Basis for a Risk-Based Approach

Existing ICH guidelines provide a framework on which to develop a risk-based methodology for TMF that is specific for each study and compliant with GCP. This paper proposes the method in Figure 1 below.

Figure 1 Based on ICH GCP (R3) Quality Management

3.2. Critical Process and “Essential” Record Identification

Guidance in ICH GCP E6 (R3) sets out how to determine which records are “essential” and therefore require filing. Risk-based management of the quality of the records and the completeness of the TMF should be governed by risk-based processes and Critical Quality Factors[1], including among them trial design, objectives, therapy, and endpoints as well as operational considerations such as countries, facilities, and equipment.

4. Risk Identification and Evaluation

4.1. TMF Risk Identification and Risk Management

Risks to TMF processes should be identified and assessed at organizational, trial, country, and site levels. This supports a practical approach to the implementation of controls, risk monitoring, communication, and reporting. Within a single trial, a combination of control measures may be applied; these may differ from trial to trial.

Higher risks associated with the lack of, or inadequate resources, systems, processes, procedures, and controls increase the likelihood that TMF records are incomplete or are not inspection ready. Incomplete, inaccurately filed, illegible, or inaccessible TMF records can prevent proper evaluation (or “reconstruction”) of the trial.

Risks can be identified and assessed by leveraging a risk log at the start of the trial, with risks linked to the TMF record production. Clearly defined TMF milestones and expected record lists can help ensure timely record collection. Ancillary TMF record repositories (e.g., those used for oversight and control) should also be identified and evaluated for risks. TMF risk assessments, like those of the study should be revisited as changes occur.

4.2. Multiple TMF Repositories

It is often the case that “essential” TMF records are retained in multiple repositories and that QC and oversight both during and after the trial are focused on the primary (e)TMF with the records retained in other repositories overlooked.

To avoid this, organizations should minimize the number of repositories used to retain “essential” TMF records. Non-primary (e)TMF repositories should be limited to those required for specialist purposes and functionality unavailable in (e)TMF; for example, laboratory data, clinical data and statistics are frequently retained in systems focused on their capture, review, interpretation, analysis, and management. These non-primary (e)TMF repositories should have defined processes to ensure the quality and completeness of their records and associated metadata. 

Where “essential” records are retained in specialist repositories, organizations should use the TMF Index to signpost the location of those “essential” records. This will

• ensure that “essential” records are routinely included in QC and oversight activities;
• include “essential” records that may not typically be considered study related such as, personal qualifications and training and ensure their archival and;
• inform monitors, auditors and inspectors of the location of “essential” records.

4.3. Organizational Level Risks

When identifying and evaluating TMF risks, a sponsor should consider

• clinical trial type;
• its resources, processes, and procedures;
• its planned use of the trial results; and
• best practice in relation to risk management.

Procedures should ensure that appropriate and sufficient levels of control are in place to support consistent decision-making, although divergence may be permitted where justifiable (e.g., for country-specific requirements). It is recommended that TMF “essential” records lists are pre-defined to reflect the requirements laid down in the protocol and / or any trial-specific processes, which may mean that some records are generated whilst others are not, for example by using and adapting the CDISC TMF Reference Model.

Each organization will have unique circumstances that potentially impact TMF management and integrity such as

• staff availability and skills (e.g., record management, QC);
• involvement of intermediaries or third parties, including record collaboration needs;
• the nature of the TMF (e.g., centralized, decentralized, outsourced);
• TMF type, capabilities, and (for paper / hybrid TMF) facilities;
• maturity of TMF set-up, maintenance, and archiving procedures; and
• use of independent Quality Assurance to routinely audit TMF systems, data, and procedures.

An example of how trial risks might influence an organization’s approach to TMF risks is presented in Table 1 below.

Table 1 Example of Trial Risk Considerations that Impact the TMF

The level of quality desired will determine

• the methods and the extent of the methods; and
• the acceptable variability permitted within the controls established.

4.4. Trial Level Risks

Initial risk management reviews should include TMF-related risks, including risk assessment for any TMF-related records stored outside of the TMF.

The study-specific TMF index, milestones, and expected records should define relevant and expected TMF records, including associated priorities and acceptable tolerances. The same rules should apply to any ad hoc or unexpected records.

4.5. Country Level Risks

Country level risks may arise from differences in local legal, regulatory, or ethical requirements, which may preclude compliance with a universally applied standard. Consequently, local expertise should be sought to determine any specific risks (e.g., European Medicines Agency guidance specifies greater detail for managing the TMF than ICH GCP [R3]).

Examples of country specific TMF requirements include

• the need to maintain TMF records in their original format;
• the need to apply and retain wet-ink signatures;
• expectations of direct TMF access for Inspectors; and
• record retention periods.

4.6. Site Level Risks

Site-level risks arise from differences in local regulatory, ethical, and institutional requirements. Site feasibility assessments should ensure these risks are identified and controlled. Sites with more site-specific requirements have a higher risk because TMF records may potentially but inadvertently be missed in standard checks.

Individual sites may also have varying levels of risk, based on any number of factors. The underlying reasons need to be identified and properly understood before risk assessment / analysis can be determined. Specifically,

• more deviations overall is not a particular risk, but a higher proportion of deviations may be;
• a high recruitment rate increases the amount of empirical evidence to the endpoint, so systematic errors at high recruiting sites may have a greater impact; 

The ability to recreate what occurred at site and to prove the appropriate level of required oversight is a component of investigator and / or sponsor oversight. An organization’s prior experience with any site may also factor into the perceived risk that a single site may represent.

4.7. Other TMF Risks

Other factors such as prior knowledge from inspections, monitoring and audit findings or other information (e.g. data analytics) may be used to identify elements that present inherent vulnerabilities and / or higher TMF risks.

The value of a risk-based approach is in the determination of risk-proportionate controls that detect when TMF records are not managed as planned, so that relevant and appropriate actions can be taken to mitigate or remediate the risks.

5. Risk-Based Controls

Organizations should have layers of controls in place that serve to pre-emptively mitigate potential risks and proactively detect issues. These controls should not be limited merely to the use of an electronic TMF or checks on records. Controls should be in place for people, processes, procedures, and the systems used to manage critical trial records and data.

5.1. People

People present both risk and control. For people to be effective controllers, it is necessary to:

• ensure roles and responsibilities are appropriately defined, assigned, and communicated (e.g., in a RACI [Responsibilities, Accountability, Consulted, and Informed] table);
• universally communicate expectations regarding the level of quality control for records, and the frequency and reporting of periodic TMF reviews for completeness and inspection readiness;
• provide role-specific training for all involved in the development, review, and approval of TMF records;
• regularly communicate metrics and indicators of status against pre-defined thresholds; and
• take action when issues with the TMF are identified to secure quality and completeness.

5.2. Processes

TMF processes (e.g., creation, upload, classification, review, approval, finalization, and QC) should be logical, simple, and user-friendly to promote timely and accurate filing in the TMF.

Where possible, tolerance limits or key performance indicators (KPIs) should be established to promote action when acceptable quality levels are not achieved. Clinical trials with similar critical processes and related TMF records can be used to create reliable threshold limits. Should KPIs not be met, a review and update of processes may be required to mitigate risks.

Periodic cross-functional reviews can be focused on the TMF sections with TMF records of the highest importance (e.g., safety data endpoint related, legal approvals and submissions). These high-risk sections should have the lowest permitted errors and should therefore receive the most frequent and contextual reviews so that documents are not reviewed in isolation. The related TMF records should have shorter processing times for upload and QC resolution to ensure best practice and compliance.

Procedures must clearly describe escalation pathways should tolerance limits not be met. The frequency and severity of not meeting expectations dictates whether to use an escalation, or if formal deviation and Corrective and Preventive Actions (CAPA) procedures should be implemented.

Trials that are not intended for regulatory submission may have a documented justification for a reduced TMF risk management process. This should be recorded in the Protocol, Risk Management Plan, or TMF plan.

5.3. Systems

eTMF systems offer a wide variety of capabilities to facilitate the implementation and management of risk-based approaches. These capabilities are captured in Table 2 below.

Table 2 TMF System Capabilities That Assist with Risk Management

Organizations should undertake periodic reviews of the effectiveness of the measures adopted to ensure the TMF is maintained as planned. When data shows a decline in TMF quality, a review of the TMF risk-based approach should be undertaken and documented to determine if a change is needed.

5.3.1. Future System Considerations

Rules-based risk scenarios can be used with artificial intelligence to assess historical situations along with current records, events, and trends. Following validation, the system could then identify risks, with probability, impact, and possible mitigations to ensure the TMF remains fit for purpose. This process can become more efficient and limit human error that might miss risks or misinterpret the risk assessments. Investing time in developing rules-based approaches is required to ensure that risk assessment and mitigation align with the regulations, GCP, and the organization's experiences, processes, and risk strategies.

6. Risk Mitigation

Regardless of the extent of controls implemented, no TMF will reach perfection, due to the volume of TMF records and data produced across multiple countries and sites, and the vast array of regulatory requirements.

Reiterating the purpose of risk-based approaches (i.e., to limit the number of significant errors), it may be useful to consider the following activities:

1. Conduct a review of predicate rule requirements and regulatory requirements as these may mandate filing over organizational requirements and procedures. (Repeated use of this mitigation may trigger revision of standard procedures).
2. Develop clear, well-structured trial planning documents and ensure staff are thoroughly trained;
3. Dedicate resources and time for maintaining milestones and expected record lists or placeholders. The management of these activities should be documented (e.g., in a the TMF Plan).
4. Define locations of the virtual TMF (including all record repositories), and ensure all records are maintained with quality and reconciled for completeness to support evaluation;
5. Incorporate records management requirements for TMF in vendor / service provider selection and contracting processes.
6. Utilize TMF specialists who can support and direct the study team and functional representatives to manage the TMF for completion and inspection readiness.
7. Implement dedicated TMF solutions that are robustly tested and proven to be fit-for-purpose.
8. Conduct periodic reviews of eTMF functionalities that support the retrieval of TMF oversight information.
9. Reduce the effort involved in managing originals and copies of records. If a record is verified and available in the right place, question the need for a duplicate copy.
10. Develop correspondence guidance to ensure that e-mails are appropriately curated. The CDISC TMF Reference Model group has created guidance on managing e-mails^[2].

Risk-based approaches do not substitute the need to ensure good record-keeping practices, or compliance with GCP, GDocP and Data Protection. The mitigations above may be leveraged in addition to elements such as system validation, system access controls (including routine and regular review), audit trail checks, vendor performance assessment and monitoring, and routine process audits. Poor TMF management will not be remediated merely by using an eTMF.

7. Risk Monitoring

The key to establishing effective risk-based approaches to TMF management and oversight is effective monitoring of tolerance limits and other identified risks. There are several critical monitoring activities that (when measured against KPIs) can serve as an escalation point for additional review and oversight.

The appropriate frequency for monitoring activities should be determined during routine risk reviews and should reflect the risk of the overall study. Ideally, organizations should employ a high frequency of monitoring during the initial stages of a trial or the use of a new contractor, process, or technology. Once acceptable quality levels are established, the frequency can be reduced.

7.1. Routine Risk Review

The risks identified at study start-up should be routinely assessed throughout the conduct of the trial to determine if they remain applicable and are being appropriately mitigated. Almost every study has at least one protocol amendment or shift in approach that warrants the need to determine if there are any new risks to monitor. This type of review should occur periodically and be documented within the risk log. Any changes to risks must be reflected in the approach to managing the TMF for the study.

A protocol amendment that requires a new type of training for site personnel may best be monitored by a scheduled completeness check a few months after the amendment to ensure all site personnel requiring training have evidence of completion. While this would be detected during a routine periodic review, the scope of the review may not cover all personnel impacted by the amendment and thus not control the risk.

7.2. TMF Record Level QC

Record level completeness checks safeguard against records being incomplete, inaccurate, and / or inappropriately filed. Record metadata reviews confirm that metadata matches the record, supports its identification, and facilitates subsequent retrieval.

Successful implementation of risk-based approaches to quality control at the record level make it possible to reduce the level from 100% (i.e., excessive) to a more appropriate level (e.g., 10%) without reducing quality targets. Examples of this exist within the International Organization for Standardization (ISO)[3] and American National Standards Institute (ANSI)[4] where sampling of TMF records may be based on the size of a lot or batch of TMF records processed over a given time. Tolerance limits are based on the standards for sample size evaluation. A lot or batch of TMF records could be defined by:

• comparable records or comparably processed records over a period of time;
• organization, functional area, or zone in the CDISC TMF Reference Model;
• criticality or risk level; and /or
• context in relation to key clinical trial milestones.

If the sample reviewed minimally meets the tolerance level, then no additional action is required (see Table 3 below). However, if the threshold is exceeded, then additional actions are required, such as additional sampling and testing. A root cause analysis of the failure to meet tolerance limits could indicate

• a deficient process;
• need for retraining; and / or
• changes needed to the threshold limits.

Table 3 Example of a TMF Quality Threshold and Action Plan

7.3. Trial-Wide Completeness QC

Trial-wide completeness and functional review is often a more daunting task because TMF record sampling may not tell the full story of the study. If a sampling methodology is used in this area, consider defining the lot / batch as a larger grouping (e.g., site, country, functional area, or key milestone/event). This helps ensure that all expected records can be considered as part of the lot / batch. Tolerance limits in this type of review are often measured by the number of issues or missing records identified as part of the review. Caution is recommended when determining the tolerance limit because calculating completeness or a missing percentage requires a significant understanding of the context of the lot / batch being reviewed.

TMFs for active studies and active sites are always a work in progress that may be reliant upon monitoring visits for collecting missing content. Milestones and events that occur throughout the study may also impact these calculations as an influx of expected content will not be immediately resolved e.g.,

If a protocol amendment occurred right before a review, it may take some time for each expected record related to that protocol amendment to go through regulatory review, ethics approval and complete TMF processing. Timelines should be established to ensure appropriate tracking of these items and should not be included in the scope of a review until those timelines have been hit.

If tolerance limits are not met for trial-wide quality, completeness and inspection readiness, escalations may involve increasing the scope of review. Dependent on how a lot / batch is defined, this may also be an opportunity to:

• identify team members who are struggling to file their records within the expected window and need retraining or resourcing support; or
• re-design the process.

8. Conclusion

Regulatory Authorities actively encourage the adoption of risk-based approaches to TMF management. This paper:

• highlights the benefits of risk-based approaches, primarily the ability to ensure the ready availability of “essential” TMF records, compliance with regulatory requirements, and enhancements in TMF management and quality;
• discusses critical TMF records and processes that should be considered, as well as the risks and mitigations associated with TMF management;
• advocates for a flexible, agile, and resource-efficient approaches to TMF management that accommodate the varying capacities of organizations, while also leveraging technology to improve quality and integrity in recordation processes; and
• emphasizes the necessity of evaluating trends from findings to prevent larger issues and to verify the effectiveness of preventative actions and mitigation plans.

Organizations are encouraged to focus their review for completeness on records and processes with a clear understanding of the risks presented to

• patient rights, well-being, safety, and dignity;
• record quality and integrity;
• TMF completeness; and
• compliance to GCP.

This will best ensure that resources are effectively and efficiently assigned to the high-risk areas. Where required, appropriate escalations (e.g., retraining, policy and SOP review, deviations, corrective and preventive actions etc.) should be applied to remediate risk in an appropriate timely manner.

9. References

9.1. Regulatory Authorities have approved taking risk-based approaches for many years and the use of a risk-based approach in the management and oversight of the TMF is well-founded within regulations and guidelines. A brief Summary of Relevant TMF Regulations is presented in Table 4.

Table 4 Regulations Pertaining to TMF Risk Management

9.2. Publications on Risk Management of the TMF

Table 5 Publications on TMF Risk Management

10. Authors

Table 6 Authors

[1] ICH E8 section 7 Critical to Quality Factors

[2] https://tmfrefmodel.com/wp-content/uploads/TMF-RM-Deliverable-eMail-Communications-Guidance-v1-2020-07-31.pdf

[3] ISO 2859-1: ISO Sampling procedures for inspection by attributes – Part 1: Sampling schemes indexed by acceptance quality limit (AQL) for lot-by-lot inspection.

[4] ASQ/ANSI Z1.4 & Z1.9: Sampling Procedures and Tables for Inspection by Attributes.