This presentation describes the metadata design and data transformation engine (DTE) software approach, that is being used in the CDISC 360 project. The objectives of automation and transparency of data flow are attained with a robust metadata design and accompanying software.
The metadata design is unchanged for all data states and standards, it can be used to store any data standard, any study or integrated data specification and any data flow. The software is also unchanged across data states and studies and "knows" how to do the data transformations and access derivations to accomplish data flows described in the metadata. The scope of application of this metadata-driven DTE includes a wide variety of relational data flows beyond current CDISC, such a DTS (data transfer specification) and DRP (data review plan). The user will describe what data flow to implement and the DTE will implement the code to perform the data flow. Much less study-level programming will be necessary.
Implementing such a design, of metadata and software, delivers automation and transparency today. Furthermore, it establishes a foundation for future development, in such technologies as AI and machine learning, to manage metadata-resident standards and project data specifications. This revolutionary change in the way we specify and perform data and reporting, with an evolutionary implementation, is described. An evolutionary step from inaccessible document-based data standards, to an industry-level standard of metadata design for storing transparent standards and data specifications is taking its next step now. It will become clear that this more robust metadata content requires the publication format to go beyond the simple define file, which still has the feel of its origins in the old pdf document format and follows the xml schema design rather than optimizing with human-friendly presentation formats. Industry-standard software and metadata must replace the study-by-study coding process we use today. Study programming is opaque, time consuming and of uneven quality. The days of the opaque CTR method of coding (copy-tweak-run) are being replaced by a transparent DTE. This DTE is intelligent enough to implement any data flow described by metadata, separating the description of data flow, to machine and human, from the mechanical code syntax to implement the data flow.
The revolution to a lesser amount of project-specific programming and more metadata management requires the kind of evolutionary steps being demonstrated in the CDISC 360 project. Our data standards must evolve to include these industry-level standards of metadata design and shared DTE software, in order to deliver the transparency, speed, quality and data privacy in data analyses. CDISC can evolve its scope of industry-level standards to metadata and software in order to lead to the future.