http://www.defenselink.mil/cio-nii/docs/Net-Centric-Data-Strategy-2003-05-092.pdfNet-Centric Data Strategy, Department of DefenseDoDNCDSAll data is advertised and available for users and applications when and where they need it. In this environment, users and applications search for and “pull” data as needed. Alternatively, users receive alerts when data to which they have subscribed is updated or changed (i.e., publish subscribe). Authorized users and applications have immediate access to data posted to the network without processing, exploitation, and dissemination delays. Users and applications “tag” data assets with metadata, or data about data, to enable discovery of data. Users and applications post all data assets to “shared” space for use by the Enterprise.This document describes the Net-Centric Data Strategy for the Department of Defense (DoD), including DoD intelligence agencies and functions. It describes a vision for a net-centric environment and the data goals for achieving that vision. It defines approaches and actions that DoD personnel will have to take as users—whether in a role as consumers and producers of data or as system and application developers. This Strategy will be followed by a subsequent directive and guidance on implementation details.3.1MAKE DATA VISIBLEAdvertise and make all data assets "visible” by providing metadata, which describes the asset.3.1.1Data SharingPost Data to Shared SpacesUsers and applications will migrate from maintaining private data (e.g., data kept within systemspecific storage) to making data available in community- and Enterprise-shared spaces (e.g., servers and services available on the Internet). These shared spaces will act as repositories where users and applications can submit, or post, data assets to the enterprise. The shared spaces will provide storage and serving mechanisms. Enterprise-shared spaces will be maintained, secured, and staged as necessary to support the Department’s missions. Data that is posted to shared spaces will be advertised via the associated metadata and will be discoverable with enterprise search tools.3.1.2Discovery MetadataAssociate Discovery Metadata With Data AssetsTo facilitate discovery of data assets, users and applications will provide discovery metadata, in accordance with the DoD Discovery Metadata Standard (DDMS), for all data posted to shared spaces. The DDMS will provide a common set of structured attributes that support discovery of data assets using search tools. COIs and asset producers determine the desired level of discovery for a data asset, e.g., discovery of a database or a record within a database, discovery of a document or a paragraph within a document. The initial focus of the DDMS is to aid in the discovery of data assets as a whole; hence, the discovery metadata in the DDMS will not always be required for individual records or elements. For exa mple, the discovery metadata will always indicate the existence of a database containing certain kinds of information but may or may not identify the contents of specific database elements. The DDMS does not preclude the use of other metadata processes or standards. For example, record- level database tagging and in-line document tagging are common practice today to support various Department objectives. These tagging initiatives will only have to enhance their existing processes to include the DDMS for Enterprise discovery. The DDMS will be adopted and implemented across the DoD components for new and existing data assets. Figure 7 illustrates the logical layers and elements of the DDMS. This standard will be registered in the DoD Metadata Registry. Enterprise visibility of a data asset is promoted when its discovery metadata complies with the DDMS. The core layer of the DDMS represents those attributes of a data asset that can be commonly described across the Department, regardless of the type of data asset or community to which it is applied. It is important to note that not all elements in the core layer are mandatory. The set of mandatory elements will be small yet robust enough to generate high value-added metadata to support Enterprise discovery. The extensible layer provides a mechanism for COIs to extend the core layer of the DDMS to support mission-specific metadata requirements.3.1.3Metadata CatalogsCreate and Maintain CatalogsMetadata catalogs will advertise the existence of shared data and will contain information about all data assets contained in the associated shared space (including databases, system output files, web pages, documents, and access services). At a minimum, the mandatory discovery metadata elements in the DDMS must be represented within metadata catalogs for any data asset posted to a shared space. COIs will establish and maintain catalogs. Each catalog may be organized according to the community-defined ontology. An Enterprise catalog will be established that links to community catalogs, effectively creating a “catalog of catalogs.” The Enterprise catalog will also contain metadata for data posted by users and applications without a specific COI affiliation. Catalogs will be searchable by applications or through user-friendly, web-based interfaces. The web-based interfaces will have a consistent look and feel and will support posting of metadata to the catalog and data to the shared space. The catalogs will also be searchable, either manually or automatically via agents, through application programming interfaces. All metadata catalogs will adhere to Enterprise discovery interface standards to allow searches within a catalog or across catalogs.3.1.4Metadata RegistryRegister Metadata Related to Structure and DefinitionThe DoD Metadata Registry will contain all metadata related to data structures, models, dictionaries, and schemas. The purpose of the Registry is to give developers and architects visibility into methods to compose and encode data and to share usage across the Department. Registration of such metadata is critical to achieve the data goals of interoperability and understanding by promoting semantic and structural understanding.3.1.5InventoryInventory Data AssetsDuring transition to the new DoD Net-Centric Data Strategy, COIs may identify and prioritize key data assets and services within their domain. These data assets or services may already exist or they may be in development. In either case, COIs will identify the data assets and services that must be made compliant with the data approaches. This prioritized list of data assets and services will provide a focus for near-term COI initiatives to create metadata to advertise the data and ensure that the data is available in shared space.Users and applications can discover the existence of data assets through catalogs, registries, and other search services. All data assets (intelligence, nonintelligence, raw, and processed) are advertised or “made visible” by providing metadata, which describes the asset.3.2Data AccessibilityMAKE DATA ACCESSIBLE3.2.1Data Access and SharingCreate Shared Spaces and Data Access ServicesShared spaces—virtual and actual—will be created to provide a “store and serve” mechanism for data assets. In addition, data access services will be created to help facilitate access to database stores, business logic processes, and system data. Data access services are any mechanisms that help expose data that is not otherwise available to users and applications. For example, a data access service may be a registered, accessible software interface that allows users and applications to extract information from an inventory database.3.2.2Security-Related MetadataAssociate Security-Related MetadataSecurity-related metadata will be provided for each data asset as defined by the Security Descriptors element set within the core layer of the DDMS (see Figure 7). Systems will be able to control access to assets based on classification metadata. GES will allow data that was typically inaccessible as a result of implementation barriers (e.g., distinct networks based on data classification and prearranged accounts for access) to be available to users and applications that have appropriate access needs. These services will permit access using security metadata, technologies such as public key infrastructure (PKI), and role- and permission-based access processes if adopted.Users and applications post data to a “shared space.” Posting data implies that (1) descriptive information about the asset (metadata) has been provided to a catalog that is visible to the Enterprise and (2) the data is stored such that users and applications in the Enterprise can access it. Data assets are made available to any user or application except when limited by policy, regulation, or security.3.3Data ManagementINSTITUTIONALIZE DATA MANAGEMENT3.3.1Leadership and GovernanceGovern Data Processes With Sustained LeadershipBest-practice studies have shown that cultural change is most effectively achieved when there is sustained leadership commitment and institutionalization of new processes. The DoD CIO will establish a DoD-wide net-centric governance process to promote and sustain successful data management practices across COIs by reviewing and sharing metrics, best practices, and incentive structures. This DoD-wide net-centric governance process will also provide oversight of net-centric infrastructure development efforts.3.3.2Processes and PracticesIncorporate Data Approaches Into Department Processes and PracticesDoD Components will recognize and fund implementation of data approaches (e.g., providing metadata, defining and registering metadata structures, and posting data). Planning, programming, and budgeting system processes and policies, as well as acquisition processes and policies, will be updated to reflect these approaches.3.3.3Advocacy, Training, and EducationAdvocate, Train, and Educate in Data PracticesBest practices show that new operating practices are assimilated more quickly when consistently promoted. The DoD CIO will continue to conduct the Data Strategy awareness campaign across the Department, promoting and educating all users about their responsibilities and the benefits of participation. The necessary tools to implement these approaches (e.g., DoD Metadata Registry, catalogs, and shared spaces) will be identified and implemented. As these tools become available, training and manuals will be provided.3.3.4Metrics and IncentivesAdopt Metrics and IncentivesThe DoD-wide net-centric governance process will identify incentives and metrics. Users and system developers will be guided by incentives to encourage and foster participation. Incentives are intended to prompt users and developers to contribute to increased data visibility across the Enterprise. Incentives may include rapid or continued funding for initiatives that implement these data approaches (e.g., providing metadata in accordance with the DDMS). Incentives may also include Enterprise-visible “top products” lists that cite specific sources that have been especially valuable to users. Metrics will be collected to track implementation and application of the approaches. Metrics will be helpful in evaluating usage to ensure participation across the Department. Metrics also serve as a means to evaluate the effectiveness of the overall Data Strategy. Measurement techniques will be developed to ensure that metrics are captured in a useful and consistent manner. Moreover, multiple publication methods such as metrics web sites and Enterprise- and COI-specific metrics reports will be made available Department-wide to promote awareness of data management successes and areas requiring improvement.Data approaches are incorporated into Department processes and practices. The benefits of Enterprise and community data are recognized throughout the Department.3.4Understandable DataENABLE DATA TO BE UNDERSTANDABLE3.4.1COI-Specific OntologiesDefine COI-Specific OntologiesCOIs will be encouraged through metrics and incentives to develop an ontology that best reflects the community understanding of their shared data. Ontologies include data categorization schemes, thesauruses, vocabularies, key word lists, and taxonomies. Ontologies promote semantic and syntactic understanding of data. For example, taxonomies enhance discovery by providing a hierarchical means of searching for data while providing users and applications with additional insights about data assets by indicating their placement among other data assets. COIdeveloped vocabularies will define terms used in describing data assets, and the thesauruses will identify related terms to assist translation services. Any community that establishes an ontology will publish it to the DoD Metadata Registry to increase understanding across the Enterprise and promote possible reuse.3.4.2Content-Related MetadataAssociate Content-Related MetadataThe summary content descriptors element set of the DDMS (see Figure 7) is specifically aimed at providing “content-related” details about data assets. Content metadata provides topics, keywords, context, and other content-related information. Content metadata gives users and applications insight into the meaning and context of the data. Content metadata provides a basis for search engines to perform searches for data assets that address specific topics.3.4.3Format-Related MetadataAssociate Format-Related MetadataThe format descriptors element set of the DDMS (see Figure 7) is used to describe details pertaining to the format of the associated data asset. The format descriptors are useful when trying to understand the physical manifestation of an asset. For example, the format descriptors will provide information regarding the type of digital file (e.g., a Joint Photographic Experts Group [JPEG] image, or an Audio Interchange File Format [AIFF]). In addition, the format descriptors contain optional information that describes the extent of the asset, such as file size, bit rate, and dimensions. Format-related metadata allows users and applications to narrow down information searches and to select products that meet their particular operating constraints (e.g., a user who is able to view only Graphic Interchange Format [GIF] images would not want to pull a JPEG image).3.4.4COI-Specific MetadataDefine COI-Specific Content-Related MetadataTo improve understanding, an extension of the discovery metadata standard is reserved for domain-specific, or COI-specific, metadata. This is represented as the extensible layer of the DDMS in Figure 7. With this extension layer, COIs will be able to provide context relevant to their particular domain area and still be able to participate in Enterprise-wide search and discovery. COIs will be required to register their COI-specific content metadata requirements in the DoD Metadata Registry. These COI-specific metadata requirements may then be integrated into appropriate Enterprise and community services such as search and mediation.Users and applications can comprehend the data, both structurally and semantically, and readily determine how the data may be used for their specific needs.3.5Trusted DataENABLE DATA TO BE TRUSTED3.5.1Pedigree and Security MetadataAssociate Data Pedigree and Security MetadataThe Resource Descriptors elements of the DDMS (see Figure 7) allow identification of the author, publisher, and sources contributing to the data, allowing users and applications to assess the derivation of the data (i.e., data pedigree). This metadata allows users and applications to select data from known sources. Reliable and quality sources will become more widely used, enhancing overall data quality throughout the Enterprise as more data sources become visible. The Security Descriptors elements of the DDMS (see Figure 7) allow security and privacy markings consistent with established standards where applicable. For information assurance (IA) and security, GES will provide auditing tools that can track access, by individual user, of each data asset. GES may also provide access control to data assets based on security markings in the metadata.3.5.2Authoritative SourcesIdentify Authoritative SourcesCOIs may identify authoritative sources for key data assets in their domain. The community will publicize their identified authoritative sources to the Enterprise, thus allowing users and applications to evaluate and understand the community- implied authority of data sources. COIs may have to resolve potentially conflicting sources and, where appropriate, coordinate with the DoD-wide governance body to identify authoritative source(s).Users and applications can determine and assess the authority of the source because the pedigree, security level, and access control level of each data asset is known and available.3.6Data InteroperabilitySUPPORT DATA INTEROPERABILITY3.6.1Metadata RegistrationRegister MetadataRegistration of metadata (e.g., eXtensible Markup Language [XML] components, database segments, and data dictionary elements) is an important activity to support interoperability in a net-centric environment. COIs will register their metadata components in the DoD Metadata Registry. Registering metadata components to the DoD Metadata Registry supports many-to-many interoperability by providing system architects and developers with insight into existing data schemas that they can employ and extend.3.6.2Format-Related MetadataAssociate Format-Related MetadataUsers and systems can employ the elements of format descriptors to specify the extent (e.g., size and dimension), type, and physical manifestation of assets. The format descriptors element set of the DDMS (see Figure 7) supports interoperability by allowing systems and users to determine the physical manifestation of data assets, which in turn helps to identify which tools and capabilities are required to use the asset.3.6.3System InterfacesIdentify Key Interfaces Between SystemsInterface engineering in an environment of many-to-many exchanges requires an unrealistic degree of interface control and an enormous commitment of resources. To facilitate interoperability within a community, COIs can determine the appropriate focus and level of data standardization within their community. This decentralized, distributed approach to interoperability ensures that key interfaces and data structures are controlled when tightly engineered interfaces are required. COIs will register metadata that results from interoperability activities in the DoD Metadata Registry. Mediation services will use the registered metadata to facilitate system interoperability between unanticipated interfaces as needed.3.6.4Interface StandardsComply With Net-Centric Interface StandardsDevelopers will be responsible for adhering to published net-centric interoperability standards, including data standards where applicable. Successful discovery and interoperability of data assets depend on compliance with metadata standards (i.e., DDMS) and data exposure standards (e.g., GES discovery interface standards). For example, data assets that are maintained by the Department’s Records Management functions must provide a means for the Enterprise discovery capability to query the inventory of their stored records. In doing so, these records management applications should employ the DDMS to respond to Enterprise discovery queries.Many-to-many exchanges of data occur between systems through interfaces that are sometimes predefined or sometimes unanticipated. Metadata is available to allow mediation or translation of data between interfaces, as needed.3.7User NeedsBE RESPONSIVE TO USER NEEDS3.7.1User InvolvementInvolve Users in COIsAs described in Section 2.1, institutional COIs in particular are focused on ensuring the implementation of these approaches. Institutional COIs will take the lead in creating catalogs, defining ontologies, and developing COI-specific metadata. To adequately reflect user needs, these COIs must engage a range of known users and developers in these activities.3.7.2User FeedbackEstablish a Process To Enable User FeedbackCOIs, under the DoD CIO Enterprise-wide net-centric governance process, will establish processes to evaluate and refine the user experience. Users may provide ratings for data sources, catalogs, or services, and content metadata usability. Ratings may include factors such as ease of use, applicability, or quality. These ratings will be published Enterprise-wide and used to promote participation in posting, identifying, and sharing data assets. Overall, this Departmentwide feedback and ratings process, coupled with improved data asset visibility, will increase the integrity and quality of data. In addition, the feedback process allows COIs and data producers to identify previously unanticipated users and applications. To improve Enterprise data visibility, the process may allow users to identify needed data by publishing a “data want ad” to a community or Enterprise collaboration space. In some cases, the data may be available but not currently visible or accessible. Hence, the source may choose to make it visible or accessible to the user or application. In other situations, providing the data may not be available or cost-effective, and the user’s “want” will remain unfilled.Perspectives of users, whether data consumers or data producers, are incorporated into data approaches via continual feedback to ensure satisfaction.