Skip to end of metadata
Go to start of metadata

The Data Management system's infrastructure consists of the hardware that stores data, and the software layer that manages the data and integrates various administrative (such as the User account database) and experiment (such as an areaDetector acquisition computer) systems.

The overall system architecture is shown in the following diagram. Individual components are described in the table below.

RolesA user may be assigned one or more of four roles in the Data Management system. These roles determine which data a user may access, and the extent to which a user may manipulate certain data in the system. More details about Roles may be found on the Ownership Management page.
On-site StorageThe APS maintains 250 TB of short-term storage on-site. Details of the APS on-site short-term data storage hardware, xstor, can be found on the APS On-site Storage page.
Petrel Data PilotPetrel is a pilot service for data management that allows researchers to store large-scale datasets and easily share that data with collaborators. Researchers from the Argonne Leadership Computing Facility (ALCF) and Globus are developing the system collaboratively. For more information visit: http://petrel.alcf.anl.gov/
Account Tools

The APS has two types of authentication systems. Access to the computing resources (cluster, workstations, etc.) is controller by an LDAP server managed by APS-IT. The APS-IS group maintains access to the Oracle server that lists all the badged Users. Most of the web systems like scheduling and proposal rely on information stored in the Oracle system for authenticating users. Synchronization tools create and maintain entries in LDAP for each badged APS User. These LDAP entries are used for authentication to the Data Management system, and by the on-site storage system for user and group ids (uid/gid). Details of the Oracle User database, and LDAP and Data Management database system synchronization tools can be found on the Data Management User Accounts page.

Data Management Database and Web Portal

All information related to data sets, data policies, and assigned permissions are kept in the Data Management system's database. A web portal allows administrators and users a convenient way to view and modify settings. More information about the web portal can be found at:

Data Storage & Acquisition ServicesThe Data Storage & Acquisition Services manage monitoring data sources, transferring data files, maintaining the integrity and security of the data, setting permissions on data, and applying policies (such as auto deletion) to data. (More details will come in the next month.)
Development & Deployment

Information about establishing a development environment and deploying the system can be found at:

The most recent software and documentation for the Data Management project can be retrieved from the following links:

Information about existing software deployments can be found at the following pages:

 

 

 

 

 

 

 

  • No labels