Building off the 1st and 2nd generation, we have reimagined what a Data Commons can be by utilizing cloud native thinking. Each microservice now lives in its own container providing isolation from the environment around it. Container orchestration is conducted dynamically to scale to the commons workload. Clouds are approached agnostically so that Gen3 can operate on any cloud, public or private.
The Commons API service is responsible for handling user submissions, and GraphQL search queries. The submissions are quality controlled against the data dictionary to ensure all required fields are present and have appropriate data values. The GraphQL service allows Commons operators and users to precisely query only the information they are most interested in from the metadata collections.
The Fence service manages all user and service authentication in the data commons. Fence controls access to the metadata, submission, indexing, and data itself. Fence implements the data commons OAuth flow, and generating and tracking JWT auth tokens by which a client may authenticate with the other commons services to access data under the projects to which the client has access.
The Indexd service provides permanent digital IDs for data objects. These IDs can be used to retrieve the data, or query the metadata associated with the object. The Indexd service tracks the locations and hash of every asset (file) in the data commons object store. It exports RESTful APIs for registering a new asset, and retrieving data for an existing asset.
The Portal service is an interactive website that allows users to explore, submit, and download data. The Portal service utilizes the Commons APIs offered by the data commons just as any other app built could.