OpenCGA Storage Overview
OpenCGA Storage is the component in charge of ingest and provide a query language over biological BigData. The use cases of this component is provide a source of data for analysis like [....] or visualization in compatible viewers like GenomeMaps. This storage-engine handles the most common file formats for NGS.
There are an increasing number of biological formats supported by OpenCGA related with the most common steps on a common NGS pipeline. Within this formats, we focus on Genomic Variants due to the complexity and analysis capabilities.
- Study oriented
- Cohort definition
- Variant annotation
More information at Variant Storage Engine
- Coverage calculation
- Alignment stats
More information at Alignment Storage Engine
- Sequence (fastA)
- Feature formats (GFF, BED, BigWIG...)
Despite the internal implementation is not using directly the GA4GH models (given that the models are still uneatable and some mandatory features for OpenCGA are missing), OpenCGA-Storage is compatible with GA4GH implementing the APIs and being able to produce or consume the ga4gh models.
Based on a common definition of the Storage Engines, depending on the requirements and the resources of the study, there are multiple implementations using different technologies. This document specifies the core functionality that all the implementations must share. Technical details or customization parameters are explained in the plugin specific section.
Index pipeline is the process of ingesting data into an OpenCGA-Storage backend. To simplify the data management, all the index pipelines have been designed following the same schema.
Read more information about index pipelines at Index Pipelines.
All the configuration needed to work with is centralized in the storage-configuration.yml file, usually located in the configuration folder.
This file contains the configuration for all the storage engines, the database connections and other cellbase and server information.
In the Storage Configuration you can find an extended explanation of the file structure and all the parameters.
There is a direct command line to interact with OpenCGA storage. This command line is not intended to be used in a full featured installation because is not connected to OpenCGA Catalog.
Use only for development or testing purposes.
See more information at Storage Command Line