Difference between revisions of "Data Portal Architecture"
(→Data flow) |
(→Data flow) |
||
| Line 10: | Line 10: | ||
[[File:GGBN Portal Architecture.jpg|center|700px]] | [[File:GGBN Portal Architecture.jpg|center|700px]] | ||
| − | '''General data architectur of the GGBN Dta Portal architecture.''' Specimen and DNA sample databases (on top left) are operated by the Network partners. Their data content is structured and provided by using BioCASe or IPT and the GGBN Data Standard extensions. The | + | '''General data architectur of the GGBN Dta Portal architecture.''' Specimen and DNA sample databases (on top left) are operated by the Network partners. Their data content is structured and provided by using BioCASe or IPT and the GGBN Data Standard extensions. The [http://wiki.bgbm.org/bhit/index.php/Main_Page Berlin Harvesting and Indexing Toolkit] is used to harvest GGBN data and store them in a MySQL database. In addition we use a SOLR instance to speed the query. After harvesting the data are cleaned and enriched by e.g. a match against certain datasets of the GBIF checklist bank. Finally the data are aggregated from multiple sources in the portal to be displayed. |
=DNA and Tissue Bank databases= | =DNA and Tissue Bank databases= | ||
Revision as of 15:37, 16 December 2015
Overview
The data architecture of the GGBN Data Portal is based on the GBIF infrastructure. The basic principle of GBIF as well as of the GGBN is to record all data sets only once. Stored at only one place they can be used as a linked reference for different applications. The GGBN Data Portal bridges the gap between sequence portals and GBIF (see Figure). More information about the GGBN Data Portal can also be found in Droege et al. 2014 Nucl Acids Res.
Data flow
Since many institutions joined GBIF applying different database structures each, the installation of wrappers has become a standard to combine different sources and integrate data easily into networks. There are two main wrapper softwares available BioCASE, and IPT. GGBN has developed the GGBN Data Standard to share DNA and tissue data via GGBN. This standard is meant to be used together with ABCD or DarwinCore.
General data architectur of the GGBN Dta Portal architecture. Specimen and DNA sample databases (on top left) are operated by the Network partners. Their data content is structured and provided by using BioCASe or IPT and the GGBN Data Standard extensions. The Berlin Harvesting and Indexing Toolkit is used to harvest GGBN data and store them in a MySQL database. In addition we use a SOLR instance to speed the query. After harvesting the data are cleaned and enriched by e.g. a match against certain datasets of the GBIF checklist bank. Finally the data are aggregated from multiple sources in the portal to be displayed.
DNA and Tissue Bank databases
You can use any database system to manage your DNA or tissue bank. IPT and BioCASe can handle most of them. Please check out []Mandatory_and_recommended_fields_for_sharing_data_with_GGBN | which parameters]] are required to share data via GGBN.
The DNA Module has been developed as an open source solution for administer a DNA and tissue bank. A new version is currently planned. Some of our partners are already using it, but you can use any suitable software.
