CKAN Harvester for openBis (OAI-PMH)
This harvester uses the OAI-PMH interface of openBis to harvest metadata. openBis is a platform for managing research data developed by ETH Zurich (CISD).
Features
- Harvests metadata from openBis instances via OAI-PMH protocol
- Supports authentication with username/password
- Allows harvesting specific sets
- License mapping support
- Resource creation based on source field
- Tag handling for openBis specifics
Dependencies
- ckanext-oaipmh
- ckanext-harvest
Installation
Use pip to install this plugin:
source /home/www-data/pyenv/bin/activate
pip install -e git+https://github.com/openresearchdata/ckanext-openbis.git#egg=ckanext-openbis --src /vagrant
cd /vagrant/ckanext-openbis
pip install -r requirements.txt
python setup.py develop
Make sure the ckanext-oaipmh and ckanext-harvest extension are installed as well.
Setup the Harvester
- Add
openbis_harvester to ckan.plugins in development.ini (or production.ini)
- Restart your webserver
- With the web browser go to
<your ckan url>/harvest/new
- As URL fill in the base URL of an OAI-PMH conforming openBis instance
- Select Source type
openBis
- If your OAI-PMH needs credentials, add the following to the “Configuration” section:
{"username": "foo", "password": "bar"}
- If you only want to harvest a specific set, add the following to the “Configuration” section:
{"set": "baz"}
- Save
- On the harvest admin click Reharvest
Run the Harvester
On the command line:
- Activate the python environment
cd to the ckan directory
- Start the consumers (NOTE: only run 1 gather and 1 fetch consumer per server):
paster --plugin=ckanext-openbis harvester gather_consumer &
paster --plugin=ckanext-openbis harvester fetch_consumer &
- Run the job:
paster --plugin=ckanext-openbis harvester run
The harvester should now start and import the OAI-PMH metadata.