Extension Generic API Harvester


Extension Basics

Title
Generic API Harvester
Name
ckanext-genericapiharvest
Type
Public extension
Description
A harvester to allow CKAN directories to keep in sync with catalogues that provide data via an API service. Part of the Open Data Monitor project.
CKAN versions
Download-Url (zip)
Last commit
10 years ago (2015-11-26 16:05:17)
Url to repo
Category
Data Management & Quality


Background Infos

Description (long)
Show details

ckanext-genericapiharvest

A harvester to allow CKAN directories to keep in sync with catalogues that provide the data via an API service.

In order to use this tool, you need to have the ODM CKAN harvester extension (https://github.com/opendatamonitor/ckanext-harvestodm) installed and loaded for your CKAN instance. Tested with CKAN v2.2.

General

The plugin uses MongoDB as metadata repository and was developed as part of the ODM project (www.opendatamonitor.eu).

Building

To build and use this plugin, simply:

git clone https://github.com/opendatamonitor/ckanext-genericapiharvest.git
cd ckanext-genericapiharvest
pip install -r requirements.txt
python setup.py develop

Then you will need to update your CKAN configuration to include the new harvester. This will mean adding the ckanext-genericapiharvest plugin as a plugin, e.g.

ckan.plugins = genericapi_harvester genericapiharvest

Also you need to add the odm_extension settings to the development.ini file in your ckan folder.

[ckan:odm_extensions]
mongoclient=localhost
mongoport=27017
log_path=/var/local/ckan/default/pyenv/src/

Using

After setting this up, you should be able to go to: http://localhost:5000/harvest

Select Register a new Catalogue

Select the GENERICAPI radiobutton

In case that you don’t have the ckanext-htmlharvest extension installed, then go to:

http://localhost:5000/genericapiharvest/generic

Licence

This work implements the ckanext-harvest template and thus licensed under the GNU Affero General Public License (AGPL) v3.0.

Version
Version release date
(not set)
Contact name
Open Data Monitor
Contakt email
(not set)
Contact Url


Installation Guide

Configuration hints

Requires MongoDB as metadata repository. Requires ckanext-harvestodm extension to be installed. Part of the Open Data Monitor (ODM) project.

Plugins to configure (ckan.ini)
genericapi_harvester genericapiharvest
CKAN Settings (ckan.ini)
# [ckan:odm_extensions]
# mongoclient = localhost
# mongoport = 27017
# log_path = /var/local/ckan/default/pyenv/src/
DB migration to be executed
(not set)
<< back to Extensions