ckanext-datapreview
This CKAN extension supplies data from local storage or via a remote download, parses CSV/XLS and provides it at a URL that can be called by Recline or other CKAN data previewer.
e.g. for a spreadsheet it returns a JSON dict such as:
{
"fields": ['Name', 'Age'],
"data": [['Bob', 42], ['Jill', 54]],
"extra_text": "This preview shows only the first 10 rows",
"max_results": 10,
"length": 435,
"url": "http://data.com/file.csv",
}
(see helpers.proxy_query() )
This extension is a modified, but local implementation of the OKFN dataproxy that runs as a CKAN extension rather than on Google AppEngine. This has been written to improve the performance on data.gov.uk and increase the maximum file size processed.
The interface to the extension::
/data/preview/<resource_id>?max-results=N&encoding=utf-8
is not exactly the same - dataproxy requires the URL instead of the resource id - the data returned is identical. Rather than always fetching the data from the remote site the new controller at the above route will first attempt to find the data in the ckanext-archiver’s local archive.
Installation
The most straightforward method of installation is::
git clone git://github.com/datagovuk/ckanext-datapreview.git
cd ckanext-datapreview
python setup.py develop
Or alternatively install directly using pip:
pip install -e git+https://github.com/datagovuk/ckanext-datapreview.git#egg=ckanext-datapreview
Once complete the datapreview should be added to your ckan.plugins property in the appropriate .ini file.
Config
In your CKAN config file, configure the following options:
limit
The ‘limit’ is the maximum size of a file downloaded or loaded into memory. If the data is not stored locally, then you don’t want