These CKAN Versions are exactely matched:

• 2.10.0
• 2.10.1
• 2.10.2
• 2.10.3
• 2.10.4
• 2.10.5
• 2.10.6
• 2.10.7
• 2.10.8
• 2.10.9
• 2.11.0
• 2.11.1
• 2.11.2
• 2.11.3
• 2.11.4
• 2.12.0
• 2.9.0
• 2.9.0a
• 2.9.1
• 2.9.10
• 2.9.11
• 2.9.13
• 2.9.2
• 2.9.3
• 2.9.4
• 2.9.5
• 2.9.6
• 2.9.7
• 2.9.8
• 2.9.9

ckanext-datastore-profiler

This is a CKAN extension for profiling datastore resources, developed for Toronto Open Data.

What is the Profiler?

The profiler will:

1. Summarize each attribute in datastore resources
2. Assign classifications to each attribute
3. Present those on the Portal’s API (and maybe frontend)

Descriptive Statistics for Integers and Floats

• mean
• min
• max
• median
• count of distinct values

Descriptive Statistics for Dates and Datetimes

• earliest date
• latest date
• earliest time
• latest time
• count of distinct values for date
• count of distinct values for date month
• count of distinct values for date year
• count of distinct values for time

Descriptive Statistics for Strings

• count of distinct values
• count of distinct “words” (string separated by spaces, ignoring things like “this”, “that”, “a” etc.)
• min and max string length
• min and max word count per string
• count of distinct “masks” (where every letter is turned into an “L” and every digit turned into a “D”)

Requirements

Compatibility with core CKAN versions:

Installation

To install ckanext-datastore-profiler:

1. Activate your CKAN virtual environment:

. /usr/lib/ckan/default/bin/activate

2. Clone the source and install:

git clone https://github.com/open-data-toronto/ckanext-datastore-profiler.git
cd ckanext-datastore-profiler
pip install -e .
pip install -r requirements.txt

3. Add datastore-profiler to the ckan.plugins setting in your CKAN config file.

4. Restart CKAN.

Tests

To run the tests:

pytest --ckan-ini=test.ini

Contact

• Reach out to opendata@toronto.ca
• Reach out on the Civic Tech Toronto Slack

License

AGPL