diff --git a/docs/api/endpoints/README.md b/docs/api/endpoints/README.md new file mode 100644 index 00000000..e8aa5c97 --- /dev/null +++ b/docs/api/endpoints/README.md @@ -0,0 +1,2 @@ +# Endpoints + diff --git a/docs/api/endpoints/admin.md b/docs/api/endpoints/admin.md new file mode 100644 index 00000000..9204e56e --- /dev/null +++ b/docs/api/endpoints/admin.md @@ -0,0 +1,353 @@ +--- +description: Use the API to managing users, keys, and access as PDAP staff +--- + +# Admin + +## Base URL + +``` +https://data-sources.pdap.io +``` + +## Login & API keys + +## Creates a new user. + +`POST` `[base-url]/user` + +Users can sign up for an account through the post function in [resources/User.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/User.py). The user's password is hashed using werkzeug.security’s generate\_pasword\_hash function. The user's email and hashed password is stored in the users table in the Data Sources database. + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------ | ------ | ----------------------------- | +| email\* | String | User's email - must be unique | +| password\* | String | User's password | + +{% tabs %} +{% tab title="201: Created User successfully created" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "Successfully added user" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} + +{% endtab %} + +{% tab title="401: Unauthorized Login failed" %} +```javascript +{ + // Response +} +``` +{% endtab %} +{% endtabs %} + + + +## Updates user password. + +`PUT` `[base-url]/user` + +Users can update their password through the put function in [resources/User.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/User.py). The user's password is hashed using werkzeug.security’s generate\_pasword\_hash function. The user's email and hashed password is stored in the users table in the Data Sources database. + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------ | ------ | ----------------------------- | +| email\* | String | User's email - must be unique | +| password\* | String | User's password | + +{% tabs %} +{% tab title="201: Created User password successfully updated" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "Successfully updated password" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} + +{% endtab %} +{% endtabs %} + +## Logs in the user. + +`POST` `[base-url]/login` + +The login function can be found through the get function in [resources/Login.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/Login.py). If the email and password match a row in the database, "Successfully logged in" will be returned. + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| email\* | String | Matches exactly with the "email" property in user's table | +| password\* | String | Checked against the password\_digest for the user with the matching "email" property using werkzeug.security’s check\_password\_hash function | + +{% tabs %} +{% tab title="200: OK Successful login" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +data string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "Successfully logged in" + "data": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDg3MDE4MjksImlhdCI6MTcwODcwMTUyOSwic3ViIjo2NX0.Fuue4oDXFlS4N_AS41N2dchvMXGEihhpVdrhwxCf8zA" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="400: Bad Request Missing or bad API key" %} + +{% endtab %} + +{% tab title="403: Forbidden " %} +```javascript +{ + // Response +} +``` +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} +```javascript +{ + // Response +} +``` +{% endtab %} +{% endtabs %} + +## Refreshes the user's session token. + +`POST` `[base-url]/refresh-session` + +The logic can be found in the post function in [resources/RefreshSession.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/RefreshSession.py). If the old session token matches a row in the database, "Successfully refreshed session token" will be returned. + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| email\* | String | Matches exactly with the "email" property in user's table | +| password\* | String | Checked against the password\_digest for the user with the matching "email" property using werkzeug.security’s check\_password\_hash function | + +{% tabs %} +{% tab title="200: OK Successful session refresh" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +data string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "Successfully refreshed session token" + "data": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDg3MDE4MjksImlhdCI6MTcwODcwMTUyOSwic3ViIjo2NX0.Fuue4oDXFlS4N_AS41N2dchvMXGEihhpVdrhwxCf8zA" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="403: Forbidden Invalid old session token" %} +```javascript +{ + // Response +} +``` +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} +```javascript +{ + // Response +} +``` +{% endtab %} +{% endtabs %} + +## Sends user a password reset link. + +`POST` `[base-url]/request-reset-password` + +This functionality can be found in the get function in [resources/RequestResetPassword.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/tree/main/resources/RequestResetPassword.py). If the email and password match a row in the database, "Successfully logged in" will be returned. + +#### Request Body + +| Name | Type | Description | +| --------------------------------------- | ------ | --------------------------------------------------------- | +| email\* | String | Matches exactly with the "email" property in user's table | + +{% tabs %} +{% tab title="200: OK Successful reset request" %} + +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} + +{% endtab %} +{% endtabs %} + +## Sends user a password reset link. + +`POST` `[base-url]/request-reset-password` + +This functionality can be found in the get function in [resources/RequestResetPassword.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/tree/main/resources/RequestResetPassword.py). If the email and password match a row in the database, "Successfully logged in" will be returned. + +#### Request Body + +| Name | Type | Description | +| --------------------------------------- | ------ | --------------------------------------------------------- | +| email\* | String | Matches exactly with the "email" property in user's table | + +{% tabs %} +{% tab title="200: OK Successful reset request" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "An email has been sent to your email address with a link to reset your password." + "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDg3MDE4MjksImlhdCI6MTcwODcwMTUyOSwic3ViIjo2NX0.Fuue4oDXFlS4N_AS41N2dchvMXGEihhpVdrhwxCf8zA" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} + +{% endtab %} +{% endtabs %} + +## Reset password token check. + +`POST` `[base-url]/reset-token-validation` + +This functionality can be found in the get function in [resources/ResetTokenValidation.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/RequestResetPassword.py). If the token matches a row in the database, "Token is valid" will be returned. + +#### Path Parameters + +| Name | Type | Description | +| --------------------------------------- | ------ | -------------------- | +| token\* | String | Reset password token | + +{% tabs %} +{% tab title="200: OK Reset password token validated" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "The submitted token is valid" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="400: Bad Request Token is invalid" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "message": "The submitted token is invalid" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} +```javascript +{ + "message": "error" +} +``` +{% endtab %} +{% endtabs %} + +## Returns an API key for a valid user and password. + +`GET` `[base-url]/api_key` + +The key generation function can be found through the get function in [resources/ApiKey](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/ApiKey.py). If the email and password match a row in the database, a new API key is created using uuid.uuid4().hex, updated in for the matching user in the users table, and the API key is sent to the user. + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| email\* | String | Matches exactly with the "email" property in user's table | +| password\* | String | Checked against the password\_digest for the user with the matching "email" property using werkzeug.security’s check\_password\_hash function | + +{% tabs %} +{% tab title="200: OK Successful login" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +api_key string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{ + "api_key": "2bd77a1d7ef24a1dad3365b8a5c6994e" +} +``` +{% endtab %} +{% endtabs %} +{% endtab %} +{% endtabs %} diff --git a/docs/api/endpoints/data-sources-database.md b/docs/api/endpoints/data-sources-database.md new file mode 100644 index 00000000..b4644bac --- /dev/null +++ b/docs/api/endpoints/data-sources-database.md @@ -0,0 +1,807 @@ +--- +description: Use the API to find, use, and manage Data Sources. +--- + +# Data Sources database + +## Base URL + +``` +https://data-sources.pdap.io +``` + +## Search Tokens + +## Generate API token for front end search + +`GET` `[base-url]/search-tokens` + +The search tokens endpoint is located in [resources/SearchTokens.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The search tokens endpoint generates an API token valid for 5 minutes and forwards the search parameters to the Quick Search endpoint. This endpoint is meant for use by the front end only. + +#### Query Parameters + +| Name | Type | Description | +| -------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| arg1\* | String | The first argument that will be forwarded on to the appropriate endpoint. Currently either "search" for quick-search or "id" for data-sources | +| arg2\* | String | The second argument that will be forwarded on to the appropriate endpoint. Currently just used for "location" for quick-search | +| endpoint | String | The endpoint that will be accessed after a search token is generated | + +{% tabs %} +{% tab title="200: OK Successful operation" %} +{% tabs %} +{% tab title="Schema" %} +
count int
+data array[object]
+ agency_name string
+ municipality string
+ state_iso string
+ data_source_name string
+ description string
+ record_type string
+ source_url string
+ record_format string
+ coverage_start string
+ coverage_end string
+ agency_supplied boolean
+
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 1,
+ "data": [
+ {
+ "agency_name": "Allegheny County Police Department - PA",
+ "municipality": "Pittsburgh",
+ "state_iso": "PA",
+ "data_source_name": "Allegheny County Police Review Board Transcripts",
+ "description": null,
+ "record_type": "Policies & Contracts",
+ "source_url": "https://www.alleghenycounty.us/county-council/police-review-board-meetings.aspx",
+ "record_format": "[\"PDF: Machine Created\"]",
+ "coverage_start": "2018-08-29",
+ "coverage_end": "2018-09-26",
+ "agency_supplied": true
+ }
+ ]
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="500: Internal Server Error Error" %}
+```javascript
+{
+ "message": error
+}
+```
+{% endtab %}
+{% endtabs %}
+
+## Search
+
+## Quick Search Data Sources by search term and location
+
+`GET` `[base-url]/quick-search/{search}/{location}`
+
+The quick search endpoint is located in [resources/QuickSearch.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The quick search endpoint executes its search using the agency\_source\_link table in the Data Sources database, which links each data source in the data\_sources table with its associated agency in the agencies table. This endpoint is meant for use by the search tokens endpoint only.
+
+#### Path Parameters
+
+| Name | Type | Description |
+| ------------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| search\* | String | Checks partial matches on any of the following properties on the data\_source table: "name", "description", "record\_type", and "tags". The search term will is case insensitive and will match singular and pluralized versions of the term. |
+| location\* | String | Checks partial matches on any of the following properties on the agencies table: "county\_name", "state\_iso", "municipality", "agency\_type", "jurisdiction\_type", "name" |
+
+#### Headers
+
+| Name | Type | Description |
+| ----------------------------------------------- | ------ | --------------------------------------------------- |
+| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” |
+
+{% tabs %}
+{% tab title="200: OK Successful operation" %}
+{% tabs %}
+{% tab title="Schema" %}
+count int
+data array[object]
+ agency_name string
+ municipality string
+ state_iso string
+ data_source_name string
+ description string
+ record_type string
+ source_url string
+ record_format string
+ coverage_start string
+ coverage_end string
+ agency_supplied boolean
+
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 1,
+ "data": [
+ {
+ "agency_name": "Allegheny County Police Department - PA",
+ "municipality": "Pittsburgh",
+ "state_iso": "PA",
+ "data_source_name": "Allegheny County Police Review Board Transcripts",
+ "description": null,
+ "record_type": "Policies & Contracts",
+ "source_url": "https://www.alleghenycounty.us/county-council/police-review-board-meetings.aspx",
+ "record_format": "[\"PDF: Machine Created\"]",
+ "coverage_start": "2018-08-29",
+ "coverage_end": "2018-09-26",
+ "agency_supplied": true
+ }
+ ]
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+
+{% tab title="500: Internal Server Error Something went wrong" %}
+```
+{
+ "count": 0,
+ "message": error
+}
+```
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+```javascript
+{
+ // Response
+}
+```
+{% endtab %}
+{% endtabs %}
+
+## Data Sources
+
+## Get all Data Sources
+
+`GET` `[base-url]/data-sources`
+
+The data sources endpoint is located in [resources/DataSources.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The data sources endpoint returns all approved rows in the corresponding Data Sources database table by default. An optional JSON object can be passed to get data sources needing identification instead.
+
+#### Headers
+
+| Name | Type | Description |
+| ----------------------------------------------- | ------ | --------------------------------------------------- |
+| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” |
+
+#### Request Body
+
+| Name | Type | Description |
+| ---- | ---- | ------------------------------------------------------------------------ |
+| Data | JSON | In order to get data sources needing identification: {"approved": False} |
+
+{% tabs %}
+{% tab title="200: OK Successful operation" %}
+{% tabs %}
+{% tab title="Schema" %}
+count int
+data array[object]
+ name string
+ submitted_name string
+ description string
+ record_type string
+ source_url string
+ agency_supplied boolean
+ supplying_entity string
+ agency_originated string
+ originating_entity string
+ agency_aggregation string
+ coverage_start string
+ coverage_end string
+ source_last_updated string
+ retention_schedule string
+ detail_level string
+ number_of_records_available string
+ size string
+ access_type array
+ data_portal_type string
+ record_format array
+ update_frequency string
+ update_method string
+ tags string
+ readme_url string
+ scraper_url string
+ data_source_created string
+ airtable_source_last_modified string
+ url_status string
+ rejection_note string
+ last_approval_editor object
+ agency_described_submitted string
+ agency_described_not_in_database string
+ approval_status string
+ record_type_other string
+ data_portal_type_other string
+ records_not_online string
+ data_source_request string
+ url_button string
+ tags_other string
+ access_notes string
+ last_cached string
+
+{% endtab %}
+
+{% tab title="Example" %}
+count: 1
+[{'name': 'Calls for Service for Chicago Police Department - IL', 'submitted_name': None, 'description': None, 'record_type': 'Calls for Service', 'source_url': 'https://informationportal.igchicago.org/911-calls-for-cpd-service/', 'agency_supplied': True, 'supplying_entity': None, 'agency_originated': None, 'originating_entity': None, 'agency_aggregation': None, 'coverage_start': '2019-01-01', 'coverage_end': None, 'source_last_updated': None, 'retention_schedule': None, 'detail_level': 'Summarized totals', 'number_of_records_available': None, 'size': None, 'access_type': ['Web page'], 'data_portal_type': None, 'record_format': None, 'update_frequency': 'Bi-weekly', 'update_method': None, 'tags': None, 'readme_url': None, 'scraper_url': None, 'data_source_created': '2022-09-28', 'airtable_source_last_modified': '2023-11-08', 'url_status': 'ok', 'rejection_note': None, 'last_approval_editor': '{"id": "usrtLIB4Vr3jTH8Ro", "email": "josh.chamberlain@pdap.io", "name": "Josh Chamberlain"}', 'agency_described_submitted': 'Chicago city, Illinois', 'agency_described_not_in_database': None, 'approval_status': 'approved', 'record_type_other': None, 'data_portal_type_other': None, 'records_not_online': None, 'data_source_request': None, 'url_button': None, 'tags_other': None, 'access_notes': None, 'last_cached': '2024-01-30', 'agency_name': 'Chicago Police Department - IL'}]
+
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+```javascript
+{
+ // Response
+}
+```
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+
+{% tab title="500: Internal Server Error Error" %}
+```javascript
+{
+ "count": 0,
+ "message": "There has been an error pulling data!"
+}
+```
+{% endtab %}
+{% endtabs %}
+
+## Data Sources for Map
+
+`GET` `[base-url]/data-sources-map`
+
+#### Headers
+
+The data sources for map endpoint is located in [resources/DataSourcesMap.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/DataSourcesMap.py). The data sources endpoint returns all approved rows in the corresponding Data Sources database table by default with only the columns relevant to mapping.
+
+| Name | Type | Description |
+| ----------------------------------------------- | ------ | --------------------------------------------------- |
+| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” |
+
+{% tabs %}
+{% tab title="200: OK Successful operation" %}
+{% tabs %}
+{% tab title="Schema" %}
+count int
+data array[object]
+ data_source_id string
+ name string
+ agency_id string
+ agency_name string
+ state_iso string
+ municipality string
+ county_name array[string]
+ record_type string
+ lat float
+ lng float
+
+{% endtab %}
+
+{% tab title="Example" %}
+count: 1
+[{'data_source_id': 'recisSIaKGoWWbC8y', 'name': 'Allegheny County Jail policies', 'agency_id': 'recNjgPW5kD573LeK', 'agency_name': 'Allegheny County Jail', 'state_iso': 'PA', 'municipality': None, 'county_name': '["Allegheny"]', 'record_type': 'Policies & Contracts', 'lat': 40.4345561, 'lng': -79.9959509}]
+
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+```javascript
+{
+ // Response
+}
+```
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+
+{% tab title="500: Internal Server Error Error" %}
+```javascript
+{
+ "count": 0,
+ "message": "There has been an error pulling data!"
+}
+```
+{% endtab %}
+{% endtabs %}
+
+## Get Data Source by Id
+
+`GET` `[base-url]/data-sources-by-id/[id]`
+
+The data sources endpoint is located in [resources/DataSources.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The data source by id endpoint returns just the row for the data source that corresponds to the id passed.
+
+#### Path Parameters
+
+| Name | Type | Description |
+| ------------------------------------ | ------ | -------------- |
+| id\* | String | Data source id |
+
+#### Headers
+
+| Name | Type | Description |
+| ----------------------------------------------- | ------ | --------------------------------------------------- |
+| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” |
+
+{% tabs %}
+{% tab title="200: OK Successful operation" %}
+{% tabs %}
+{% tab title="Schema" %}
+```plsql
+count int
+data array[object]
+ name string
+ submitted_name string
+ description string
+ record_type string
+ source_url string
+ agency_supplied boolean
+ supplying_entity string
+ agency_originated string
+ originating_entity string
+ agency_aggregation string
+ coverage_start string
+ coverage_end string
+ source_last_updated string
+ retention_schedule string
+ detail_level string
+ number_of_records_available string
+ size string
+ access_type array
+ data_portal_type string
+ record_format array
+ update_frequency string
+ update_method string
+ tags string
+ readme_url string
+ scraper_url string
+ data_source_created string
+ airtable_source_last_modified string
+ url_status string
+ rejection_note string
+ last_approval_editor object
+ agency_described_submitted string
+ agency_described_not_in_database string
+ approval_status string
+ record_type_other string
+ data_portal_type_other string
+ records_not_online string
+ data_source_request string
+ url_button string
+ tags_other string
+ access_notes string
+ last_cached string
+ homepage_url string
+ count_data_sources string
+ agency_type string
+ multi_agency string
+ submitted_name string
+ jurisdiction_type string
+ state_iso string
+ municipality string
+ zip_code string
+ county_fips string
+ county_name array
+ lat float
+ lng float
+ data_sources array
+ no_web_presence string
+ airtable_agency_last_modified string
+ data_sources_last_updated string
+ approved boolean
+ rejection_reason string
+ last_approval_editor object
+ agency_created string
+ county_airtable_uid string
+ defunct_year string
+ data_source_id string
+ agency_id string
+ agency_name string
+```
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "data": {'name': 'Calls for Service for Asheville Police Department - NC', 'submitted_name': 'Asheville Police Department', 'description': None, 'record_type': 'Calls for Service', 'source_url': 'https://services.arcgis.com/aJ16ENn1AaqdFlqx/arcgis/rest/services/APD_CAD_911_Calls_2006/FeatureServer/0', 'agency_supplied': True, 'supplying_entity': None, 'agency_originated': None, 'originating_entity': None, 'agency_aggregation': None, 'coverage_start': '2006-01-01', 'coverage_end': '2006-12-31', 'source_last_updated': None, 'retention_schedule': None, 'detail_level': None, 'number_of_records_available': None, 'size': None, 'access_type': ['API', 'Download'], 'data_portal_type': 'ArcGIS', 'record_format': ['GIS / Shapefile'], 'update_frequency': None, 'update_method': None, 'tags': None, 'readme_url': 'https://docs.google.com/document/d/143a0LoGwNwmmHxJu1msxjOFAfAXPk7otQSWkrLtUDk0/edit?usp=sharing', 'scraper_url': 'https://pypi.org/project/openpolicedata/', 'data_source_created': '2023-03-02', 'airtable_source_last_modified': '2023-11-08', 'url_status': 'ok', 'rejection_note': None, 'last_approval_editor': '{"id": "usrtLIB4Vr3jTH8Ro", "email": "josh.chamberlain@pdap.io", "name": "Josh Chamberlain"}', 'agency_described_submitted': None, 'agency_described_not_in_database': None, 'approval_status': 'approved', 'record_type_other': None, 'data_portal_type_other': None, 'records_not_online': None, 'data_source_request': None, 'url_button': None, 'tags_other': None, 'access_notes': None, 'last_cached': '0001-01-01', 'homepage_url': 'https://www.ashevillenc.gov/department/police', 'count_data_sources': 18, 'agency_type': 'law enforcement/police', 'multi_agency': None, 'jurisdiction_type': 'local', 'state_iso': 'NC', 'municipality': 'Asheville', 'zip_code': '28801', 'county_fips': '37021', 'county_name': ['Buncombe'], 'lat': 35.594677, 'lng': -82.54986, 'data_sources': ['recpWxJ9JVa6BtLi5', 'reczwxaH31Wf9gRjS', 'recBd7NrWsvfTyDk0', 'recy24QB6I8FCVrQr', 'recW6aQGuNyzedIDl', 'recmrXPvQn9Gtfpba', 'recsojoTxJ3g08qKl', 'recTUW1QUZpsGVxoJ', 'reckqhpMEvDgiDGXF', 'recjnLeosesVTaW2r', 'reccrwbvTL6Ttd8XT', 'reckiy2nuY5iRptBm', 'recyJMD6eYF9Ln98B', 'recLdzmQMXC3XuPWU', 'recRNvjKBo5LkUBS9', 'recKFoiPMOmWFZvqM', 'recordd7DcM3raQF7', 'recV35fyFlof4pQXP'], 'no_web_presence': None, 'airtable_agency_last_modified': '2023-05-16', 'data_sources_last_updated': '2023-03-02', 'approved': True, 'rejection_reason': None, 'agency_created': '2022-08-18', 'county_airtable_uid': 'recrCy8hHYuxC8ZhU', 'defunct_year': None, 'data_source_id': 'reczwxaH31Wf9gRjS', 'agency_id': 'recJDGmbd7UMFcfa0', 'agency_name': 'Asheville Police Department - NC'}
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+{% tabs %}
+{% tab title="Schema" %}
+```plsql
+count int
+data array[object]
+```
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 0,
+ "data": []
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+
+{% tab title="500: Internal Server Error Error" %}
+```javascript
+{
+ "message": "There has been an error pulling data!"
+}
+```
+{% endtab %}
+
+{% tab title="404: Not Found Id not found" %}
+```javascript
+{
+ "message": "Data source not found."
+}
+```
+{% endtab %}
+{% endtabs %}
+
+## Create Data Source
+
+`POST` `[base-url]/data-sources`
+
+The data sources endpoint is located in [resources/DataSources.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The create data source endpoint posts a new data source to the database and returns True if successful and False if not.
+
+#### Headers
+
+| Name | Type | Description |
+| ----------------------------------------------- | ------ | --------------------------------------------------- |
+| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” |
+
+#### Request Body
+
+| Name | Type | Description |
+| ---- | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Data | JSON | A JSON object of the data source information. Refer to the Data Source dictionary for available fields: https://docs.pdap.io/activities/data-dictionaries/data-sources-data-dictionary. However, the following fields cannot be edited: rejection_note, data_source_request, approval_status, airtable_uid, airtable_source_last_modified
Below is an example of an acceptable body:
{ "name": "Calls for Service for Chicago Police Department - IL",
"record_type": "Calls for Service",
"source_url": "https://informationportal.igchicago.org/911-calls-for-cpd-service",
"coverage_start": "2019-01-01"
}
| + +{% tabs %} +{% tab title="200: OK Successful operation" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +message string +``` +{% endtab %} + +{% tab title="Example" %} +```json +{"message": "Data source added successfully."} +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="403: Forbidden Invalid API key" %} + +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} +```javascript +{ + "message": "There has been an error adding the data source" +} +``` +{% endtab %} + +{% tab title="400: Bad Request Missing or bad API key" %} +```javascript +{ + // Response +} +``` +{% endtab %} +{% endtabs %} + +## Update Data Source by Id + +`PUT` `[base-url]/data-sources-by-id/[id]` + +The data sources endpoint is located in [resources/DataSources.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The update data source by id endpoint updates a data source and returns a status to confirm a successful update. + +#### Path Parameters + +| Name | Type | Description | +| ------------------------------------ | ------ | -------------- | +| id\* | String | Data source id | + +#### Headers + +| Name | Type | Description | +| ----------------------------------------------- | ------ | --------------------------------------------------- | +| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” | + +#### Request Body + +| Name | Type | Description | +| ---- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Data | JSON |A JSON object of the data to be updated. Refer to the Data Source dictionary for available fields: https://docs.pdap.io/activities/data-dictionaries/data-sources-data-dictionary. However, the following fields cannot be edited: data_source_request, airtable_uid, airtable_source_last_modified
Below is an example of an acceptable body:
{ "name": "Calls for Service for Chicago Police Department - IL",
"record_type": "Calls for Service",
"source_url": "https://informationportal.igchicago.org/911-calls-for-cpd-service",
"coverage_start": "2019-01-01"
}
| + +{% tabs %} +{% tab title="200: OK Successful operation" %} +```json +{"message": "Data source successfully updated."} +``` +{% endtab %} + +{% tab title="500: Internal Server Error Error" %} +```javascript +{ + "message": "There has been an error updating the data source" +} +``` +{% endtab %} + +{% tab title="403: Forbidden Invalid API key" %} +```javascript +{ + // Response +} +``` +{% endtab %} + +{% tab title="400: Bad Request Missing or bad API key" %} +```javascript +{ + // Response +} +``` +{% endtab %} +{% endtabs %} + +## Archives + +## Get all Archived urls + +`GET` `[base-url]/archives` + +The archives endpoint is located in [resources/Archives.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/Archives.py). The get method on the archives endpoint returns all rows for urls that the [automatic archives](https://github.com/Police-Data-Accessibility-Project/automatic-archives/blob/main/cache\_url.py) script has cached in the Internet Archive. + +#### Headers + +| Name | Type | Description | +| ----------------------------------------------- | ------ | --------------------------------------------------- | +| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” | + +{% tabs %} +{% tab title="200: OK Successful operation" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +id string +source_url string +update_frequency string +last_cached string +``` +{% endtab %} + +{% tab title="Example" %} +```json +[{'id': 'rec0Fh8dMrlfxlpoH', 'source_url': 'https://www.portlandmaine.gov/999/Annual-Reports', 'update_frequency': 'Annually', 'last_cached': '2024-01-30'}] +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="400: Bad Request Missing or bad API key" %} +{% tabs %} +{% tab title="Schema" %} +```plsql +string +``` +{% endtab %} + +{% tab title="Example" %} +```json +"There has been an error pulling data!" +``` +{% endtab %} +{% endtabs %} +{% endtab %} + +{% tab title="403: Forbidden Invalid API key" %} + +{% endtab %} +{% endtabs %} + +## Get all Archived urls + +`PUT` `[base-url]/archives` + +The archives endpoint is located in [resources/Archives.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/Archives.py). The put method on the archives endpoint updates the data source matching the passed id, updating the last\_cached date if it alone is passed, or it and the broken\_source\_url\_as\_of field and the url\_status to 'broken'. + +#### Headers + +| Name | Type | Description | +| ----------------------------------------------- | ------ | --------------------------------------------------- | +| Authorization\* | String | Value formatted as "Bearer \[access token/api key]” | + +#### Request Body + +| Name | Type | Description | +| ------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------- | +| id\* | String | The airtable uid for the data source that was cached | +| broken\_source\_url\_as\_of\* | Date | The current date if the url is no longer active, otherwise None | +| last\_cached\* | Date | The current date since the data source url was just cached in the Internet Archive | + +{% tabs %} +{% tab title="200: OK Successful operation" %} +{% tabs %} +{% tab title="Schema" %} +count int
+data array[object]
+ agency_name string
+ municipality string
+ state_iso string
+ data_source_name string
+ description string
+ record_type string
+ source_url string
+ record_format string
+ coverage_start string
+ coverage_end string
+ agency_supplied boolean
+
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 1,
+ "data": [
+ {
+ "agency_name": "Allegheny County Police Department - PA",
+ "municipality": "Pittsburgh",
+ "state_iso": "PA",
+ "data_source_name": "Allegheny County Police Review Board Transcripts",
+ "description": null,
+ "record_type": "Policies & Contracts",
+ "source_url": "https://www.alleghenycounty.us/county-council/police-review-board-meetings.aspx",
+ "record_format": "[\"PDF: Machine Created\"]",
+ "coverage_start": "2018-08-29",
+ "coverage_end": "2018-09-26",
+ "agency_supplied": true
+ }
+ ]
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+{% endtabs %}
+
+## Agencies
+
+## Get all Agencies
+
+`GET` `[base-url]/agencies/{page}`
+
+The agencies endpoint is located in [resources/Agencies.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/resources/QuickSearch.py). The agencies endpoint returns 1000 rows from the corresponding Data Sources database table offset by the page number passed.
+
+#### Path Parameters
+
+| Name | Type | Description |
+| -------------------------------------- | ------ | -------------------------------------------------------------------------------------------- |
+| page\* | String | Passing 1 will return the first 1000 rows. Subsequent page number return subsequent results |
+
+#### Headers
+
+| Name | Type | Description |
+| ------------- | ------ | --------------------------------------------------- |
+| Authorization | String | Value formatted as "Bearer \[access token/api key]” |
+
+{% tabs %}
+{% tab title="200: OK Successful operation" %}
+{% tabs %}
+{% tab title="Schema" %}
+count int
+data array[object]
+ agency_name string
+ municipality string
+ state_iso string
+ data_source_name string
+ description string
+ record_type string
+ source_url string
+ record_format string
+ coverage_start string
+ coverage_end string
+ agency_supplied boolean
+
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 1,
+ "data": [
+ {
+ "agency_name": "Allegheny County Police Department - PA",
+ "municipality": "Pittsburgh",
+ "state_iso": "PA",
+ "data_source_name": "Allegheny County Police Review Board Transcripts",
+ "description": null,
+ "record_type": "Policies & Contracts",
+ "source_url": "https://www.alleghenycounty.us/county-council/police-review-board-meetings.aspx",
+ "record_format": "[\"PDF: Machine Created\"]",
+ "coverage_start": "2018-08-29",
+ "coverage_end": "2018-09-26",
+ "agency_supplied": true
+ }
+ ]
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="400: Bad Request Missing or bad API key" %}
+{% tabs %}
+{% tab title="Schema" %}
+```plsql
+count int
+data array[object]
+```
+{% endtab %}
+
+{% tab title="Example" %}
+```json
+{
+ "count": 0,
+ "data": []
+}
+```
+{% endtab %}
+{% endtabs %}
+{% endtab %}
+
+{% tab title="403: Forbidden Invalid API key" %}
+
+{% endtab %}
+{% endtabs %}
diff --git a/docs/api/introduction.md b/docs/api/introduction.md
new file mode 100644
index 00000000..0d89b6eb
--- /dev/null
+++ b/docs/api/introduction.md
@@ -0,0 +1,133 @@
+# Introduction
+
+## Overview
+
+The PDAP API is how internal and external users programmatically access information and make changes to the PDAP [Data Sources database](broken-reference).
+
+## Get access
+
+Reach out to [contact@pdap.io](mailto:contact@pdap.io) or make noise in Discord if you'd like access to the API.
+
+See the [administration page](endpoints/admin.md) for information on user creation with the API.
+
+## Authentication
+
+API routes that read and modify the Data Sources database should be protected through authentication. The API has an @api\_required decorator (located in [/middleware/security.py](https://github.com/Police-Data-Accessibility-Project/data-sources-app/blob/main/middleware/security.py)) that can be added to each route so that only authenticated users can access the database. To protect a route with this decorator, add @api\_required on the line above a given route.
+
+### @api\_required decorator
+
+The @api\_required decorator requires a request header to include an `Authorization` key with the value formatted as `Bearer [jwt_token]`. The api\_required function parses the request header, decodes and extracts the API key, and checks to see if the api\_key is defined. If it isn’t, it’ll return a message requesting an API key.
+
+### Validating the API key
+
+If there is an API key, it’s passed to the is\_valid function. This function connects to the Data Source database’s users table and finds the user in the table with the matching API key. The function then checks that a valid user was returned from the database. If not, the function returns `False` to `api_required`, which sends a response stating that the API key was invalid.
+
+## Rate limits
+
+The rate limit for querying the database currently maxes out at 3000 rows, which is larger than the row count of any table in the database right now.
+
+## Examples
+
+{% tabs %}
+{% tab title="Python" %}
+**Quick Search Data Sources**
+
+```python
+import requests
+base_url = "https://data-sources.pdap.io/"
+search_term = "review" # Replace with your search term
+location = "Pittsburgh" # Replace with your desired location
+api_key = "YOUR_API_KEY_HERE" # Replace with your actual API key
+
+url = f"{base_url}quick-search/{search_term}/{location}"
+
+# Create the Authorization header
+headers = {
+ "Authorization": f"Bearer {api_key}"
+}
+
+# Make the GET request with the Authorization header
+response = requests.get(url, headers=headers)
+```
+
+**GET Agencies**
+
+```python
+import requests
+base_url = "https://data-sources.pdap.io/"
+api_key = "YOUR_API_KEY_HERE" # Replace with your actual API key
+
+url = f"{base_url}agencies"
+
+# Create the Authorization header
+headers = {
+ "Authorization": f"Bearer {api_key}"
+}
+
+# Make the GET request with the Authorization header
+response = requests.get(url, headers=headers)
+```
+{% endtab %}
+
+{% tab title="JavaScript" %}
+### Quick Search Data Sources
+
+const axios = require('axios');
+
+const baseUrl = "https://data-sources.pdap.io/";
+const search_term = "review"; // Replace with your search term
+const location = "Pittsburgh"; // Replace with your desired location
+const api_key = "YOUR_API_KEY_HERE"; // Replace with your actual API key
+
+const url = `${baseUrl}quick-search/${search_term}/${location}`;
+
+// Create the Authorization header
+const headers = {
+ 'Authorization': `Bearer ${api_key}`
+};
+
+// Make the GET request with the Authorization header
+axios.get(url, { headers })
+ .then(response => {
+ console.log("Search results:");
+ response.data.forEach(item => {
+ console.log(`Data Source Name: ${item.name}`);
+ });
+ })
+ .catch(error => {
+ console.error("An error occurred:", error.message);
+ }
+ });
+
+
+### GET agencies
+
+```javascript
+const axios = require('axios');
+
+const baseUrl = "https://data-sources.pdap.io/";
+const api_key = "YOUR_API_KEY_HERE"; // Replace with your actual API key
+
+const url = `${baseUrl}agencies`;
+
+// Create the Authorization header
+const headers = {
+ 'Authorization': `Bearer ${api_key}`
+};
+
+// Make the GET request with the Authorization header
+axios.get(url, { headers })
+ .then(response => {
+ console.log("Search results:");
+ response.data.forEach(item => {
+ console.log(`Agency: ${item.name}`);
+ });
+ })
+ .catch(error => {
+ console.error("An error occurred:", error.message);
+ }
+ });
+```
+{% endtab %}
+{% endtabs %}
+
diff --git a/docs/data-dictionaries/README.md b/docs/data-dictionaries/README.md
new file mode 100644
index 00000000..7a7715e1
--- /dev/null
+++ b/docs/data-dictionaries/README.md
@@ -0,0 +1,2 @@
+# Data Dictionaries
+
diff --git a/docs/data-dictionaries/agencies-data-dictionary.md b/docs/data-dictionaries/agencies-data-dictionary.md
new file mode 100644
index 00000000..dbb78756
--- /dev/null
+++ b/docs/data-dictionaries/agencies-data-dictionary.md
@@ -0,0 +1,40 @@
+# Agencies data dictionary
+
+{% hint style="info" %}
+To see which options are available for select fields, consult the [submission form](https://airtable.com/shrzxLdSsYmBvIWMH).
+{% endhint %}
+
+### Required properties for submission
+
+`submitter_contact`, `submitted_name`, `homepage_url`, `jurisdiction_type`, `agency_type`, + geographic info dependent on the `jurisdiction_type`.
+
+## Identification & web presence
+
+| Property | Type | Description |
+| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| name | string | If a `state` is present, concatenates `submitted_name` + `" - "` + `state_iso` to help differentiate agencies with similar names in some Airtable menus. Otherwise, uses `submitted_name`. |
+| submitted\_name | string | What does the agency call itself? |
+| agency\_type | string | Options include `law enforcement/police`, `court`, and `corrections`. |
+| multi\_agency | boolean | True for utility agencies, used to geolocate data sources which cover multiple agencies. For example, a source about the "Allegheny County Aggregated" multi-agencies contains records about all agencies in Allegheny County. |
+| defunct\_year | integer | If present, denotes an agency which has defunct but may still have relevant records. |
+| homepage\_url | string | A URL for the homepage of this agency. |
+| no\_web\_presence | boolean | True when an agency does not have a dedicated website. |
+
+## Geographic
+
+| Property | Type | Description |
+| ------------------ | --------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
+| jurisdiction\_type | string | What is the highest level of jurisdiction for the agency? Can be an option like `local` or `county`. |
+| state\_iso | string | 2-character ISO code. Does not apply to federal agencies. |
+| county\_fips | string (not an integer because of leading 0s) | Unique 5-digit code to identify counties. Does not apply to state or federal agencies. |
+| county\_name | string | Official County name. Does not apply to state or federal agencies. |
+| municipality | string | Official municipality name. Does not apply to county, state, or federal agencies. |
+| zip\_code | string (not an integer because of leading 0s) | Postal code for the agency's headquarters. |
+| lat | float | Geographic latitude for the agency's headquarters. |
+| lng | float | Geographic longitude for the agency's headquarters. |
+
+## Meta & utility
+
+| Property | Type | Description |
+| ------------- | ------ | --------------------------------------------------------- |
+| airtable\_uid | string | The Airtable-generated UID of this particular data source |
diff --git a/docs/data-dictionaries/data-sources-data-dictionary.md b/docs/data-dictionaries/data-sources-data-dictionary.md
new file mode 100644
index 00000000..52a6bc4e
--- /dev/null
+++ b/docs/data-dictionaries/data-sources-data-dictionary.md
@@ -0,0 +1,80 @@
+# Data Sources data dictionary
+
+{% hint style="info" %}
+To see which options are available for select fields, consult the [submission form](https://airtable.com/shrJafakrcmTxHU2i).
+{% endhint %}
+
+### Required properties for submission
+
+`submitted_name`, `submitter_contact_info`, `record_type`, `agency_supplied` (+ other "[provenance](data-sources-data-dictionary.md#provenance)" properties, if "no")
+
+## What is it?
+
+| Property | Type | Description |
+| --------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| name | string | Uses `submitted_name` if present or concatenates `record_type` + `" for "` + `agency_described`; can get weird when one or both are not present |
+| submitted\_name | string | Required for individual Data Source submissions for clarity. |
+| record\_type | string | What kind of data is accessible from this source? For more info, see the [Record Types taxonomy](record-types-taxonomy.md). |
+| tags | array | Are there any keyword descriptors which might help people find this in a search? Try to limit tags to information which can't be contained in other properties. |
+| description | string (textarea) | Information to give clarity and confidence about what this source is, how it was processed, and whether the person reading the description might want to use it. Especially important if the source is difficult to preview or categorize. |
+
+## Agency
+
+| Property | Type | Description |
+| ------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| agency\_described | array (foreignkey based on an agency's ID within Airtable) | To which criminal legal system agency or agencies does this Data Source refer? |
+| agency\_aggregation | array | If present, the Data Source describes multiple agencies. Can be an item like `local` or `county`. |
+| state | string | 2-character ISO code, related to the associated Agency object, if present. |
+| county | string | Related to the associated Agency object, if present. |
+| municipality | string | Related to the associated Agency object, if present. |
+| agency\_type | string | Related to the associated Agency object, if present. |
+| jurisdiction\_type | array | Related to the associated Agency object, if present. What is the highest level of jurisdiction for the agency? Can be an item like `local` or `county`. |
+
+## Provenance
+
+_Where did it come from?_
+
+| Property | Type | Description |
+| ------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| agency\_supplied | boolean | Is the relevant Agency also the entity supplying the data? This may be "no" if the Agency or local government contracted with a third party to publish this data, or if a third party was the original record-keeper. |
+| supplying\_entity | string | If the Agency didn't publish this, who did? |
+| agency\_originated | boolean | Is the relevant Agency also the original record-keeper? This is usually "yes", unless a third party collected data about a police Agency.
table | property to hide | reason/notes |
---|---|---|
data sources | submitter_contact_info | PII |
data sources | submission_notes | intended to be private |
data sources | private_access_instructions | intended to be private |
agencies | submitter_contact | PII |
data requests | submitter_contact_info | PII |
data requests | internal_notes | visible to those with write access |
data requests | connected_volunteers | PII |
volunteers | name | PII |
volunteers | internal_notes | visible to those with write access |
volunteers | discord | PII |
volunteers | github | PII |
volunteers | email | PII |