Merge branch 'main' into gcalendar-connector

CODEOWNERS

@@ -0,0 +1 @@
+* @cohere-ai/connectors

gcalendar/.env-template

@@ -1,4 +1 @@
-GCALENDAR_SERVICE_ACCOUNT_INFO=
-GCALENDAR_SEARCH_LIMIT=
-# Connector Authorization
-GCALENDAR__CONNECTOR_API_KEY=
+GCALENDAR_CONNECTOR_API_KEY=

gcalendar/README.md

@@ -2,68 +2,19 @@
 
 This connects Cohere to Google Calendar, allowing searching events.
 
-## Authentication
+## Credentials
 
-This connector supports two types of authentication: Service Account and OAuth.
+To use this connector, you will need to enable the Google Calendar API in a Google Cloud project. There are several steps to follow, please refer to the following [guide.](https://developers.google.com/calendar/api/quickstart/python)
 
-### Service Account
+You will need to:
 
-For service account authentication this connector requires two environment variables:
+1. Enable the API
+2. Configure OAuth consent: use the Google account that will be used for the Calendar integration
+3. Create credentials for your app, making sure to select the "Desktop app" option, then download the `credentials.json` file
 
-#### `GCALENDAR_SERVICE_ACCOUNT_INFO`
-#### `GCALENDAR_CALENDAR_ID`
+Once this is done, save the `credentials.json` file into this directory.
 
-The `GCALENDAR_SERVICE_ACCOUNT_INFO` variable should contain the JSON content of the service account credentials file.
-To get the credentials file, follow these steps:
-
-1. [Create a project in Google Cloud Console](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
-2. [Create a service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
-   and [activate the Google Calendar API](https://console.cloud.google.com/apis/library/calendar-json.googleapis.com) in
-   the Google Cloud Console.
-3. [Create a service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) and download
-   the credentials file as JSON. The credentials file should look like this:
-
-```json
-{
-  "type": "service_account",
-  "project_id": "{project id}",
-  "private_key_id": "{private_key_id}",
-  "private_key": "{private_key}",
-  "client_email": "{client_email}",
-  "client_id": "{client_id}",
-  "auth_uri": "{auth_uri}",
-  "token_uri": "{token_uri}",
-  "auth_provider_x509_cert_url": "{auth_provider_x509_cert_url}",
-  "client_x509_cert_url": "{client_x509_cert_url}",
-  "universe_domain": "{universe_domain}"
-}
-```
-
-4. Convert the JSON credentails to a string through `json.dumps(credentials)` and save the result in
-   the `GCALENDAR_SERVICE_ACCOUNT_INFO` environment variable.
-5. Make sure
-   to [share the calendar you want to search with the service account email address](https://support.google.com/a/answer/7337554?hl=en).
-6. Assign the `GCALENDAR_CALENDAR_ID` environment variable the value of the calendar owner's email address.
-
-#### `GCALENDAR_CONNECTOR_API_KEY`
-
-The `GCALENDAR_CONNECTOR_API_KEY` should contain an API key for the connector. This value must be present in
-the `Authorization` header for all requests to the connector.
-
-### OAuth
-
-When using OAuth for authentication, the connector does not require any additional environment variables. Instead, the
-OAuth flow should occur outside of the Connector and Cohere's API will forward the user's access token to this connector
-through the `Authorization` header.
-
-
-## Optional Configuration
-
-This connector also supports a few optional environment variables to configure the search:
-
-1. `GCALENDAR_SEARCH_LIMIT` - Number of results to return. Default is 10.
-2. `GCALENDAR_CALENDAR_ID` - ID of the calendar to search in. If not provided, the search will be performed in the primary calendar.
-For the service account authentication method, this value should be the email address of the calendar owner.
+This connector will also require a `token.json` file that is generated automatically once you authorize the app. After you deploy it, head to your hosted connector server's `/ui` page. For example, `https://myconnector.com/ui`, and trigger an initial `/search` request. This will bring up the Google OAuth page, where you can login to the same email as step 2 earlier.
 
 ## Development
 

gcalendar/credentials.json-template

@@ -0,0 +1,11 @@
+{
+  "installed": {
+    "client_id": "myclientid",
+    "project_id": "myproject",
+    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
+    "token_uri": "https://oauth2.googleapis.com/token",
+    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
+    "client_secret": "mysecret",
+    "redirect_uris": ["http://localhost"]
+  }
+}

gcalendar/provider/app.py

@@ -14,10 +14,6 @@ def search(body):
     except UpstreamProviderError as error:
         logger.error(f"Upstream search error: {error.message}")
         abort(502, error.message)
-    except AssertionError as error:
-        logger.error(f"GCalendar connector config error: {error}")
-        abort(502, f"GCalendar connector config error: {error}")
-
     return {"results": data}
 
 

gcalendar/provider/client.py

@@ -1,52 +1,51 @@
-import datetime
 import logging
+import os
+import datetime
 
-from flask import request, current_app as app
+from flask import current_app as app
 from google.auth.transport.requests import Request
-from google.oauth2 import service_account
 from google.oauth2.credentials import Credentials
+from google_auth_oauthlib.flow import InstalledAppFlow
 from googleapiclient.discovery import build
 from googleapiclient.errors import HttpError
 
 from . import UpstreamProviderError
 
+
 logger = logging.getLogger(__name__)
-AUTHORIZATION_HEADER = "Authorization"
-BEARER_PREFIX = "Bearer "
-DEFAULT_SEARCH_LIMIT = 20
+
+client = None
 
 
 class GoogleCalendarClient:
+    DEFAULT_SEARCH_LIMIT = 20
     SCOPES = [
         "https://www.googleapis.com/auth/calendar.readonly",
     ]
 
-    def __init__(self, service_account_info, access_token, calendar_id, search_limit):
-        self.search_limit = search_limit
-        self.calendar_id = calendar_id
-        self.service = build(
-            "calendar",
-            "v3",
-            credentials=self._request_credentials(service_account_info, access_token),
-        )
-
-    def _request_credentials(self, service_account_info=None, access_token=None):
-        if service_account_info is not None:
-            logger.debug("Using service account credentials")
-            credentials = service_account.Credentials.from_service_account_info(
-                service_account_info, scopes=self.SCOPES
-            )
-            if credentials.expired or not credentials.valid:
-                credentials.refresh(Request())
-
-            return credentials
-        elif access_token is not None:
-            logger.debug("Using oauth credentials")
-            return Credentials(access_token)
-        else:
-            raise UpstreamProviderError(
-                "No service account or oauth credentials provided"
-            )
+    def __init__(self):
+        creds = None
+        # Handle Authentication
+        if os.path.exists("token.json"):
+            logger.debug("Found token.json file")
+            creds = Credentials.from_authorized_user_file("token.json", self.SCOPES)
+
+        # If there are no (valid) credentials available, let the user log in.
+        if not creds or not creds.valid:
+            logger.debug("No valid credentials found")
+            if creds and creds.expired and creds.refresh_token:
+                creds.refresh(Request())
+            else:
+                flow = InstalledAppFlow.from_client_secrets_file(
+                    "credentials.json", self.SCOPES
+                )
+                creds = flow.run_local_server(port=0)
+
+            # Save the credentials for the next run
+            with open("token.json", "w") as token:
+                token.write(creds.to_json())
+
+        self.service = build("calendar", "v3", credentials=creds)
 
     def _request(self, request):
         try:
@@ -56,11 +55,10 @@ def _request(self, request):
 
     def search_events(self, query):
         now = datetime.datetime.utcnow().isoformat() + "Z"  # 'Z' indicates UTC time
-
         request = self.service.events().list(
-            calendarId=self.calendar_id,
+            calendarId="primary",
             timeMin=now,
-            maxResults=self.search_limit,
+            maxResults=self.DEFAULT_SEARCH_LIMIT,
             singleEvents=True,
             orderBy="startTime",
             q=query,
@@ -72,20 +70,8 @@ def search_events(self, query):
 
 
 def get_client():
-    service_account_info = app.config.get("SERVICE_ACCOUNT_INFO", None)
-    access_token = get_access_token()
-    calendar_id = app.config.get("CALENDAR_ID", "primary")
-    search_limit = app.config.get("SEARCH_LIMIT", DEFAULT_SEARCH_LIMIT)
-    if service_account_info is None and access_token is None:
-        raise AssertionError("No service account or oauth credentials provided")
-
-    return GoogleCalendarClient(
-        service_account_info, access_token, calendar_id, search_limit
-    )
-
-
-def get_access_token():
-    authorization_header = request.headers.get(AUTHORIZATION_HEADER, "")
-    if authorization_header.startswith(BEARER_PREFIX):
-        return authorization_header.removeprefix(BEARER_PREFIX)
-    return None
+    global client
+    if client is None:
+        client = GoogleCalendarClient()
+
+    return client

gcalendar/pyproject.toml

@@ -6,7 +6,7 @@ authors = ["Scott Mountenay <scott@lightsonsoftware.com>"]
 readme = "README.md"
 
 [tool.poetry.dependencies]
-python = "^3.11"
+python = "^3.10"
 connexion = {version = "2.14.2", extras = ["swagger-ui"]}
 python-dotenv = "^1.0.0"
 flask = "2.2.5"

gdrive/README.md

@@ -49,6 +49,29 @@ The `GDRIVE_CONNECTOR_API_KEY` should contain an API key for the connector. This
 
 When using OAuth for authentication, the connector does not require any additional environment variables. Instead, the OAuth flow should occur outside of the Connector and Cohere's API will forward the user's access token to this connector through the `Authorization` header.
 
+To use OAuth, you must first create a Google OAuth client ID and secret. You can follow Google's [guide](https://developers.google.com/identity/protocols/oauth2/web-server#creatingcred) to get started. When creating your application use `https://api.cohere.com/v1/connectors/oauth/token` as the redirect URI.
+
+Once your Google OAuth credentials are ready, you can register the connector in Cohere's API with the following configuration:
+
+```bash
+curl  -X POST \
+  'https://api.cohere.ai/v1/connectors' \
+  --header 'Accept: */*' \
+  --header 'Authorization: Bearer {COHERE-API-KEY}' \
+  --header 'Content-Type: application/json' \
+  --data-raw '{
+  "name": "GDrive with OAuth",
+  "url": "{YOUR_CONNECTOR-URL}",
+  "oauth": {
+    "client_id": "{GOOGLE-OAUTH-CLIENT-ID}",
+    "client_secret": "{GOOGLE-OAUTH-CLIENT-SECRET}",
+    "authorize_url": "https://accounts.google.com/o/oauth2/auth",
+    "token_url": "https://oauth2.googleapis.com/token",
+    "scope": "https://www.googleapis.com/auth/drive.readonly"
+  }
+}'
+```
+
 With OAuth the connector will be able to search any Google Drive folders that the user has access to.
 
 ## Optional Configuration
@@ -70,8 +93,7 @@ Create a virtual environment and install dependencies with poetry. We recommend
 Next, start up the search connector server:
 
 ```bash
-  $ poetry shell
-  $ flask --app provider --debug run --port 5000
+  $ poetry flask --app provider --debug run --port 5000
 ```
 
 and check with curl to see that everything works:

gdrive/provider/provider.py

@@ -144,4 +144,4 @@ def search(query, access_token=None):
     except HttpError as http_error:
         raise UpstreamProviderError(message=str(http_error)) from http_error
 
-    return process_data_with_service(search_results, request_credentials)
+    return process_data_with_service(search_results, request_credentials(access_token))