Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronization with high number of labels with long strings #206

Merged
merged 29 commits into from
Jan 5, 2024

Conversation

cubeddu
Copy link
Contributor

@cubeddu cubeddu commented Oct 17, 2023

Issues

Switching from GET to POST in fetch calls allows the handling of long labels or large data efficiently. Unlike GET, which has URL length limits, POST carries data in the request body, preventing data truncation and ensuring accurate data transmission. This change resolves issues with long labels in URL queries, eliminating risks tied to URL string parameter limits.

Description of changes:

  • Updated fetch functionality for templated queries to use the POST method.
  • Resolved linter and pre-commit issues.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@codecov
Copy link

codecov bot commented Oct 17, 2023

Codecov Report

Attention: 900 lines in your changes are missing coverage. Please review.

Comparison is base (90aec5d) 10.00% compared to head (bcaf1df) 10.01%.
Report is 2 commits behind head on main.

Files Patch % Lines
...s/graph-explorer/src/connector/sparql/useSPARQL.ts 0.00% 242 Missing and 1 partial ⚠️
...es/graph-explorer/src/connector/useGEFetchTypes.ts 0.00% 192 Missing and 1 partial ⚠️
...ackages/graph-explorer/src/connector/useGEFetch.ts 0.00% 86 Missing and 1 partial ⚠️
...graph-explorer/src/connector/gremlin/useGremlin.ts 0.00% 76 Missing and 1 partial ⚠️
...explorer/src/connector/openCypher/useOpenCypher.ts 0.00% 72 Missing and 1 partial ⚠️
...r/src/core/ConnectorProvider/ConnectorProvider.tsx 0.00% 70 Missing and 1 partial ⚠️
...nector/openCypher/templates/oneHopTemplateEdges.ts 0.00% 38 Missing and 1 partial ⚠️
...nector/openCypher/queries/fetchVertexTypeCounts.ts 0.00% 24 Missing and 1 partial ⚠️
packages/graph-explorer/src/hooks/useSchemaSync.ts 0.00% 15 Missing ⚠️
...lorer/src/workspaces/DataExplorer/DataExplorer.tsx 0.00% 13 Missing ⚠️
... and 24 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #206      +/-   ##
==========================================
+ Coverage   10.00%   10.01%   +0.01%     
==========================================
  Files         406      407       +1     
  Lines       30137    30089      -48     
  Branches      654      655       +1     
==========================================
  Hits         3014     3014              
+ Misses      26789    26740      -49     
- Partials      334      335       +1     
Flag Coverage Δ
unittests 10.01% <5.36%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cubeddu cubeddu changed the title Fixes for Templated Queries and Linter Issues Synchronization with high number of labels with long strings Oct 17, 2023
Cole-Greer
Cole-Greer previously approved these changes Oct 19, 2023
Copy link

@Cole-Greer Cole-Greer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. As long as there are no concerns with the schema request I think this is good to go.

Copy link
Member

@michaelnchin michaelnchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cubeddu ! Nice job on finding a workaround for the header limit. Couple of changes before we merge this in...

@cubeddu
Copy link
Contributor Author

cubeddu commented Nov 15, 2023

@michaelnchin PR is ready for review.

@cubeddu cubeddu requested a review from michaelnchin November 15, 2023 19:03
@charlesivie charlesivie self-requested a review November 24, 2023 17:44
charlesivie
charlesivie previously approved these changes Nov 24, 2023
michaelnchin

This comment was marked as resolved.

@cubeddu

This comment was marked as resolved.

Juan Cubeddu added 7 commits December 12, 2023 14:29
This commit refactors the `useOpenCypher`, `useGremlin`, `useSPARQL`, and `useGEFetch` functions. It updates the imports to include the `useConfiguration` function from the `core` module. It also modifies the function signatures to remove the `connection` parameter and instead retrieves the connection configuration using `useConfiguration`.
Copy link
Member

@michaelnchin michaelnchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested and working well. Thanks @cubeddu !

@michaelnchin michaelnchin merged commit 4313ad0 into aws:main Jan 5, 2024
2 of 3 checks passed
cubeddu added a commit to cubeddu/graph-explorer that referenced this pull request Jan 8, 2024
* updating fetch for templated queries to be POST

* fix linter and pre commit issues

* fix linter SelectListBox node.value can be null

* changelog

* removed method get from fetch as it's defaults to get

* Bump Babel, PostCSS, and Vite to latest (aws#210)

* Bump Babel and PostCSS versions

* update changelog

* Bump crypto-js to 4.2.0 (aws#214)

* Bump crypto-js to 4.2.0

* update changelog

* merge

* Bye abstract class hello React hooks, test passing

* Fix error messages on schema fetch

* server does post calls instead of gets calls to neptune

* Fix type imports and update schema state

* Fix headers in useGEFetch and useSPARQL, and add
GET method to useOpenCypher and useGremlin

* Add body-parser middleware, created middleware to sign headers

* Refactor ConnectorProvider

* Add new connection configuration options

* Refactor useOpenCypher, useGremlin, useSPARQL, and useGEFetch

This commit refactors the `useOpenCypher`, `useGremlin`, `useSPARQL`, and `useGEFetch` functions. It updates the imports to include the `useConfiguration` function from the `core` module. It also modifies the function signatures to remove the `connection` parameter and instead retrieves the connection configuration using `useConfiguration`.

* Update fetch request timeout value

* Remove unused import in ConnectorProvider.tsx

* Add compression middleware to node-server.js

* Refactor aws4SigningMiddleware to use async/await

* adjusting header signature

* update

* IAM api call works for POST

* GET call return undefined

* Update IAM headers and fix query parameter in useGremlin

---------

Co-authored-by: Michael Chin <chnmch@amazon.com>
Co-authored-by: Michael Chin <mchin188@yahoo.com>
Co-authored-by: Juan Cubeddu <vcubjuan@amzon.com>
@kmcginnes kmcginnes mentioned this pull request Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

4 participants