Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LG-15217 Add submit_attempts property to doc auth troubleshooting option actions #11682

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jennyverdeyen
Copy link
Member

🎫 Ticket

Link to the relevant ticket:
LG-15217

🛠 Summary of changes

We want to be able to track metrics for users who choose to "try again online" or fall back to "try in person" after not being able to verify their ID in the remote flow. In particular we want to know how many submission attempts they've made before eventually trying in person. This PR adds submit_attempts to the analytics for these events: "Frontend: IdV: Capture troubleshooting dismissed" and "Frontend: IdV: Capture troubleshooting dismissed"

📜 Testing Plan

Provide a checklist of steps to confirm the changes.

  • Run make watch_events in your terminal to view the analytics events

  • Enter the application through Sinatra, choosing "identity-verified"

  • Begin the remote flow by choosing "Continue online" on /verify/how_to_verify

  • Choose "upload photos" to reach the document capture step

  • Upload failing images and submit

  • Choose "Try Again"

  • Observe that the event "Frontend: IdV: Capture troubleshooting dismissed" has been logged, with submit_attempts included under event_properties. It should accurately reflect how many times you've tried to submit the doc. (So far, 1)

  • Back on the document capture page, upload another erroring/failing ID (you may need a separate failing YML or reload the page to upload the same ones you just uploaded)

  • This time, choose "Try in person"

  • Observe that the event "IdV: verify in person troubleshooting option clicked" has been logged, with submit_attempts included under event_properties. It should accurately reflect how many times you've tried to submit the doc (This time, 2)

👀 Screenshots

Screen.Recording.2024-12-20.at.11.17.17.AM.mov

…shooting_option_clicked and idv_capture_troubleshooting_dismissed

changelog: Internal, In-person proofing, add submit_attempts property to try again and IPP fallback doc auth troubleshooting events
@jennyverdeyen jennyverdeyen requested review from a team and gina-yamada December 20, 2024 18:09
@gina-yamada
Copy link
Contributor

gina-yamada commented Dec 23, 2024

📓 Testing Notes:

❓If a user starts over, should the count start over? It makes sense that is does not start over- your comments say user so I bet not but just checking
✅ Count increases with every doc auth attempt and is as I'd expect
✅ Count increased when I was using hybrid (tested w/ telephony) and is as I'd expect
✅ Tested if a user fails doc auth, then starts ID-IPP and decides they don't want to do it- and tries doc auth again (only to fail) and then starts ID-IPP - count as I'd expect
✏️ I think the last sentence in Summary of changes needs a small update. You state This PR adds submit_attempts to the analytics for these events: "Frontend: IdV: Capture troubleshooting dismissed" and "Frontend: IdV: Capture troubleshooting dismissed" You mentioned Frontend: IdV: Capture troubleshooting dismissed twice but not IdV: Verify in person troubleshooting option clicked.
❌ I think the name submit_attempts is vague, especially on IdV: Verify in person troubleshooting option clicked. Can we give this a more descriptive name? (I recently had to look up key/value pairs on a handful of events to confirm what each was collecting.)

@@ -80,6 +81,11 @@ export interface UploadErrorResponse {
*/
remaining_submit_attempts?: number;

/**
* Number of submit attempts for user.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Number of doc auth submit attempts for user

@@ -1198,12 +1198,14 @@ def idv_cancellation_visited(
# @param ["hybrid","standard"] flow_path Document capture user flow
# @param [String] use_alternate_sdk
# @param [Boolean] liveness_checking_required
# @param [Integer] submit_attempts Times that user has tried submitting
Copy link
Contributor

@gina-yamada gina-yamada Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe try... number of doc auth submit_attempts for user

@@ -5268,16 +5271,19 @@ def idv_verify_by_mail_enter_code_visited(

# @param ["hybrid","standard"] flow_path Document capture user flow
# @param [Boolean] opted_in_to_in_person_proofing User opted into in person proofing
# @param [Integer] submit_attempts Times that user has tried submitting
# The user clicked the troubleshooting option to start in-person proofing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you update the description above, maybe do so here too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants