Retest V2.0 - Interactive Panel (released)

[AlexB52]

Hi everyone,

I've released a POC for Retest V2 with an interactive command. You can install the preview with: gem install retest --pre. Then run it with retest. Use retest --help to get info on available parameters when launching retest. For example retest --notify will give you a sound feedback when tests are passing or failing. Highly recommend it!

EDIT: Retest v2 has officially been released and can now be installed with gem install retest. More features and improvements to come in the next releases but this is in a state for an initial release with a complete new interactive shell.

The gem works the same with the addition of a new prompt that lets you make a few commands (to be expanded in other releases)
Entering 'h' prints the current usage of the interactive POC.

Setup identified: [RSPEC]. Using command: 'bin/rspec <test>'
Launching Retest...
Ready to refactor! You can make file changes now

Type interactive command and press enter. Enter 'h' for help.
> h

* 'h', 'help'              # Prints help.
* 'p', 'pause'             # Pauses Retest. Tests aren't run on file change events until unpaused.
* 'u', 'unpause'           # Unpauses Retest.
* <ENTER>                  # Runs last changed triggered command.
* 'ra, 'run all'           # Runs all tests.
* 'f', 'force'             # Forces a selection of test to run on every file change.
* 'r', 'reset'             # Disables forced selection.
* 'd', 'diff' [GIT BRANCH] # Runs matching specs that changed from a target branch.
* 'c'                      # Clears window.
* 'e', 'exit'              # Exits Retest.

Note: It's currently impossible to alter the starting arguments through the interactive command. For example retest --all --notify will always default the main test command to running all tests with sound feedback.

I recommend using retest --notify on a large codebase and retest --all --notify on projects with a very fast test suite.

Question: What do people think? Feedback would be appreciated.

Thanks, everyone.

[AlexB52]

First observation, when using debugger or byebug stdin isn't available anymore and retest gets stuck.

EDIT: This can currently be solved by remote debugging since the test is run in another process

To use debugger with retest v2.0.0.pre

# In your app
require "debug/open_nonstop"
debugger # somewhere in your code

Then when retest is waiting for a connection do rdbg --attach in another terminal window to debug there.

[AlexB52]

I had a breakthrough, and connecting remotely to a debugger session might not be necessary with some IO.pipe management.

[AlexB52]

New pre-release should solve this problem across cases 🤞

[AlexB52]

New observation
When running retest --notify 'ruby <changed>' and pressing enter after a few runs, the runner returns

Running last command
404 - File Not Found
Retest could not find a changed file to run.

Also when entering ra for run all

Running all tests
404 - File Not Found
Retest could not find a changed file to run.

[clupprich]

Thanks for creating and maintaining this gem, @AlexB52 - I love it!

I observed a very minor issue this morning: I ran a session with retest and upon exiting with Ctrl + C, an NoMethodError was raised in

retest/lib/retest/prompt.rb

Line 31 in cd8ab0b

options.values[input.gets.chomp.to_i]

because input.gets returned nil. Unfortunately I cleared the output, so no logs, etc.

[AlexB52]

Thanks @clupprich
To understand the issue, tests have just run and you want to exit. You press ctrl + c and get this error instead.
Is that correct?
How has the last test run been triggered with retest?
Let me know. Cheers

[clupprich]

That's exactly what happened, @AlexB52. The last test runs was triggered automatically when I saved one of the source files.

[AlexB52]

I'm unable to reproduce the error but I will put a #to_s after #gets and it should do the trick.

[AlexB52]

@clupprich I have pushed a new pre-release for retest v2.0. You can install it with gem install retest --pre.
The new pre-version is 2.0.0.pre1 2.0.0.pre2

Let me know if this fixed your issue. Also let me know if the debugger works as expected.

Note: Some log info statement remain
There are some logs I forgot to clean up like input is: and waiting for input
These should be removed in the next pre release

[clupprich]

I updated to the latest pre-release and have been running that for over a week now - didn't happen again. Thanks so much @AlexB52!

[AlexB52]

I've pushed the new pre-release, 2.0.0.pre1.
Install the new version with gem install retest --pre

It should allow the use of the debugger without requiring a remote rdbg connection.
If there are cases where your inputs don't seem to do anything this is a bug.
Please let me know here in this discussion so I can track them.
Thanks

[AlexB52]

The diffing feature on the interactive command was missing the STDIN support for synchronous runs.
Should be fixed once this is merged: #221

[AlexB52]

Debugger works but now there is no possible way to use up arrow for example to reuse the input from the history. Every command needs to be written or copied.

[AlexB52]

Fixed in 2.0.0.pre3

[AlexB52]

Pausing the program from automatic listening is reenabled after a spec is run manually, it shouldn't until we manually reenable it.

[AlexB52]

Fixed in 2.0.0.pre3

[AlexB52]

Major milestone achieved. I think we're nearly ready for proper v2 release. Things left to do

1. Create some feature tests of the interactive command.
2. Support and use fswatch as the default listener when installed on the computer.

You can play with the retest interactive command by installing the latest pre version with gem install retest --pre
Latest version is v2.0.0.pre4

[AlexB52]

I decided to support watchexec instead for v2.0.
fswatch hasn't been active for 2 years but watchexec seems active. fswatch will be added in 2.1 instead if there is interest.

[paulodeon]

Hey @AlexB52 thanks for the awesome gem, I have wanted this for a long time!

Quick question for you - I have some cases where the rspec file is very large so I want to only run an example at a specific line number.

How would I do that?

I've tried the force command , then selected a file, but couldnt figure out how to add a line number onto the end

[AlexB52]

Thanks @paulodeon

There is currently no way to force a specific line to run within Retest functionality. This could be looked at in the future as a new feature but is currently impossible just with retest.

In rspec I currently use the :force meta tag or using fit, fdescribe, fcontext to run specific examples. You can still force a file with Retest then the RSpec runner will pick up the meta tag and run the specific examples as it normally would. This is how I currently force some example when using Retest with RSpec.

See RSpec docs: filter_run_when_matching

[JoeDupuis]

+1 to that, forcing certain tests instead of a whole file would be really useful when adding breakpoints.

I was looking into adding a way to select tests by names after forcing a file, but it looks like we can't list test names out the box with rails default test runner (unsure with rspec). I wonder if they would accept a PR upstream to add a listing option 🤔

Alternatively, since we can set the test command when starting retest, it is probably easier to just set --name or add the line number in the command. (ex restest 'bin/rails test <test>:123' or restest 'bin/rails test <test> --name "test something"').

It doesn't look like we can change the command without restarting restest. I might look into adding a way to change the command instead 👀

Thank you for working on this! it's my new favorite thing! 😁

[AlexB52]

Thank you for working on this! it's my new favorite thing! 😁

Thank you. Really happy you like retest

It doesn't look like we can change the command without restarting restest. I might look into adding a way to change the command instead 👀

Changing launching settings without a restart would be a nice addition to retest yes. PRs are welcomed. Thanks

+1 to that, forcing certain tests instead of a whole file would be really useful when adding breakpoints.

This isn't trivial as it should support the different setups when possible, rake TEST=<test> rails test <test>, ruby <test> and rspec <test>.

One way would be to ask for specific lines during the forced selection on rspec and rails setup only. Hopefully something not too intrusive for the user as well.

It would be a lot friendlier if Minitest had a way to force specs like rspec with :force tag out of the box instead of skipping tests.