Update README

[tdrhq]

fiveam-asdf seems very useful, but the README seems outdated. For one, it seems like :class needs to be ":fiveam-tester-system", for another, seems like it needs something called :test-names.

Would really appreciate it to have an updated docs. Thanks!

[rpgoldman]

This is not a full answer, but I have moved fiveam-asdf to the contrib/ subdirectory of asdf.

But your point is a good one; I will try to get documentation done for this.

[jeosol]

Hi Rob,

I hope you are doing well.

Thanks for putting the fiveam-asdf library together. I was trying to get a better integrated test workflow and I already use fiveam for my tests. I did some search and found fiveam-asdf and wanted to understand it and give a try.

I want to make some comments related to the README and usage description. This may still be related to the initial comment by tdrhq.

Basically, it is related to the options passed to the test system. I put together a very simple hello-world defsystem with the aim of documenting proper usage (for my understanding and when cleaned up, we can update the documentation).

Below is the hello-world/tests defsystem linked to hello-world defsystem (not shown) so we can make calls like (asdf:test-system :hello-world) to perform tests:

(defsystem "hello-world"
...)

(defsystem "hello-world/tests"
  :license "v.0.0.1"
  :class :package-inferred-fiveam-tester-system
  :defsystem-depends-on ("fiveam-asdf")
  :depends-on ("fiveam"
               "hello-world"
               "hello-world/tests/hello-world-tests"
               "hello-world/tests/package")
  :test-package :hello-world/tests/package
  :test-names '(:test-hello-world)
  :num-checks 2
  :description "Test system for hello-world"
  :perform (test-op (op c) (uiop:symbol-call :fiveam :run!
                                             (find-symbol* :test-hello-world :hello-world/tests/package))))

---

My main questions are related to the options. First of all, I am not sure I am using the library correctly as when I commented out the test-names and test-packages and class options (same as old setup with fiveam-asdf) things work as expected.

In my setup, I was not able to get it to work without the program-op line at the end, and I was thinking that fiveam-asdf does the program-op bit behind the scenes if it has already information about test-names and test-packages, which is what I specified to the program-op bit in the hello-world/tests defsystem.

So there is some redundancy which is why I feel, I may be missing something due to my lack of understand of proper usage or some of the options are not specified correctly in my test defsystem.

Thanks

[rpgoldman]

Hi. I am a little busy today, but I will try to get answers to you tomorrow and likely also some fixes to docstrings and/or README.

[rpgoldman]

Below is the hello-world/tests defsystem linked to hello-world defsystem (not shown) so we can make calls like (asdf:test-system :hello-world) to perform tests:

(defsystem "hello-world"
...)

(defsystem "hello-world/tests"
 :license "v.0.0.1"
 :class :package-inferred-fiveam-tester-system
 :defsystem-depends-on ("fiveam-asdf")
 :depends-on ("fiveam"
              "hello-world"
              "hello-world/tests/hello-world-tests"
              "hello-world/tests/package")
 :test-package :hello-world/tests/package
 :test-names '(:test-hello-world)
 :num-checks 2
 :description "Test system for hello-world"
 :perform (test-op (op c) (uiop:symbol-call :fiveam :run!
                                            (find-symbol* :test-hello-world :hello-world/tests/package))))

My main questions are related to the options. First of all, I am not sure I am using the library correctly as when I commented out the test-names and test-packages and class options (same as old setup with fiveam-asdf) things work as expected.

That's because you added the :perform (test-op ()...) method definition. The :test-package and :test-names slots are used by the built in perform method for test-op on the fiveam-tester systems. See

fiveam-asdf/code.lisp

Lines 131 to 135 in 624670e

	(defmethod perform ((op test-op) (sys fiveam-tester-system))
	(let* ((results (run-tests sys)))
	(explain-results results)
	(verify-num-checks results sys)
	(verify-success results sys)))

In my setup, I was not able to get it to work without the program-op line at the end, and I was thinking that fiveam-asdf does the program-op bit behind the scenes if it has already information about test-names and test-packages, which is what I specified to the program-op bit in the hello-world/tests defsystem.

Yes, that is what should happen. Note that there's a minor mistake in your defsystem: the :test-names argument should not be quoted. That might have broken things for you.

If that does not fix things, please try the following:

(fiveam-asdf::test-syms (asdf:find-system "hello-world/tests"))

and report what it returns.

[jeosol]

Hi Rob, Thanks for your follow email and addressing this issue. From your email, I realized I mixed things up and adding the :perform option appears redundant and it is indeed the case. I checked the link and I see there is a perform defmethod specialized on fiveam-test-system. Apparently the error is due to the quoted list passed to the :test-names option. Yes, this broke things indeed. I removed the quote and remove :perform option and everything works fine and correctly now. So this fixes the error I was having. The fiveam-asdf method is much better and cleaner. Thanks again for your help in looking into this. The output you requested is shown below: * (fiveam-asdf::test-syms (asdf:find-system "hello-world/tests")) (HELLO-WORLD/TESTS/HELLO-WORLD-TESTS:TEST-HELLO-WORLD)

…

On Wed, Nov 3, 2021 at 10:07 PM rpgoldman ***@***.***> wrote: Below is the *hello-world/tests* defsystem linked to *hello-world* defsystem (not shown) so we can make calls like (asdf:test-system :hello-world) to perform tests: (defsystem "hello-world" ...) (defsystem "hello-world/tests" :license "v.0.0.1" :class :package-inferred-fiveam-tester-system :defsystem-depends-on ("fiveam-asdf") :depends-on ("fiveam" "hello-world" "hello-world/tests/hello-world-tests" "hello-world/tests/package") :test-package :hello-world/tests/package :test-names '(:test-hello-world) :num-checks 2 :description "Test system for hello-world" :perform (test-op (op c) (uiop:symbol-call :fiveam :run! (find-symbol* :test-hello-world :hello-world/tests/package)))) My main questions are related to the options. First of all, I am not sure I am using the library correctly as when I commented out the *test-names* and *test-packages* and class options (same as old setup with fiveam-asdf) things work as expected. That's because you added the :perform (test-op ()...) method definition. The :test-package and :test-names slots are used by the built in perform method for test-op on the fiveam-tester systems. See https://github.com/rpgoldman/fiveam-asdf/blob/624670e105d8f87966027b86704d6363450d097e/code.lisp#L131-L135 In my setup, I was not able to get it to work without the program-op line at the end, and I was thinking that fiveam-asdf does the program-op bit behind the scenes if it has already information about *test-names* and *test-packages*, which is what I specified to the program-op bit in the *hello-world/tests* defsystem. Yes, that is what should happen. Note that there's a minor mistake in your defsystem: the :test-names argument should *not* be quoted. That might have broken things for you. If that does not fix things, please try the following: (fiveam-asdf::test-syms (asdf:find-system "hello-world/tests")) and report what it returns. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACEFQ4MV627AFNUNMPBMCCLUKIBHNANCNFSM4I5VZNXA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.