Support in Wasp for external/community starter templates #2402

Martinsos · 2024-12-08T23:22:59Z

TODO:

Update e2e tests
Remove the -t github: feature?

I started this PR with the idea to add support for external starter templates to Wasp CLI. I imagined that we would allow users to use them in the similar fashion as right now we do for our official templates, it is just that they would specify them with sometihng like github:<repo_owner>/<repo_name>[/template/dir] and we would pull the template from there.

I implemented ability to this via wasp new -t <repo_owner>/<repo_name>[/template/dir], and also ended up refactoring a bit bigger parts of the starter templates logic in CLI while at it, both to support it and to improve the code in general.

However, I then at some point realized that I am not sure if this is useful at all! When you think about it, what do they get this way compared to just going to a github repo of that starter template and cloning that or using github's "clone as a template" feature? For official templates, we want to make sure they can be discovered and easily used from the CLI, and that makes sense, and we also insert name of the app into them and correct Wasp version. But for community tempaltes, that isn't that important I think -> We would expected users to go out of Wasp CLI and check out that template anyway, read the instrutions there, ... . They probably found out about it in some other fashion also. And I think maintainers of these community templates are not likely to be interested in using the rules that come with how our CLI pulls in the tempaltes -> I don't think they will use the template placeholders for wasp version and app name, nor can I see them updating templates immediately on every wasp version (which wasp CLI expects from official tempaltes, since it pulls them by specific version tag). I expect community templates will instead on their main be at the latest wasp version that they managed to update to and that is it, and they will be late a bit after each Wasp release, that is good enough and simple enough, I would actually recommend doing it that way.

So, I concluded that having this ability to pull in the repo from our CLI via github:... is actually not useful! What do you think, do you agree?

However, while implementing this PR, it turned out that most work was other stuff, which is improvments to the rest of the code / docs. Everything taht I implmented in this PR:

Added ability to do wasp new -t github:....
Refactored and improved the code of StarterTemplates modules in wasp CLI Haskell code.
Updated CLI instructions on tempaltes to be correct and up to date.
Updated docs on starter templates to have a "Community Templates" section that lists Roke, the template by @wardbox .
Added an additional interactive option to the starter template menu in wasp new that is called "community template" that informs users about existence of community templates, to help with discovery.

As it turns out, (1) is actually only a small part of it.

So, what I did at the end is create this PR that contains all the changes from above, although I would suggest that I remove (1) (which is mostly changes in TemplateId.hs), but I would merge the rest.

What I would like you to do:

Review the PR.
Let me know if you would keep the github: functionality or not. I vote for not, and will kick it out if you agree.
Let me know how you find the addition of community templates link in interactive menu, is that all right. Once picked, it throws an error, but I felt like making that not be an error is not worth it at the moment, and that it is clear enough.

Additional option in interactive menu:

Addition to docs:

infomiho · 2024-12-10T11:24:34Z

Great analysis @Martinsos!

I think you are right that if we concluded that a feature is not really useful (github: templates), we should remove it. We don't want to maintain features we don't use or even get stuck with a iteration of a feature that might be okay now, but later on with new info - we find it we want to reimplement it.
I dislike the community template option in the menu. Somebody will want to try out community templates and then they get an error. "Oh well, this guys have some sort of a bug". I'd rather invest some time into outputting some extra text to say "Hey, these are our official templates - check out the community templates as well" in some sort of "Footer" of the listed options.

I'll review the code next :)

infomiho

LGTM

Left some minor comments. I like the github: implementation - but - as I said, let's remove features we don't believe would be used by our users.

infomiho · 2024-12-10T11:25:45Z

waspc/ChangeLog.md

@@ -1,11 +1,15 @@
 # Changelog

-## Next
+## [WIP] 0.16.0


What was wrong with Next? 😢 I kinda liked it, you don't have to think about what the next version will be

Next is cool! But in this case I knew what next version is going to be, so I thought why not write it down? But yeah I could have left Next you are right, wouldn't be an issue. I can also roll with Next in the future.

infomiho · 2024-12-10T11:28:05Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject.hs

+        case waspAppAiGenerator of
+          WaspAI -> AI.createNewProjectInteractiveOnDisk absWaspProjectDir appName


Why did we generalise this, do we expect different generators in the future?

Maybe :D! Not likely though. The reason why I generalized it is that it allowed me to treat this template the same way as other templates, most importantly I could put it into the "FeaturedStarterTemplates.hs" file in a nice way. As an "overengineering" it is really a tiny step, but it allows me that + I found it ok that it is explicit this will use Wasp AI, I don't think that makes code more complex.

infomiho · 2024-12-10T11:30:52Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/ArgumentsParser.hs

@@ -33,7 +33,7 @@ parseNewProjectArgs newArgs = parserResultToEither $ execParserPure defaultPrefs
      Opt.strOption $
        Opt.long "template"
          <> Opt.short 't'
-          <> Opt.metavar "TEMPLATE_NAME"
+          <> Opt.metavar "TEMPLATE"


Do we maybe want to go with TEMPLATE_ID to keep it consistent with the record above?

I kind of didn't want to bother them with the details, I thought I will just put TEMPLATE and they will figure it out. TEMPLATE_ID sounded a bit low-level. I can put it thought if you think it is better DX / clearer.

infomiho · 2024-12-10T11:46:31Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/Skeleton.hs

+
+readWaspProjectSkeletonFiles :: IO [(Path System (Rel WaspProjectDir) File', Text)]
+readWaspProjectSkeletonFiles = do
+  skeletonFilesDir <- (</> [reldir|Cli/templates/skeleton|]) <$> Data.getAbsDataDirPath


I had trouble parsing this line at first, can write in way that it reads like a path i.e. from left to right?

Sure good idea, I did this:

skeletonFilesDir <- Data.getAbsDataDirPath <&> (</> [reldir|Cli/templates/skeleton|])

<&> is flipped <$>.

Btw this is old code actually, I just moved this file from one place to another. I commented on that when creating a PR, but I forgot to sumbit those comments :(.

infomiho · 2024-12-10T11:48:59Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/StarterTemplate.hs

+  }
+
+-- | AI generators that can generate Wasp apps.
+data WaspAppAiGenerator = WaspAI


Are we overengineerring here a bit? Are we working on other AI generators for Wasp or looking to support other AI external AI generators? Sorry if I missed that we are doing something like that.

Yes, that is my secret project :D! Kidding, it is a tiny bit of overengienerg that allowed me to treat AI starter template same as the rest while keeping compile time safety. Commented on it also in another comment in more details.

infomiho · 2024-12-10T11:51:47Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/StarterTemplateId.hs

+
+-- | Given a template id (as string), it will obtain the information on the template that this id references.
+-- It will throw if the id is invalid (can't be parsed, or information on the template can't be obtain based on it).
+getStarterTemplateByIdOrThrow :: (MonadError String m) => [ST.StarterTemplate] -> String -> m ST.StarterTemplate


Cool way of resolving the Github based templates 👍

Thanks! Will likely be throwing away most of this though.

Martinsos · 2024-12-08T23:23:45Z

waspc/ChangeLog.md


 ### ⚠️ Breaking Changes

 - Renamed and split `deserializeAndSanitizeProviderData` to `getProviderData` and `getProviderDataWithPassword` so it's more explicit if the resulting data will contain the hashed password or not.

+### 🎉 New Features and improvements
+
+- Added support for third-party starter templates via `wasp new -t github:<repo_owner>/<repo_name>[/dir/to/template]`.


TODO: I should remove this if we decide to drop -t github: support.

Martinsos · 2024-12-08T23:24:57Z

waspc/cli/exe/Main.hs

@@ -161,8 +161,7 @@ printUsage =
        title "  GENERAL",
        cmd   "    new [<name>] [args]   Creates a new Wasp project. Run it without arguments for interactive mode.",
              "      OPTIONS:",
-              "        -t|--template <template-name>",
-              "           Check out the templates list here: https://github.com/wasp-lang/starters",


This was pointing to a repo that contained only one or two starters, so I instead removed this, and you instead get more info on valid template names if you make a mistake while using -t, which is more maintainable and less crowded here in the top level cli info.

Martinsos · 2024-12-08T23:27:37Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/Common.hs

This is just refactoring, before we had one big file, I split it into multiple files.

Martinsos · 2024-12-08T23:27:53Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/FeaturedStarterTemplates.hs

Also just refactoring of splitting into multiple files.

Martinsos · 2024-12-08T23:28:20Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/Skeleton.hs

Also just refactoring of splitting into multiple files.

Martinsos · 2024-12-08T23:28:36Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/FeaturedStarterTemplates.hs

+    )
+
+{- HLINT ignore aiGeneratedStarterTemplate "Redundant $" -}
+waspAiGeneratedStarterTemplate :: ST.StarterTemplate


Wasp AI generator is also here now, and has its own normal entry which is better. Refactoring.

Martinsos · 2024-12-08T23:29:09Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/StarterTemplate.hs

Also just refactoring.

Martinsos · 2024-12-08T23:30:02Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/StarterTemplateId.hs

In this file I added the notion of github:.... So removing it would mostly come down to delegint big part of this file.

TODO: I should remove github: support from there if we end up not going with it.

Martinsos · 2024-12-08T23:30:37Z

waspc/cli/src/Wasp/Cli/Command/CreateNewProject/StarterTemplates/StarterTemplateId.hs

+  | IdTypeGhRepoTemplateUri
+  deriving (Enum, Bounded)
+
+-- | This function serves its purpose just by being defined, even if not used anywhere, because it


@sodic you might find this interesting as a type/compiler trick, we discussed sometihng similar some time ago, and I ended up using it here.

Martinsos · 2024-12-12T08:59:03Z

So when I wrote this PR I also wrote 9 comments for the reviewers to make reviewing easier, but they stayed on "pending"! Too bad, I pushed them now.

vincanger · 2024-12-12T10:23:21Z

Yeah @Martinsos I agree it's not really useful. What could be cool though is to have a section on our docs or webpage that features community templates to give those creators some recognition and perhaps inspire others to make their own.

Martinsos · 2024-12-12T11:04:29Z

web/docs/project/starter-templates.md

+
+
+## Community templates #{community-templates}


This id in #{} got rendered in docs! See what is with that.

What do you mean? How does it look like?

Martinsos added 10 commits December 4, 2024 23:06

Added support for external / 3rd-party starter templates.

e3ffedf

Reorganized and split into smaller modules.

8dbc612

Took care of some TODOs.

7b61e2e

More TODOs solved.

aaeda4f

fix

d0fc526

fix

5fec0d0

fix

9fc1ae0

fix

5c8e7ad

fix

2d60856

Added docs an interactive option.

35e71a8

Martinsos requested review from infomiho and sodic December 8, 2024 23:22

fix

d298377

infomiho approved these changes Dec 10, 2024

View reviewed changes

Martinsos commented Dec 12, 2024

View reviewed changes

fix

98a4d3d

Martinsos commented Dec 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support in Wasp for external/community starter templates #2402

Support in Wasp for external/community starter templates #2402

Martinsos commented Dec 8, 2024 •

edited

Loading

infomiho commented Dec 10, 2024

infomiho left a comment

infomiho Dec 10, 2024

Martinsos Dec 12, 2024 •

edited

Loading

infomiho Dec 10, 2024

Martinsos Dec 12, 2024

infomiho Dec 10, 2024

Martinsos Dec 12, 2024

infomiho Dec 10, 2024

Martinsos Dec 12, 2024

infomiho Dec 10, 2024

Martinsos Dec 12, 2024

infomiho Dec 10, 2024

Martinsos Dec 12, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos Dec 8, 2024

Martinsos commented Dec 12, 2024

vincanger commented Dec 12, 2024

Martinsos Dec 12, 2024

infomiho Dec 12, 2024

		case waspAppAiGenerator of
		WaspAI -> AI.createNewProjectInteractiveOnDisk absWaspProjectDir appName

Support in Wasp for external/community starter templates #2402

Are you sure you want to change the base?

Support in Wasp for external/community starter templates #2402

Conversation

Martinsos commented Dec 8, 2024 • edited Loading

infomiho commented Dec 10, 2024

infomiho left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martinsos Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martinsos commented Dec 12, 2024

vincanger commented Dec 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martinsos commented Dec 8, 2024 •

edited

Loading

Martinsos Dec 12, 2024 •

edited

Loading