Getting Started instructions aren't working

[DaveMeade]

I'm hoping someone can help me figure this out. I can't get the 'getting started from scratch' instructions to work.

• The instructions use import syntax for payload.config.js
• Thus node server.js errors out unless I set type: module in the package.json
• The instructions however use require() syntax for server.js - which will then error out
• I updated the server.js to use import syntax despite the instructions but then get
• Error [ERR_REQUIRE_ESM]: require() error in node_modules\payload\dist\config\load.js which is where I give up.

I do want to do this from scratch, but for what it's worth the create-payload-app tool didn't work for me either (constantly failing to find .ts files it didn't generate but still wanted somewhere, and misnaming js files as ts files).

[denolfe]

We've actually removed the "from scratch" instructions in the latest docs that are yet to be deployed because there's now some complex tsconfig configuration to allow for the best DX. I would recommend running create-payload-app and transfering all relevant files.

It sounds like you're running into issues that we've tried to mitigate by having create-payload-app. What issues were you facing when going that route?

[DaveMeade]

Sorry for the delayed response.

Initially, I couldn't get the create-payload-app to work. (I selected the blank-js option). This option still doesn't work for me.

I've done some troubleshooting on my side since then (and before reading your reply), including trying to get ts-node working in general. I don’t know if what I did matters, but now if I select the "blank" option the create-payload-app creates a working project. Selecting the blank-js option still does not work.

Unfortunately, even with a 'working' install, I find the project that is created very difficult to work with. I wanted to use payload because it seemed it would be easy to install into an existing express project I'm working on. That doesn't seem to be the case.

The created project rather forcibly imposes some very opinionated (and needless) views as to what my devops workflow/tooling should be. Perhaps it does this to ensure the generated docker-compose file will work, but that too is problematic.

Why include a docker-compose file and not the handful of additional lines required to add a mongo server which is also required to run? For that matter, why not docker-ize just mongo but not the payload app? At the very least there are some changes one might want to make here even just for basic 'getting it running'.

Unfortunately, the project breaks if one does too much customization on this set of docker assumptions. For example, maybe my project would be well suited to have a couple of custom dockerfiles, and perhaps even some startup scripts to initialize a mongo server etc (and, honestly, lord only knows what else one might include in their project).

No problem: I'll just create a /build folder drop some docker files in there with the scripts I need, script out the dockerfiles, tweak the docker-compose, and shazam - we're good to go. Right? No, unfortunately, this will break things in the app-generated project.

The project's built-in npm commands call some payload build process that ends up copying a whole bunch of USELESS files into the build directory. I have no idea what else this payload build process might be doing, so I've no idea if I can simply remove it.

Fundamentally - I don't need or want a CMS to force me to re-code my devops from the ground up. I use gulp, and I have a different set of docker paths/setups. If and when I need a /dist folder, I will script that.

All I need from payload is documentation on how payload interacts with express, and the BARE MINIMUM needed to install payload into an EXISTING project with its own existing set of devop tools, workflows, and resources already wired up.

[jmikrut]

Hey there - I can chime in on this.

create-payload-app

Your version of npx create-payload-app must be cached / out of date, because there is no longer an option for blank-js. Try running npx create-payload-app@latest to force npx to use the newest copy.

Setting up Payload from scratch

I think you're pointing to the fact that we should probably explain how to set up a Payload project from scratch once again in our docs. We omitted this because we felt like it wasn't relevant, but this is still 100% possible to do and honestly we don't impose anything on your dev workflow. You are free to take things and run with them however you want.

Really all you need is a few things:

• Import Payload into your Express server (import payload from 'payload') or require if using cjs
• Create a payload.config.ts (or js) file, and make sure that you have an env variable PAYLOAD_CONFIG_PATH that passes the path to where you place / name your Payload config
• Run payload.init() and pass it all required options, including your Express app

That's it.

The reason that we suggest using ts-node is that for most of our users, they'll want to use ts and they'll also want to have React components in their Payload config. And we also ship with nodemon because 99.9% of the time people will end up wanting that.

But, to have either TS or JSX in your Payload config (or anywhere in your project for that matter), you need to transpile your project somehow. So we suggest using ts-node. But you 100% don't need to. You could use babel/register or @swc/register before starting your server, or really whatever you want - as long as Node can parse your own files. Payload doesn't care.

Matter of fact, we actually don't have any specific requirements at all, besides maybe node version and the fact that we need a Payload config available and accessible either in the default location (./payload.config.js) or via an env variable.

Docker

RE: Docker, I just took a look at the docker-compose that ships with create-payload-app and it does indeed ship a Mongo server as well as a Node server. Are you seeing something different?

also, to note, is that the usage of a docker compose file that we ship is only needed if you want to. Personally, I’m not a docker guy. I don’t use it at all. It’s just there for reference and it’s a good starting point so that people can customize it as they need.

payload build

Now, for the payload build command, that is used to build the Payload admin UI and is absolutely necessary. Those files are not useless—those are the Payload admin files. I think that it's strange to instantly assume that files are useless but that's me.

I think there are some wires being crossed but ultimately Payload doesn't care at all about how you use it and the actual requirements are as minimal as we can get. What ships with create-payload-app are sensible defaults but you don't need any of it. You are free to use Gulp ;)

[DaveMeade]

Thanks @jmikrut I will try the above to see if I can get it running this weekend perhaps.

Now, for the payload build command, that is used to build the Payload admin UI and is absolutely necessary. Those files are not useless—those are the Payload admin files. I think that it's strange to instantly assume that files are useless but that's me.

I mean that they are useless in the /build directory I created to house dockerfiles. I have no doubt they are needed somewhere, but they are serving no purpose (being useless) in the /build folder. The package ships with default commands that have some sort of assumption relating to /dist folders and docker contexts that ended up putting those files someplace useless.

I do need some documentation to describe where to place payload build output ... but the app-generated project isn't helping by obfuscating that away behind some path assumptions that appear to somehow be related to docker context (?). My express app would have other endpoints, they might use ejs views, and/or have other middleware pieces. My output folder has some structure to it, and my dockerfiles are in a /build subfolder. The default commands of the app-created project are not cutting it (and frankly putting output in crazy/useless places); without 'from scratch' instructions I can't workout how to plug it all into my project.

[DaveMeade]

Ok so I went back to try and add payload to an existing project based on the above post.

I did the following:

• npm install --save --legacy-peer-deps payload
• npm install --save-dev cross-env (for setting PAYLOAD_CONFIG_PATH ala the app-generated package)
• updated server.js per: https://payloadcms.com/docs/getting-started/installation#adding-to-an-existing-app
  • I had to remove the async/await wrapper on the payload.init() command or it would give an error
• added a payload.config.ts to /src/ts (my project doesn't use ts otherwise)
• updated npm start command such that: "start": "cross-env PAYLOAD_CONFIG_PATH=src/ts/payload.config.ts node server.js",
• At this point nothing is calling a 'payload build' command, but nothing in the instructions for adding it to an existing project or the instructions above mention it - saying rather that it's time to fire it up and test. I do so, but I get:

import { buildConfig } from "payload/config";
^^^^^^
SyntaxError: Cannot use import statement outside a module

• The package.json does have "type": "module", and I'm using import syntax everywhere else just fine.
• It seems to be finding the payload.config.ts file ok.
• I'm not exactly sure why I'm getting this error.

Seperately:

In an attempt to work out on my own how to add in the payload build command, I added an npm script to test:
"payload-build": "cross-env PAYLOAD_CONFIG_PATH=src/ts/payload.config.ts payload build"

When I run npm run payload-build, it dumps a bunch of files in my projects /build folder (where they can do nothing for anyone in my project). When I saw this before I had believed it must have had something to do with the docker context - but it seems that's just where payload build puts things no matter what. This makes no sense in my project. /build is only used for docker stuff and my gulp workflow doesn't otherwise care about /build.

This is what I mean by opinionated - why can't I specify where this should go within my project? (My expectation is that it should go to the /views/admin folder since my express server uses ejs middleware, but in anycase - in my project - it certainly shouldn't go in /build.)

The stuff this command is creating here (in my build folder) are all just (admin) front end ui stuff (I think). My normal build workflows successfully find and copy such assets from other node_modules vendors over to my express app folder all the time, easily, and I can simply state where they should go ... there's no documentation on how to do that for payload that I can find, and the pre-packaged builder seems to just shoves things into the 'build' folder without any knowledge of how my project is actually structured.

So I have at least 2 blockers:

1. I'm still getting some sort of 'not a module' type of error
2. how do I state where the output of payload build should actually go?

[CallMeLaNN]

For number 1, I believe it require understanding of esm module in order to use "type": "module" for your project. payload build is clearly a commonjs cli and commonjs can't import esm module. Payload repo doesn't support esm yet. So until then, you can't use esm if you want to mix/merge the project into a node server. If I understand correctly, you will save a lot from above after moving back to use commonjs. I just came across to find the topic, just like this #1439

Probably your requirement is quite advanced or otherwise unique. It will be better to separate the projects so it will be clear to isolate the issue in payload vs your setup and have the freedom you need. This will also avoid complicated discussion. Please also reply/discuss the the right places here for clarity.

As we trying the payload, my opinion is if our setup closed to what was recommended and tested by the team, it will be easier for us in the future and easier reproducing any bug for them too. Anyway, otherwise it's our own choices and consequences - extra customization require extra effort and deeper understanding.

[ckerimoglu]

Was solution to this issue found maybe in the meantime? I am currently having the exact same problem as described in the first part.

import { buildConfig } from "payload/config";
^^^^^^
SyntaxError: Cannot use import statement outside a module