Is [partial] sequential plugin execution possible?

[atomicpages]

Summary

How can you wait for createNode from one plugin to complete before starting the execution of another plugin?

Relevant information

In my case, I want to wait for gatsby-source-filesystem to complete before starting so I have the full list of sourced files that I can delegate to another plugin for post-processing. I found a few leads in the docs like getNodesByType or getNodes but the issue is the map series which allows for parallel execution of all the plugins.

1. Is is possible to wait for one plugin to complete before starting the execution of another?
2. If not, can I make a GraphQL query inside of onCreateNode?
   Note: I'm guessing no since schemas are generated after this lifecycle hook

Unfortunately the plugin needs access to the list of files and ideally I'd like to source them from gatsby-source-filesystem to avoid adding something like globby

Other wild ideas included:

• Seeing is there's an event emitted I can wait for

• Using something like globby and let the user of the plugin specify where to read files from:

  {
    resolve: `my-plugin`,
    options: { files: `src/**/*` }
  }

File contents

gatsby-config.js:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: "pages",
        path: `${__dirname}/src/pages/`,
      },
  },
  "my-plugin"
  ]
};

my-plugin/gatsby-node.js:

exports.onCreateNode = ({ actions }, options) => {
  // do something
};

package.json: N/A
gatsby-node.js: N/A
gatsby-browser.js: N/A
gatsby-ssr.js: N/A

[pieh]

Hi @atomicpages

the issue is the map series which allows for parallel execution of all the plugins.

If you are referring to

gatsby/packages/gatsby/src/utils/api-runner-node.js

Line 525 in 0db2074

Promise.mapSeries(implementingPlugins, plugin => {

Then map series is actually not serial - it's sequential

But, it depends on API you would to use - with sourceNodes you could order the plugin in gatsby-config to run after gatsby-source-filestem and then File nodes should all be created when sourceNodes of your plugin is called. This is not the case with onCreateNode because those do run as soon as nodes are created (so before sourceNodes is finished).

Can you talk more about what specifically you are trying to do with your plugin? Maybe we can figure out a way to make it work. Need of access to File nodes reminds me of remark plugin case where it needs access to those - but it can defer that to query running (and not data sourcing), so at the time of running queries all File nodes are guaranteed to be there (this might or might not work for your case)

[atomicpages]

@pieh thanks for the reply! I was really probing to see if there was an option to synchronously execute (some or all) plugins.

Of course! I'm building a typescript plugin for Gatsby, but unlike other tools where we can use loadNodeContent the ts compiler wants file paths

Edit:

I'm trying to build a plugin that uses react-docgen-typescript instead of react-docgen. There are two issues:

• The issue with the ts compiler is it wants file paths instead of file contents which by itself is not really an issue
• Performance of tsc (this is where issues arise)

Most of the code from gatsby-transformer-react-docgen can be recycled with the exception of:

gatsby/packages/gatsby-transformer-react-docgen/src/on-node-create.js

Lines 110 to 114 in 7688f59

	const content = await loadNodeContent(node)
	let components
	try {
	components = parseMetadata(content, node, pluginOptions)

I tried something similar with react-docgen-typescript that leads to a dizzying array of crazy errors:

try {
  const ast = parseMetadata(node.absolutePath);
} catch (e) {
  reporter.error(/* ... */);
}