v1.0.0-beta8.4

Highlights

• Replace Chromium with the new "Chrome for Testing" distribution. #53
• Support node 20
• Drop node 14
• Update some dependencies

Updating

Update to the latest version by running:

npm update -g ayakashi

v1.0.0-beta8.3

Highlights

• Support node 16 & 18
• Drop node 12
• Support generating and running typescript projects. More info here
• New headless chromium detection patches that pass all current tests
• New commands to update subcomponents
  • update-chrome Improved chromium downloader. See command options here
  • update-ua Updates the builtin database of user agent strings
  • update-stealth Updates the headless chromium detection patches
• Documentation moved to github pages here: https://ayakashi-io.github.io

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running
npm --save-dev @ayakashi/types

v1.0.0-beta7.2

Highlights

• add support for node 14
• drop support for node 8, 10 and 13
• dependency updates

Updating

Update to the latest version by running:

npm update -g ayakashi

A new recommended Chromium revision is also available. It can be updated by running
ayakashi update-chrome

v1.0.0-beta7.1

Highlights

• support node 13
  • LTS versions (10 or 12) are still the recommended versions
• trackMissingChildren() will now cascade to nested child props
• some type improvements

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running
npm --save-dev @ayakashi/types

A new recommended Chromium revision is also available. It can be updated by running
ayakashi update-chrome

v1.0.0-beta7

A maintenance release focusing mainly on bug fixes, small improvements and test coverage.

Highlights

• support node 12
• evaluate()
  • parameter and return types are now fully inferred
  • improved serialization allows passing and returning complex objects and functions
  • evaluateAsync() now supports async functions
• support default exports on actions, preloaders, extractors and props
• type improvements
• new integration test suite
• many bug fixes

Updating

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running
npm --save-dev @ayakashi/types

v1.0.0-beta6

• Cookies are now automatically shared between page loads, renderless loads and HTTP requests. New cookie methods for manual manipulation have also been added. Read more here
• Persistent sessions have been completely revamped. They now allow chromium contexts, cookies and other scraper settings to be shared between multiple steps of a pipeline. Read more here
• User agents have been improved and they now include more settings to tweak. The options have been moved to the emulatorOptions section of the config
• New workers and workerConcurrency config options have been added to allow manual control of the thread count and concurrency of the system
• domQL data attribute querying is now more flexible. Read more here
• Simple mode is now available for renderlessScrapers and apiScrapers with the --simpleRenderless and --simpleApi CLI flags
• Bug fixes and improvements

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running
npm --save-dev @ayakashi/types

v1.0.0-beta5

• The extraction API has been revamped
  • extract() will now return just the extracted values instead of wrapper objects.
    Learn more in the updated documentation
  • HTML attributes and data-attributes are now supported
  • Multiple extractions in the same extract() call and nested extractions have been removed
  • New methods extractFirst() and extractLast() can be used to extract a single value of the first or the last match respectively
  • New method join() can be used to properly group multiple sets of related data
  • Inferred return types have been greatly improved
• The builtin scripts have been updated to reflect the new extraction semantics
• Primitive values can now be yielded as output from scrapers or scripts without the need to be wrapped in objects
• Random protocol and bridge ports can be used by setting them to 0 in the config
• Multiple projects can now be run at the same time without interfering with each other
• Multiple resumable sessions of the same project can exist and run (even at the same time) by using the
  --sessionKey CLI argument

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running
npm --save-dev @ayakashi/types

v1.0.0-beta4

• Add resume support. Projects can now be resumed in case of interruption or failure.
  Documentation: https://ayakashi.io/docs/reference/cli-commands.html#run
• Add the .loadHtml() method in renderlessScrapers.
  Documentation: https://ayakashi.io/docs/guide/renderless-scrapers.html
• Add support for script generators.
  Documentation: https://ayakashi.io/docs/going_deeper/script-generators.html
• Request methods (.get(), .post() etc) are now available in all scraper types.
  Documentation: https://ayakashi.io/docs/guide/api-scrapers.html
• Improve memory usage for long running operations.
• Fixes and improvements

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm update.

v1.0.0-beta3.1

• Add the --jsonConfig CLI argument which allows passing a config as a json string to ayakashi run. Learn more here: https://ayakashi.io/docs/reference/cli-commands.html#run-arguments

Update to the latest version by running:

npm update -g ayakashi

v1.0.0-beta3

• Each scrapper invocation now runs in its own browser context (like an incognito window) unless persistentSession is on or not running in headless mode.
• Add support for renderless scrappers. Learn more here: https://ayakashi.io/docs/guide/renderless-scrappers.html
• Add support for API scrappers. Learn more here: https://ayakashi.io/docs/guide/api-scrappers.html
• saveToSQL script will now re-use the database connection and won't create a new one each time it is run.
• Add recursiveYield() and recursiveYieldEach() methods, which allow to recursively re-run the scrapper by yielding the extracted data to itself.
• domQL tagName queries will now accept both lowercase and uppercase tag names.

Update to the latest version by running:

npm update -g ayakashi

A new @ayakashi/types package is also available. It can be updated in existing projects by running npm update.