Skip to content

Commit

Permalink
[GR-47943] Backport: Update documentation to use standalone distribut…
Browse files Browse the repository at this point in the history
…ions and Maven dependencies.

PullRequest: graalpython/2975
  • Loading branch information
olyagpl committed Sep 20, 2023
2 parents 344fc63 + 65d7f1d commit 69b7b7a
Show file tree
Hide file tree
Showing 4 changed files with 99 additions and 35 deletions.
2 changes: 2 additions & 0 deletions docs/user/Interoperability.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ permalink: /reference-manual/python/Interoperability/
---
# Interoperability

The best way to embed GraalPy is to use the [GraalVM SDK Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html).

## The Polyglot API

Since GraalVM supports several other programming languages including JavaScript, R,
Expand Down
2 changes: 1 addition & 1 deletion docs/user/PythonNativeImages.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ These are:
Another useful option to reduce the size of the native executable is to omit a pre-initialized Python context from the executable.
By default, a default Python context is already pre-initialized and ready for immediate execution.
In embeddings that use a custom polyglot engine to allow context sharing, the pre-initialized context cannot be used, however.
It can be omitted by explicitly passing
It can be omitted by explicitly passing:

```bash
-Dimage-build-time.PreinitializeContexts=
Expand Down
11 changes: 4 additions & 7 deletions docs/user/PythonStandaloneBinaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,11 @@ permalink: /reference-manual/python/standalone-binaries/
With GraalPy, you can distribute Python applications or libraries as standalone binaries or JAR files without any external dependencies.
The [Truffle framework](https://github.com/oracle/graal/tree/master/truffle) on which GraalPy is built, and the [Sulong LLVM runtime](https://github.com/oracle/graal/tree/master/sulong) that GraalPy leverages for managed execution of Python's native extensions enables users to completely virtualize all filesystem accesses of Python programs, including those to the standard library and installed packages.

GraalPy comes with a module that can create standalone binaries or Java-Python polyglot project skeletons.
The binaries bundle everything into one native executable.
The polyglot skeletons are set up with Maven to to generate a standalone binary for a simple java - python hello world example and can be used as a starting point or inspiration for further java-python polyglot development.
GraalPy comes with a module that can create Python binaries for Linux, Windows, and macOS.
The binaries bundle everything into a single-file native executable.

### Prerequisite

A GraalPy distribution.
TODO - add link
The tool can also generate a skeleton Maven project that sets up a polyglot embedding of Python packages into Java.
The polyglot skeletons are set up with Maven to to generate a standalone binary for a simple Java-Python HelloWorld example and can be used as a starting point or inspiration for further Java-Python polyglot development.

## Creating GraalPy Native Binaries

Expand Down
119 changes: 92 additions & 27 deletions docs/user/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,51 +9,94 @@ redirect_from: /docs/reference-manual/python/
# GraalVM Python Runtime

GraalPy provides a Python 3.10 compliant runtime.
A primary goal is to support PyTorch, SciPy, and their constituent libraries, as well as to work with other data science and machine learning libraries from the rich Python ecosystem..
A primary goal is to support PyTorch, SciPy, and their constituent libraries, as well as to work with other data science and machine learning libraries from the rich Python ecosystem.
GraalPy can usually execute pure Python code faster than CPython, and nearly match CPython performance when C extensions are involved.
While many workloads run fine, any Python program that uses external packages could hit something unsupported.
At this point, the Python implementation is made available for experimentation and curious end-users.
See the [FAQ](FAQ.md) for commonly asked questions about this implementation.

## Installing GraalPy
## GraalPy Distributions

As of GraalVM for JDK 21, the Python runtime (GraalPy) is available as a standalone distribution.

### Linux and macOS
A GraalPy standalone built on top of Oracle GraalVM (Oracle GraalPy) is licensed under the [GraalVM Free Terms and Conditions (GFTC)](https://www.oracle.com/downloads/licenses/graal-free-license.html) license, which permits use by any user including commercial and production use. Redistribution is permitted as long as it is not for a fee.
Oracle GraalPy provides the best experience: it comes with additional optimizations, is significantly faster and more memory-efficient.

The easiest way to install GraalPy on Linux and macOS platforms is to use [pyenv](https://github.com/pyenv/pyenv/), the Python version manager.
For example, to install version 22.3.0, for example, run the following commands:
A GraalPy standalone built on top of GraalVM Community Edition (GraalPy Community) is fully open-source.
To distinguish between them, GraalPy Community has the suffix `-community` in the name.

```bash
pyenv install graalpy-22.3.0
pyenv shell graalpy-22.3.0
# Oracle GraalPy
graalpy-VERSION-PLATFORM.tar.gz
# GraalPy Community
graalpy-community-VERSION-PLATFORM.tar.gz
```

Another option is to use [Conda-Forge](https://conda-forge.org/).
To get an environment with the latest version of GraalPy, use the following command:
Two language runtime options are available for both Oracle and Community GraalPy: Native and JVM.
In the Native configuration, GraalPy is ahead-of-time compiled to a standalone native executable.
This means that you do not need a JVM installed on your system to use it and it is size-compact.
In the JVM configuration, you can use Java interoperability easily, and peak performance may be higher than the native configuration.
A JVM standalone that comes with a JVM has the `-jvm` suffix in a name: `graalpy-jvm-<version>-<os>-<arch>.tar.gz`.

| Configuration: | Native (default) | JVM |
| ------------------ | ---------------: | ------------: |
| Time to start | faster | slower |
| Time to reach peak performance | faster | slower |
| Peak performance (also considering GC) | good | best |
| Java host interoperability | needs configuration | works |

## Installing GraalPy

You can install GraalPy either using a Python manager or downloading a compressed GraalPy tarball appropriate for your platform.

### Downloading

1. Navigate to [GitHub releases](https://github.com/oracle/graalpython/releases/) and select a desired standalone for your operating system.
2. Uncompress the archive:

> Note: If you are using macOS Catalina and later, first remove the quarantine attribute:
```shell
sudo xattr -r -d com.apple.quarantine <archive>.tar.gz
```
Now extract:
```shell
tar -xzf <archive>.tar.gz
```
Alternatively, open the file in the Finder.
3. Check the version to see if the runtime is active:
```shell
./path/to/bin/graalpy --version
```

### Using `pyenv`
#### Linux and macOS

The other way to install GraalPy on Linux and macOS platforms is to use [pyenv](https://github.com/pyenv/pyenv/), the Python version manager.
For example, to install version 23.1.1, run the following command:

```bash
conda create -c conda-forge -n graalpy graalpy
# To install Oracle GraalPy
pyenv install graalpy-23.1.1
# To install GraalPy Community
pyenv install graalpy-community-23.1.1
```

Alternatively, [download](https://github.com/oracle/graalpython/releases) a compressed GraalPy installation file appropriate for your platform.
For example, for Linux, download a file that matches the pattern _graalpy-XX.Y.Z-linux-amd64.tar.gz_.
Uncompress the file and update your PATH variable as necessary.
If you are using macOS Catalina or later, you may need to remove the quarantine attribute.
To do this, run the following command:
Another option is to use [Conda-Forge](https://conda-forge.org/).
To get an environment with the latest version of GraalPy, use the following command:

```bash
sudo xattr -r -d com.apple.quarantine /path/to/GRAALPY_HOME
conda create -c conda-forge -n graalpy graalpy
```

To try GraalPy with a full GraalVM, including support for Java embedding and interoperability with other languages, you can use the bundled releases from [www.graalvm.org](https://www.graalvm.org/downloads/).
#### Windows

### Windows
There is a GraalPy preview build for Windows that you can [download](https://github.com/oracle/graalpython/releases/).
It supports installation of pure Python packages via `pip`. Native extensions are a work in progress.

There is a preview binary build of Windows that you can download via [www.graalvm.org](https://www.graalvm.org/downloads/).
The Windows build has several known issues:

- JLine treats Windows a dumb terminal, no autocomplete and limited editing capabilities in the REPL
- Interactive help() in the REPL doesn't work
- Oracle GraalPy builds cannot create venvs or install packages, use community GraalPy builds to do those things.
- Inside venvs:
- graalpy.cmd and graalpy.exe are broken
- pip.exe cannot be used directly
Expand Down Expand Up @@ -127,13 +170,35 @@ These can be viewed using the following command:
graalpy --help --help:tools --help:languages
```

## Native Image and JVM Runtime
## Interoperability with Java

The best way to embed GraalPy is to use the [GraalVM SDK Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html).

As of GraalVM for JDK 21, all necessary artifacts can be downloaded directly from Maven Central.
All artifacts relevant to embedders can be found in the Maven dependency group [`org.graalvm.polyglot`](https://central.sonatype.com/namespace/org.graalvm.polyglot).

To embed GraalPy into a Java host application, add GraalPy as a Maven dependency or explicitly put the JAR on the module path. Below is the Maven configuration for a Python embedding:
```xml
<dependency>
<groupId>org.graalvm.polyglot</groupId>
<artifactId>polyglot</artifactId>
<version>23.1.0</version>
</dependency>
<dependency>
<groupId>org.graalvm.polyglot</groupId>
<artifactId>python</artifactId>
<version>23.1.0</version>
<scope>runtime</scope>
<type>pom</type>
</dependency>
```

The `<scope>runtime</scope>` parameter is only necessary if you need the runtime dependency.

By default, GraalVM runs GraalPy from a binary, compiled ahead-of-time with [Native Image](https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/README.md), yielding faster startup time and lower footprint.
Although the ahead-of-time compiled binary includes the Python and LLVM interpreters, to interoperate with
other languages you must supply the `--jvm` option.
This instructs the launcher to run on the JVM instead of in Native Image mode.
Thus, you will notice a longer startup time.
Depending on which supported JDK you run embedded GraalPy, the level of optimizations varies, as described [here](https://www.graalvm.org/reference-manual/embed-languages/#runtime-optimization-support).

Learn more in a dedicated [GraalPy Interoperability guide](Interoperability.md). See also the [Embedding Languages documentation](https://www.graalvm.org/reference-manual/embed-languages/) on how a guest language like Python can possibly interract with Java.

## Related Documentation
* [Installing Supported Packages](Packages.md)

* [Installing Supported Packages](Packages.md)

0 comments on commit 69b7b7a

Please sign in to comment.