How to create a custom MuPDF Device in Python

[Rodrigodd]

Is your feature request related to a problem? Please describe.

This would help implement a workaround for issue #3604, where I need to discover which clip mask is being applied to an image. One suggested alternative was to use mutool trace, which outputs the drawing commands as XML. However, using this method requires invoking an external tool and then parsing XML, which is undesirable. Alternatively, I could use the MuPDF C API directly to implement a derived class of Device, where I could manually keep track of clip masks as needed (see the implementation for trace-device for example).

Ideally, I would like to do all this in Python.

Describe the solution you'd like

Expose the methods fill_path, fill_image, clip_path, etc., of Device in the Python API. Allow subclassing it with custom overrides of these methods, and enable passing this custom class to Page.run and similar functions.

Describe alternatives you've considered

Implement what I need in C and write my own bindings for Python.

Additional context

The JavaScript API contains the methods of Device and allows passing a custom JavaScript object with the same methods as a Device, as per its documentation.

[JorjMcKie]

You can do that already now ... in Python!
Please be aware that MuPDF's functions are available via pymupdf.mupdf. So fight your way through that universe of possibilities.

[Rodrigodd]

Didn't know that, good to know! As a first try I run the following code:

import fitz

def hello():
    print('hello')

class MyDevice:
    def __init__(self):
        self.device = fitz.mupdf.fz_new_device_of_size(512) # I don't know the exact size
        self.device.fz_fill_path = hello
        self.device.fz_fill_image = hello
        self.device.fz_clip_path = hello

doc = fitz.open('drawing.pdf')
page = doc[0]
page.run(MyDevice(), fitz.Identity)

But it does not appear to work. I am not sure if this is even suppose to work, that I can override the functions pointers in fz_device with Python methods, and the python API automagically convert them to function pointers in the C side.

If you have any ideia, please let me know. But I will try to investigate this more at my side.

[julian-smith-artifex-com]

There is some information about this at: https://mupdf.readthedocs.io/en/latest/language-bindings.html#making-mupdf-function-pointers-call-python-code

You also might like to look at PyMuPDF's __init__.py, either in the Github source https://github.com/pymupdf/PyMuPDF/blob/main/src/__init__.py or in your own installed PyMuPDF.

Look for classes that inherit mupdf.FzDevice2() - these are custom devices written in Python.

Debugging custom devices can be tricky but hopefully the above information will help. Feel free to ask questions here or our discord channel (see top of any page on https://pymupdf.readthedocs.io/en/latest/)

[Rodrigodd]

Yeah, this works!

import fitz

def my_fill_path( dev, ctx, path, even_odd, ctm, colorspace, color, alpha, color_params):
    print('fill')

def my_fill_image( dev, ctx, image, ctm, alpha, color_params):
    print('image')

def my_clip_path(dev, ctx, path, even_odd, ctm, scissor):
    print('clip')

class MyDevice(fitz.mupdf.FzDevice2):
    def __init__(self):
        super().__init__()
        self.use_virtual_fill_path()
        self.use_virtual_fill_image()
        self.use_virtual_clip_path()

    fill_path = my_fill_path
    fill_image = my_fill_image
    clip_path = my_clip_path

doc = fitz.open('drawing.pdf')
page = doc[0]
# page.run(MyDevice(), fitz.Identity)
fitz.mupdf.fz_run_page(page.this, MyDevice(), fitz.mupdf.FzMatrix(), fitz.mupdf.FzCookie())

clip
image
clip
clip
image

I think this will be enough for me.

But I will still not close this issue, because it would be nicer if this was expose in the main pyMuPDF API (but fell free to close it if you disagree). Something like this, I believe:

import fitz

class MyDevice(fitz.Device):
    def fill_path(self, path, even_odd, ctm, colorspace, color, alpha, color_params):
        print('fill')
    
    def fill_image(self, image, ctm, alpha, color_params):
        print('image')
    
    def clip_path(self, path, even_odd, ctm, scissor):
        print('clip')
  
doc = fitz.open('drawing.pdf')
page = doc[0]
page.run(MyDevice(), fitz.Identity)

[JorjMcKie]

Let us put this in discussions - it no longer fulfils the criteria of an issue.

[JorjMcKie]

We might want to write recipes some day that implement tasks not available with the high level API.

[julian-smith-artifex-com]

Great, i'm glad you've got it working.

A couple of points:

• You can use def <fnname>(): ... inside the class definition, instead of things like fill_path = my_fill_path - we only use the latter way in __init__.py because some of the functions are reused in different devices.

• I agree it would be good to avoid the need to do things like self.use_virtual_fill_path(). I'd like to implement this in the C++ later so it's also available to the C# bindings, but i haven't figured out a way to do it efficiently yet. Maybe a Python-specific approach would be more practical, and certainly Python's introspection is much better.