ndv live view how-to guide

[jacopoabramo]

• ndv version: latest branch
• Python version: 3.10
• Operating System: Windows 11

Description

I want to stream continously to ndv from multiple channels. I'm starting with a single channel in this case (a synthetic data generator using openwfs). On my system it works (this time I installed pyqt) but the display is a bit wobbly. I'm attaching a video example.

continous-example.mp4

I'm fairly certain that I can do better, maybe change the timer timeout signal or something.

What I Did

Reference script attached

continous_example.py.txt

Requirements

latest ndv branch[pyqt,vispy]
openwfs

[gselzer]

Heh, I was trying to do the same thing a few months back - it's a performance problem, and we should try to rectify this (cc @tlambert03). I think it was some problem around the data wrapper being recreated - when you clear the old images, it removes and repaints the data.

A simple fix is to reuse the data wrapper by providing a buffer to the viewer that you update, instead of providing a new buffer every time. I modified your file to add this:

continous_example.py.txt

Note also that I added a few functions to your SyntheticManager class. You may obtain better performance by calling on_timer when your QTimer times out.

[jacopoabramo]

I was thinking that the solution was relatively simple but I checked the source code for _data and the setter raises an exception, so I thought that maybe I was missing something.

[tlambert03]

yep, just want to chime in here and say the fast streaming in ndv has so far taken a back seat to higher dimensional viewing. however, if we can make some assumptions about the data coming in (for example, if we know that we'll only ever be looking at one frame at a time, and we can always just update the "current" buffer), then I'm sure we can get this to be as fast as vispy itself can be by just directly passing the incoming data buffer to the vispy texture

[jacopoabramo]

yep, just want to chime in here and say the fast streaming in ndv has so far taken a back seat to higher dimensional viewing. however, if we can make some assumptions about the data coming in (for example, if we know that we'll only ever be looking at one frame at a time, and we can always just update the "current" buffer), then I'm sure we can get this to be as fast as vispy itself can be by just directly passing the incoming data buffer to the vispy texture

That would be my approach. In the end I will be forcing the refresh rate around 30/60 Hz for the visualization since I don't need to play cyberpunk. I will be using psygnal so I'm detached from the Qt event loop and the only Qt stuff I'll be using are for widgets. One thing I'm considering is if it's worth to combine msgpack with psygnal in order to have faster transmission but my gut feeling is that I would make it worse.

[jacopoabramo]

@gselzer just a question, is the @ensure_main_thread decorator strictly necessary?

[gselzer]

@gselzer just a question, is the @ensure_main_thread decorator strictly necessary?

Uh, no, it isn't. I saw errors earlier on in my debugging about QObject parents and threads, so I threw it in. That ended up not being the fix, but I forgot to remove it. You can delete it.

[tlambert03]

btw, @jacopoabramo, this is the "simple image preview" widget that we've been using in pymmcore-widgets:

https://github.com/pymmcore-plus/pymmcore-widgets/blob/main/src/pymmcore_widgets/views/_image_widget.py

much faster than ndv for this use case, but doesn't have the other features. so, need to merge the two. but it might give you additional ideas

[jacopoabramo]

I'm tempted to play around with vispy directly at this point but I wanted to explore the option of ndv first - I'm lazy at that, but I also like to support projects by being a customer and providing feedback.

[gselzer]

I'm tempted to play around with vispy directly at this point

@jacopoabramo what are you looking for in using vispy directly?

[jacopoabramo]

@jacopoabramo what are you looking for in using vispy directly?

The most critical thing I would like to have as soon as possible is the first point I mentioned in #60, and that would be viewing datasets adjacently.

[gselzer]

The most critical thing I would like to have as soon as possible is the first point I mentioned in #60, and that would be viewing datasets adjacently.

I mean... if you want to take a stab at it, I think a PR here would be very welcome! 😉

[jacopoabramo]

The most critical thing I would like to have as soon as possible is the first point I mentioned in #60, and that would be viewing datasets adjacently.

I mean... if you want to take a stab at it, I think a PR here would be very welcome! 😉

I mentioned it to @tlambert03 that I would be willing to help though limited I am in my capacity; if you can give me some pointers on where to start looking for that specific feature I can have a look. I was honestly waiting for the MVC V2 API before giving a crack to it; I don't like it too much when too many features confluence into a single branch (past experiences taught me that) but if it's something that can be somehow pushed in the new model I can try to help

[tlambert03]

I was honestly waiting for the MVC V2 API before giving a crack to it

yeah... i was gonna say this might be a good idea. I definitely don't want to squelch any potential enthusiasm for contributions!! Nor do I want to keep important features on hold while we work on a refactor... But i also don't want to frustrate you by having work you spend time on have to be very thoroughly reworked in the near (?🤞) future. If you do try to add something, let's just be very sure to add a test for any public APIs that you may want to ensure keep working after the refactor

thank you!

[jacopoabramo]

I was honestly waiting for the MVC V2 API before giving a crack to it

yeah... i was gonna say this might be a good idea. I definitely don't want to squelch any potential enthusiasm for contributions!! Nor do I want to keep important features on hold while we work on a refactor... But i also don't want to frustrate you by having work you spend time on have to be very thoroughly reworked in the near (?🤞) future. If you do try to add something, let's just be very sure to add a test for any public APIs that you may want to ensure keep working after the refactor

thank you!

I absolutely agree. I had my share of experience with overenthusiastic contributions which I just did not have the heart to block and I don't want to fall victim of the same situation.