Fixing lag when first accessing an image's RGBA values

[tbell5]

Note: the solution I point out here is unnecessary, specifically the use of the pierrre/imageutil library. See the comment by @Zyko0 below. All you need to do is to use an image.Image object for retrieveing the Alpha values instead of an *ebiten.Image

Hi All,

I was about to write a post to get help with the problem mentioned in the title, but I was able to find an effective solution, so I figured I would share my problem/solution, as it could be a common problem (or maybe not, don't hesitate to let me know if this is an error on my part or has already been addressed, I'm still a noob with both go and ebiten). I should mention I'm working on a 2015 MacBook Pro, in case that's relevant... I haven't tested whether the problem occurs on any other platforms.

Anyway, my problem started when implementing drag-and-drop functionality via the method in this example. That method uses the following code to determine if a user clicks on a given sprite (comments are from the example, not my own):

// In returns true if (x, y) is in the sprite, and false otherwise.
func (s *Sprite) In(x, y int) bool {
	// Check the actual color (alpha) value at the specified position
	// so that the result of In becomes natural to users.
	//
	// Note that this is not a good manner to use At for logic
	// since color from At might include some errors on some machines.
	// As this is not so important logic, it's ok to use it so far.
	return s.image.At(x-s.x, y-s.y).(color.RGBA).A > 0
}

This works great most of the time, but in the game I was making, sprite.At() caused a significant lag of 1-2 seconds the first time it was being called (i.e. when the user first clicks). This issue doesn't occur in the example, but my hypothesis is that it occurs if you have more image data being loaded, as this may cause the sprite images to be un-cached (or something like that, I'm out of my depth here).

The solution I found was using this library by @pierrre https://github.com/pierrre/imageutil. An in-depth discussion and writeup can be found here golang/go#15759. The gist of it is that the library can be used to explicitly load the RGBA value from an image into memory using myAtFunc := imageutil.NewAtFunc(img), then can instantly access these value anytime afterwards using myAtFunc(x, y). Thus the problem of unexpected lag is fixed, as you can load the RGBA value before the game starts.

My solution as applied to the Ebiten drag-and-drop code is shown below. A few specific sticking points to mention: first, unlike image.At(x,y), the AtFunc(x,y) call from the pierre/imageutil library will cause an error if the (x,y) coordinates are out of bounds for the image, so you need to explicitly check that they are within the image's width/height bounds beforehand. Second, the call to imageutil.NewAtFunc(img) needs to be passed a golang image object; if you pass it the *ebiten.Image, the lag problem still persists, at least in my code (I'm not sure why this is; perhaps because you need to pass the image object itself, and not a pointer? I don't know enough about the image library or the ebiten implementation to really say).

import(
    ...
    "image"
    _ "image/png"

    "github.com/hajimehoshi/ebiten/v2"
    "github.com/pierrre/imageutil"
    ...
)

var (
    spriteImage *ebiten.Image

    at imageutil.AtFunc()
)

func (s *Sprite) In(x, y int) bool {
    sWidth, sHeight = spriteImage.Size()
    dx, dy = x-s.x, y-s.y

    if dx < 0 || dx >= sWidth || dy < 0 || dy >= iHeight {
        return false
    }

    _, _, _, A := at(dx, dy)
   return A > 0
}

func main() {
    img, _, err := image.Decode(...) 
    if err != nil {
        ...
    }
    // initialize the at function
    at = imageutil.NewAtFunc(img)
    ballImage = ebiten.NewImageFromImage(img)
}

Anyway, hope this helps someone in the future, let me know any thoughts/errors/corrections. Cheers!

edit 1: "pierrre", not "pierre"

[Zyko0]

The lag you're mentioning certainly comes from .At() being called on an *ebiten.Image.
For you to know, whenever you call .At() on an ebiten image, there is a backend call to retrieve the color value of the pixel from GPU which costs time, because it's not directly available in memory.

If I were you (with or without the suggested library), I would keep another copy of your image as a go image and perform alpha checks on this one.
Try to store img variable from your main function as an image.Image in your sprite struct for the checks, and the ebiten.Image for drawing 😄

If this is a behavior you want to keep for your sprite, you can also declare a slice or a different kind of storage for alpha values to be retrieved by x,y also.

[tbell5]

Ohhh, I see. I figured I was missing something. I can confirm, when calling At() on an image.Image, as opposed to ebiten.Image, there is no lag. So actually my solution was only working because I was circuitously calling At() on an image.Image through the use of that imageutil library.

So my solution here is moot; I'll make a note of that at the top of the post. Thanks for your help!

[Zyko0]

You're welcome ! 😄