StreamResource doesn't work with input stream from IP camera

[F-Kamp]

Description of the bug

Hello,

I'm trying to consume a mjpg stream from an AXIS P1245 MK II IP camera, create a StreamResource of it and then distribute that camera stream into multiple browser instances using a single StreamResource for the source of an Image element (I don't want every browser instance to connect to the camera directly).

Unfortunately the underlying input stream is immediately closed with the following exception:

java.io.IOException.txt

I've thought of the following possible issues:

1. The http connection to the camera could be wrong
   To check this, I've entered the connection URL in the browser. The browser displays the camera stream without any problems.

2. My logic to connect to the camera using Kotlin code could be wrong
   To check this, I've added a background task that connects to the camera and keeps reading the input from it. This loop keeps running and proves that the connection logic works.

3. The content of the stream might be wrong (i.e. can't be parsed into jpg images)
   Using a custom library (https://github.com/sarxos/webcam-capture/tree/master) to parse the mjpg input stream content into individual jpgs works and I can show individual images

Since all of the above possibilities have been checked and disproven, I've come to the conclusion that something Vaadin-related must be faulty.
Experimentally I've also tried to use the URL to the camera stream directly as Image src. This works only in Firefox; not in Chrome or Edge. When I inspect the generated HTML and compare it to the HTML when I enter the camera's stream URL directly into the browser I can't find any differences (apart from some styling). But within the Vaadin context the stream isn't displayed at all.

Please note, I don't want to set the Image's URL directly to the camera in production. That was only to test if there are any problems accessing the camera's stream when running within the Vaadin context (which has been confirmed).

Also, to remove any kind of side-effects with my existing application I've run these tests in a skeleton-start-kotlin-spring project.

Expected behavior

StreamResource should be able to handle an InputStream where data is added continuously. It should be possible to use that StreamResource as the source for Image instances in different UI instances.

Minimal reproducible example

@Route
class MainView : VerticalLayout() {

    private val logger = KotlinLogging.logger { }

    private val url = "http://${Secrets.USER}:${Secrets.PASSWORD}@${Secrets.IP}/axis-cgi/mjpg/video.cgi"
    private val imgHolder = Image("", "uninitialized")

    init {

        val streamResource = StreamResource("test-image", InputStreamFactory { createStream() })

        // doesn't work at all
        add(Image(streamResource, "error-alt-text"))

        // works in Firefox, but not in Chrome & Edge
        add(Image(url, "error-alt-text2"))


    }

    override fun onAttach(attachEvent: AttachEvent) {
        super.onAttach(attachEvent)

        // works (i.e. the stream remains open and content keeps being pushed into the stream)
        Thread { openAndReadBytes() }.start()

        // works but is obviously a very bad way to do this
        parseAndShowIndividualImagesAsync(attachEvent.ui)
    }

    /**
     * To test whether opening the stream and reading images from it works at all, we try to parse the stream manually,
     * extract the images from it and then update an Image's source whenever we receive new data from the stream.
     * Parsing of the stream is handled by https://github.com/sarxos/webcam-capture/tree/master
     */
    private fun parseAndShowIndividualImagesAsync(ui: UI) {
        add(imgHolder)

        Thread {
            try {

                // Uses https://github.com/sarxos/webcam-capture/blob/master/webcam-capture-drivers/driver-mjpeg/src/main/java/com/github/sarxos/webcam/ds/mjpeg/MjpegCaptureDevice.java
                // and https://github.com/sarxos/webcam-capture/blob/master/webcam-capture/src/main/java/com/github/sarxos/webcam/util/MjpegInputStream.java

                val device = MjpegCaptureDevice(URI.create(url).toURL())
                device.open()
                var counter = 0
                while (this@MainView.isAttached) {

                    device.image?.let {
                        val byteArrayOutputStream = ByteArrayOutputStream()
                        ImageIO.write(it, "jpg", byteArrayOutputStream)
                        val bytes: ByteArray = byteArrayOutputStream.toByteArray()
                        ui.access {

                            imgHolder.setSrc(
                                StreamResource(
                                    "Img${counter++}",
                                    InputStreamFactory { bytes.inputStream() })
                            )
                        }
                    }
                }
            } catch (ex: Throwable) {
                logger.error(ex) { "failure" }
            }
        }.start()
    }

    /**
     * Tests whether the stream remains open and keeps receiving content when running in the background
     */
    private fun openAndReadBytes() {
        createStream().use { inputStream ->
            while (this@MainView.isAttached) {
                val byte = inputStream.readNBytes(80000)
                logger.info { "Read ${byte.size} bytes" }
            }
        }
    }

    private fun createStream(): InputStream {
        logger.info { "Opening stream..." }
        val url = URI.create(url).toURL()
        val connection = url.openConnection()
        val stream = connection.getInputStream()
        logger.info { "Created stream" }
        return stream
    }

}

Versions

• Vaadin / Flow version: 24.5.8
• Java version: 17
• OS version: Windows 11
• Browser version (if applicable): Chrome 131.0.6778.140, Firefox 133.0.3, Edge 131.0.2903.99
• Application Server (if applicable): Tomcat from spring-boot-starter-parent v3.3.6
• IDE (if applicable): Intellij

[mshabarov]

Two thoughts on this topic:

1. Whether the width and height properties are involved in the error here, because Vaadin apparently doesn't use them.
2. Can we get some public .cgi url with which we can test the app on our side.

[mshabarov]

Some references that may be helpful on this topic:

• https://vaadin.com/blog/efficiently-serving-video-files-in-java-web-apps-with-http-range-requests
• https://vaadin.com/directory/component/vcamera

Ping @mstahv if he has any insights about using StreamResource with video streams.

[F-Kamp]

We've been able to provide access to the camera: http://motives-lb.de/axis-cgi/mjpg/video.cgi

It currently points to the display of a humidity sensor in our office, so don't expect frequent changes. Also there's a red-light filter in front of the camera, so colors are a bit off.

The image's width and height are irrelevant as far as I can tell; I can see the image's error texts instead of the webcam content. And I can see that the underlying connection is killed when I try to consume it as a StreamResource.

[F-Kamp]

I've taken a look at your links @mshabarov and it certainly sounds like I'm missing something to make the stream work. An additional obstacle might be that my real application is running on WildFly (and not Spring) - so any examples that detail how to get it to work with Spring won't help much.
I've only used the spring starter to demonstrate the problem that I've encountered while developing my real application.

[mstahv]

I would suggest to by-pass the StreamResource abstraction here altogether. It probably just complicates things here and it has fundamentally mixed input and output streams.

Instead I'd create a proxy with some lower level abstraction for your camera stream to your Vaadin application and then give that proxy address to the Image, instead of the StreamResource.

[mstahv]

I have previously couple of times used jetty's generic servlets, but this was brought up by google now. Never used this myself, but based on its readme I'd expect it to work fine in Widfly: https://github.com/mitre/HTTP-Proxy-Servlet

[F-Kamp]

Thank you @mstahv very much for pointing out https://github.com/mitre/HTTP-Proxy-Servlet. With that library I've been able to create a proxy URL which works as URL for an Image in the Vaadin UI.

But there's something that still bothers me. While it's now possible to display the camera stream in the UI, my initial idea was to consume the camera stream only once and then re-distribute that to multiple UIs (to prevent the camera from servicing too many clients). Using a proxy however results in a new connection to the camera for every UI that is opened.

Isn't there a better way to do this?

[mstahv]

Currently it (most likely) indeed creates a new connection with httpclient for every user. To optimise that you probably need a custom proxy that only opens one backend (webcam) connection and that is then "multiplexes" different end user connections to that. Maybe buffer one images from the "multipart/x-mixed-replace" connection to the server, so that a new user doesn't need to wait for next image but gets the previous right away (I have no idea how often it is updated).

[F-Kamp]

I see, thank you again @mstahv. After discussing it with our customer we decided to keep it simple for now and simply live with multiple connections to the webcam.

Now back to my original bug report: Is anyone going to investigate the behavior of StreamResource?
We can't keep the camera stream publicly available indefinitely. We only borrowed the camera for testing purposes from our customer and will have to return it in a few weeks.

[mshabarov]

Our team is approaching the Christmas vacation season, so the investigation likely won't happen until January 2025, but I think we can use some publicly exposed axis camera streams for future investigations, e.g. from here https://github.com/fury999io/public-ip-cams?tab=readme-ov-file .