AbstractIOUringStreamChannel iovArray litter

[pveentjer]

Every time there is a writev, an IovArray instance is created. Also, there is the overhead of pooling the memory of the iovArrayBuffer.

 @Override
        protected int scheduleWriteMultiple(ChannelOutboundBuffer in) {
            assert iovArray == null;
            int numElements = Math.min(in.size(), Limits.IOV_MAX);
            ByteBuf iovArrayBuffer = alloc().directBuffer(numElements * IovArray.IOV_SIZE);
            iovArray = new IovArray(iovArrayBuffer); <-----
            try {
                int offset = iovArray.count();
                in.forEachFlushedMessage(iovArray);
                submissionQueue().addWritev(socket.intValue(),
                        iovArray.memoryAddress(offset), iovArray.count() - offset, (short) 0);
            } catch (Exception e) {
                // This should never happen, anyway fallback to single write.
                iovArray.release();
                iovArray = null;
                scheduleWriteSingle(in.current());
            }
            return 1;
        }

Perhaps it would be better to preallocate the iovArray with ByteBuff of size:

ByteBuf iovArrayBuffer = alloc().directBuffer(IOV_MAX * IovArray.IOV_SIZE);
iovArray = new IovArray(iovArrayBuffer); 

This way the iovArray is always big enough for any number of buffers.

So the usage would look something like this:

 @Override
        protected int scheduleWriteMultiple(ChannelOutboundBuffer in) {
            iovArray.clear();
            try {
                int offset = iovArray.count();
                in.forEachFlushedMessage(iovArray);
                submissionQueue().addWritev(socket.intValue(),
                        iovArray.memoryAddress(offset), iovArray.count() - offset, (short) 0);
            } catch (Exception e) {
                // This should never happen, anyway fallback to single write.
                scheduleWriteSingle(in.current());
            }
            return 1;
        }

One drawback of preallocating is that it will consume extra memory. So with a 64 bit JVM and 1024 as IOV_MAX, this would be around '2 * 8B * 1024 = 16KB'. So perhaps that would be a reason to not pool by default.

[franz1981]

I would use unpooled direct memory there too, and yes, agree on your point

@1Jo1 any concern why the iov array was treated as immutable and allocated/released for each writev?
I remember that we cannot have more the one in-flight write (vectored or not), hence it shouldn't happen that iovArray will be used while its memory still used for a previously submitted (and never completed) one

[normanmaurer]

Yes I think we can pre-allocate.

[franz1981]

@pveentjer do you think you can send a PR? I can help if needed, obviously :)