Vulkan raytracing plumbing

[Fahien]

Here's a bunch of code adding some Vulkan raytracing stuff to the rendering device:

• Vulkan implementations in RenderingDeviceDriverVulkan
• Raytracing instruction list in RenderingDeviceGraph
• Functions to create acceleration structures and raytracing pipelines in RenderingDevice

There's more in the fahien/raytracing-test branch, with code handling raygen, miss, and closest-hit shaders, but it's a hack on top of the forward clustered renderer, and it needs some "guided" refactoring.

Here's a sample which uses GDScript to drive the renderer: raytracing-gdscript-demo

I hope this changes would look useful to jump-start raytracing support.

Relevant for godotengine/godot-proposals#5162

[Chaosus]

Commits needs to be squashed (see https://docs.godotengine.org/en/latest/contributing/workflow/pr_workflow.html).

[DarioSamo]

I had a very brief look and it looks pretty good to me, although I'd hold off on making changes that add a new rendering method. Are you planning on sticking to only supporting the hit shader workflow or would you also like to add ray query support as well?

If you'd like to test your rendering without adding a new rendering method, keep in mind you can also use GDScript to drive the renderer and it should be pretty sufficient to test it out. This is also a simple way you can provide us with a demo project to test it out.

[octanejohn]

i see that you started on a renderer too on that repo, i think your renderer could be tested/based on the godot ideas

https://gist.github.com/reduz/c5769d0e705d8ab7ac187d63be0099b5

[Fahien]

@DarioSamo

I had a very brief look and it looks pretty good to me, although I'd hold off on making changes that add a new rendering method. Are you planning on sticking to only supporting the hit shader workflow or would you also like to add ray query support as well?

Just focusing on the hit shader for now.

If you'd like to test your rendering without adding a new rendering method, keep in mind you can also use GDScript to drive the renderer and it should be pretty sufficient to test it out. This is also a simple way you can provide us with a demo project to test it out.

There you go: https://github.com/Fahien/godot-raytracing-gdscript-demo

[a-johnston]

This is very cool to see! Last night I took a look at how hard it might be adding metal support and while it's definitely out of my wheelhouse here's a commit to at least get it building on macos + a new method to guard user code from invoking raytracing when it isn't supported (also includes some precommit changes) a-johnston@a220083

ed: I have some (probably very misguided and) incomplete changes on https://github.com/a-johnston/godot/tree/raytracing-metal but giving up where it is. Currently it blows up for spirv_cross::CompilerError: A memory declaration object must be used in TraceRayKHR. when in the raytracing renderer but there are other issues past that point. Also notable that this has a lot of conflicts with the hddagi branch, although that's too be expected, if one or the other is going to be merged soon.

[DarioSamo]

Thanks to @Fahien for making the changes we agreed to. I won't have much time to review this until I get back in a few weeks but I'd like to know if you're willing to extend this PR to cover the other backends or if the intention is only for Vulkan to be merged first.

If you wish to tackle the other APIs, I can leave you some relevant documentation on how to replicate the drivers functionality in the meantime.

[Fahien]

@DarioSamo: I'd like to know if you're willing to extend this PR to cover the other backends or if the intention is only for Vulkan to be merged first.

I can try, but I wonder if it would be better doing this in subsequent PRs.

[Fahien]

@Calinou: I wonder if this should also be exposed as a shader preprocessor define in the long run (for Godot's shader language), so that you can perform different shader compilations depending on support.

Done

[Bonkahe]

Sorry if I'm a little bit in-experienced, but out of curiosity I pulled the branch and built it, then ran it on the example gdscript project, it was obviously out of date with the latest version of the branch, so I updated the render() function in raytracing.gd with the one found in the raytracing_list_begin description, with the addition of the last two lines actually getting the texture and assigning it to the viewport. Unfortunately this is the result:

Any idea why this is occurring? I assume I missed something but it's not currently throwing any errors.

The full script is here:

extends Node3D

@onready
var rd := RenderingServer.create_local_rendering_device()
@onready
var screen_texture := get_node("TextureRect")

var raytracing_texture: RID
var shader: RID
var raytracing_pipeline: RID
var vertex_buffer: RID
var vertex_array: RID
var index_buffer: RID
var index_array: RID
var transform_buffer: RID
var blas: RID
var tlas: RID
var uniform_set: RID

func _cleanup():
	if rd == null:
		return

	rd.free_rid(uniform_set)
	rd.free_rid(tlas)
	rd.free_rid(blas)
	rd.free_rid(transform_buffer)
	rd.free_rid(index_array)
	rd.free_rid(index_buffer)
	rd.free_rid(vertex_array)
	rd.free_rid(vertex_buffer)
	rd.free_rid(raytracing_pipeline)
	rd.free_rid(shader)
	rd.free_rid(raytracing_texture)
	rd.free()
	rd = null

func _notification(what: int):
	if what == NOTIFICATION_PREDELETE:
		_cleanup()

func _ready():
	if rd.raytracing_is_supported():
		_initialise_screen_texture()
		_initialize_raytracing_texture()
		_initialize_scene()
		_initialize_raytracing_pipeline()

func _process(_delta):
	if rd.raytracing_is_supported():
		_render()

func _initialize_raytracing_texture():
	# Create texture for raytracing rendering.
	var texture_format := RDTextureFormat.new()
	texture_format.texture_type = RenderingDevice.TEXTURE_TYPE_2D
	texture_format.format = RenderingDevice.DATA_FORMAT_R32G32B32A32_SFLOAT
	texture_format.width = get_viewport().size.x
	texture_format.height = get_viewport().size.y
	# It needs storage bit for the raytracing pipeline and can copy from for the presentation graphics pipeline.
	texture_format.usage_bits = RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT | RenderingDevice.TEXTURE_USAGE_STORAGE_BIT
	var texture_view := RDTextureView.new()
	raytracing_texture = rd.texture_create(texture_format, texture_view)

func _initialise_screen_texture():
	var image_size = get_viewport().size
	var image = Image.create(image_size.x, image_size.y, false, Image.FORMAT_RGBAF)
	var image_texture = ImageTexture.create_from_image(image)
	screen_texture.texture = image_texture

func _set_screen_texture_data(data: PackedByteArray):
	var image_size = get_viewport().size
	var image := Image.create_from_data(image_size.x, image_size.y, false, Image.FORMAT_RGBAF, data)
	screen_texture.texture.update(image)

func _initialize_raytracing_pipeline():
	# Create raytracing shaders.
	var shader_file := load("res://ray.glsl")
	var shader_spirv: RDShaderSPIRV = shader_file.get_spirv()
	shader = rd.shader_create_from_spirv(shader_spirv)
	raytracing_pipeline = rd.raytracing_pipeline_create(shader)

	var image_uniform := RDUniform.new()
	image_uniform.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
	image_uniform.binding = 0
	image_uniform.add_id(raytracing_texture)
	
	var as_uniform := RDUniform.new()
	as_uniform.uniform_type = RenderingDevice.UNIFORM_TYPE_ACCELERATION_STRUCTURE
	as_uniform.binding = 1
	as_uniform.add_id(tlas)
	
	uniform_set = rd.uniform_set_create([image_uniform, as_uniform], shader, 0)

func _initialize_scene():
	# Vertex buffer for a triangle
	# Prepare our data. We use floats in the shader, so we need 32 bit.
	var points := PackedFloat32Array([
			 0.0, -0.7, 1.0,
			 0.5, -0.7, 1.0,
			 0.0,  0.5, 1.0,
			-0.5, -0.7, 1.0,
			 0.5, -0.7, 1.0,
			-0.5,  0.5, 1.0,
		])
	var point_bytes := points.to_byte_array()
	vertex_buffer = rd.vertex_buffer_create(point_bytes.size(), point_bytes, true)
	var vertex_desc := RDVertexAttribute.new()
	vertex_desc.format = RenderingDevice.DATA_FORMAT_R32G32B32_SFLOAT
	vertex_desc.location = 0
	vertex_desc.stride = 4 * 3
	var vertex_format := rd.vertex_format_create([vertex_desc])
	vertex_array = rd.vertex_array_create(points.size() / 3, vertex_format, [vertex_buffer], [3*3*4])

	# Index buffer
	var indices := PackedInt32Array([0, 2, 1])
	var index_bytes := indices.to_byte_array()
	index_buffer = rd.index_buffer_create(indices.size(), RenderingDevice.INDEX_BUFFER_FORMAT_UINT32, index_bytes)
	index_array = rd.index_array_create(index_buffer, 0, indices.size())

	# Transform buffer
	var transform_matrix := PackedFloat32Array([
		1.0, 0.0, 0.0, 0.0,
		0.0, 1.0, 0.0, 0.0,
		0.0, 0.0, 1.0, 0.0,
	])
	var transform_bytes := transform_matrix.to_byte_array()
	transform_buffer = rd.storage_buffer_create(transform_bytes.size(), transform_bytes, RenderingDevice.STORAGE_BUFFER_USAGE_SHADER_DEVICE_ADDRESS | RenderingDevice.STORAGE_BUFFER_USAGE_ACCELERATION_STRUCTURE_BUILD_INPUT_READ_ONLY)

	# Create a BLAS for a mesh
	blas = rd.blas_create(vertex_array, index_array, transform_buffer)
	# Create TLAS with BLASs.
	tlas = rd.tlas_create([blas])

func _render():
	# Build acceleration structures.
	rd.acceleration_structure_build(blas)
	rd.acceleration_structure_build(tlas)
	
	var raylist = rd.raytracing_list_begin()
	
	# Bind pipeline and uniforms.
	rd.raytracing_list_bind_raytracing_pipeline(raylist, raytracing_pipeline)
	rd.raytracing_list_bind_uniform_set(raylist, uniform_set, 0)
	
	# Trace rays.
	var width = get_viewport().size.x
	var height = get_viewport().size.y
	rd.raytracing_list_trace_rays(raylist, width, height)
	
	rd.raytracing_list_end()
	
	var byte_data := rd.texture_get_data(raytracing_texture, 0)
	_set_screen_texture_data(byte_data)

[Fahien]

@Bonkahe, thank you for testing this, and do not worry for your experience level, we all started from zero and improved with time and dedication :)

I suppose you expected to see the geometries in the scene, but the demo is currently not able to get them to build raytracing acceleration structures from GDscript. If you take a look at the _initialize_scene() function, you'll notice that it's creating vertices and indices for a simple geometry "manually", which is the triangle you see. Then, the raytracing shaders in the demo do not do anything fancy: green when a ray hits a geometry, red when it does not. So, everything works as expected!

P.S. I have been experimenting with a compositor effect in other branches, but there are still some issues to solve, but this is a story for another PR:

• https://github.com/Fahien/godot/tree/fahien/raytracing-vertex-arrays
• https://github.com/Fahien/godot-raytracing-gdscript-demo/tree/compositor