Add documentation about the compositor

tutorials/rendering/compositor.rst

@@ -0,0 +1,369 @@
+.. _doc_compositor:
+
+The Compositor
+==============
+
+The compositor is a new feature in Godot 4 that allows control over
+the rendering pipeline when rendering the contents of a :ref:`Viewport <class_Viewport>`.
+
+It can be configured on a :ref:`WorldEnvironment <class_WorldEnvironment>`
+node where it applies to all Viewports, or it can be configured on
+a :ref:`Camera3D <class_Camera3D>` and apply only to
+the Viewport using that camera.
+
+The :ref:`Compositor <class_Compositor>` resource is used to configure
+the compositor. To get started, simply create a new compositor on
+the appropriate node:
+
+.. image:: img/new_compositor.webp
+
+.. note::
+    The compositor is currently a feature that is only supported by
+    the Mobile and Forward+ renderers.
+
+Compositor effects
+------------------
+
+Compositor effects allow you to insert additional logic into the rendering
+pipeline at various stages. This is an advanced feature that requires
+a high level of understanding of the rendering pipeline to use to
+its best advantage.
+
+As the core logic of the compositor effect is called from the rendering
+pipeline it is important to note that this logic will thus run within
+the thread on which rendering takes place.
+Care needs to be taken to ensure we don't run into threading issues.
+
+To illustrate how to use compositor effects we'll create a simple
+post processing effect that allows you to write your own shader code
+and apply this full screen through a compute shader.
+You can find the finished demo project `here <https://github.com/godotengine/godot-demo-projects/tree/master/compute/post_shader>`_.
+
+We start by creating a new script called ``post_process_shader.gd``.
+We'll make this a tool script so we can see the compositor effect work in the editor.
+We need to extend our node from :ref:`CompositorEffect <class_CompositorEffect>`.
+We must also give our script a class name.
+
+.. code-block:: gdscript
+
+    @tool
+    extends CompositorEffect
+    class_name PostProcessShader
+
+Next we're going to define a constant for our shader template code.
+This is the boilerplate code that makes our compute shader work.
+
+.. code-block:: gdscript
+
+    const template_shader : String = "#version 450
+
+    // Invocations in the (x, y, z) dimension
+    layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
+
+    layout(rgba16f, set = 0, binding = 0) uniform image2D color_image;
+
+    // Our push constant
+    layout(push_constant, std430) uniform Params {
+        vec2 raster_size;
+        vec2 reserved;
+    } params;
+
+    // The code we want to execute in each invocation
+    void main() {
+        ivec2 uv = ivec2(gl_GlobalInvocationID.xy);
+        ivec2 size = ivec2(params.raster_size);
+
+        if (uv.x >= size.x || uv.y >= size.y) {
+            return;
+        }
+
+        vec4 color = imageLoad(color_image, uv);
+
+        #COMPUTE_CODE
+
+        imageStore(color_image, uv, color);
+    }"
+
+For more information on how compute shaders work,
+please check :ref:`Using compute shaders <doc_compute_shaders>`.
+
+The important bit here is that for every pixel on our screen,
+our ``main`` function is executed and inside of this we load
+the current color value of our pixel, execute our user code,
+and write our modified color back to our color image.
+
+``#COMPUTE_CODE`` gets replaced by our user code.
+
+In order to set our user code, we need an export variable.
+We'll also define a few script variables we'll be using:
+
+.. code-block:: gdscript
+
+    @export_multiline var shader_code : String = "":
+        set(value):
+            mutex.lock()
+            shader_code = value
+            shader_is_dirty = true
+            mutex.unlock()
+
+    var rd : RenderingDevice
+    var shader : RID
+    var pipeline : RID
+
+    var mutex : Mutex = Mutex.new()
+    var shader_is_dirty : bool = true
+
+
+Note the use of a :ref:`Mutex <class_Mutex>` in our code.
+Most of our implementation gets called from the rendering engine
+and thus runs within our rendering thread.
+
+We need to ensure that we set our new shader code, and mark our
+shader code as dirty, without our render thread accessing this
+data at the same time. 
+
+Next we initialize our effect.
+
+.. code-block:: gdscript
+
+    # Called when this resource is constructed.
+    func _init():
+        effect_callback_type = EFFECT_CALLBACK_TYPE_POST_TRANSPARENT
+        rd = RenderingServer.get_rendering_device()
+
+
+The main thing here is setting our ``effect_callback_type`` which tells
+the rendering engine at what stage of the render pipeline to call our code.
+
+.. note::
+
+    Currently we only have access to the stages of the 3D rendering pipeline!
+
+We also get a reference to our rendering device, which will come in very handy.
+
+We also need to clean up after ourselves, for this we react to the
+``NOTIFICATION_PREDELETE`` notification:
+
+.. code-block:: gdscript
+
+    # System notifications, we want to react on the notification that
+    # alerts us we are about to be destroyed.
+    func _notification(what):
+        if what == NOTIFICATION_PREDELETE:
+            if shader.is_valid():
+                # Freeing our shader will also free any dependents such as the pipeline!
+                RenderingServer.free_rid(shader)
+
+Note that we do not use our mutex here even though we create our shader inside
+of our render thread.
+The methods on our rendering server are thread safe and ``free_rid`` will
+be postponed cleaning up the shader until after any frames currently being
+rendered are finished.
+
+Also note that we are not freeing our pipeline. The rendering device does
+dependency tracking and as the pipeline is dependent on the shader, it will
+be automatically freed when the shader is destructed.
+
+From this point onwards our code will run on the rendering thread.
+
+Our next step is a helper function that will recompile the shader if the user
+code was changed.
+
+.. code-block:: gdscript
+
+    # Check if our shader has changed and needs to be recompiled.
+    func _check_shader() -> bool:
+        if not rd:
+            return false
+
+        var new_shader_code : String = ""
+
+        # Check if our shader is dirty.
+        mutex.lock()
+        if shader_is_dirty:
+            new_shader_code = shader_code
+            shader_is_dirty = false
+        mutex.unlock()
+
+        # We don't have a (new) shader?
+        if new_shader_code.is_empty():
+            return pipeline.is_valid()
+
+        # Apply template.
+        new_shader_code = template_shader.replace("#COMPUTE_CODE", new_shader_code);
+
+        # Out with the old.
+        if shader.is_valid():
+            rd.free_rid(shader)
+            shader = RID()
+            pipeline = RID()
+
+        # In with the new.
+        var shader_source : RDShaderSource = RDShaderSource.new()
+        shader_source.language = RenderingDevice.SHADER_LANGUAGE_GLSL
+        shader_source.source_compute = new_shader_code
+        var shader_spirv : RDShaderSPIRV = rd.shader_compile_spirv_from_source(shader_source)
+
+        if shader_spirv.compile_error_compute != "":
+            push_error(shader_spirv.compile_error_compute)
+            push_error("In: " + new_shader_code)
+            return false
+
+        shader = rd.shader_create_from_spirv(shader_spirv)
+        if not shader.is_valid():
+            return false
+
+        pipeline = rd.compute_pipeline_create(shader)
+        return pipeline.is_valid()
+
+At the top of this method we again use our mutex to protect accessing our
+user shader code and our is dirty flag.
+We make a local copy of the user shader code if our user shader code is dirty.
+
+If we don't have a new code fragment, we return true if we already have a
+valid pipeline.
+
+If we do have a new code fragment we embed it in our template code and then
+compile it.
+
+.. warning::
+    The code shown here compiles our new code in runtime.
+    This is great for prototyping as we can immediately see the effect
+    of the changed shader.
+
+    This prevents precompiling and caching this shader which may be an issues
+    on some platforms such as consoles.
+    Note that the demo project comes with an alternative example where
+    a ``glsl`` file contains the entire compute shader and this is used.
+    Godot is able to precompile and cache the shader with this approach.
+
+Finally we need to implement our effect callback, the rendering engine will call
+this at the right stage of rendering.
+
+.. code-block:: gdscript
+
+    # Called by the rendering thread every frame.
+    func _render_callback(p_effect_callback_type, p_render_data):
+        if rd and p_effect_callback_type == EFFECT_CALLBACK_TYPE_POST_TRANSPARENT and _check_shader():
+            # Get our render scene buffers object, this gives us access to our render buffers.
+            # Note that implementation differs per renderer hence the need for the cast.
+            var render_scene_buffers : RenderSceneBuffersRD = p_render_data.get_render_scene_buffers()
+            if render_scene_buffers:
+                # Get our render size, this is the 3D render resolution!
+                var size = render_scene_buffers.get_internal_size()
+                if size.x == 0 and size.y == 0:
+                    return
+
+                # We can use a compute shader here 
+                var x_groups = (size.x - 1) / 8 + 1
+                var y_groups = (size.y - 1) / 8 + 1
+                var z_groups = 1
+
+                # Push constant
+                var push_constant : PackedFloat32Array = PackedFloat32Array()
+                push_constant.push_back(size.x)
+                push_constant.push_back(size.y)
+                push_constant.push_back(0.0)
+                push_constant.push_back(0.0)
+
+                # Loop through views just in case we're doing stereo rendering. No extra cost if this is mono.
+                var view_count = render_scene_buffers.get_view_count()
+                for view in range(view_count):
+                    # Get the RID for our color image, we will be reading from and writing to it.
+                    var input_image = render_scene_buffers.get_color_layer(view)
+
+                    # Create a uniform set, this will be cached, the cache will be cleared if our viewports configuration is changed.
+                    var uniform : RDUniform = RDUniform.new()
+                    uniform.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
+                    uniform.binding = 0
+                    uniform.add_id(input_image)
+                    var uniform_set = UniformSetCacheRD.get_cache(shader, 0, [ uniform ])
+
+                    # Run our compute shader.
+                    var compute_list := rd.compute_list_begin()
+                    rd.compute_list_bind_compute_pipeline(compute_list, pipeline)
+                    rd.compute_list_bind_uniform_set(compute_list, uniform_set, 0)
+                    rd.compute_list_set_push_constant(compute_list, push_constant.to_byte_array(), push_constant.size() * 4)
+                    rd.compute_list_dispatch(compute_list, x_groups, y_groups, z_groups)
+                    rd.compute_list_end()
+
+At the start of this method we check if we have a rendering device,
+if our callback type is the correct one, and check if we have our shader.
+
+.. note::
+
+    The check for the effect type is only a safety mechanism.
+    We've set this in our ``_init`` function, however it is possible
+    for the user to change this in the UI.
+
+Our ``p_render_data`` parameter gives us access to an object that holds
+data specific to the frame we're currently rendering. We're currently only
+interested in our render scene buffers, which provide us access to all the
+internal buffers used by the rendering engine.
+Note that we cast this to :ref:`RenderSceneBuffersRD <class_RenderSceneBuffersRD>`
+to expose the full API to this data.
+
+Next we obtain our ``internal size`` which is the resolution of our 3D render
+buffers before they are upscaled (if applicable), upscaling happens after our
+post processes have run.
+
+From our internal size we calculate our group size, see our local size in our
+template shader.
+
+We also populate our push constant so our shader knows our size.
+Godot does not support structs here **yet** so we use a
+``PackedFloat32Array`` to store this data into. Note that we have
+to pad this array with a 16 byte alignment. In other words, the
+length of our array needs to be a multiple of 4.
+
+Now we loop through our views, this is in case we're using multiview rendering
+which is applicable for stereo rendering (XR). In most cases we will only have
+one view.
+
+.. note::
+
+    There is no performance benefit to use multiview for post processing
+    here, handling the views separately like this will still enable the GPU
+    to use parallelism if beneficial.
+
+Next we obtain the color buffer for this view. This is the buffer into which
+our 3D scene has been rendered.
+
+We then prepare a uniform set so we can communicate the color buffer to our
+shader.
+
+Note the use of our :ref:`UniformSetCacheRD <class_UniformSetCacheRD>` cache
+which ensures we can check for our uniform set each frame.
+As our color buffer can change from frame to frame and our uniform cache
+will automatically clean up uniform sets when buffers are freed, this is
+the safe way to ensure we do not leak memory or use an outdated set.
+
+Finally we build our compute list by binding our pipeline,
+binding our uniform set, pushing our push constant data,
+and calling dispatch for our groups.
+
+With our compositor effect completed, we now need to add it to our compositor.
+
+On our compositor we expand the compositor effects property
+and press ``Add Element``.
+
+Now we can add our compositor effect:
+
+.. image:: img/add_compositor_effect.webp
+
+After selecting our ``PostProcessShader`` we need to set our user shader code:
+
+.. code-block:: glsl
+
+    float gray = color.r * 0.2125 + color.g * 0.7154 + color.b * 0.0721;
+    color.rgb = vec3(gray);
+
+With that all done, our output is in grayscale.
+
+.. image:: img/post_process_shader.webp
+
+.. note::
+
+    For a more advanced example of post effects, check out the
+    `Radial blur based sky rays <https://github.com/BastiaanOlij/RERadialSunRays>`_
+    example project created by Bastiaan Olij.

tutorials/rendering/index.rst

@@ -10,3 +10,4 @@ Rendering
    viewports
    multiple_resolutions
    jitter_stutter
+   compositor