Improve benchmark registration

- Automatically register benchmarks based on `.tscn` files found in the
  `benchmarks/` folder.
  - `manager.gd` no longer needs to be edited every time a benchmark is added.
- Use full path for include/exclude glob instead of just the last category's name.
- Move benchmarks to a subfolder.
- Position the benchmark label away from the window's edges for better readability,
  and increase its outline size.
- Run the dynamic light cull with shadows benchmark
  (there was a `.tscn` file but it was unused).

CONTRIBUTING.md

@@ -2,9 +2,10 @@
 
 Thank you for your interest in contributing!
 
-**Note:** This project only supports Godot's `master` branch (4.0's development
-branch), not Godot 3.x. Attempting to open this project in Godot 3.x will result
-in errors.
+> **Note**
+>
+> This project only supports Godot's `master` branch (4.0's development branch),
+> not Godot 3.x. Attempting to open this project in Godot 3.x will result in errors.
 
 ## Adding new benchmarks
 
@@ -12,15 +13,19 @@ You can add new test cases by following these steps:
 
 ### Create new benchmark
 
-- Open the Godot editor and import this project.
-- Create a new scene with `snake_case` naming with a root node suited to the
-  benchmark[^1], then save it in one of the existing folders depending on its
-  category. If your benchmark does not suit any existing category or
-  subcategory, you can also create a new folder in the repository's root folder.
-  The root node's name does not have any bearing on functionality, but it's
-  recommended to use a PascalCase version of the scene name. For example, if
-  your scene file name is `typed_int_array.tscn`, the root node name should be
-  `TypedIntArray`.
+1. Open the Godot editor and import this project.
+2. Create a new scene with `snake_case` naming with a root node suited to the
+   benchmark[^1], then save it in one of the existing folders depending on its
+   category. If your benchmark does not suit any existing category or
+   subcategory, you can also create a new folder in the repository's root folder.
+   The root node's name does not have any bearing on functionality, but it's
+   recommended to use a PascalCase version of the scene name. For example, if
+   your scene file name is `typed_int_array.tscn`, the root node name should be
+   `TypedIntArray`.
+
+Instanced scenes that are used in other benchmark scenes **must** have their file
+name start with an underscore (e.g. `_particles.tscn`).
+This will exclude the scene from benchmark registration.
 
 [^1]: For 3D rendering benchmarks, the root node should be Node3D. For 2D
 rendering benchmarks, the root node should be Node2D. For UI benchmarks, the
@@ -29,33 +34,29 @@ node should be Node.
 
 ### Configure the benchmark
 
-- Create a Label node, rename it to `Benchmark` (case-sensitive) and attach
-  `benchmark.gd` as a script. Select the Benchmark node to configure its
-  properties. There are 5 properties available which control the *metrics* that
-  will be displayed in the results table:
-  - **Test Render Cpu:** Enable this for rendering benchmarks. Leave it disabled
-    for other benchmarks.
-  - **Test Render Gpu:** Enable this for rendering benchmarks. Leave it disabled
-    for other benchmarks.
-  - **Test Idle:** Enable this for non-rendering CPU-intensive benchmarks. Leave
-    it disabled for other benchmarks.
-  - **Test Physics:** Enable this for physics benchmarks. Leave it disabled for
-    other benchmarks.
-  - **Time Limit:** Enable this for rendering or physics benchmarks (which
-    measure the time spent processing things every frame instead of a total time
-    to process a set amount of items). For other benchmarks such as scripting,
-    disable this.
-- Attach a script to the scene's root node, with the same name as the scene file
-  (except the file extension will be `.gd` instead of `.tscn`). In the script's
-  `_ready()` function, perform the benchmark tasks. You don't need to surround
-  the benchmarked code with time measurement functions, as time is measured
-  automatically.
-- For benchmarks that do not have a fixed time limit (such as scripting benchmarks),
-  call `Manager.end_test()` at the end of the `_ready()` function.
-- Modify `manager.gd` in the repository root to include your new benchmark
-  (`tests` array variable). The new benchmark should be listed at the bottom of
-  the category it was added to. For clarity, make sure to separate each category
-  with a blank line within the `tests` array.
+1. Create a Label node, rename it to `Benchmark` (case-sensitive) and attach
+   `benchmark.gd` as a script. Select the Benchmark node to configure its
+   properties. There are 5 properties available which control the *metrics* that
+   will be displayed in the results table:
+   - **Test Render Cpu:** Enable this for rendering benchmarks. Leave it disabled
+     for other benchmarks.
+   - **Test Render Gpu:** Enable this for rendering benchmarks. Leave it disabled
+     for other benchmarks.
+   - **Test Idle:** Enable this for non-rendering CPU-intensive benchmarks. Leave
+     it disabled for other benchmarks.
+   - **Test Physics:** Enable this for physics benchmarks. Leave it disabled for
+     other benchmarks.
+   - **Time Limit:** Enable this for rendering or physics benchmarks (which
+     measure the time spent processing things every frame instead of a total time
+     to process a set amount of items). For other benchmarks such as scripting,
+     disable this.
+2. Attach a script to the scene's root node, with the same name as the scene file
+   (except the file extension will be `.gd` instead of `.tscn`). In the script's
+   `_ready()` function, perform the benchmark tasks. You don't need to surround
+   the benchmarked code with time measurement functions, as time is measured
+   automatically.
+3. For benchmarks that do not have a fixed time limit (such as scripting benchmarks),
+   call `Manager.end_test()` at the end of the `_ready()` function.
 
 Remember to follow the
 [GDScript style guide](https://docs.godotengine.org/en/latest/tutorials/scripting/gdscript/gdscript_styleguide.html)
@@ -64,12 +65,12 @@ unless you are specifically benchmarking non-typed scripts.
 
 ### Test the benchmark
 
-- Run the project in the editor.
-- Check the checkbox next to your newly added benchmark and click **Run** in the
-  bottom-right corner.
-- Wait for the benchmark to complete.
-- If all goes well, the benchmark completion time should appear in the table for
-  all enabled benchmarks (and all relevant metrics). Metrics that were disabled
-  on a specific benchmarks will not display any result.
+1. Run the project in the editor.
+2. Check the checkbox next to your newly added benchmark and click **Run** in the
+   bottom-right corner.
+3. Wait for the benchmark to complete.
+4. If all goes well, the benchmark completion time should appear in the table for
+   all enabled benchmarks (and all relevant metrics). Metrics that were disabled
+   on a specific benchmarks will not display any result.
 
-If the benchmark works as expected, congratulations! You can now open a pull request for it :)
+If the benchmark works as expected, congratulations! You can now open a pull request for it 🙂

README.md

@@ -51,20 +51,20 @@ The `--include-benchmarks` CLI argument can be used to specify the name.
 The project will print a message to acknowledge that your argument was taken
 into account for filtering benchmarks.
 
-Benchmark names all follow `category/some_name` naming, with `category` being the
-name of the *last* path component (folder) and `some_name` being the name of the
-benchmark's scene file.
+Benchmark names all follow `category/subcategory/some_name` naming, with
+`category/subcategory` being the name *all* path components (folders) and
+`some_name` being the name of the benchmark's scene file without the `.tscn` extension.
 
-```
-godot -- --run-benchmarks --include-benchmarks="culling/static_cull"
+```bash
+godot -- --run-benchmarks --include-benchmarks="rendering/culling/basic_cull"
 ```
 
 #### Run a category of benchmarks
 
 Use glob syntax (with `*` acting as a wildcard) to run a category of benchmarks:
 
-```
---include-benchmarks="culling/static_cull"
+```bash
+--include-benchmarks="rendering/culling/basic_cull"
 ```
 
 You can exclude specific benchmarks using the `--exclude-benchmarks` command line argument.

benchmark.gd

@@ -15,11 +15,13 @@ func _ready() -> void:
 		hide()
 	add_theme_color_override("font_color", Color(1, 1, 1))
 	add_theme_color_override("font_outline_color", Color(0, 0, 0))
-	add_theme_constant_override("outline_size", 2)
+	add_theme_constant_override("outline_size", 8)
+	# Position away from the screen edges for better readability.
+	position = Vector2(20, 20)
 
 
-func _process(delta: float) -> void:
-	var txt: String
+func _process(_delta: float) -> void:
+	var txt: String = ""
 	if test_render_cpu:
 		txt += "CPU: %s ms (incl. %s ms frame setup time)\n" % [
 			str(RenderingServer.viewport_get_measured_render_time_cpu(get_tree().root.get_viewport_rid()) + RenderingServer.get_frame_setup_time_cpu()).pad_decimals(2),

benchmarks/gdscript/packed_string_array.tscn

@@ -0,0 +1,13 @@
+[gd_scene load_steps=3 format=3 uid="uid://dyo16udaurxui"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/gdscript/packed_string_array.gd" id="2"]
+
+[node name="PackedStringArray" type="Node"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 14.0
+script = ExtResource("1")
+time_limit = false

benchmarks/gdscript/typed_int_array.tscn

@@ -0,0 +1,13 @@
+[gd_scene load_steps=3 format=3 uid="uid://7qhexqvebn6s"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/gdscript/typed_int_array.gd" id="2"]
+
+[node name="TypedIntArray" type="Node"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 14.0
+script = ExtResource("1")
+time_limit = false

benchmarks/gdscript/typed_string_array.tscn

@@ -0,0 +1,13 @@
+[gd_scene load_steps=3 format=3 uid="uid://bcrdxqm7xedqd"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/gdscript/typed_string_array.gd" id="2"]
+
+[node name="TypedStringArray" type="Node"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 14.0
+script = ExtResource("1")
+time_limit = false

benchmarks/gdscript/untyped_int_array.tscn

@@ -0,0 +1,13 @@
+[gd_scene load_steps=3 format=3 uid="uid://b88jxheg5d2dp"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/gdscript/untyped_int_array.gd" id="2"]
+
+[node name="UntypedIntArray" type="Node"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 14.0
+script = ExtResource("1")
+time_limit = false

benchmarks/gdscript/untyped_string_array.tscn

@@ -0,0 +1,13 @@
+[gd_scene load_steps=3 format=3 uid="uid://bqtiknifdkwms"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/gdscript/untyped_string_array.gd" id="2"]
+
+[node name="UntypedStringArray" type="Node"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 14.0
+script = ExtResource("1")
+time_limit = false

benchmarks/rendering/culling/basic_cull.gd

@@ -1,4 +1,4 @@
-extends "res://rendering/culling/culling_base.gd"
+extends "res://benchmarks/rendering/culling/culling_base.gd"
 
 const NUMBER_OF_OBJECTS = 10_000
 

benchmarks/rendering/culling/basic_cull.tscn

@@ -0,0 +1,19 @@
+[gd_scene load_steps=3 format=3 uid="uid://u718n0qh57cd"]
+
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/basic_cull.gd" id="1"]
+[ext_resource type="Script" path="res://benchmark.gd" id="2"]
+
+[node name="Node3D" type="Node3D"]
+script = ExtResource("1")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+script = ExtResource("2")
+test_render_cpu = true
+test_render_gpu = true
+
+[node name="Camera3D" type="Camera3D" parent="."]
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
+current = true
+far = 200.0

benchmarks/rendering/culling/culling_base.gd

@@ -52,7 +52,7 @@ void fragment() {
 		objects.append(ins)
 		object_xforms.append(xf)
 
-func fill_with_omni_lights(amount: int, use_shadows: bool = true) -> void:
+func fill_with_omni_lights(amount: int, use_shadows: bool) -> void:
 	var cam := $Camera3D as Camera3D
 	var zn := 2
 	var zextent := cam.far - zn

benchmarks/rendering/culling/directional_light_cull.gd

@@ -1,4 +1,4 @@
-extends "res://rendering/culling/culling_base.gd"
+extends "res://benchmarks/rendering/culling/culling_base.gd"
 
 const NUMBER_OF_OBJECTS = 10_000
 

benchmarks/rendering/culling/directional_light_cull.tscn

@@ -1,29 +1,24 @@
-[gd_scene load_steps=3 format=3 uid="uid://bjyxhd3fw5jfy"]
+[gd_scene load_steps=3 format=3 uid="uid://8owc7v5tw052"]
 
-[ext_resource type="Script" path="res://rendering/culling/directional_light_cull.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/directional_light_cull.gd" id="1"]
 [ext_resource type="Script" path="res://benchmark.gd" id="2"]
 
 [node name="Node3D" type="Node3D"]
-script = ExtResource( "1" )
+script = ExtResource("1")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+theme_override_font_sizes/font_size = 16
+script = ExtResource("2")
+test_render_cpu = true
+test_render_gpu = true
 
 [node name="Camera3D" type="Camera3D" parent="."]
 transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
 current = true
 far = 200.0
 
-[node name="Benchmark" type="Label" parent="."]
-offset_left = 16.0
-offset_top = 16.0
-offset_right = 56.0
-offset_bottom = 56.0
-theme_override_font_sizes/font_size = 16
-script = ExtResource( "2" )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true
-
 [node name="DirectionalLight3D" type="DirectionalLight3D" parent="."]
 transform = Transform3D(0.933154, 0, 0.359476, -0.242196, 0.738961, 0.628711, -0.265639, -0.673748, 0.689565, 2.09069, 0, 0)
 shadow_enabled = true

benchmarks/rendering/culling/dynamic_cull.gd

@@ -1,4 +1,4 @@
-extends "res://rendering/culling/culling_base.gd"
+extends "res://benchmarks/rendering/culling/culling_base.gd"
 
 const NUMBER_OF_OBJECTS = 10_000
 

benchmarks/rendering/culling/dynamic_cull.tscn

@@ -0,0 +1,19 @@
+[gd_scene load_steps=3 format=3 uid="uid://ddecopbjk0pd3"]
+
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/dynamic_cull.gd" id="1"]
+[ext_resource type="Script" path="res://benchmark.gd" id="2"]
+
+[node name="Node3D" type="Node3D"]
+script = ExtResource("1")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+script = ExtResource("2")
+test_render_cpu = true
+test_render_gpu = true
+
+[node name="Camera3D" type="Camera3D" parent="."]
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
+current = true
+far = 200.0

benchmarks/rendering/culling/dynamic_light_cull.gd

@@ -1,4 +1,4 @@
-extends "res://rendering/culling/culling_base.gd"
+extends "res://benchmarks/rendering/culling/culling_base.gd"
 
 const NUMBER_OF_OBJECTS = 10_000
 const NUMBER_OF_OMNI_LIGHTS = 100

benchmarks/rendering/culling/dynamic_light_cull.tscn

@@ -0,0 +1,19 @@
+[gd_scene load_steps=3 format=3 uid="uid://caal1eu6emwn1"]
+
+[ext_resource type="Script" path="res://benchmark.gd" id="1"]
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/dynamic_light_cull.gd" id="2"]
+
+[node name="Node3D" type="Node3D"]
+script = ExtResource("2")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+script = ExtResource("1")
+test_render_cpu = true
+test_render_gpu = true
+
+[node name="Camera3D" type="Camera3D" parent="."]
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
+current = true
+far = 200.0

benchmarks/rendering/culling/dynamic_light_cull_with_shadows.tscn

@@ -0,0 +1,20 @@
+[gd_scene load_steps=3 format=3 uid="uid://d3rchghob37sn"]
+
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/dynamic_light_cull.gd" id="1_d0xom"]
+[ext_resource type="Script" path="res://benchmark.gd" id="2_txiip"]
+
+[node name="Node3D" type="Node3D"]
+script = ExtResource("1_d0xom")
+use_shadows = true
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+script = ExtResource("2_txiip")
+test_render_cpu = true
+test_render_gpu = true
+
+[node name="Camera3D" type="Camera3D" parent="."]
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
+current = true
+far = 200.0

benchmarks/rendering/culling/static_light_cull.gd

@@ -1,4 +1,4 @@
-extends "res://rendering/culling/culling_base.gd"
+extends "res://benchmarks/rendering/culling/culling_base.gd"
 
 const NUMBER_OF_OBJECTS = 10_000
 const NUMBER_OF_OMNI_LIGHTS = 100
@@ -6,4 +6,4 @@ const NUMBER_OF_OMNI_LIGHTS = 100
 
 func _ready() -> void:
 	fill_with_objects(NUMBER_OF_OBJECTS)
-	fill_with_omni_lights(NUMBER_OF_OMNI_LIGHTS)
+	fill_with_omni_lights(NUMBER_OF_OMNI_LIGHTS, false)

benchmarks/rendering/culling/static_light_cull.tscn

@@ -0,0 +1,19 @@
+[gd_scene load_steps=3 format=3 uid="uid://bceclc3bopcwu"]
+
+[ext_resource type="Script" path="res://benchmarks/rendering/culling/static_light_cull.gd" id="1"]
+[ext_resource type="Script" path="res://benchmark.gd" id="2"]
+
+[node name="Node3D" type="Node3D"]
+script = ExtResource("1")
+
+[node name="Benchmark" type="Label" parent="."]
+offset_right = 40.0
+offset_bottom = 40.0
+script = ExtResource("2")
+test_render_cpu = true
+test_render_gpu = true
+
+[node name="Camera3D" type="Camera3D" parent="."]
+transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609)
+current = true
+far = 200.0

gdscript/packed_string_array.tscn

@@ -1,17 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://gdscript/packed_string_array.gd" type="Script" id=2]
-
-[node name="PackedStringArray" type="Node"]
-script = ExtResource( 2 )
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 14.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-time_limit = false

gdscript/typed_int_array.tscn

@@ -1,17 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://gdscript/typed_int_array.gd" type="Script" id=2]
-
-[node name="TypedIntArray" type="Node"]
-script = ExtResource( 2 )
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 14.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-time_limit = false

gdscript/typed_string_array.tscn

@@ -1,17 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://gdscript/typed_string_array.gd" type="Script" id=2]
-
-[node name="TypedStringArray" type="Node"]
-script = ExtResource( 2 )
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 14.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-time_limit = false

gdscript/untyped_int_array.tscn

@@ -1,17 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://gdscript/untyped_int_array.gd" type="Script" id=2]
-
-[node name="UntypedIntArray" type="Node"]
-script = ExtResource( 2 )
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 14.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-time_limit = false

gdscript/untyped_string_array.tscn

@@ -1,17 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://gdscript/untyped_string_array.gd" type="Script" id=2]
-
-[node name="UntypedStringArray" type="Node"]
-script = ExtResource( 2 )
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 14.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-time_limit = false

main.gd

@@ -27,6 +27,7 @@ func _ready() -> void:
 		var test_name := Manager.get_test_name(i)
 		var category := Manager.get_test_category(i)
 		var results := Manager.get_test_result(i)
+		var path := Manager.get_test_path(i)
 
 		if category not in categories:
 			var c := tree.create_item(root)
@@ -37,6 +38,8 @@ func _ready() -> void:
 		item.set_cell_mode(0, TreeItem.CELL_MODE_CHECK)
 		item.set_text(0, test_name)
 		item.set_editable(0, true)
+		# Store the full scene path each TreeItem points towards (for include/exclude glob checking).
+		item.set_meta("path", path)
 
 		if results:
 			if results.render_cpu:
@@ -61,13 +64,15 @@ func _ready() -> void:
 	for argument in OS.get_cmdline_user_args():
 		if argument.begins_with("--include-benchmarks="):
 			var key_value := argument.split("=")
-			# Remove quotes around the argument's value, so that "`culling/*`" becomes `culling/*` for globbing.
+			# Remove quotes around the argument's value, so that "`rendering/culling/*`"
+			# becomes `rendering/culling/*` for globbing.
 			include_benchmarks_glob = key_value[1].trim_prefix('"').trim_suffix('"').trim_prefix("'").trim_suffix("'")
 			print("Using benchmark include glob specified on command line: %s" % include_benchmarks_glob)
 
 		if argument.begins_with("--exclude-benchmarks="):
 			var key_value := argument.split("=")
-			# Remove quotes around the argument's value, so that "`culling/*`" becomes `culling/*` for globbing.
+			# Remove quotes around the argument's value, so that "`rendering/culling/*`"
+			# becomes `rendering/culling/*` for globbing.
 			exclude_benchmarks_glob = key_value[1].trim_prefix('"').trim_suffix('"').trim_prefix("'").trim_suffix("'")
 			print("Using benchmark exclude glob specified on command line: %s" % exclude_benchmarks_glob)
 
@@ -99,17 +104,20 @@ func _on_Run_pressed() -> void:
 	var index := 0
 	var paths := []
 	for item in items:
-		var path := str(item.get_parent().get_text(0) + "/" + item.get_text(0)).to_lower().replace(" ", "_")
+		var path: String = item.get_meta("path").trim_prefix("res://benchmarks/").trim_suffix(".tscn")
+		
 		if not include_benchmarks_glob.is_empty():
 			if not path.match(include_benchmarks_glob):
-				continue
+				item.set_checked(0, false)
+				
 		if not exclude_benchmarks_glob.is_empty():
 			if path.match(exclude_benchmarks_glob):
-				continue
+				item.set_checked(0, false)
 
 		if item.is_checked(0):
 			queue.push_back(index)
 			paths.push_back(path)
+			print(queue)
 
 		index += 1
 
@@ -120,8 +128,8 @@ func _on_Run_pressed() -> void:
 	else:
 		print_rich("[color=red][b]ERROR:[/b] No benchmarks to run.[/color]")
 		if Manager.run_from_cli:
-			print("Double-check the syntax of the benchmarks include/exclude glob (quotes are required).")
-			print_rich('Example usage: [code]godot --run-benchmarks --include-benchmarks="culling/*" --exclude-benchmarks="culling/static_cull"[/code]')
+			print("       Double-check the syntax of the benchmarks include/exclude glob (quotes are required).")
+			print_rich('       Example usage: [code]godot -- --run-benchmarks --include-benchmarks="rendering/culling/*" --exclude-benchmarks="rendering/culling/basic_cull"[/code]')
 			get_tree().quit(1)
 
 

manager.gd

@@ -17,19 +17,8 @@ class Test:
 		category = p_category
 		path = p_path
 
-var tests := [
-	Test.new("Static Cull", "Culling", "res://rendering/culling/basic_cull.tscn"),
-	Test.new("Dynamic Cull", "Culling", "res://rendering/culling/dynamic_cull.tscn"),
-	Test.new("Static Lights Cull", "Culling", "res://rendering/culling/static_light_cull.tscn"),
-	Test.new("Dynamic Lights Cull", "Culling", "res://rendering/culling/dynamic_light_cull.tscn"),
-	Test.new("Directional Light Cull", "Culling", "res://rendering/culling/directional_light_cull.tscn"),
-
-	Test.new("Untyped Int Array", "GDScript", "res://gdscript/untyped_int_array.tscn"),
-	Test.new("Typed Int Array", "GDScript", "res://gdscript/typed_int_array.tscn"),
-	Test.new("Untyped String Array", "GDScript", "res://gdscript/untyped_string_array.tscn"),
-	Test.new("Typed String Array", "GDScript", "res://gdscript/typed_string_array.tscn"),
-	Test.new("Packed String Array", "GDScript", "res://gdscript/packed_string_array.tscn"),
-]
+# List of benchmarks populated in `_ready()`.
+var tests: Array[Test] = []
 
 var frames_captured := 0
 var results: Results = null
@@ -51,10 +40,43 @@ var record_physics := false
 var time_limit := true
 
 
+## Returns file paths ending with `.tscn` within a folder, recursively.
+func dir_contents(path: String, contents: PackedStringArray = PackedStringArray()) -> PackedStringArray:
+
+	var dir := Directory.new()
+	if dir.open(path) == OK:
+		dir.list_dir_begin()
+		var file_name = dir.get_next()
+		while file_name != "":
+			if dir.current_is_dir():
+				dir_contents(path.plus_file(file_name), contents)
+			elif file_name.ends_with(".tscn"):
+				contents.push_back(path.plus_file(file_name))
+			file_name = dir.get_next()
+	else:
+		print("An error occurred when trying to access the path: %s" % path)
+
+	return contents
+
+
 func _ready():
 	RenderingServer.viewport_set_measure_render_time(get_tree().root.get_viewport_rid(),true)
 	set_process(false)
 
+	# Register benchmarks automatically based on `.tscn` file paths in the `benchmarks/` folder.
+	# Scene names starting with `_` are excluded, as this denotes an instanced scene that is
+	# referred to in another scene.
+	var benchmark_paths := dir_contents("res://benchmarks/")
+	benchmark_paths.sort()
+	for benchmark_path in benchmark_paths:
+		var benchmark_name := benchmark_path.get_file().get_basename()
+		# Capitalize only after checking whether the name begins with `_`, as `capitalize()`
+		# removes underscores.
+		if not benchmark_name.begins_with("_"):
+			benchmark_name = benchmark_name.capitalize()
+			var category := benchmark_path.get_base_dir().trim_prefix("res://benchmarks/").replace("/", " > ").capitalize()
+			tests.push_back(Test.new(benchmark_name, category, benchmark_path))
+
 
 func _process(delta: float) -> void:
 	if not recording:
@@ -98,6 +120,10 @@ func get_test_result(index: int) -> Results:
 	return tests[index].results
 
 
+func get_test_path(index: int) -> String:
+	return tests[index].path
+
+
 func benchmark(queue: Array, time: float, return_path: String) -> void:
 	tests_queue = queue
 	if tests_queue.size() == 0:
@@ -116,26 +142,30 @@ func begin_test() -> void:
 	print("Running benchmark %d of %d: %s" % [
 			tests_queue_initial_size - tests_queue.size() + 1,
 			tests_queue_initial_size,
-			tests[tests_queue[0]].path.trim_prefix("res://").trim_suffix(".tscn")]
+			tests[tests_queue[0]].path.trim_prefix("res://benchmarks/").trim_suffix(".tscn")]
 	)
 
 	results = Results.new()
-	recording = true
-	results = Results.new()
-	begin_time = Time.get_ticks_usec() * 0.001
-	remaining_time = test_time
 	set_process(true)
 	get_tree().change_scene(tests[tests_queue[0]].path)
 
-	var benchmark_group := get_tree().get_nodes_in_group("benchmark_config")
+	recording = true
+	begin_time = Time.get_ticks_usec() * 0.001
+	remaining_time = test_time
 
-	if benchmark_group.size() >= 1:
-		var benchmark: Node = benchmark_group[0]
-		record_render_cpu = benchmark.test_render_cpu
-		record_render_gpu = benchmark.test_render_gpu
-		record_idle = benchmark.test_idle
-		record_physics = benchmark.test_physics
-		time_limit = benchmark.time_limit
+	# Wait for the scene tree to be ready (required for `benchmark_config` group to be available).
+	# This requires waiting for 2 frames to work consistently (1 frame is flaky).
+	for i in 2:
+		await get_tree().process_frame
+
+	var benchmark_node := get_tree().get_first_node_in_group("benchmark_config")
+
+	if benchmark_node:
+		record_render_cpu = benchmark_node.test_render_cpu
+		record_render_gpu = benchmark_node.test_render_gpu
+		record_idle = benchmark_node.test_idle
+		record_physics = benchmark_node.test_physics
+		time_limit = benchmark_node.time_limit
 	else:
 		record_render_cpu = true
 		record_render_gpu = true

rendering/culling/basic_cull.tscn

@@ -1,24 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://rendering/culling/basic_cull.gd" type="Script" id=1]
-[ext_resource path="res://benchmark.gd" type="Script" id=2]
-
-[node name="Node3D" type="Node3D"]
-script = ExtResource( 1 )
-
-[node name="Camera3D" type="Camera3D" parent="."]
-transform = Transform( 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609 )
-current = true
-far = 200.0
-script = null
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 40.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 2 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true

rendering/culling/dynamic_cull.tscn

@@ -1,24 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://rendering/culling/dynamic_cull.gd" type="Script" id=1]
-[ext_resource path="res://benchmark.gd" type="Script" id=2]
-
-[node name="Node3D" type="Node3D"]
-script = ExtResource( 1 )
-
-[node name="Camera3D" type="Camera3D" parent="."]
-transform = Transform( 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609 )
-current = true
-far = 200.0
-script = null
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 40.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 2 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true

rendering/culling/dynamic_light_cull.tscn

@@ -1,24 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://rendering/culling/dynamic_light_cull.gd" type="Script" id=2]
-
-[node name="Node3D" type="Node3D"]
-script = ExtResource( 2 )
-
-[node name="Camera3D" type="Camera3D" parent="."]
-transform = Transform( 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609 )
-current = true
-far = 200.0
-script = null
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 40.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true

rendering/culling/dynamic_light_cull_withs_shadows.tscn

@@ -1,25 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://benchmark.gd" type="Script" id=1]
-[ext_resource path="res://rendering/culling/dynamic_light_cull.gd" type="Script" id=2]
-
-[node name="Node3D" type="Node3D"]
-script = ExtResource( 2 )
-use_shadows = true
-
-[node name="Camera3D" type="Camera3D" parent="."]
-transform = Transform( 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609 )
-current = true
-far = 200.0
-script = null
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 40.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 1 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true

rendering/culling/static_light_cull.tscn

@@ -1,24 +0,0 @@
-[gd_scene load_steps=3 format=2]
-
-[ext_resource path="res://rendering/culling/static_light_cull.gd" type="Script" id=1]
-[ext_resource path="res://benchmark.gd" type="Script" id=2]
-
-[node name="Node3D" type="Node3D"]
-script = ExtResource( 1 )
-
-[node name="Camera3D" type="Camera3D" parent="."]
-transform = Transform( 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0.873609 )
-current = true
-far = 200.0
-script = null
-
-[node name="Benchmark" type="Label" parent="."]
-offset_right = 40.0
-offset_bottom = 40.0
-structured_text_bidi_override_options = [  ]
-script = ExtResource( 2 )
-__meta__ = {
-"_edit_use_anchors_": false
-}
-test_render_cpu = true
-test_render_gpu = true