diff --git a/index.rst b/index.rst index c5c33d6..126b80f 100644 --- a/index.rst +++ b/index.rst @@ -80,14 +80,3 @@ for your topic of interest. You can also use the search function in the top-left other/website other/demos other/benchmarks - -.. toctree:: - :hidden: - :maxdepth: 1 - :caption: Maintainers documentation - :name: sec-maintainers - - maintainers/how_to_become_a_maintainer - maintainers/introduction - maintainers/merge_queue - maintainers/building_releases diff --git a/maintainers/building_releases.rst b/maintainers/building_releases.rst deleted file mode 100644 index a94555a..0000000 --- a/maintainers/building_releases.rst +++ /dev/null @@ -1,241 +0,0 @@ -Building releases -================= - -Guidelines for release coordinators to build official releases. - -- Target users: **Release coordinators**. -- Relevant reading for: **Advanced engine users and contributors**, to - understand how official builds are made. -- Other relevant docs: `Building from source `__. - -This document starts by presenting how to make release builds in the -most succinct way possible, and later sections give more details on -relevant steps or components. - -Build process -------------- - -We’ll use ``4.4-beta1`` as the dev snapshot being built in this example. -The process is the same for dev snapshots and stable releases. - -Login to the build server via SSH. Hostname IP and port are shared in -private. - -Then the following commands are the exact set needed to make an official -build: - -.. code:: bash - - screen - cd godot-build-scripts/4.4 - cd git - git checkout master -b 4.4-beta1 - git pull origin master - cd .. - - # Start the build. - sh build.sh -v 4.4-beta1 -g 4.4-beta1 - # Here it takes several hours, you can detach from the `screen` session with 'Ctrl+A' then 'D'. - # When you resume with `screen -r`, confirm that all builds succeeded with: - tail -n 1 out/logs/* - - # Package the release (rename and zip files, sign, notarize, etc.). - sh build-release.sh -v 4.4-beta1 -t 4.4.beta1 - - # If the above step succeeds, and only if this is meant to be published officially: - sh publish-release.sh -v 4.4-beta1 - -Context to go deeper --------------------- - -Reproducible builds via containers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Our build pipeline is set up on a physical build server running Fedora. -All platforms are cross-compiled from Fedora using relevant -cross-compilation toolchains. - -The Podman/Docker build containers (or images, we tend to use the terms -interchangeably) are built using the Dockerfiles and scripts in this -repository: - -https://github.com/godotengine/build-containers - -The build server already has the latest version of these containers -prebuilt. You can review the list with ``sudo podman images | less``, -for example: - -.. code:: bash - - REPOSITORY TAG IMAGE ID CREATED SIZE - localhost/godot-web 4.3-f40 ff11195607d6 4 months ago 2.42 GB - localhost/godot-ios 4.3-f40 a6d60e6ad94a 4 months ago 5.49 GB - localhost/godot-osx 4.3-f40 e82d7cc9a7af 4 months ago 4.78 GB - localhost/godot-android 4.3-f40 5b6d3f00e1b4 4 months ago 4.29 GB - localhost/godot-windows 4.3-f40 0e4d07925ffe 4 months ago 2.46 GB - localhost/godot-linux 4.3-f40 1b4e60457cee 4 months ago 2.8 GB - localhost/godot-fedora 4.3-f40 5244d1fe13e6 4 months ago 1.01 GB - registry.fedoraproject.org/fedora 40 e442870eb306 4 months ago 233 MB - -The “TAG” column is a version string defined at build time when calling -the ``build.sh`` script, in the form -``-``. - -Each image can then be referred to as ``localhost/:`` -e.g. \ ``localhost/godot-windows/4.3-f40``. - -Once a minor version is released, we stick to the same prebuilt images -during its lifetime, to avoid introducing regressions from toolchain -updates. - -So in the above example, the latest image is ``4.3-f40`` (used for Godot -4.3.x and based on Fedora 40), and there would likely be a set -eventually named ``4.4-f41`` with Fedora 41 as base and all toolchains -updated. - -Check the README file in ``build-containers`` for more details, and a -list of the toolchains currently in use. - -In-container build scripts --------------------------- - -Once the containers are built locally, they can be used via the scripts -hosted here: - -https://github.com/godotengine/godot-build-scripts - -On the build server, we cloned this repository multiple times for each -Godot branch, in the ``~/godot-build-scripts//`` folders: - -.. code:: bash - - $ ls ~/godot-build-scripts/ - 3.5 3.6 4.0 4.1 4.2 4.3 4.4 - -Each of these is checked out on the relevant branch of -``godot-build-scripts``. Older branches may have slight variations to -the workflow described above. - -Some files in these folders are ``.gitignore``\ d but quite important: - -* ``config.sh`` is generated by copying ``config.sh.in`` and filling in the blanks. It contains all the details needed to sign, notarize and publish our builds. It’s also the file which defines which image version/tag we’re using for those builds (e.g. ``4.3-f40``). -* Signing keys for various platforms. - -Building the binaries -~~~~~~~~~~~~~~~~~~~~~ - -The entry point to make builds is ``build.sh``, which is responsible -for: - -* Downloading the dependencies in ``deps`` if missing. - * Note: If you update dependencies in the script or at their download location, the easiest way to ensure things are up-to-date is to delete ``deps`` and let ``build.sh`` download everything again. -* Cloning the Godot Git repo in ``git`` (only needed the first time). -* Validating that the specified branch or commit matches the version we claim to build with ``-v``. -* Creating a tarball of the Godot branch or commit we specify with ``-g``. -* Starting builds for all platforms sequentially. - -Builds are done by invoking ``podman`` (RedHat’s better Docker -interface) with relevant arguments and the respective scripts found in -``build-windows/build.sh``, ``build-linux/build.sh``, etc. Review those -scripts for an overview of what SCons options are used for each -platform. - - [!TIP] In a situation where you want to build for one specific - platform or a subset of platforms and not all of them, the easiest - way currently is to manually edit the main ``build.sh`` file (with - ``nano`` or command line editor of choice), and comment out the - ``${podman_run}`` lines you don’t want to build (or ``exit 0`` - early). Remember to reset those changes before making a full official - build. - - You can also build only “classical” (no .NET) or “mono” builds with - the ``-b classical|mono`` option. - -Packaging the binaries -~~~~~~~~~~~~~~~~~~~~~~ - -The next step is to run ``build-release.sh``, which basically takes care -of copying, renaming, signing, notarizing, and zipping all editor and -template binaries in the proper way. - - [!TIP] Like ``build.sh``, this file can be hacked together if you - ever need to do test builds only for some platforms. Just remember to - reset the changes before making a full official build. - -Publishing the binaries -~~~~~~~~~~~~~~~~~~~~~~~ - -Finally, the ``publish-release.sh`` script takes care of: - -- Uploading binaries to GitHub at - https://github.com/godotengine/godot-builds - - - It does this using the scripts in ``~/upload/godot-builds/``, and - a GitHub write token saved in ``~/.bashrc`` for the @GodotBuilder - account, which needs to be renewed occasionally. - -- Uploading the Web editor build to our web server. If the Web editor - build should be marked as the latest stable one, pass the ``-l`` - argument to ``publish-release.sh``. -- Publish the NuGet packages to NuGet Gallery. -- Publish the Android library to MavenCentral. - -For stable builds only, it additionally uploads the release to: - -* GitHub at https://github.com/godotengine/godot - * This requires getting a GitHub write token for one of our accounts. @GodotBuilder doesn’t have write access to godotengine/godot for safety. -* Steam -* EGS -* itch.io - -On Steam, EGS, and itch.io, the builds need to be published manually in -their respective backends (including writing a release post for Steam -and itch.io). This is outside the scope of this howto. - - [!NOTE] Stable releases of the editor should also be uploaded to - Google Play and the Horizon Store. Until we automate this, this - should be synced with @m4gr3d. - -Updating documentation -~~~~~~~~~~~~~~~~~~~~~~ - -The documentation needs version updates in various locations: - -* Update the class reference from `godot `__. -* Create a new branch of the current version in `godot-docs `__. -* Create a new version for this branch on `readthedocs `__. -* Update the version number in `conf.py `__ to the upcoming version on ``master``. - -Using ``screen`` ----------------- - -``screen`` is a “window manager” for the terminal, which lets you have -one or multiple interactive shells which can persist even when logging -off from your main session. - -This is useful over SSH, as the connection might disconnect -unexpectedly, or you might simply need to turn off the computer to go to -bed. - -If you used ``screen`` before starting a build, you can “detach” from -the ``screen`` session, disconnect, and later on reconnect and re-attach -to the ongoing ``screen`` session which will have continued on the -remote host. - -Basic commands to know: - -* ``screen``: Start a session. -* ``screen -r``: Resume a previously detached session. If more than one are detached, you’ll need to specify a PID after ``-r``. -* ``screen -r -d``: Resume a session that wasn’t properly detached (e.g. due to the connection timing out). -* ``exit`` (within a ``screen`` session): Terminate the session, or if multiple virtual terminals were open, close the current one. - -Within a ``screen`` session, commands can be issued using the Ctrl+A -hotkey. This hotkey starts an input mode without visual feedback, which -can take notably these commands: - -* ``C``: Create a new virtual terminal in the current session. -* ``D``: Detach from the current session. -* ``0-9``: Go to the n-th virtual terminal. When creating a new session, there is only one virtual terminal (0), but after using the C command, you can cycle between 0 and 1, etc. - * This is useful when one virtual terminal is busy with a multi-hour build, but you still want to do some checks in a separate terminal (e.g. ``tail -n 1 out/logs/*`` to review the progress). - -See ``man screen`` for more details and possibly cool tricks. diff --git a/maintainers/how_to_become_a_maintainer.rst b/maintainers/how_to_become_a_maintainer.rst deleted file mode 100644 index 55f5f11..0000000 --- a/maintainers/how_to_become_a_maintainer.rst +++ /dev/null @@ -1,4 +0,0 @@ -How to become a maintainer -========================== - -TODO diff --git a/maintainers/introduction.rst b/maintainers/introduction.rst deleted file mode 100644 index a8cfe5f..0000000 --- a/maintainers/introduction.rst +++ /dev/null @@ -1,4 +0,0 @@ -Introduction for new maintainers -================================ - -TODO diff --git a/maintainers/merge_queue.rst b/maintainers/merge_queue.rst deleted file mode 100644 index dce1e10..0000000 --- a/maintainers/merge_queue.rst +++ /dev/null @@ -1,176 +0,0 @@ -Pull request merge queue -======================== - -Guidelines for release coordinators to use the PRs merge queue project -to track, review, and merge PRs during development phases. - -- Target users: **Release coordinators**, and to some extent **team - leaders / area maintainers**. -- Relevant reading for: **All engine contributors**, to understand how - merges are scheduled. -- Other relevant docs: `PR review process `__. - - - Note: This page currently says that either team leaders or - production team members (here “release coordinators”) can merge - PRs. This is slightly outdated and the current process has only - release coordinators doing merges (with exceptions for - e.g. critical CI or build fixes). - -Rationale ---------- - -Godot’s development currently requires that PRs are first approved by -maintainers, and then merged by release coordinators. This is for -multiple reasons: - -- Release coordinators are in charge of making engine builds (dev - snapshots, beta, RC, stable), and need to control the timing of - merges in the ``master`` branch, to ensure the snapshots they build - are suitable for end user testing. -- The PR review process has a number of requirements for both technical - assessment by competent maintainers, and stylistic or organization - considerations. Release coordinators are trained to assess whenever - all relevant reviews have been done, and can sign off on merging the - PR. -- Godot’s CI process is heavy, and to avoid wasting resources and CI - congestion, PRs are merged in batches. This is currently a manual - process done by release coordinators. -- Adding some delay between a “ready to merge” maintainer approval and - the actual merge gives time to other maintainers to voice concerns - they may have. - -Process -------- - -The internal process for merging a PR is: - -- A PR is opened, ``CODEOWNERS`` assigns relevant reviewers, and - reviews are done. An unreviewed PR should *not* be added to the merge - queue project. - - - An exception to this is if the PR is a must-have for the next dev - snapshot (e.g. regression fix, or very important to merge timely - to fit the roadmap for that release), in which case it can be - added to the merge queue even before approval. - - TODO: Should we add a “Priority review” state for this? - -- At least one engine maintainer approves the PR, i.e. it becomes an - “approved PR”. This is the cue to add the PR to the merge queue - project, to hand it over to release coordinators. -- Release coordinators ensure that all approved PRs get tracked in the - merge queue project. See below for how to do this. -- Approved (or priority) PRs can have the following statuses in the - merge queue project: - - - To Review: Despite having one approval, more approvals are still - needed for it to be ready. This should be reflected in the review - requests, and clarified to other release coordinators with the - comment field. - - Needs Work or Consensus: These PRs have been approved at least by - one maintainer, but more work is needed, or a consensus still - needs to be formed. The reason for being shelved here needs to be - written in the comment field. - - Approved: These PRs are approved and a priori ready to merge. - Release coordinators do a second review and write in the comment - field whether it’s indeed ready to merge (with their name), or - still needs some action (e.g. rebase or squashing). During that - second review, PRs can be sent back to “To Review” (if more - maintainers need to approve) or “Needs Work or Consensus”. - - TODO: The state of “approved by maintainers AND approved for merge - by release coordinators” could warrant its own status instead of - relying on a comment. But we still need to know *who* approved for - merge. - -- At least once per day, release coordinators do a merge batch with all - the approved PRs queued for the “Next Wave” and where a release - coordinator has confirmed that they’re ready to go. - -Keeping track of all approved PRs ---------------------------------- - -As a goal, all PRs which have been approved by a maintainer should be -tracked in the merge queue project, and be assessed by release -coordinators in a timely manner. - -This implies: - -- Making sure that PRs that get approved by maintainers get added to - the merge queue project. Approved PRs that haven’t yet been added to - the merge queue can be found with this search: - https://github.com/godotengine/godot/issues?q=is%3Apr%20state%3Aopen%20sort%3Aupdated-desc%20review%3Aapproved%20-project%3Agodotengine%2F90%20(milestone%3A4.4%20OR%20milestone%3A4.x%20OR%20no%3Amilestone) - - - Release coordinators should make it a habit of reviewing this list - daily to find newly approved PRs. - - TODO: The above list is specific to 4.4 (project “90”) but can be - adjusted if/when we switch to a new project for future releases. - -- Assessing PRs added to the merge queue by other maintainers, which - may not have been seen yet by release management. Currently, a lot - can be found in the “Mislabeled” tab as some maintainers don’t set - the wave or status when adding to the merge queue. Likewise, some - maintainers add unapproved PRs to the merge queue (typically their - own, which they want reviewed), and that’s a misuse of the project - that we need to clarify. -- Reviewing PRs ourselves as part of our daily routine, and adding them - right away to the merge queue project with the right wave and status. -- (Re)-Assessing the items in all statuses in the “Next Wave” tab - regularly, to ensure that we don’t miss when a PR that was close to - ready is now fully ready. PRs that were put in “To Review” or “Needs - Work or Consensus” which get a new maintainer review should - automatically be moved to “Approved”, which helps put them back in - the “hot” queue. - -Merging a PR batch ------------------- - -One purpose of the merge queue, aside from doing all the review -validations described above, is to be able to merge batches of multiple -PRs all at once. This helps reduce the strain on CI resources by -starting only one CI workflow for a whole batch being merged. - -The process to make a PR batch is along those lines: - -- Go through the “Approved” PRs with a “Ready to merge” comment from a - release coordinator, and open them all in tabs. Keep those tabs open - through the process. -- Make sure that those PRs are actually ready to merge (there might - have been new comments/reviews made after a release coordinator - approval, or new merge conflicts, etc.). -- Make sure that each PR has the proper milestone (e.g. ``4.4`` if - merged during the 4.4 release cycle), properly references the issues - it closes with a closing keyword, and that those issues also have the - corresponding milestone. -- Copy the PR numbers of all PRs meant for a merge batch in some file. -- Merge them all locally with ``git merge --no-ff`` and a merge commit - message matching what GitHub would generate (see the script below). - **DO NOT REBASE** after this, as it would flatten the merge commits - and lose the association to the original PRs. -- Build once and run the editor to make sure no obvious bug is being - introduced. -- Push to the ``master`` branch. -- Go through the now merged open tabs for each PR and thank the - contributor(s) for their work. - -.. - - [!NOTE] To merge PRs locally in the same way that GitHub would do it, - we use this - ```git-local-merge.py`` `__ - Python script. Make it executable and place it in the ``PATH``, so - that a batch of PRs can be merged with: - - .. code:: bash - - git-local-merge.py 100001 100002 100003 ... - -Future work ------------ - -The above procedure is based on the current workflow with the merge -queue project, and the various usage that different maintainers make of -it. - -It’s meant to clarify for release coordinators how they should use it, -and in that process we should keep re-evaluating how to improve both the -merge queue project and the guidelines further, so that things are -clearer for both maintainers and new release coordinators.