bin dist related docs (#2684)

* bin dist related docs

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docker/README.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/binaries.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/distribution_internals.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/distribution_internals.md

Co-authored-by: 0xmiel <sacha@status.im>

* Update docs/the_nimbus_book/src/binaries.md

Co-authored-by: 0xmiel <sacha@status.im>

Co-authored-by: 0xmiel <sacha@status.im>

docker/README.md

@@ -0,0 +1,23 @@
+## directory layout
+
+### dist/base_image/
+
+Base Docker images for building distributable binaries. Uploaded to
+Docker Hub (we need them to reproduce officially released builds).
+
+### dist/
+
+Dockerfiles used to build local Docker images based on the base images
+described above. Only used for generating distributable binaries. Not uploaded
+to Docker Hub.
+
+### dist/binaries/
+
+Docker images for end-users, obtained by copying distributable binaries inside
+official Debian images. Uploaded to Docker Hub as part of the CI release process.
+
+Also contains some example `docker-compose` configuration files.
+
+## more details
+
+See the ["Binary distribution internals"](https://nimbus.guide/distribution_internals.html) page of the Nimbus book.

docker/dist/entry_point.sh

@@ -192,7 +192,7 @@ for BINARY in ${BINARIES}; do
   fi
   cd - >/dev/null
 done
-sed -e "s/GIT_COMMIT/${GIT_COMMIT}/" docker/dist/README.md > "${DIST_PATH}/README.md"
+sed -e "s/GIT_COMMIT/${GIT_COMMIT}/" docker/dist/README.md.tpl > "${DIST_PATH}/README.md"
 
 if [[ "${PLATFORM}" == "Linux_amd64" ]]; then
   sed -i -e 's/^make dist$/make dist-amd64/' "${DIST_PATH}/README.md"
@@ -202,7 +202,7 @@ elif [[ "${PLATFORM}" == "Linux_arm64v8" ]]; then
   sed -i -e 's/^make dist$/make dist-arm64/' "${DIST_PATH}/README.md"
 elif [[ "${PLATFORM}" == "Windows_amd64" ]]; then
   sed -i -e 's/^make dist$/make dist-win64/' "${DIST_PATH}/README.md"
-  cp -a docker/dist/README-Windows.md "${DIST_PATH}/"
+  cp -a docker/dist/README-Windows.md.tpl "${DIST_PATH}/README-Windows.md"
 elif [[ "${PLATFORM}" == "macOS_amd64" ]]; then
   sed -i -e 's/^make dist$/make dist-macos/' "${DIST_PATH}/README.md"
 elif [[ "${PLATFORM}" == "macOS_arm64" ]]; then

docs/README.md

@@ -10,7 +10,7 @@ cargo install mdbook mdbook-toc mdbook-open-on-gh
 cd docs/the_nimbus_book
 mdbook serve -p 4000
 
-# Publish book using makefile (in top-level)
+# Publish book using makefile (in the top-level dir)
 make publish-book
 ```
 

docs/the_nimbus_book/src/binaries.md

@@ -1,18 +1,21 @@
 # Nimbus binaries
 
-We currently have binaries available for Linux `AMD64`, `ARM32` and `ARM64`, Windows and  MacOS (`AMD64` and `ARM64`).
+We currently have binaries available for Linux `AMD64`, `ARM` and `ARM64`, Windows `AMD64` and macOS (`AMD64` and `ARM64`).
 
 You can find the latest release here: [https://github.com/status-im/nimbus-eth2/releases](https://github.com/status-im/nimbus-eth2/releases)
 
-Scroll to the bottom of the first release you see, and click on `Assets`. You should see a list that looks like the following:
+Scroll to the bottom of the first release you see (no, not the nightly pre-release), and click on `Assets`. You should see a list that looks like the following:
 
-![](https://i.imgur.com/R33o4MG.png)
+![](https://i.imgur.com/6wuvM2d.png)
 
 Click on the `tar.gz` file that corresponds to your OS and architecture, unpack the archive, read the README and run the binary directly (or through one of our provided wrapper scripts).
 
 We've designed the build process to be reproducible. In practice, this means that anyone can verify that these exact binaries were produced from the corresponding source code commits. For more about the philosophy and importance of this feature see [reproducible-builds.org](https://reproducible-builds.org/).
 
-For instructions on how to reproduce the build, [see here](https://github.com/status-im/nimbus-eth2/blob/master/docker/dist/README.md#reproducing-the-build).
-
-> **N.B.** Binaries are built with Docker. As such they do not support the `log-file` [option](./options.md). We recommend capturing the console output with a [log rotation helper](./log-rotate.md) instead. 
+For instructions on how to reproduce those binaries, see "README.md" inside the archive.
 
+> **N.B.** These binaries are meant to run as a service. As such, they do not
+support the `log-file` [option](./options.md) - duplicating stdout to a
+log file would complicate things. We recommend capturing
+their output (stdout and stderr) with a [log rotation
+helper](./log-rotate.md) instead.

docs/the_nimbus_book/src/distribution_internals.md

@@ -11,16 +11,17 @@ for software that deals with digital tokens of significant value.
 
 ## Docker containers for internal use
 
-The easiest way to guarantee that users are able to replicate 
+The easiest way to guarantee that users are able to replicate
 our binaries for themselves is to give them the same software environment we used in CI. Docker
 containers fit the bill, so everything starts with the architecture- and
-OS-specific containers in `docker/dist/base\_image/`.
+OS-specific containers in `docker/dist/base_image/`.
 
 These images contain all the packages we need, are built and published once (to
 Docker Hub), and are then reused as the basis for temporary Docker
 images where the `nimbus-eth2` build is carried out.
 
-These temporary images are controlled by Dockerfiles in `docker/dist/`. Since we're not publishing them anywhere, we can customize them to the system
+These temporary images are controlled by Dockerfiles in `docker/dist/`. Since
+we're not publishing them anywhere, we can customize them to the system
 they run on (we ensure they use the host's UID/GID, the host's QEMU static
 binaries, etc); they get access to the source code through the use of external volumes.
 
@@ -30,25 +31,25 @@ It all starts from the GitHub actions in `.github/workflows/release.yml`. There
 is a different job for each supported OS-architecture combination and they all
 run in parallel (ideally).
 
-The `build-amd64` CI job is special, because it creates a new
-GitHub release draft, as soon as possible. All the other jobs will upload their
-binary distribution tarballs to this draft release, but, since it's not feasible
-to communicate between CI jobs, they simply use GitHub APIs to find out what
-the latest release is, check that it has the right Git tag, and use that as their
-last step.
+Once all those CI jobs complete successfully, a GitHub release draft is created
+and all the distributable archives are uploaded to it. A list of checksums for
+the main binaries is inserted in the release description. That draft needs to
+be manually published.
 
-The build itself is triggered by a Make target: `make dist-amd64`. This invokes
-`scripts/make\_dist.sh` which builds the corresponding Docker container from
+The build itself is triggered by a Make target. E.g.: `make dist-amd64`. This invokes
+`scripts/make_dist.sh` which builds the corresponding Docker container from
 `docker/dist/` and runs it with the Git repository's top directory as an external
 volume.
 
-The entry point for that container is `docker/dist/entry\_point.sh` and that's
+The entry point for that container is `docker/dist/entry_point.sh` and that's
 where you'll find the Make invocations needed to finally build the software and
 create distributable tarballs.
 
 ## Docker images for end users
 
-Configured in `.github/workflows/release.yml` (exclusively for the `build-amd64` job):  we unpack the distribution tarball and copy its content into a third type of Docker image - this one meant for end users and defined by `docker/dist/binaries/Dockerfile.amd64`.
+Configured in `.github/workflows/release.yml` (only for Linux AMD64, ARM and
+ARM64): we unpack the distribution tarball and copy its content into a third
+type of Docker image - meant for end users and defined by
+`docker/dist/binaries/Dockerfile.amd64` (and related).
 
 We then publish that to [Docker Hub](https://hub.docker.com/r/statusim/nimbus-eth2).
-