Fixes #1690: Server docker image hard-codes 4500 in a few places

This makes the default public port for starting FDB processes the same as the FDB_PORT. This is probably necessary given #1714, especially for coordinators, though it might not be necessary for other processes in the cluster. This can *almost* be used to start up multiple FDB processes locally and then access them from the same machine, but that (unfortunately) requires both the other processes in the docker compose network and the host machine to agree on what IP to use for the coordinator. But as that machine has different IPs in those networks, they cannot be made to agree.

packaging/docker/Dockerfile

@@ -70,5 +70,6 @@ ENV FDB_PORT 4500
 ENV FDB_CLUSTER_FILE /var/fdb/fdb.cluster
 ENV FDB_NETWORKING_MODE container
 ENV FDB_COORDINATOR ""
+ENV FDB_COORDINATOR_PORT 4500
 ENV FDB_CLUSTER_FILE_CONTENTS ""
 ENV FDB_PROCESS_CLASS unset

packaging/docker/README.md

@@ -57,6 +57,13 @@ helpful when setting up a larger cluster inside a docker network, for instance
 when using Docker Compose. The name you provide must be resolvable through the
 DNS on the container you are running.
 
+### FDB_COORDINATOR_PORT
+
+The port to use for connecting to the FDB coordinator process. This should be
+set by other processes in a multi-process cluster to the same value as the
+`FDB_PORT` environment variable of the coordinator process. It will default
+to 4500, which is also the default for `FDB_PORT`.
+
 # Copying Into Other Images
 
 You can also use this image to provide files for images that are clients of a
@@ -68,4 +75,4 @@ files you may want to copy are:
 	library, which you can use if you are setting up a multiversion client.
 *	`/var/fdb/scripts/create_cluster_file.bash`: A script for setting up the
 	cluster file based on an `FDB_COORDINATOR` environment variable.
-*	`/usr/bin/fdbcli`: The FoundationDB CLI.
+*	`/usr/bin/fdbcli`: The FoundationDB CLI.

packaging/docker/create_cluster_file.bash

@@ -39,7 +39,7 @@ function create_cluster_file() {
 			echo "Failed to look up coordinator address for $FDB_COORDINATOR" 1>&2
 			exit 1
 		fi
-		echo "docker:docker@$coordinator_ip:4500" > $FDB_CLUSTER_FILE
+		echo "docker:docker@$coordinator_ip:$FDB_COORDINATOR_PORT" > $FDB_CLUSTER_FILE
 	else
 		echo "FDB_COORDINATOR environment variable not defined" 1>&2
 		exit 1
@@ -48,4 +48,4 @@ function create_cluster_file() {
 
 if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
     create_cluster_file "$@"
-fi
+fi

packaging/docker/create_server_environment.bash

@@ -43,4 +43,4 @@ function create_server_environment() {
 	fi
 
 	create_cluster_file
-}
+}

packaging/docker/fdb.bash

@@ -23,7 +23,7 @@
 source /var/fdb/scripts/create_server_environment.bash
 create_server_environment
 source /var/fdb/.fdbenv
-echo "Starting FDB server on $PUBLIC_IP:4500"
-fdbserver --listen_address 0.0.0.0:$FDB_PORT --public_address $PUBLIC_IP:4500 \
+echo "Starting FDB server on $PUBLIC_IP:$FDB_PORT"
+fdbserver --listen_address 0.0.0.0:$FDB_PORT --public_address $PUBLIC_IP:$FDB_PORT \
 	--datadir /var/fdb/data --logdir /var/fdb/logs \
-	--locality_zoneid=`hostname` --locality_machineid=`hostname` --class $FDB_PROCESS_CLASS
+	--locality_zoneid=`hostname` --locality_machineid=`hostname` --class $FDB_PROCESS_CLASS

packaging/docker/samples/local/README.md

@@ -0,0 +1,45 @@
+# Local Docker-based FoundationDB Cluster
+
+This contains a sample `docker-compose.yaml` and some simple startup and teardown
+scripts for running a simple single-instance FoundationDB using the Docker image
+specified in this repository. This uses the `host` networking option to expose
+the server process to its host machine.
+
+This depends on having the FoundationDB client installed on your host machine
+to work properly. This can be done using one of the client packages available
+on our [Download](https://www.foundationdb.org/download/) page. The startup
+scripts included here depend on `fdbcli` from one of those packages, and any
+client that wishes to connect will need a copy of the FoundationDB native client
+in addition to its binding of choice. Both the CLI and the native client
+are installed in all of our client packages
+
+Once those dependencies are installed, one can build the FoundationDB Docker
+image:
+
+```
+docker build --build-arg FDB_VERSION=6.1.8 -t foundationdb:6.1.8 ../..
+```
+
+Then one can start the cluster by running:
+
+```
+./start.bash
+```
+
+This starts up a single instance FoundationDB cluster using the `docker-compose.yaml`
+and configures it as a new database. This will write the cluster file information to
+`docker.cluster`. One should then be able to access the cluster through the CLI
+or one of the bindings by using this cluster file. For example:
+
+```
+fdbcli --exec status -C docker.cluster
+```
+
+To stop the cluster, one can run:
+
+```
+./stop.bash
+```
+
+Note that all data are lost between reboots of the processes as they have not
+been configured to use a persistent volume (but write to Docker's temporary file system).

packaging/docker/samples/local/docker-compose.yml

@@ -0,0 +1,32 @@
+# docker-compose.yaml
+#
+# This source file is part of the FoundationDB open source project
+#
+# Copyright 2013-2018 Apple Inc. and the FoundationDB project authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Specification for a one node cluster than can be accessed from the host.
+# The user must specify the FDB_PORT on which it is run.
+
+version: '3'
+services:
+  fdb:
+    image: foundationdb:6.1.8
+    ports:
+      - $FDB_PORT:$FDB_PORT/tcp
+    environment:
+      FDB_NETWORKING_MODE: host
+      FDB_COORDINATOR_PORT: $FDB_PORT
+      FDB_PORT: $FDB_PORT

packaging/docker/samples/local/start.bash

@@ -0,0 +1,39 @@
+#! /bin/bash
+
+#
+# start.bash
+#
+# This source file is part of the FoundationDB open source project
+#
+# Copyright 2013-2018 Apple Inc. and the FoundationDB project authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -eu
+
+FDB_CLUSTER_FILE="${FDB_CLUSTER_FILE:-docker.cluster}"
+FDB_PORT="${FDB_PORT:-4550}"
+
+FDB_PORT=$FDB_PORT docker-compose up -d fdb
+echo "docker:docker@127.0.0.1:$FDB_PORT" > $FDB_CLUSTER_FILE
+
+# Attempt to connect. Configure the database if necessary.
+if ! fdbcli -C $FDB_CLUSTER_FILE --exec status --timeout 1 ; then
+    if ! fdbcli -C $FDB_CLUSTER_FILE --exec "configure new single memory ; status" --timeout 10 ; then 
+        echo "Unable to configure new FDB cluster."
+        exit 1
+    fi
+fi
+
+echo "Can now connect to docker-based FDB cluster using $FDB_CLUSTER_FILE."

packaging/docker/samples/local/stop.bash

@@ -0,0 +1,28 @@
+#! /bin/bash
+
+#
+# stop.bash
+#
+# This source file is part of the FoundationDB open source project
+#
+# Copyright 2013-2018 Apple Inc. and the FoundationDB project authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -eu
+
+FDB_PORT="${FDB_PORT:-4550}"
+
+FDB_PORT=$FDB_PORT docker-compose down
+echo "Docker-based FDB cluster is now down."

packaging/docker/samples/python/app/Dockerfile

@@ -24,9 +24,9 @@ RUN apt-get update; apt-get install -y dnsutils
 RUN mkdir -p /app
 WORKDIR /app
 
-COPY --from=foundationdb:5.2.5 /usr/lib/libfdb_c.so /usr/lib
-COPY --from=foundationdb:5.2.5 /usr/bin/fdbcli /usr/bin/
-COPY --from=foundationdb:5.2.5 /var/fdb/scripts/create_cluster_file.bash /app
+COPY --from=foundationdb:6.1.8 /usr/lib/libfdb_c.so /usr/lib
+COPY --from=foundationdb:6.1.8 /usr/bin/fdbcli /usr/bin/
+COPY --from=foundationdb:6.1.8 /var/fdb/scripts/create_cluster_file.bash /app
 
 COPY requirements.txt /app
 RUN pip install -r requirements.txt
@@ -38,4 +38,4 @@ RUN chmod u+x /app/start.bash
 CMD /app/start.bash
 
 ENV FLASK_APP=server.py
-ENV FLASK_ENV=development
+ENV FLASK_ENV=development

packaging/docker/samples/python/docker-compose.yml

@@ -19,18 +19,34 @@
 
 version: '3'
 services:
-  fdb:
-    image: foundationdb:5.2.5
-    environment:
-      FDB_COORDINATOR: fdb-coordinator
+  # Specify three fdbserver processes.
   fdb-coordinator:
-    image: foundationdb:5.2.5
+    image: foundationdb:6.1.8
     environment:
       FDB_COORDINATOR: fdb-coordinator
+  fdb-server-1:
+    depends_on:
+      - fdb-coordinator
+    image: foundationdb:6.1.8
+    environment:
+      FDB_COORDINATOR: fdb-coordinator
+  fdb-server-2:
+    depends_on:
+      - fdb-coordinator
+    image: foundationdb:6.1.8
+    environment:
+      FDB_COORDINATOR: fdb-coordinator
+
+  # Bring up the application so that it depends on the cluster.
   app:
+    depends_on:
+      - fdb-coordinator
+      - fdb-server-1
+      - fdb-server-2
     build:
       context: app
     ports:
       - 5000:5000
     environment:
-      FDB_COORDINATOR: fdb-coordinator
+      FDB_COORDINATOR: fdb-coordinator
+      FDB_COORDINATOR_PORT: 4550