Merge pull request #5060 from sfc-gh-ljoswiak/docs/special-keys-global-config-tracing

Add documentation for global configuration and tracing special key space modules

documentation/sphinx/source/api-reference.rst

@@ -17,3 +17,4 @@ The following documents give detailed descriptions of the API for each language:
    api-c
    api-error-codes
    special-keys
+   global-configuration

documentation/sphinx/source/global-configuration.rst

@@ -0,0 +1,130 @@
+.. _global-configuration:
+.. default-domain:: cpp
+.. highlight:: cpp
+
+====================
+Global Configuration
+====================
+
+The global configuration framework is an eventually consistent configuration
+mechanism to efficiently make runtime changes to all clients and servers. It
+works by broadcasting updates made to the global configuration key space,
+relying on individual machines to store existing configuration in-memory.
+
+The global configuration framework provides a key-value interface to all
+processes and clients in a FoundationDB cluster.
+
+The global configuration framework is internal to FoundationDB and clients will
+usually have no need to interact with it. The API is provided here for
+reference.
+
+Reading data
+------------
+
+The global configuration framework is exposed through the
+``GlobalConfig::globalConfig()`` static function. There are separate ways to
+read a value, depending on if it is an object or a primitive.
+
+.. function:: template<class T> const T get(KeyRef name, T defaultVal)
+
+   Returns the value associated with ``name`` stored in global configuration,
+   or ``defaultVal`` if no key matching ``name`` exists. This templated
+   function is enabled only when the ``std::is_arithmetic<T>`` specialization
+   returns true.
+
+   .. code-block:: cpp
+   
+      auto& config = GlobalConfig::globalConfig();
+      double value = config.get<double>("path/to/key", 1.0);
+
+.. function:: const Reference<ConfigValue> get(KeyRef name)
+
+   Returns the value associated with ``name`` stored in global configuration.
+
+   .. code-block:: cpp
+   
+      auto& config = GlobalConfig::globalConfig();
+      auto configValue = config.get("path/to/key");
+   
+      // Check if value exists
+      ASSERT(configValue.value.has_value());
+      // Cast to correct type
+      auto str = std::any_cast<StringRef>(configValue.value);
+
+.. function:: const std::map<KeyRef, Reference<ConfigValue>> get(KeyRangeRef range)
+
+   Returns all values in the specified range.
+
+.. type:: ConfigValue
+
+   Holds a global configuration value and the arena where it lives. ::
+
+     struct ConfigValue : ReferenceCounted<ConfigValue> {
+        Arena arena;
+        std::any value;
+     }
+
+   ``arena``
+       The arena where the value (and the associated key) lives in memory.
+
+   ``value``
+       The stored value.
+
+Writing data
+------------
+
+Writing data to global configuration is done using transactions written to the
+special key space range ``\xff\xff/global_config/ - \xff\xff/global_config/0``.
+Values must always be encoded according to the :ref:`api-python-tuple-layer`.
+
+.. code-block:: cpp
+
+   // In GlobalConfig.actor.h
+   extern const KeyRef myGlobalConfigKey;
+   // In GlobalConfig.actor.cpp
+   const KeyRef myGlobalConfigKey = LiteralStringRef("config/key");
+   
+   // When you want to set the value..
+   Tuple value = Tuple().appendDouble(1.5);
+   
+   FDBTransaction* tr = ...;
+   tr->setOption(FDBTransactionOptions::SPECIAL_KEY_SPACE_ENABLE_WRITES);
+   tr->set(GlobalConfig::prefixedKey(myGlobalConfigKey), value.pack());
+   // commit transaction
+
+The client is responsible for avoiding conflicts with other global
+configuration keys. For most uses, it is recommended to create a new key space.
+For example, an application that wants to write configuration data should use
+the ``global_config/config/`` namespace, instead of storing keys in the top
+level ``global_config/`` key space.
+
+^^^^^^^^^^^^^
+Clearing data
+^^^^^^^^^^^^^
+
+Data can be removed from global configuration using standard transaction
+semantics. Submit a clear or clear range to the appropriate global
+configuration keys in the special key space to clear data.
+
+Watching data
+-------------
+
+Global configuration provides functionality to watch for changes.
+
+.. function:: Future<Void> onInitialized()
+
+   Returns a ``Future`` which will be triggered when global configuration has
+   been successfully initialized and populated with data.
+
+.. function:: Future<Void> onChange()
+
+   Returns a ``Future`` which will be triggered when any key-value pair in
+   global configuration changes.
+
+.. function:: void trigger(KeyRef key, std::function<void(std::optional<std::any>)> fn)
+
+   Registers a callback to be called when the value for global configuration
+   key ``key`` is changed. The callback function takes a single argument, an
+   optional which will be populated with the updated value when ``key`` is
+   changed, or an empty optional if the value was cleared. If the value is an
+   allocated object, its memory remains in control of global configuration.

documentation/sphinx/source/special-keys.rst

@@ -234,6 +234,27 @@ command                    string   The fdbcli command corresponding to this ope
 message                    string   Help text explaining the reason this operation failed
 ========================== ======== ===============
 
+Global configuration module
+---------------------------
+
+The global configuration module provides an interface to read and write values
+to :doc:`global-configuration`. In general, clients should not read and write
+the global configuration special key space keys directly, but should instead
+use the global configuration functions.
+
+#. ``\xff\xff/global_config/<key> := <value>`` Read/write. Reading keys in the range will return a tuple decoded string representation of the value for the given key. Writing a value will update all processes in the cluster with the new key-value pair. Values must be written using the :ref:`api-python-tuple-layer`.
+
+Tracing module
+--------------
+
+The tracing module provides read and write access to a transactions' tracing
+data. Every transaction contains a unique identifier which follows the
+transaction through the system. By providing access to set this identifier,
+clients can connect FoundationDB transactions to outside events.
+
+#. ``\xff\xff/tracing/transaction_id := <transaction_id>`` Read/write. A 64-bit integer transaction ID which follows the transaction as it moves through FoundationDB. All transactions are assigned a random transaction ID on creation, and this key can be read to surface the randomly generated ID. Alternatively, set this key to provide a custom identifier. When setting this key, provide a string in the form of a 64-bit integer, which will be automatically converted to the appropriate type.
+#. ``\xff\xff/tracing/token := <tracing_enabled>`` Read/write. Set to true/false to enable or disable tracing for the transaction, respectively. If read, returns a 64-bit integer set to 0 if tracing has been disabled, or a random 64-bit integer otherwise (this integers value has no meaning to the client other than to determine whether the transaction will be traced).
+
 .. [#conflicting_keys] In practice, the transaction probably committed successfully. However, if you're running multiple resolvers then it's possible for a transaction to cause another to abort even if it doesn't commit successfully.
 .. [#max_read_transaction_life_versions] The number 5000000 comes from the server knob MAX_READ_TRANSACTION_LIFE_VERSIONS
 .. [#special_key_space_enable_writes] Enabling this option enables other transaction options, such as ``ACCESS_SYSTEM_KEYS``. This may change in the future.