Documentation for special-key-space

design/special-key-space.md

@@ -0,0 +1,81 @@
+# Special-Key-Space
+This document discusses why we need the proposed special-key-space framwork. And for what problems the framework aims to solve and in what scenarios a developer should use it.   
+
+## Motivation
+Currently, there are several client functions implemented as FDB calls by passing through special keys(`prefixed with \xff\xff`). Below are all existing features:
+- **status/json**: `get("\xff\xff/status/json")`
+- **cluster_file_path**: `get("\xff\xff/cluster_file_path)`
+- **connection_string**: `get("\xff\xff/connection_string)`
+- **worker_interfaces**: `getRange("\xff\xff/worker_interfaces", <any_key>)`
+- **conflicting-keys**: `getRange("\xff\xff/transaction/conflicting_keys/", "\xff\xff/transaction/conflicting_keys/\xff")`
+
+At present, implementions are hard-coded and the pain points are obvious:
+- **Maintainability**: As more features added, the hard-coded snippets are hard to maintain 
+- **Granularity**: It is impossible to scale up and down. For example, you want a cheap call like `get("\xff\xff/status/json/<certain_field>")` instead of calling `status/json` and parsing the results. In the constrast, sometime you want to aggregate results from several similiar features like `getRange("\xff\xff/transaction/, \xff\xff/transaction/\xff")` to get all transaction related info. Both of them are not achievable at present.
+- **Consistency**: While using FDB calls like `get` or `getRange`, the behavior that the result of `get("\xff\xff/B")` is not included in `getRange("\xff\xff/A", "\xff\xff/C")` is inconsistent with general FDB calls.
+
+Consequently, the special-key-space framework wants to integrate all client functions using special keys(`prefixed with \xff`) and solve the pain points listed above.
+
+## When
+If your feature is exposing information to clients and the results are easily formatted as key-value pairs, then you can use special-key-space to implement your client function.
+
+## How
+If you choose to use, you need to implement a function class that inherits from `SpecialKeyRangeBaseImpl`, which has an abstract method `Future<Standalone<RangeResultRef>> getRange(Reference<ReadYourWritesTransaction> ryw, KeyRangeRef kr)`.
+This method can be treated as a callback, whose implementation details are determined by the developer.
+Once you fill out the method, register the function class to the corresponding key range.
+Below is a detailed example.
+```c++
+// Implement the function class,
+// the corresponding key range is [\xff\xff/example/, \xff\xff/example/\xff)
+class SKRExampleImpl : public SpecialKeyRangeBaseImpl {
+public:
+    explicit SKRExampleImpl(KeyRangeRef kr): SpecialKeyRangeBaseImpl(kr) {
+        // Our implementation is quite simple here, the key-value pairs are formatted as:
+        // \xff\xff/example/<country_name> : <capital_city_name>
+        CountryToCapitalCity[LiteralStringRef("USA")] = LiteralStringRef("Washington, D.C.");
+        CountryToCapitalCity[LiteralStringRef("UK")] = LiteralStringRef("London");
+        CountryToCapitalCity[LiteralStringRef("Japan")] = LiteralStringRef("Tokyo");
+        CountryToCapitalCity[LiteralStringRef("China")] = LiteralStringRef("Beijing");
+    }
+    // Implement the getRange interface
+    Future<Standalone<RangeResultRef>> getRange(Reference<ReadYourWritesTransaction> ryw,
+                                            KeyRangeRef kr) const override {
+        
+        Standalone<RangeResultRef> result;
+        for (auto const& country : CountryToCapitalCity) {
+            // the registered range here: [\xff\xff/example/, \xff\xff/example/\xff]
+            Key keyWithPrefix = country.first.withPrefix(range.begin);
+            // check if any valid keys are given in the range
+            if (kr.contains(keyWithPrefix)) {
+                result.push_back(result.arena(), KeyValueRef(keyWithPrefix, country.second));
+                result.arena().dependsOn(keyWithPrefix.arena());
+            }
+        }
+        return result;
+    }
+private:
+    std::map<Key, Value> CountryToCapitalCity;
+};
+// Instantiate the function object
+// In development, you should have a function object pointer in DatabaseContext(DatabaseContext.h) and initialize in DatabaseContext's constructor(NativeAPI.actor.cpp)
+const KeyRangeRef exampleRange(LiteralStringRef("\xff\xff/example/"), LiteralStringRef("\xff\xff/example/\xff"));
+SKRExampleImpl exampleImpl(exampleRange);
+// Assuming the database handler is `cx`, register to special-key-space
+// In development, you should register all function objects in the constructor of DatabaseContext(NativeAPI.actor.cpp)
+cx->specialKeySpace->registerKeyRange(exampleRange, &exampleImpl);
+// Now any ReadYourWritesTransaction associated with `cx` is able to query the info
+state ReadYourWritesTransaction tr(cx);
+// get
+Optional<Value> res1 = wait(tr.get("\xff\xff/example/Japan"));
+ASSERT(res1.present() && res.getValue() == LiteralStringRef("Tokyo"));
+// getRange
+// Note: for getRange(key1, key2), both key1 and key2 should prefixed with \xff\xff
+// something like getRange("normal_key", "\xff\xff/...") is not supported yet
+Standalone<RangeResultRef> res2 = wait(tr.getRange(LiteralStringRef("\xff\xff/example/U"), LiteralStringRef("\xff\xff/example/U\xff")));
+// res2 should contain USA and UK
+ASSERT(
+    res2.size() == 2 &&
+    res2[0].value == LiteralStringRef("London") &&
+    res2[1].value == LiteralStringRef("Washington, D.C.")
+);
+```