add key-value-sqlite host component
This is intended to be the default key-value store for Spin. Eventually, we'll provide others based on different backends (e.g. Redis) which may be selected and configured as part of spin.toml, a runtime config file, or a CLI option. Since I was wondering what overhead the SQLite dependency would add to Spin, I did some informal comparisons. It increases the stripped Spin release binary size by about 3% and increases the clean release build time by about 1.5%. simplify `KeyValueSqlite::get` fix type name in comment remove unused error variant case from key-value.wit various key-value store updates - Allow the key-value host component dynamically dispatch to multiple implementations (e.g. allowing an app to use SQLite and Redis simultaneously) - Enable configuring the key-value implementations when creating the host component - Rename `namespace` to `store` - Use `tokio::task::block_in_place` when doing SQLite I/O fix grammar in comment key-value: only support the default (unnamed) store by default This also adds a `rust-key-value` example. add documentation to table.rs remove unnecessary package.metadata.component directive from example address review feedback - Add more detailed comment about default KV store config and future runtime config - Add `Store::default` to SDK rename `sqlite::Config` to `sqlite::DatabaseLocation` address clippy warnings and fix doc grammar use local app dir for default KV database For remote apps and local apps in a read-only database, fall back to an in-memory database. These defaults may be overridden using the `--key-value-file` option. sync implementation with SIP - change `error::runtime` to `error::io` - add `get-keys` function to interface - remove `--key-value-file` CLI option in favor of a runtime config CLI rename default store to "default" add `key_value_stores` key to component config This tells Spin which stores, if any, a component is allowed to access. move SQLite KV implementation into its own crate This makes the `key-value` crate entirely implementation-agnostic, allowing runtimes to plug in whatever they want, without pulling in an SQLite dependency which might not be relevant. use "origin" metadata from app to find spin.toml refactor key-value host component to avoid deep clones update rust-key-value example Signed-off-by: Joel Dice <joel.dice@fermyon.com>
This commit is contained in:
parent
2808827891
commit
7020a0c47b
|
@ -8,7 +8,8 @@ main.wasm
|
||||||
.vscode/*.log
|
.vscode/*.log
|
||||||
tests/**/Cargo.lock
|
tests/**/Cargo.lock
|
||||||
crates/**/Cargo.lock
|
crates/**/Cargo.lock
|
||||||
Cargo.lock
|
examples/**/Cargo.lock
|
||||||
tests/testcases/*-generated
|
tests/testcases/*-generated
|
||||||
spin-plugin-update.lock
|
spin-plugin-update.lock
|
||||||
package-lock.json
|
package-lock.json
|
||||||
|
.spin
|
||||||
|
|
|
@ -1357,6 +1357,12 @@ version = "0.2.0"
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
|
checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "fallible-streaming-iterator"
|
||||||
|
version = "0.1.9"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "fastrand"
|
name = "fastrand"
|
||||||
version = "1.8.0"
|
version = "1.8.0"
|
||||||
|
@ -1754,6 +1760,15 @@ dependencies = [
|
||||||
"ahash",
|
"ahash",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "hashlink"
|
||||||
|
version = "0.8.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "69fe1fcf8b4278d860ad0548329f892a3631fb63f82574df68275f34cdbe0ffa"
|
||||||
|
dependencies = [
|
||||||
|
"hashbrown 0.12.3",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "heck"
|
name = "heck"
|
||||||
version = "0.3.3"
|
version = "0.3.3"
|
||||||
|
@ -2131,6 +2146,30 @@ dependencies = [
|
||||||
"sha2 0.10.6",
|
"sha2 0.10.6",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "key-value"
|
||||||
|
version = "0.8.0"
|
||||||
|
dependencies = [
|
||||||
|
"anyhow",
|
||||||
|
"spin-app",
|
||||||
|
"spin-core",
|
||||||
|
"tokio",
|
||||||
|
"tracing",
|
||||||
|
"wit-bindgen-wasmtime",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "key-value-sqlite"
|
||||||
|
version = "0.1.0"
|
||||||
|
dependencies = [
|
||||||
|
"anyhow",
|
||||||
|
"key-value",
|
||||||
|
"once_cell",
|
||||||
|
"rusqlite",
|
||||||
|
"tokio",
|
||||||
|
"wit-bindgen-wasmtime",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "kstring"
|
name = "kstring"
|
||||||
version = "1.0.6"
|
version = "1.0.6"
|
||||||
|
@ -2259,6 +2298,17 @@ dependencies = [
|
||||||
"winapi",
|
"winapi",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "libsqlite3-sys"
|
||||||
|
version = "0.25.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "29f835d03d717946d28b1d1ed632eb6f0e24a299388ee623d0c23118d3e8a7fa"
|
||||||
|
dependencies = [
|
||||||
|
"cc",
|
||||||
|
"pkg-config",
|
||||||
|
"vcpkg",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "libz-sys"
|
name = "libz-sys"
|
||||||
version = "1.1.8"
|
version = "1.1.8"
|
||||||
|
@ -3579,6 +3629,20 @@ dependencies = [
|
||||||
"winapi",
|
"winapi",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "rusqlite"
|
||||||
|
version = "0.28.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "01e213bc3ecb39ac32e81e51ebe31fd888a940515173e3a18a35f8c6e896422a"
|
||||||
|
dependencies = [
|
||||||
|
"bitflags",
|
||||||
|
"fallible-iterator",
|
||||||
|
"fallible-streaming-iterator",
|
||||||
|
"hashlink",
|
||||||
|
"libsqlite3-sys",
|
||||||
|
"smallvec",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "rust_decimal"
|
name = "rust_decimal"
|
||||||
version = "1.28.0"
|
version = "1.28.0"
|
||||||
|
@ -4143,6 +4207,8 @@ dependencies = [
|
||||||
"hippo-openapi",
|
"hippo-openapi",
|
||||||
"hyper",
|
"hyper",
|
||||||
"is-terminal",
|
"is-terminal",
|
||||||
|
"key-value",
|
||||||
|
"key-value-sqlite",
|
||||||
"lazy_static",
|
"lazy_static",
|
||||||
"nix 0.24.3",
|
"nix 0.24.3",
|
||||||
"openssl",
|
"openssl",
|
||||||
|
@ -4445,6 +4511,8 @@ dependencies = [
|
||||||
"ctrlc",
|
"ctrlc",
|
||||||
"dirs 4.0.0",
|
"dirs 4.0.0",
|
||||||
"futures",
|
"futures",
|
||||||
|
"key-value",
|
||||||
|
"key-value-sqlite",
|
||||||
"oci-distribution",
|
"oci-distribution",
|
||||||
"outbound-http",
|
"outbound-http",
|
||||||
"outbound-mysql",
|
"outbound-mysql",
|
||||||
|
|
|
@ -33,6 +33,8 @@ lazy_static = "1.4.0"
|
||||||
nix = { version = "0.24", features = ["signal"] }
|
nix = { version = "0.24", features = ["signal"] }
|
||||||
outbound-http = { path = "crates/outbound-http" }
|
outbound-http = { path = "crates/outbound-http" }
|
||||||
outbound-redis = { path = "crates/outbound-redis" }
|
outbound-redis = { path = "crates/outbound-redis" }
|
||||||
|
key-value = { path = "crates/key-value" }
|
||||||
|
key-value-sqlite = { path = "crates/key-value-sqlite" }
|
||||||
path-absolutize = "3.0.11"
|
path-absolutize = "3.0.11"
|
||||||
rand = "0.8"
|
rand = "0.8"
|
||||||
regex = "1.5.5"
|
regex = "1.5.5"
|
||||||
|
@ -101,6 +103,8 @@ members = [
|
||||||
"crates/manifest",
|
"crates/manifest",
|
||||||
"crates/outbound-http",
|
"crates/outbound-http",
|
||||||
"crates/outbound-redis",
|
"crates/outbound-redis",
|
||||||
|
"crates/key-value",
|
||||||
|
"crates/key-value-sqlite",
|
||||||
"crates/plugins",
|
"crates/plugins",
|
||||||
"crates/redis",
|
"crates/redis",
|
||||||
"crates/templates",
|
"crates/templates",
|
||||||
|
|
2
build.rs
2
build.rs
|
@ -9,6 +9,7 @@ use cargo_target_dep::build_target_dep;
|
||||||
|
|
||||||
const RUST_HTTP_INTEGRATION_TEST: &str = "tests/http/simple-spin-rust";
|
const RUST_HTTP_INTEGRATION_TEST: &str = "tests/http/simple-spin-rust";
|
||||||
const RUST_HTTP_INTEGRATION_ENV_TEST: &str = "tests/http/headers-env-routes-test";
|
const RUST_HTTP_INTEGRATION_ENV_TEST: &str = "tests/http/headers-env-routes-test";
|
||||||
|
const RUST_HTTP_INTEGRATION_KEY_VALUE_TEST: &str = "tests/http/key-value";
|
||||||
const RUST_HTTP_VAULT_CONFIG_TEST: &str = "tests/http/vault-config-test";
|
const RUST_HTTP_VAULT_CONFIG_TEST: &str = "tests/http/vault-config-test";
|
||||||
const RUST_OUTBOUND_REDIS_INTEGRATION_TEST: &str = "tests/outbound-redis/http-rust-outbound-redis";
|
const RUST_OUTBOUND_REDIS_INTEGRATION_TEST: &str = "tests/outbound-redis/http-rust-outbound-redis";
|
||||||
const RUST_OUTBOUND_PG_INTEGRATION_TEST: &str = "tests/outbound-pg/http-rust-outbound-pg";
|
const RUST_OUTBOUND_PG_INTEGRATION_TEST: &str = "tests/outbound-pg/http-rust-outbound-pg";
|
||||||
|
@ -71,6 +72,7 @@ error: the `wasm32-wasi` target is not installed
|
||||||
|
|
||||||
cargo_build(RUST_HTTP_INTEGRATION_TEST);
|
cargo_build(RUST_HTTP_INTEGRATION_TEST);
|
||||||
cargo_build(RUST_HTTP_INTEGRATION_ENV_TEST);
|
cargo_build(RUST_HTTP_INTEGRATION_ENV_TEST);
|
||||||
|
cargo_build(RUST_HTTP_INTEGRATION_KEY_VALUE_TEST);
|
||||||
cargo_build(RUST_HTTP_VAULT_CONFIG_TEST);
|
cargo_build(RUST_HTTP_VAULT_CONFIG_TEST);
|
||||||
cargo_build(RUST_OUTBOUND_REDIS_INTEGRATION_TEST);
|
cargo_build(RUST_OUTBOUND_REDIS_INTEGRATION_TEST);
|
||||||
cargo_build(RUST_OUTBOUND_PG_INTEGRATION_TEST);
|
cargo_build(RUST_OUTBOUND_PG_INTEGRATION_TEST);
|
||||||
|
|
|
@ -92,6 +92,15 @@ impl AppLoader {
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Consumes `self` and `parts`, producing an [`OwnedApp`].
|
||||||
|
pub fn into_owned(self, parts: AppParts) -> OwnedApp {
|
||||||
|
OwnedApp::new(self, move |loader| App {
|
||||||
|
loader,
|
||||||
|
uri: parts.uri,
|
||||||
|
locked: parts.locked,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
/// Loads an [`OwnedApp`] from the given `Loader`-implementation-specific
|
/// Loads an [`OwnedApp`] from the given `Loader`-implementation-specific
|
||||||
/// `uri`; the [`OwnedApp`] takes ownership of this [`AppLoader`].
|
/// `uri`; the [`OwnedApp`] takes ownership of this [`AppLoader`].
|
||||||
pub async fn load_owned_app(self, uri: String) -> Result<OwnedApp> {
|
pub async fn load_owned_app(self, uri: String) -> Result<OwnedApp> {
|
||||||
|
@ -130,7 +139,21 @@ pub struct App<'a> {
|
||||||
locked: LockedApp,
|
locked: LockedApp,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The parts of an app exclusive of the [`AppLoader`]
|
||||||
|
pub struct AppParts {
|
||||||
|
uri: String,
|
||||||
|
locked: LockedApp,
|
||||||
|
}
|
||||||
|
|
||||||
impl<'a> App<'a> {
|
impl<'a> App<'a> {
|
||||||
|
/// Consume `self`, producing an [`AppParts`]
|
||||||
|
pub fn into_parts(self) -> AppParts {
|
||||||
|
AppParts {
|
||||||
|
uri: self.uri,
|
||||||
|
locked: self.locked,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Returns a [`Loader`]-implementation-specific URI for this app.
|
/// Returns a [`Loader`]-implementation-specific URI for this app.
|
||||||
pub fn uri(&self) -> &str {
|
pub fn uri(&self) -> &str {
|
||||||
&self.uri
|
&self.uri
|
||||||
|
|
|
@ -0,0 +1,12 @@
|
||||||
|
[package]
|
||||||
|
name = "key-value-sqlite"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
anyhow = "1"
|
||||||
|
once_cell = "1"
|
||||||
|
rusqlite = { version = "0.28.0", features = [ "bundled" ] }
|
||||||
|
tokio = "1"
|
||||||
|
wit-bindgen-wasmtime = { workspace = true }
|
||||||
|
key-value = { path = "../key-value" }
|
|
@ -0,0 +1,213 @@
|
||||||
|
use anyhow::Result;
|
||||||
|
use key_value::{key_value::Error, log_error, Impl, ImplStore};
|
||||||
|
use once_cell::sync::OnceCell;
|
||||||
|
use rusqlite::Connection;
|
||||||
|
use std::{
|
||||||
|
path::PathBuf,
|
||||||
|
sync::{Arc, Mutex},
|
||||||
|
};
|
||||||
|
use tokio::task;
|
||||||
|
use wit_bindgen_wasmtime::async_trait;
|
||||||
|
|
||||||
|
pub enum DatabaseLocation {
|
||||||
|
InMemory,
|
||||||
|
Path(PathBuf),
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct KeyValueSqlite {
|
||||||
|
location: DatabaseLocation,
|
||||||
|
connection: OnceCell<Arc<Mutex<Connection>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl KeyValueSqlite {
|
||||||
|
pub fn new(location: DatabaseLocation) -> Self {
|
||||||
|
Self {
|
||||||
|
location,
|
||||||
|
connection: OnceCell::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl Impl for KeyValueSqlite {
|
||||||
|
async fn open(&self, name: &str) -> Result<Box<dyn ImplStore>, Error> {
|
||||||
|
let connection = task::block_in_place(|| {
|
||||||
|
self.connection.get_or_try_init(|| {
|
||||||
|
let connection = match &self.location {
|
||||||
|
DatabaseLocation::InMemory => {
|
||||||
|
println!("Using in-memory key-value store");
|
||||||
|
Connection::open_in_memory()
|
||||||
|
}
|
||||||
|
DatabaseLocation::Path(path) => {
|
||||||
|
println!("Using {} for key-value store", path.display());
|
||||||
|
Connection::open(path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.map_err(log_error)?;
|
||||||
|
|
||||||
|
connection
|
||||||
|
.execute(
|
||||||
|
"CREATE TABLE IF NOT EXISTS spin_key_value (
|
||||||
|
store TEXT NOT NULL,
|
||||||
|
key TEXT NOT NULL,
|
||||||
|
value BLOB NOT NULL,
|
||||||
|
|
||||||
|
PRIMARY KEY (store, key)
|
||||||
|
)",
|
||||||
|
(),
|
||||||
|
)
|
||||||
|
.map_err(log_error)?;
|
||||||
|
|
||||||
|
Ok(Arc::new(Mutex::new(connection)))
|
||||||
|
})
|
||||||
|
})?;
|
||||||
|
|
||||||
|
Ok(Box::new(SqliteStore {
|
||||||
|
name: name.to_owned(),
|
||||||
|
connection: connection.clone(),
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct SqliteStore {
|
||||||
|
name: String,
|
||||||
|
connection: Arc<Mutex<Connection>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl ImplStore for SqliteStore {
|
||||||
|
async fn get(&self, key: &str) -> Result<Vec<u8>, Error> {
|
||||||
|
task::block_in_place(|| {
|
||||||
|
self.connection
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.prepare_cached("SELECT value FROM spin_key_value WHERE store=$1 AND key=$2")
|
||||||
|
.map_err(log_error)?
|
||||||
|
.query_map([&self.name, key], |row| row.get(0))
|
||||||
|
.map_err(log_error)?
|
||||||
|
.next()
|
||||||
|
.ok_or(Error::NoSuchKey)?
|
||||||
|
.map_err(log_error)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(&self, key: &str, value: &[u8]) -> Result<(), Error> {
|
||||||
|
task::block_in_place(|| {
|
||||||
|
self.connection
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.prepare_cached(
|
||||||
|
"INSERT INTO spin_key_value (store, key, value) VALUES ($1, $2, $3)
|
||||||
|
ON CONFLICT(store, key) DO UPDATE SET value=$3",
|
||||||
|
)
|
||||||
|
.map_err(log_error)?
|
||||||
|
.execute(rusqlite::params![&self.name, key, value])
|
||||||
|
.map_err(log_error)
|
||||||
|
.map(drop)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&self, key: &str) -> Result<(), Error> {
|
||||||
|
task::block_in_place(|| {
|
||||||
|
self.connection
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.prepare_cached("DELETE FROM spin_key_value WHERE store=$1 AND key=$2")
|
||||||
|
.map_err(log_error)?
|
||||||
|
.execute([&self.name, key])
|
||||||
|
.map_err(log_error)
|
||||||
|
.map(drop)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn exists(&self, key: &str) -> Result<bool, Error> {
|
||||||
|
match self.get(key).await {
|
||||||
|
Ok(_) => Ok(true),
|
||||||
|
Err(Error::NoSuchKey) => Ok(false),
|
||||||
|
Err(e) => Err(e),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_keys(&self) -> Result<Vec<String>, Error> {
|
||||||
|
task::block_in_place(|| {
|
||||||
|
self.connection
|
||||||
|
.lock()
|
||||||
|
.unwrap()
|
||||||
|
.prepare_cached("SELECT key FROM spin_key_value WHERE store=$1")
|
||||||
|
.map_err(log_error)?
|
||||||
|
.query_map([&self.name], |row| row.get(0))
|
||||||
|
.map_err(log_error)?
|
||||||
|
.map(|r| r.map_err(log_error))
|
||||||
|
.collect()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod test {
|
||||||
|
use super::*;
|
||||||
|
use key_value::{KeyValue, KeyValueDispatch};
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 1)]
|
||||||
|
async fn all() -> Result<()> {
|
||||||
|
let mut kv = KeyValueDispatch::new(Arc::new(
|
||||||
|
[(
|
||||||
|
"default".to_owned(),
|
||||||
|
Box::new(KeyValueSqlite::new(DatabaseLocation::InMemory)) as Box<dyn Impl>,
|
||||||
|
)]
|
||||||
|
.into_iter()
|
||||||
|
.collect(),
|
||||||
|
));
|
||||||
|
|
||||||
|
kv.allowed_stores = ["default", "foo"]
|
||||||
|
.into_iter()
|
||||||
|
.map(ToOwned::to_owned)
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
assert!(matches!(
|
||||||
|
kv.exists(42, "bar").await,
|
||||||
|
Err(Error::InvalidStore)
|
||||||
|
));
|
||||||
|
|
||||||
|
assert!(matches!(kv.open("foo").await, Err(Error::NoSuchStore)));
|
||||||
|
assert!(matches!(
|
||||||
|
kv.open("forbidden").await,
|
||||||
|
Err(Error::AccessDenied)
|
||||||
|
));
|
||||||
|
|
||||||
|
let store = kv.open("default").await?;
|
||||||
|
|
||||||
|
assert!(!kv.exists(store, "bar").await?);
|
||||||
|
|
||||||
|
assert!(matches!(kv.get(store, "bar").await, Err(Error::NoSuchKey)));
|
||||||
|
|
||||||
|
kv.set(store, "bar", b"baz").await?;
|
||||||
|
|
||||||
|
assert!(kv.exists(store, "bar").await?);
|
||||||
|
|
||||||
|
assert_eq!(b"baz" as &[_], &kv.get(store, "bar").await?);
|
||||||
|
|
||||||
|
kv.set(store, "bar", b"wow").await?;
|
||||||
|
|
||||||
|
assert_eq!(b"wow" as &[_], &kv.get(store, "bar").await?);
|
||||||
|
|
||||||
|
assert_eq!(&["bar".to_owned()] as &[_], &kv.get_keys(store).await?);
|
||||||
|
|
||||||
|
kv.delete(store, "bar").await?;
|
||||||
|
|
||||||
|
assert!(!kv.exists(store, "bar").await?);
|
||||||
|
|
||||||
|
assert_eq!(&[] as &[String], &kv.get_keys(store).await?);
|
||||||
|
|
||||||
|
assert!(matches!(kv.get(store, "bar").await, Err(Error::NoSuchKey)));
|
||||||
|
|
||||||
|
kv.close(store).await;
|
||||||
|
|
||||||
|
assert!(matches!(
|
||||||
|
kv.exists(store, "bar").await,
|
||||||
|
Err(Error::InvalidStore)
|
||||||
|
));
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
|
@ -0,0 +1,16 @@
|
||||||
|
[package]
|
||||||
|
name = "key-value"
|
||||||
|
version = { workspace = true }
|
||||||
|
authors = { workspace = true }
|
||||||
|
edition = { workspace = true }
|
||||||
|
|
||||||
|
[lib]
|
||||||
|
doctest = false
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
anyhow = "1.0"
|
||||||
|
tokio = { version = "1", features = [ "macros" ] }
|
||||||
|
spin-core = { path = "../core" }
|
||||||
|
spin-app = { path = "../app" }
|
||||||
|
tracing = { workspace = true }
|
||||||
|
wit-bindgen-wasmtime = { workspace = true }
|
|
@ -0,0 +1,47 @@
|
||||||
|
use crate::{Impl, KeyValueDispatch};
|
||||||
|
use spin_app::DynamicHostComponent;
|
||||||
|
use spin_core::HostComponent;
|
||||||
|
use std::{collections::HashMap, sync::Arc};
|
||||||
|
|
||||||
|
pub const KEY_VALUE_STORES_METADATA_KEY: &str = "key_value_stores";
|
||||||
|
|
||||||
|
pub struct KeyValueComponent {
|
||||||
|
impls: Arc<HashMap<String, Box<dyn Impl>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl KeyValueComponent {
|
||||||
|
pub fn new(impls: Arc<HashMap<String, Box<dyn Impl>>>) -> Self {
|
||||||
|
Self { impls }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl HostComponent for KeyValueComponent {
|
||||||
|
type Data = KeyValueDispatch;
|
||||||
|
|
||||||
|
fn add_to_linker<T: Send>(
|
||||||
|
linker: &mut spin_core::Linker<T>,
|
||||||
|
get: impl Fn(&mut spin_core::Data<T>) -> &mut Self::Data + Send + Sync + Copy + 'static,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
super::key_value::add_to_linker(linker, get)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn build_data(&self) -> Self::Data {
|
||||||
|
KeyValueDispatch::new(self.impls.clone())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DynamicHostComponent for KeyValueComponent {
|
||||||
|
fn update_data(
|
||||||
|
&self,
|
||||||
|
data: &mut Self::Data,
|
||||||
|
component: &spin_app::AppComponent,
|
||||||
|
) -> anyhow::Result<()> {
|
||||||
|
data.allowed_stores = component
|
||||||
|
.get_metadata::<Vec<String>>(KEY_VALUE_STORES_METADATA_KEY)?
|
||||||
|
.unwrap_or_default()
|
||||||
|
.into_iter()
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
|
@ -0,0 +1,126 @@
|
||||||
|
use anyhow::Result;
|
||||||
|
use std::{
|
||||||
|
collections::{HashMap, HashSet},
|
||||||
|
sync::Arc,
|
||||||
|
};
|
||||||
|
use table::Table;
|
||||||
|
use wit_bindgen_wasmtime::async_trait;
|
||||||
|
|
||||||
|
mod host_component;
|
||||||
|
mod table;
|
||||||
|
|
||||||
|
pub use host_component::KeyValueComponent;
|
||||||
|
|
||||||
|
wit_bindgen_wasmtime::export!({paths: ["../../wit/ephemeral/key-value.wit"], async: *});
|
||||||
|
|
||||||
|
pub use key_value::{Error, KeyValue, Store};
|
||||||
|
|
||||||
|
impl std::fmt::Display for Error {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
write!(f, "{:?}", self)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::error::Error for Error {}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait Impl: Sync + Send {
|
||||||
|
async fn open(&self, name: &str) -> Result<Box<dyn ImplStore>, Error>;
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
pub trait ImplStore: Sync + Send {
|
||||||
|
async fn get(&self, key: &str) -> Result<Vec<u8>, Error>;
|
||||||
|
|
||||||
|
async fn set(&self, key: &str, value: &[u8]) -> Result<(), Error>;
|
||||||
|
|
||||||
|
async fn delete(&self, key: &str) -> Result<(), Error>;
|
||||||
|
|
||||||
|
async fn exists(&self, key: &str) -> Result<bool, Error>;
|
||||||
|
|
||||||
|
async fn get_keys(&self) -> Result<Vec<String>, Error>;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct KeyValueDispatch {
|
||||||
|
pub allowed_stores: HashSet<String>,
|
||||||
|
impls: Arc<HashMap<String, Box<dyn Impl>>>,
|
||||||
|
stores: Table<Box<dyn ImplStore>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl KeyValueDispatch {
|
||||||
|
pub fn new(impls: Arc<HashMap<String, Box<dyn Impl>>>) -> Self {
|
||||||
|
Self {
|
||||||
|
allowed_stores: HashSet::new(),
|
||||||
|
impls,
|
||||||
|
stores: Table::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl KeyValue for KeyValueDispatch {
|
||||||
|
async fn open(&mut self, name: &str) -> Result<Store, Error> {
|
||||||
|
if self.allowed_stores.contains(name) {
|
||||||
|
self.stores
|
||||||
|
.push(
|
||||||
|
self.impls
|
||||||
|
.get(name)
|
||||||
|
.ok_or(Error::NoSuchStore)?
|
||||||
|
.open(name)
|
||||||
|
.await?,
|
||||||
|
)
|
||||||
|
.map_err(|()| Error::StoreTableFull)
|
||||||
|
} else {
|
||||||
|
Err(Error::AccessDenied)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get(&mut self, store: Store, key: &str) -> Result<Vec<u8>, Error> {
|
||||||
|
self.stores
|
||||||
|
.get(store)
|
||||||
|
.ok_or(Error::InvalidStore)?
|
||||||
|
.get(key)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn set(&mut self, store: Store, key: &str, value: &[u8]) -> Result<(), Error> {
|
||||||
|
self.stores
|
||||||
|
.get(store)
|
||||||
|
.ok_or(Error::InvalidStore)?
|
||||||
|
.set(key, value)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn delete(&mut self, store: Store, key: &str) -> Result<(), Error> {
|
||||||
|
self.stores
|
||||||
|
.get(store)
|
||||||
|
.ok_or(Error::InvalidStore)?
|
||||||
|
.delete(key)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn exists(&mut self, store: Store, key: &str) -> Result<bool, Error> {
|
||||||
|
self.stores
|
||||||
|
.get(store)
|
||||||
|
.ok_or(Error::InvalidStore)?
|
||||||
|
.exists(key)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_keys(&mut self, store: Store) -> Result<Vec<String>, Error> {
|
||||||
|
self.stores
|
||||||
|
.get(store)
|
||||||
|
.ok_or(Error::InvalidStore)?
|
||||||
|
.get_keys()
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn close(&mut self, store: Store) {
|
||||||
|
self.stores.remove(store);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn log_error(err: impl std::fmt::Debug) -> Error {
|
||||||
|
tracing::warn!("key-value error: {err:?}");
|
||||||
|
Error::Io(format!("{err:?}"))
|
||||||
|
}
|
|
@ -0,0 +1,62 @@
|
||||||
|
// TODO: there's nothing key-value-specific about this utility, so it could be moved elsewhere, e.g. to a utility
|
||||||
|
// crate of some kind.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
/// This is a table for generating unique u32 identifiers for each element in a dynamically-changing set of
|
||||||
|
/// resources.
|
||||||
|
///
|
||||||
|
/// This is inspired by the `Table` type in
|
||||||
|
/// [wasi-common](https://github.com/bytecodealliance/wasmtime/tree/main/crates/wasi-common) and serves the same
|
||||||
|
/// purpose: allow opaque resources and their lifetimes to be managed across an interface boundary, analogous to
|
||||||
|
/// how file handles work across the user-kernel boundary.
|
||||||
|
///
|
||||||
|
pub struct Table<V> {
|
||||||
|
next_key: u32,
|
||||||
|
tuples: HashMap<u32, V>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<V> Table<V> {
|
||||||
|
/// Create a new, empty table.
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self {
|
||||||
|
next_key: 0,
|
||||||
|
tuples: HashMap::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Add the specified resource to this table.
|
||||||
|
///
|
||||||
|
/// If the table is full (i.e. there already are 2^32 resources inside), this returns `Err(())`. Otherwise, a
|
||||||
|
/// new, unique identifier is allocated for the resource and returned.
|
||||||
|
///
|
||||||
|
/// This function will attempt to avoid reusing recently closed identifiers, but after 2^32 calls to this
|
||||||
|
/// function they will start repeating.
|
||||||
|
pub fn push(&mut self, value: V) -> Result<u32, ()> {
|
||||||
|
if self.tuples.len() == u32::MAX as usize {
|
||||||
|
Err(())
|
||||||
|
} else {
|
||||||
|
loop {
|
||||||
|
let key = self.next_key;
|
||||||
|
self.next_key = self.next_key.wrapping_add(1);
|
||||||
|
if self.tuples.contains_key(&key) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
self.tuples.insert(key, value);
|
||||||
|
return Ok(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Get a reference to the resource identified by the specified `key`, if it exists.
|
||||||
|
pub fn get(&self, key: u32) -> Option<&V> {
|
||||||
|
self.tuples.get(&key)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Remove the resource identified by the specified `key`, if present.
|
||||||
|
///
|
||||||
|
/// This makes the key eligible for eventual reuse (i.e. for a newly-pushed resource).
|
||||||
|
pub fn remove(&mut self, key: u32) -> Option<V> {
|
||||||
|
self.tuples.remove(&key)
|
||||||
|
}
|
||||||
|
}
|
|
@ -47,6 +47,8 @@ pub struct RawWasmConfig {
|
||||||
pub files: Option<String>,
|
pub files: Option<String>,
|
||||||
/// Optional list of HTTP hosts the component is allowed to connect.
|
/// Optional list of HTTP hosts the component is allowed to connect.
|
||||||
pub allowed_http_hosts: Option<Vec<String>>,
|
pub allowed_http_hosts: Option<Vec<String>>,
|
||||||
|
/// Optional list of key-value stores the component is allowed to use.
|
||||||
|
pub key_value_stores: Option<Vec<String>>,
|
||||||
/// Environment variables to be mapped inside the Wasm module at runtime.
|
/// Environment variables to be mapped inside the Wasm module at runtime.
|
||||||
pub environment: Option<HashMap<String, String>>,
|
pub environment: Option<HashMap<String, String>>,
|
||||||
}
|
}
|
||||||
|
|
|
@ -128,10 +128,12 @@ async fn core(
|
||||||
};
|
};
|
||||||
let environment = raw.wasm.environment.unwrap_or_default();
|
let environment = raw.wasm.environment.unwrap_or_default();
|
||||||
let allowed_http_hosts = raw.wasm.allowed_http_hosts.unwrap_or_default();
|
let allowed_http_hosts = raw.wasm.allowed_http_hosts.unwrap_or_default();
|
||||||
|
let key_value_stores = raw.wasm.key_value_stores.unwrap_or_default();
|
||||||
let wasm = WasmConfig {
|
let wasm = WasmConfig {
|
||||||
environment,
|
environment,
|
||||||
mounts,
|
mounts,
|
||||||
allowed_http_hosts,
|
allowed_http_hosts,
|
||||||
|
key_value_stores,
|
||||||
};
|
};
|
||||||
let config = raw.config.unwrap_or_default();
|
let config = raw.config.unwrap_or_default();
|
||||||
Ok(CoreComponent {
|
Ok(CoreComponent {
|
||||||
|
|
|
@ -114,6 +114,8 @@ pub struct RawWasmConfig {
|
||||||
pub exclude_files: Option<Vec<String>>,
|
pub exclude_files: Option<Vec<String>>,
|
||||||
/// Optional list of HTTP hosts the component is allowed to connect.
|
/// Optional list of HTTP hosts the component is allowed to connect.
|
||||||
pub allowed_http_hosts: Option<Vec<String>>,
|
pub allowed_http_hosts: Option<Vec<String>>,
|
||||||
|
/// Optional list of key-value stores the component is allowed to use.
|
||||||
|
pub key_value_stores: Option<Vec<String>>,
|
||||||
/// Environment variables to be mapped inside the Wasm module at runtime.
|
/// Environment variables to be mapped inside the Wasm module at runtime.
|
||||||
pub environment: Option<HashMap<String, String>>,
|
pub environment: Option<HashMap<String, String>>,
|
||||||
}
|
}
|
||||||
|
|
|
@ -240,10 +240,12 @@ async fn core(
|
||||||
};
|
};
|
||||||
let environment = raw.wasm.environment.unwrap_or_default();
|
let environment = raw.wasm.environment.unwrap_or_default();
|
||||||
let allowed_http_hosts = raw.wasm.allowed_http_hosts.unwrap_or_default();
|
let allowed_http_hosts = raw.wasm.allowed_http_hosts.unwrap_or_default();
|
||||||
|
let key_value_stores = raw.wasm.key_value_stores.unwrap_or_default();
|
||||||
let wasm = WasmConfig {
|
let wasm = WasmConfig {
|
||||||
environment,
|
environment,
|
||||||
mounts,
|
mounts,
|
||||||
allowed_http_hosts,
|
allowed_http_hosts,
|
||||||
|
key_value_stores,
|
||||||
};
|
};
|
||||||
let config = raw.config.unwrap_or_default();
|
let config = raw.config.unwrap_or_default();
|
||||||
Ok(CoreComponent {
|
Ok(CoreComponent {
|
||||||
|
|
|
@ -262,6 +262,8 @@ pub struct WasmConfig {
|
||||||
pub mounts: Vec<DirectoryMount>,
|
pub mounts: Vec<DirectoryMount>,
|
||||||
/// Optional list of HTTP hosts the component is allowed to connect.
|
/// Optional list of HTTP hosts the component is allowed to connect.
|
||||||
pub allowed_http_hosts: Vec<String>,
|
pub allowed_http_hosts: Vec<String>,
|
||||||
|
/// Optional list of key-value stores the component is allowed to use.
|
||||||
|
pub key_value_stores: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Directory mount for the assets of a component.
|
/// Directory mount for the assets of a component.
|
||||||
|
|
|
@ -127,6 +127,7 @@ async fn bindle_component_manifest(
|
||||||
environment: local.wasm.environment.clone(),
|
environment: local.wasm.environment.clone(),
|
||||||
files: asset_group,
|
files: asset_group,
|
||||||
allowed_http_hosts: local.wasm.allowed_http_hosts.clone(),
|
allowed_http_hosts: local.wasm.allowed_http_hosts.clone(),
|
||||||
|
key_value_stores: local.wasm.key_value_stores.clone(),
|
||||||
},
|
},
|
||||||
trigger: local.trigger.clone(),
|
trigger: local.trigger.clone(),
|
||||||
config: local.config.clone(),
|
config: local.config.clone(),
|
||||||
|
|
|
@ -16,6 +16,8 @@ outbound-http = { path = "../outbound-http" }
|
||||||
outbound-redis = { path = "../outbound-redis" }
|
outbound-redis = { path = "../outbound-redis" }
|
||||||
outbound-pg = { path = "../outbound-pg" }
|
outbound-pg = { path = "../outbound-pg" }
|
||||||
outbound-mysql = { path = "../outbound-mysql" }
|
outbound-mysql = { path = "../outbound-mysql" }
|
||||||
|
key-value = { path = "../key-value" }
|
||||||
|
key-value-sqlite = { path = "../key-value-sqlite" }
|
||||||
sanitize-filename = "0.4"
|
sanitize-filename = "0.4"
|
||||||
serde = "1.0"
|
serde = "1.0"
|
||||||
serde_json = "1.0"
|
serde_json = "1.0"
|
||||||
|
|
|
@ -31,10 +31,10 @@ where
|
||||||
{
|
{
|
||||||
/// Log directory for the stdout and stderr of components.
|
/// Log directory for the stdout and stderr of components.
|
||||||
#[clap(
|
#[clap(
|
||||||
name = APP_LOG_DIR,
|
name = APP_LOG_DIR,
|
||||||
short = 'L',
|
short = 'L',
|
||||||
long = "log-dir",
|
long = "log-dir",
|
||||||
)]
|
)]
|
||||||
pub log: Option<PathBuf>,
|
pub log: Option<PathBuf>,
|
||||||
|
|
||||||
/// Disable Wasmtime cache.
|
/// Disable Wasmtime cache.
|
||||||
|
@ -61,14 +61,14 @@ where
|
||||||
name = FOLLOW_LOG_OPT,
|
name = FOLLOW_LOG_OPT,
|
||||||
long = "follow",
|
long = "follow",
|
||||||
multiple_occurrences = true,
|
multiple_occurrences = true,
|
||||||
)]
|
)]
|
||||||
pub follow_components: Vec<String>,
|
pub follow_components: Vec<String>,
|
||||||
|
|
||||||
/// Print all component output to stdout/stderr
|
/// Print all component output to stdout/stderr
|
||||||
#[clap(
|
#[clap(
|
||||||
long = "follow-all",
|
long = "follow-all",
|
||||||
conflicts_with = FOLLOW_LOG_OPT,
|
conflicts_with = FOLLOW_LOG_OPT,
|
||||||
)]
|
)]
|
||||||
pub follow_all_components: bool,
|
pub follow_all_components: bool,
|
||||||
|
|
||||||
/// Set the static assets of the components in the temporary directory as writable.
|
/// Set the static assets of the components in the temporary directory as writable.
|
||||||
|
|
|
@ -6,13 +6,16 @@ mod stdio;
|
||||||
|
|
||||||
use std::{
|
use std::{
|
||||||
collections::HashMap,
|
collections::HashMap,
|
||||||
|
fs,
|
||||||
marker::PhantomData,
|
marker::PhantomData,
|
||||||
path::{Path, PathBuf},
|
path::{Path, PathBuf},
|
||||||
|
sync::Arc,
|
||||||
};
|
};
|
||||||
|
|
||||||
use anyhow::{anyhow, Context, Result};
|
use anyhow::{anyhow, Context, Result};
|
||||||
pub use async_trait::async_trait;
|
pub use async_trait::async_trait;
|
||||||
use serde::de::DeserializeOwned;
|
use serde::de::DeserializeOwned;
|
||||||
|
use url::Url;
|
||||||
|
|
||||||
use spin_app::{App, AppComponent, AppLoader, AppTrigger, Loader, OwnedApp};
|
use spin_app::{App, AppComponent, AppLoader, AppTrigger, Loader, OwnedApp};
|
||||||
use spin_config::{
|
use spin_config::{
|
||||||
|
@ -23,6 +26,8 @@ use spin_core::{Config, Engine, EngineBuilder, Instance, InstancePre, Store, Sto
|
||||||
|
|
||||||
const SPIN_HOME: &str = ".spin";
|
const SPIN_HOME: &str = ".spin";
|
||||||
const SPIN_CONFIG_ENV_PREFIX: &str = "SPIN_APP";
|
const SPIN_CONFIG_ENV_PREFIX: &str = "SPIN_APP";
|
||||||
|
const DEFAULT_SQLITE_DB_DIRECTORY: &str = ".spin";
|
||||||
|
const DEFAULT_SQLITE_DB_FILENAME: &str = "sqlite_key_value.db";
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
pub trait TriggerExecutor: Sized {
|
pub trait TriggerExecutor: Sized {
|
||||||
|
@ -88,6 +93,16 @@ impl<Executor: TriggerExecutor> TriggerExecutorBuilder<Executor> {
|
||||||
where
|
where
|
||||||
Executor::TriggerConfig: DeserializeOwned,
|
Executor::TriggerConfig: DeserializeOwned,
|
||||||
{
|
{
|
||||||
|
let key_value_file;
|
||||||
|
let app_parts;
|
||||||
|
{
|
||||||
|
let app = self.loader.load_app(app_uri.clone()).await?;
|
||||||
|
|
||||||
|
key_value_file = key_value_file_location(&app);
|
||||||
|
|
||||||
|
app_parts = app.into_parts();
|
||||||
|
}
|
||||||
|
|
||||||
let engine = {
|
let engine = {
|
||||||
let mut builder = Engine::builder(&self.config)?;
|
let mut builder = Engine::builder(&self.config)?;
|
||||||
|
|
||||||
|
@ -95,6 +110,30 @@ impl<Executor: TriggerExecutor> TriggerExecutorBuilder<Executor> {
|
||||||
builder.add_host_component(outbound_redis::OutboundRedisComponent)?;
|
builder.add_host_component(outbound_redis::OutboundRedisComponent)?;
|
||||||
builder.add_host_component(outbound_pg::OutboundPg::default())?;
|
builder.add_host_component(outbound_pg::OutboundPg::default())?;
|
||||||
builder.add_host_component(outbound_mysql::OutboundMysql::default())?;
|
builder.add_host_component(outbound_mysql::OutboundMysql::default())?;
|
||||||
|
|
||||||
|
self.loader.add_dynamic_host_component(
|
||||||
|
&mut builder,
|
||||||
|
key_value::KeyValueComponent::new(
|
||||||
|
// TODO: Once we have runtime configuration for key-value stores, the user will be able to
|
||||||
|
// both change the default store configuration (e.g. use Redis, or an SQLite in-memory
|
||||||
|
// database, or use a different path) and add other named stores with their own
|
||||||
|
// configurations.
|
||||||
|
Arc::new(
|
||||||
|
[(
|
||||||
|
"default".to_owned(),
|
||||||
|
Box::new(key_value_sqlite::KeyValueSqlite::new(
|
||||||
|
if let Some(key_value_file) = key_value_file {
|
||||||
|
key_value_sqlite::DatabaseLocation::Path(key_value_file)
|
||||||
|
} else {
|
||||||
|
key_value_sqlite::DatabaseLocation::InMemory
|
||||||
|
},
|
||||||
|
)) as Box<dyn key_value::Impl>,
|
||||||
|
)]
|
||||||
|
.into_iter()
|
||||||
|
.collect(),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)?;
|
||||||
self.loader.add_dynamic_host_component(
|
self.loader.add_dynamic_host_component(
|
||||||
&mut builder,
|
&mut builder,
|
||||||
outbound_http::OutboundHttpComponent,
|
outbound_http::OutboundHttpComponent,
|
||||||
|
@ -111,7 +150,8 @@ impl<Executor: TriggerExecutor> TriggerExecutorBuilder<Executor> {
|
||||||
builder.build()
|
builder.build()
|
||||||
};
|
};
|
||||||
|
|
||||||
let app = self.loader.load_owned_app(app_uri).await?;
|
let app = self.loader.into_owned(app_parts);
|
||||||
|
|
||||||
let app_name = app.borrowed().require_metadata("name")?;
|
let app_name = app.borrowed().require_metadata("name")?;
|
||||||
|
|
||||||
self.hooks.app_loaded(app.borrowed())?;
|
self.hooks.app_loaded(app.borrowed())?;
|
||||||
|
@ -341,3 +381,23 @@ fn decode_preinstantiation_error(e: anyhow::Error) -> anyhow::Error {
|
||||||
|
|
||||||
e
|
e
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn key_value_file_location(app: &App) -> Option<PathBuf> {
|
||||||
|
let url = Url::parse(&app.get_metadata::<String>("origin").ok()??).ok()?;
|
||||||
|
let local_spin_toml = if url.scheme() == "file" {
|
||||||
|
url.path()
|
||||||
|
} else {
|
||||||
|
// Use in-memory store for remote apps.
|
||||||
|
return None;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Attempt to create or reuse a database file inside the app directory. If that's not successful, we'll use an
|
||||||
|
// in-memory database.
|
||||||
|
let directory = Path::new(&local_spin_toml)
|
||||||
|
.parent()?
|
||||||
|
.join(DEFAULT_SQLITE_DB_DIRECTORY);
|
||||||
|
|
||||||
|
fs::create_dir_all(&directory).ok()?;
|
||||||
|
|
||||||
|
Some(directory.join(DEFAULT_SQLITE_DB_FILENAME))
|
||||||
|
}
|
||||||
|
|
|
@ -136,6 +136,7 @@ impl LockedAppBuilder {
|
||||||
let metadata = ValuesMapBuilder::new()
|
let metadata = ValuesMapBuilder::new()
|
||||||
.string_option("description", component.description)
|
.string_option("description", component.description)
|
||||||
.string_array("allowed_http_hosts", component.wasm.allowed_http_hosts)
|
.string_array("allowed_http_hosts", component.wasm.allowed_http_hosts)
|
||||||
|
.string_array("key_value_stores", component.wasm.key_value_stores)
|
||||||
.take();
|
.take();
|
||||||
|
|
||||||
let source = {
|
let source = {
|
||||||
|
|
|
@ -0,0 +1,2 @@
|
||||||
|
[build]
|
||||||
|
target = "wasm32-wasi"
|
|
@ -0,0 +1,20 @@
|
||||||
|
[package]
|
||||||
|
name = "spin-key-value"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[lib]
|
||||||
|
crate-type = [ "cdylib" ]
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
# Useful crate to handle errors.
|
||||||
|
anyhow = "1"
|
||||||
|
# Crate to simplify working with bytes.
|
||||||
|
bytes = "1"
|
||||||
|
# General-purpose crate with common HTTP types.
|
||||||
|
http = "0.2"
|
||||||
|
spin-sdk = { path = "../../sdk/rust"}
|
||||||
|
# The wit-bindgen-rust dependency generates bindings for interfaces.
|
||||||
|
wit-bindgen-rust = { git = "https://github.com/bytecodealliance/wit-bindgen", rev = "cb871cfa1ee460b51eb1d144b175b9aab9c50aba" }
|
||||||
|
|
||||||
|
[workspace]
|
|
@ -0,0 +1,15 @@
|
||||||
|
spin_version = "1"
|
||||||
|
authors = ["Fermyon Engineering <engineering@fermyon.com>"]
|
||||||
|
description = "A simple application that exercises key-value storage."
|
||||||
|
name = "spin-key-value"
|
||||||
|
trigger = {type = "http", base = "/test"}
|
||||||
|
version = "1.0.0"
|
||||||
|
|
||||||
|
[[component]]
|
||||||
|
id = "hello"
|
||||||
|
source = "target/wasm32-wasi/release/spin_key_value.wasm"
|
||||||
|
key_value_stores = ["default"]
|
||||||
|
[component.trigger]
|
||||||
|
route = "/..."
|
||||||
|
[component.build]
|
||||||
|
command = "cargo build --target wasm32-wasi --release"
|
|
@ -0,0 +1,46 @@
|
||||||
|
use anyhow::Result;
|
||||||
|
use http::{Method, StatusCode};
|
||||||
|
use spin_sdk::{
|
||||||
|
http::{Request, Response},
|
||||||
|
http_component,
|
||||||
|
key_value::{Error, Store},
|
||||||
|
};
|
||||||
|
|
||||||
|
#[http_component]
|
||||||
|
fn handle_request(req: Request) -> Result<Response> {
|
||||||
|
// Open the default key-value store
|
||||||
|
let store = Store::open_default()?;
|
||||||
|
|
||||||
|
let (status, body) = match req.method() {
|
||||||
|
&Method::POST => {
|
||||||
|
// Add the request (URI, body) tuple to the store
|
||||||
|
store.set(req.uri().path(), req.body().as_deref().unwrap_or(&[]))?;
|
||||||
|
(StatusCode::OK, None)
|
||||||
|
}
|
||||||
|
&Method::GET => {
|
||||||
|
// Get the value associated with the request URI, or return a 404 if it's not present
|
||||||
|
match store.get(req.uri().path()) {
|
||||||
|
Ok(value) => (StatusCode::OK, Some(value.into())),
|
||||||
|
Err(Error::NoSuchKey) => (StatusCode::NOT_FOUND, None),
|
||||||
|
Err(error) => return Err(error.into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
&Method::DELETE => {
|
||||||
|
// Delete the value associated with the request URI, if present
|
||||||
|
store.delete(req.uri().path())?;
|
||||||
|
(StatusCode::OK, None)
|
||||||
|
}
|
||||||
|
&Method::HEAD => {
|
||||||
|
// Like GET, except do not return the value
|
||||||
|
match store.exists(req.uri().path()) {
|
||||||
|
Ok(true) => (StatusCode::OK, None),
|
||||||
|
Ok(false) => (StatusCode::NOT_FOUND, None),
|
||||||
|
Err(error) => return Err(error.into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// No other methods are currently supported
|
||||||
|
_ => (StatusCode::METHOD_NOT_ALLOWED, None),
|
||||||
|
};
|
||||||
|
|
||||||
|
Ok(http::Response::builder().status(status).body(body)?)
|
||||||
|
}
|
|
@ -0,0 +1,78 @@
|
||||||
|
//! Spin key-value persistent storage
|
||||||
|
//!
|
||||||
|
//! This module provides a generic interface for key-value storage, which may be implemented by the host various
|
||||||
|
//! ways (e.g. via an in-memory table, a local file, or a remote database). Details such as consistency model and
|
||||||
|
//! durability will depend on the implementation and may vary from one to store to the next.
|
||||||
|
|
||||||
|
wit_bindgen_rust::import!("../../wit/ephemeral/key-value.wit");
|
||||||
|
|
||||||
|
use key_value::Store as RawStore;
|
||||||
|
|
||||||
|
/// Errors which may be raised by the methods of `Store`
|
||||||
|
pub type Error = key_value::Error;
|
||||||
|
|
||||||
|
/// Represents a store in which key value tuples may be placed
|
||||||
|
#[derive(Clone, Debug)]
|
||||||
|
pub struct Store(RawStore);
|
||||||
|
|
||||||
|
impl Store {
|
||||||
|
/// Open the specified store.
|
||||||
|
///
|
||||||
|
/// If `name` is empty, open the default store. Other stores may also be available depending on the app
|
||||||
|
/// configuration.
|
||||||
|
pub fn open(name: impl AsRef<str>) -> Result<Self, Error> {
|
||||||
|
key_value::open(name.as_ref()).map(Self)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Open the default store.
|
||||||
|
///
|
||||||
|
/// This is equivalent to `Store::open("default")`.
|
||||||
|
pub fn open_default() -> Result<Self, Error> {
|
||||||
|
Self::open("default")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Get the value, if any, associated with the specified key in this store.
|
||||||
|
///
|
||||||
|
/// If no value is found, this will return `Err(Error::NoSuchKey)`.
|
||||||
|
pub fn get(&self, key: impl AsRef<str>) -> Result<Vec<u8>, Error> {
|
||||||
|
key_value::get(self.0, key.as_ref())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Set the value for the specified key.
|
||||||
|
///
|
||||||
|
/// This will overwrite any previous value, if present.
|
||||||
|
pub fn set(&self, key: impl AsRef<str>, value: impl AsRef<[u8]>) -> Result<(), Error> {
|
||||||
|
key_value::set(self.0, key.as_ref(), value.as_ref())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Delete the tuple for the specified key, if any.
|
||||||
|
///
|
||||||
|
/// This will have no effect and return `Ok(())` if the tuple was not present.
|
||||||
|
pub fn delete(&self, key: impl AsRef<str>) -> Result<(), Error> {
|
||||||
|
key_value::delete(self.0, key.as_ref())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Check whether a tuple exists for the specified key.
|
||||||
|
pub fn exists(&self, key: impl AsRef<str>) -> Result<bool, Error> {
|
||||||
|
key_value::exists(self.0, key.as_ref())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Get the set of keys in this store.
|
||||||
|
pub fn get_keys(&self) -> Result<Vec<String>, Error> {
|
||||||
|
key_value::get_keys(self.0)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for Store {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
key_value::close(self.0)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Display for Error {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
write!(f, "{self:?}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::error::Error for Error {}
|
|
@ -5,6 +5,9 @@
|
||||||
/// Outbound HTTP request functionality.
|
/// Outbound HTTP request functionality.
|
||||||
pub mod outbound_http;
|
pub mod outbound_http;
|
||||||
|
|
||||||
|
/// Key/Value storage.
|
||||||
|
pub mod key_value;
|
||||||
|
|
||||||
/// Exports the procedural macros for writing handlers for Spin components.
|
/// Exports the procedural macros for writing handlers for Spin components.
|
||||||
pub use spin_macro::*;
|
pub use spin_macro::*;
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,2 @@
|
||||||
|
[build]
|
||||||
|
target = "wasm32-wasi"
|
|
@ -0,0 +1,24 @@
|
||||||
|
[package]
|
||||||
|
name = "spin-key-value"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[lib]
|
||||||
|
crate-type = [ "cdylib" ]
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
# Useful crate to handle errors.
|
||||||
|
anyhow = "1"
|
||||||
|
# Crate to simplify working with bytes.
|
||||||
|
bytes = "1"
|
||||||
|
# General-purpose crate with common HTTP types.
|
||||||
|
http = "0.2"
|
||||||
|
spin-sdk = { path = "../../../sdk/rust"}
|
||||||
|
# The wit-bindgen-rust dependency generates bindings for interfaces.
|
||||||
|
wit-bindgen-rust = { git = "https://github.com/bytecodealliance/wit-bindgen", rev = "cb871cfa1ee460b51eb1d144b175b9aab9c50aba" }
|
||||||
|
|
||||||
|
[workspace]
|
||||||
|
|
||||||
|
# Metadata about this component.
|
||||||
|
[package.metadata.component]
|
||||||
|
name = "spinhelloworld"
|
|
@ -0,0 +1,13 @@
|
||||||
|
spin_version = "1"
|
||||||
|
authors = ["Fermyon Engineering <engineering@fermyon.com>"]
|
||||||
|
description = "A simple application that exercises key/value storage."
|
||||||
|
name = "spin-key-value"
|
||||||
|
trigger = {type = "http", base = "/test"}
|
||||||
|
version = "1.0.0"
|
||||||
|
|
||||||
|
[[component]]
|
||||||
|
id = "hello"
|
||||||
|
key_value_stores = ["default", "foo"]
|
||||||
|
source = "target/wasm32-wasi/release/spin_key_value.wasm"
|
||||||
|
[component.trigger]
|
||||||
|
route = "/..."
|
|
@ -0,0 +1,42 @@
|
||||||
|
use anyhow::{ensure, Result};
|
||||||
|
use spin_sdk::{
|
||||||
|
http::{Request, Response},
|
||||||
|
http_component,
|
||||||
|
key_value::{Error, Store},
|
||||||
|
};
|
||||||
|
|
||||||
|
#[http_component]
|
||||||
|
fn handle_request(_req: Request) -> Result<Response> {
|
||||||
|
ensure!(matches!(Store::open("foo"), Err(Error::NoSuchStore)));
|
||||||
|
ensure!(matches!(Store::open("forbidden"), Err(Error::AccessDenied)));
|
||||||
|
|
||||||
|
let store = Store::open_default()?;
|
||||||
|
|
||||||
|
store.delete("bar")?;
|
||||||
|
|
||||||
|
ensure!(!store.exists("bar")?);
|
||||||
|
|
||||||
|
ensure!(matches!(store.get("bar"), Err(Error::NoSuchKey)));
|
||||||
|
|
||||||
|
store.set("bar", b"baz")?;
|
||||||
|
|
||||||
|
ensure!(store.exists("bar")?);
|
||||||
|
|
||||||
|
ensure!(b"baz" as &[_] == &store.get("bar")?);
|
||||||
|
|
||||||
|
store.set("bar", b"wow")?;
|
||||||
|
|
||||||
|
ensure!(b"wow" as &[_] == &store.get("bar")?);
|
||||||
|
|
||||||
|
ensure!(&["bar".to_owned()] as &[_] == &store.get_keys()?);
|
||||||
|
|
||||||
|
store.delete("bar")?;
|
||||||
|
|
||||||
|
ensure!(&[] as &[String] == &store.get_keys()?);
|
||||||
|
|
||||||
|
ensure!(!store.exists("bar")?);
|
||||||
|
|
||||||
|
ensure!(matches!(store.get("bar"), Err(Error::NoSuchKey)));
|
||||||
|
|
||||||
|
Ok(http::Response::builder().status(200).body(None)?)
|
||||||
|
}
|
|
@ -19,6 +19,7 @@ mod integration_tests {
|
||||||
use tokio::{net::TcpStream, time::sleep};
|
use tokio::{net::TcpStream, time::sleep};
|
||||||
|
|
||||||
const RUST_HTTP_INTEGRATION_TEST: &str = "tests/http/simple-spin-rust";
|
const RUST_HTTP_INTEGRATION_TEST: &str = "tests/http/simple-spin-rust";
|
||||||
|
const RUST_HTTP_KEY_VALUE_TEST: &str = "tests/http/key-value";
|
||||||
|
|
||||||
const DEFAULT_MANIFEST_LOCATION: &str = "spin.toml";
|
const DEFAULT_MANIFEST_LOCATION: &str = "spin.toml";
|
||||||
|
|
||||||
|
@ -601,6 +602,21 @@ mod integration_tests {
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn test_key_value_local() -> Result<()> {
|
||||||
|
let s = SpinTestController::with_manifest(
|
||||||
|
&format!("{}/{}", RUST_HTTP_KEY_VALUE_TEST, DEFAULT_MANIFEST_LOCATION),
|
||||||
|
&[],
|
||||||
|
&[],
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
assert_status(&s, "/test", 200).await?;
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(feature = "outbound-pg-tests")]
|
#[cfg(feature = "outbound-pg-tests")]
|
||||||
mod outbound_pg_tests {
|
mod outbound_pg_tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
|
@ -96,6 +96,7 @@ pub mod all {
|
||||||
tc.run(controller).await.unwrap()
|
tc.run(controller).await.unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[allow(unused)]
|
||||||
pub async fn http_grain_works(controller: &dyn Controller) {
|
pub async fn http_grain_works(controller: &dyn Controller) {
|
||||||
fn checks(metadata: &AppMetadata) -> Result<()> {
|
fn checks(metadata: &AppMetadata) -> Result<()> {
|
||||||
return assert_http_response(metadata.base.as_str(), 200, &[], Some("Hello, World\n"));
|
return assert_http_response(metadata.base.as_str(), 200, &[], Some("Hello, World\n"));
|
||||||
|
|
|
@ -0,0 +1,78 @@
|
||||||
|
// A handle to an open key-value store
|
||||||
|
type store = u32
|
||||||
|
|
||||||
|
// The set of errors which may be raised by functions in this interface
|
||||||
|
variant error {
|
||||||
|
// Too many stores have been opened simultaneously. Closing one or more
|
||||||
|
// stores prior to retrying may address this.
|
||||||
|
store-table-full,
|
||||||
|
|
||||||
|
// The host does not recognize the store name requested. Defining and
|
||||||
|
// configuring a store with that name in a runtime configuration file
|
||||||
|
// may address this.
|
||||||
|
no-such-store,
|
||||||
|
|
||||||
|
// The requesting component does not have access to the specified store
|
||||||
|
// (which may or may not exist).
|
||||||
|
access-denied,
|
||||||
|
|
||||||
|
// The store handle provided is not recognized, i.e. it was either never
|
||||||
|
// opened or has been closed.
|
||||||
|
invalid-store,
|
||||||
|
|
||||||
|
// No key-value tuple exists for the specified key in the specified
|
||||||
|
// store.
|
||||||
|
no-such-key,
|
||||||
|
|
||||||
|
// Some implementation-specific error has occurred (e.g. I/O)
|
||||||
|
io(string)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Open the store with the specified name.
|
||||||
|
//
|
||||||
|
// If `name` is "default", the default store is opened. Otherwise,
|
||||||
|
// `name` must refer to a store defined and configured in a runtime
|
||||||
|
// configuration file supplied with the application.
|
||||||
|
//
|
||||||
|
// `error::no-such-store` will be raised if the `name` is not recognized.
|
||||||
|
open: func(name: string) -> expected<store, error>
|
||||||
|
|
||||||
|
// Get the value associated with the specified `key` from the specified
|
||||||
|
// `store`.
|
||||||
|
//
|
||||||
|
// `error::invalid-store` will be raised if `store` is not a valid handle
|
||||||
|
// to an open store, and `error::no-such-key` will be raised if there is no
|
||||||
|
// tuple for `key` in `store`.
|
||||||
|
get: func(store: store, key: string) -> expected<list<u8>, error>
|
||||||
|
|
||||||
|
// Set the `value` associated with the specified `key` in the specified
|
||||||
|
// `store`, overwriting any existing value.
|
||||||
|
//
|
||||||
|
// `error::invalid-store` will be raised if `store` is not a valid handle
|
||||||
|
// to an open store.
|
||||||
|
set: func(store: store, key: string, value: list<u8>) -> expected<unit, error>
|
||||||
|
|
||||||
|
// Delete the tuple with the specified `key` from the specified `store`.
|
||||||
|
//
|
||||||
|
// `error::invalid-store` will be raised if `store` is not a valid handle
|
||||||
|
// to an open store. No error is raised if a tuple did not previously
|
||||||
|
// exist for `key`.
|
||||||
|
delete: func(store: store, key: string) -> expected<unit, error>
|
||||||
|
|
||||||
|
// Return whether a tuple exists for the specified `key` in the specified
|
||||||
|
// `store`.
|
||||||
|
//
|
||||||
|
// `error::invalid-store` will be raised if `store` is not a valid handle
|
||||||
|
// to an open store.
|
||||||
|
exists: func(store: store, key: string) -> expected<bool, error>
|
||||||
|
|
||||||
|
// Return a list of all the keys in the specified `store`.
|
||||||
|
//
|
||||||
|
// `error::invalid-store` will be raised if `store` is not a valid handle
|
||||||
|
// to an open store.
|
||||||
|
get-keys: func(store: store) -> expected<list<string>, error>
|
||||||
|
|
||||||
|
// Close the specified `store`.
|
||||||
|
//
|
||||||
|
// This has no effect if `store` is not a valid handle to an open store.
|
||||||
|
close: func(store: store)
|
Loading…
Reference in New Issue