Fluvio is a distributed, programmable streaming platform written in Rust.
Welcome to the 52nd edition of this week in Fluvio.This one is going to be short and sweet.
We have released connector secrets and here are the details:
Specify user interfaces to cloud connector secrets. This includes a cli for populating the secrets, as well as how a connector configuration can refer to the secrets.
Customers need to be able to provide secrets to connectors in a protected way in order to access their own or third-party services with the confidence that Infynyon infrastructure provides protection from loss of the secrets.
Cloud secrets are set via cli. Each secret is a named value with all secrets sharing a secrets namespace per account. Connector configuration files can refer to secrets by name, and the cloud connector infrastructure will provision the connector with the named secrets.
Due to security concerns, listing actual secret values or downloading them after they have been set is not allowed. However, a listing of secret names as well as what date they were last set is described in the interface.
The secrets cli is an added subcommand to ‘fluvio cloud’ with the following cli ui interface:
fluvio cloud secret set --connector <NAME> <VALUE> fluvio cloud secret set --connector <NAME> --file <FILENAME> # e.g. a tls cert file fluvio cloud secret delete <NAME> fluvio cloud secret list
Note: The current implementation limits the scope of the secrets to connectors only. Also see open questions below regarding set, update, create.
One security principle in effect with secret list is that we never want to send custoemr secrets out of the cloud. They are only decrypted at the point of use inside the connector. But users still need to see what named secrets have been set, and potentially when they were last updated.
$ fluvio cloud secret list SecretNames LastUpdate CAT_FACTS_CLIENT_ID 12-10-2022 1:07pm CAT_FACTS_SECRET 01-02-2023 12:01am
The connector config files can reference cloud secrets by NAME as follows:
meta: version: 0.1.0 name: my-connector type: package-name topic: a-topic create-topic: true <CUSTOM>: # named section for custom config parameters, usually a short name like "http", or "mqtt" param_client_id: secret: name: CAT_FACTS_CLIENT_ID param_client_secret: secret: name: CAT_FACTS_SECRET
We are working on a few interesting problems around deduplication and stream processing. We have also made solid progress on a public roadmap to share with the community.
We would love to know what do you recommend we build next to make your development flow easier. Please comment in our Discord channel and let us know. Tag Deb (DRC) on your feedback and feature requests.
For the full list of changes this week, be sure to check out our CHANGELOG.