Skip to main content
Version: 0.11.12 (stable)

Generate

Generate a Connectorโ€‹

CDK generate helps developers build a sample Connector project by answering a few simple questions.

Use cdk generate to create a new connector project:

$ cdk generate
๐Ÿคท Project Name: my-connector
๐Ÿคท Please set a group name: acme
๐Ÿคท Which type of Connector would you like [source/sink]? ยท source
๐Ÿคท Will your Connector be public? ยท false
[1/8] Done: .gitignore
[2/8] Done: Cargo.toml
[3/8] Done: Connector.toml
[4/8] Done: README.md
[5/8] Done: sample-config.yaml
[6/8] Done: src/config.rs
[7/8] Done: src/main.rs
[8/8] Done: src

The generator created Rust project ready to compile:

$ tree
.
โ”œโ”€โ”€ Cargo.toml
โ”œโ”€โ”€ Connector.toml
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ sample-config.yaml
โ””โ”€โ”€ src
โ”œโ”€โ”€ config.rs
โ””โ”€โ”€ main.rs

This a simple connector with the code in src/main.rs:

mod config;
use config::CustomConfig;

use fluvio::{RecordKey, TopicProducerPool};
use fluvio_connector_common::{
connector,
Result
};

#[connector(source)]
async fn start(config: CustomConfig, producer: TopicProducerPool) -> Result<()> {
println!("Starting my-connector source connector with {config:?}");
for i in 1..1000 {
let value = format!("Hello, Fluvio - {i}");
producer.send(RecordKey::NULL, value).await?;
producer.flush().await?;
std::thread::sleep(std::time::Duration::from_millis(1000));
}
Ok(())
}

Connectors may also have define configuration parameters as shown in src/config.rs:

use fluvio_connector_common::connector;

#[connector(config)]
#[derive(Debug)]
pub(crate) struct CustomConfig {
#[allow(dead_code)]
foo: String,
}

The Connector.toml file contains the definition of the Connector parameters required to load the file in the Cluster and publish it to Connector Hub.

[package]
name = "my-connector"
group = "acme"
version = "0.1.0"
apiVersion = "0.1.0"
fluvio = "0.10.0"
description = ""
license = "Apache-2.0"
visibility = "private"

[direction]
source = true

[deployment]
binary = "my-connector"

[custom.properties.foo]
title = "Foo"
description = "Foo"
type = "string"

Sectionsโ€‹

  • package is used to build the connector FQDN acme/my-connector@0.1.0, and the description to publish to Hub. The group name is equivalent to the package owner in the Hub.
  • direction is used to declare the direction data flows through the connector, with respect to the Fluvio cluster. An inbound connector uses source = true, and and outbound connector uses sink = true
  • custom.properties.foo defines a user configuration key foo that can be used in the logic of the connector.

The project is ready to build and test.