We are hiring! Apply here
SmartModule Hub
 

Introduction to the SmartModule Hub Service

The SmartModule Hub allows users to start building their data streaming pipeline with SmartModules which have been uploaded into the Hub as ready to use packages. The Hub allows Fluvio pipeline construction focusing on the business logic by providing already built, reliable and tested modules. The modules contain high level configurable inputs which can be applied without needing to rebuild the module. Pipelines can employ a mix of Hub downloaded packages as well as custom developed SmartModules.

A view of available and installed SmartModules is accessible from both the fluvio cli as well as on the Fluvio cloud dashboard.

 

Fluvio Hub Commands

Commands for users to interact with the SmartModule hub are under the fluvio hub set of commands.

  • fluvio hub download group/package@version
  • fluvio hub list
 

fluvio hub download

Download a SmartModule from the hub
Usage: fluvio hub download [OPTIONS] <name>
Arguments:
  <name>  SmartModule name: e.g. infinyon/jolt@v0.0.1
Options:
  -c, --cluster <host:port>        Address of cluster
      --tls                        Enable TLS
      --enable-client-cert         TLS: use client cert
      --domain <DOMAIN>            Required if client cert is used
      --ca-cert <CA_CERT>          Path to TLS ca cert, required when client cert is enabled
      --client-cert <CLIENT_CERT>  Path to TLS client certificate
      --client-key <CLIENT_KEY>    Path to TLS client private key
  -P, --profile <profile>          
      --local                      just download package to local filesystem
      --ipkg                       given local package file, download to cluster
  -h, --help                       Print help (see more with '--help')

The command fluvio hub download group/package@version will download a hub smartmodule into your cluster so it can be configured and used in your pipeline.

$ fluvio hub download infinyon/egex-filter2@0.1.0

downloading infinyon/regex-filter2@0.1.0 to infinyon-regex-filter2-0.1.0.ipkg
... downloading complete
... checking package
trying connection to fluvio router.dev.infinyon.cloud:9003
... cluster smartmodule install complete

Once a package is downloaded, a fluvio smartmodule list command will show the installation

$ flvio sm list
  SMARTMODULE                   SIZE     
  infinyon/regex-filter@0.1.0   316.1 KB 

Once a SmartModule is download, it can be configured and applied in a pipleine. See the SmartModule page for more details on how work with SmartModules once they are downloaded in your cluster.

 

fluvio hub list

The command fluvio hub list will list SmartModules and versions available for download

List available SmartModules in the hub
Usage: fluvio hub list [OPTIONS]
Options:
  -O, --output <type>  Output [default: table] [possible values: table, yaml, json]
  -h, --help           Print help (see more with '--help')

The list command shows Smartmodules by group/package@version in the SMARTMODULE column. The Visibility column describes if the package is accessible to public or private views. If package visibility is private only your user login can list or download the package. Packages uploaded by InfinyOn will be in the infinyon group.

$ fluvio hub list 

  SMARTMODULE                  Visibility 
  mypriv/foo-priv@0.1.1        private    
  infinyon/jolt@0.1.0          public     
  infinyon/json-sql@0.1.0      public     
  infinyon/regex-filter@0.1.0  public
...
 

SmartModule Packages

SmartModule Packages are listed in the hub as in the form of <group>/<package name>@version.

Group infinyon packages are published by InfinyOn, the maker of Fluvio. Third-party developers may also upload public packages which appear in other groups. The Hub service enforces signing and publishing restrictions on uploaded packages. Once a signed package of given group has been published, only the group owner may update that package or add new packages within that group.

 

Private and Public Packages

Published packages are by default private. In order to make a published package publicly available, the package should be set to public. The cli method to do this is below:

cd smartmodule-dev-dir
smdk set-public

After the package is set to public, any publish will ask to verify the publish operation. Once a package version is publicly published, the package cannot revoked.