Security

TLS

NiFi sets up TLS encryption for the http endpoints that serve the UI. By default, this interface is secured using certificates generated to work with the default SecretClass tls.

Nifi can be configured to use a different SecretClass as shown below:

apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
spec:
  # ...
  clusterConfig:
    tls:
      serverSecretClass: non-default-secret-class (1)

1	The name of the SecretClass used for certificates for the NiFi UI.

Authentication

Every user has to authenticate themselves before using NiFI. There are multiple options to set up the authentication of users. All authentication related parameters are configured under spec.clusterConfig.authentication.

Single user

The Single user allows the creation of one admin user for NiFi. This is a rudimentary authentication method to quickly test and log in to the canvas. However, due to it being a single user with all rights, this is not recommended in production.

apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: simple-nifi-users (1)
spec:
  provider:
    static:
      userCredentialsSecret:
        name: nifi-admin-credentials (2)

1	The name of the AuthenticationClass referenced in the NiFi cluster.
2	The name of the Secret containing the admin credentials.

apiVersion: v1
kind: Secret
metadata:
  name: nifi-admin-credentials (1)
stringData:
  admin: admin (2)
  bob: bob (3)

1	The name of the Secret containing the admin user credentials.
2	The user and password combination of the admin user. The username must be "admin" and cannot be changed. The NiFi Pods will not start if they cannot mount the "admin" entry from the Secret. The password can be adapted.
3	The Secret maybe used by other products of the Stackable Data Platform that allow more than one user. The Stackable operator for Apache NiFi will ignore all users except for "admin".

spec:
  clusterConfig:
    authentication:
      - authenticationClass: simple-nifi-users (1)

1	The reference to an AuthenticationClass. NiFi only supports one authentication mechanism at a time.

LDAP

NiFi supports authentication of users against an LDAP server. This requires setting up an AuthenticationClass for the LDAP server. The AuthenticationClass is then referenced in the NifiCluster resource as follows:

apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
  name: test-nifi
spec:
  clusterConfig:
    authentication:
      - authenticationClass: ldap (1)

1	The reference to an AuthenticationClass called `ldap`

You can follow the Authentication with OpenLDAP tutorial to learn how to set up an AuthenticationClass for an LDAP server, as well as consulting the AuthenticationClass reference .

OIDC

NiFi supports authentication of users against an OIDC provider. This requires setting up an AuthenticationClass for the OIDC provider and specifying a Secret containing the OIDC client id and client secret as part of the NiFi configuration. The AuthenticationClass and the OIDC client credentials Secret are then referenced in the NifiCluster resource:

apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
  name: test-nifi
spec:
  clusterConfig:
    authentication:
      - authenticationClass: oidc (1)
        oidc:
          clientCredentialsSecret: nifi-oidc-client (2)

1	The reference to an AuthenticationClass called `oidc`
2	The reference to an existing Secret called `nifi-oidc-client`

apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: oidc
spec:
  provider:
    oidc:
      hostname: keycloak.example.com
      rootPath: /realms/test/ (1)
      principalClaim: preferred_username
      scopes:
        - openid
        - email
        - profile
      port: 8080
      tls: null
  [...]

1	A trailing slash in `rootPath` is necessary.

apiVersion: v1
kind: Secret
metadata:
  name: nifi-oidc-client
stringData:
  clientId: <client-id>
  clientSecret: <client-secret>

Authorization

The Stackable Operator for Apache NiFi supports multiple authorization methods, the available authorization methods depend on the chosen authentication method. Using Open Policy Agent for authorization is independent of the authentication method.

Single user

With this authorization method, a single user has administrator capabilities.

LDAP

The operator uses the FileUserGroupProvider and FileAccessPolicyProvider to bind the LDAP user to the NiFi administrator group. This user is then able to create and modify groups and policies in the web interface. These changes local to the Pod running NiFi and are not persistent.

OIDC

With this authorization method, all authenticated users have administrator capabilities.

An admin user with an auto-generated password is created that can access the NiFi API. The password for this user is stored in a Kubernetes Secret called <nifi-name>-oidc-admin-password.

Open Policy Agent (OPA)

NiFi can be configured to delegate authorization decisions to an Open Policy Agent (OPA) instance. More information on the setup and configuration of OPA can be found in the OPA Operator documentation.

A NiFi cluster can be configured with OPA authorization by adding this section to the configuration:

spec:
  clusterConfig:
    authorization:
      opa:
        configMapName: simple-opa (1)
        package: my-nifi-rules (2)
        cache:
          entryTimeToLive: 5s (3)
          maxEntries: 10 (4)

1	The name of your OPA Stacklet (`simple-opa` in this case)
2	The rego rule package to use for policy decisions. The package needs to contain an `allow` rule. This is optional and defaults to the name of the NiFi Stacklet.
3	TTL for items in the cache in NiFi. Optional, defaults to 30 seconds.
4	Maximum number of concurrent entries in the cache in NiFi. Optional, defaults to 10000 entries.

Defining rego rules

For a general explanation of how rules are written, please refer to the OPA documentation. Authorization with OPA is done using a custom authorizer provided by a plugin for NiFi.

OPA Inputs

The payload sent by NiFi with each request to OPA, that is accessible within the rego rules, has the following structure:

Payload Field

Description

Possible Values

action.name

The action taken against the resource.

read, write

resource.id

The unique identifier of the resource that is being authorized. This might be a parent component in the case of resourceNotFound is set to true.

resource.name

The name of the resource that is being authorized. This might be a parent component in the case of resourceNotFound is set to true.

resource.safeDescription

The description of the resource that is being authorized.

requestedResource.id

The unique identifier of the original resource that was requested (see Component Level Access Policies and Access Policy Inheritance).

requestedResource.name

The name of the original resource that is being authorized on (see Component Level Access Policies and Access Policy Inheritance).

requestedResource.safeDescription

The description of the original resource that is being authorized on (see Component Level Access Policies and Access Policy Inheritance).

identity.name

The name of the identity/user accessing the resource.

identity.groups

Comma-separated list of groups that the identity/user accessing the resource belongs to.

properties.isAccessAttempt

Whether this is a direct access attempt of the resource or if it’s being checked as part of another response.

true, false (String)

isAnonymous

Whether the entity accessing the resource is anonymous.

true, false (String)

resourceContext

Object containing the event attributes to make additional access decisions for provenance events.

{"": ""} if empty

userContext

Additional context for the user to make additional access decisions.

{"": ""} if empty

OPA Result

The OPA authorizer plugin expects rego rules to be named allow and to return a result following this schema:

{
    "allowed": <Boolean>, (1)
    "resourceNotFound": <Boolean>, (2)
    "dumpCache": <Boolean>, (3)
    "message": <String>, (4)
}

1	Whether the action against the resource is allowed. Optional, defaults to false.
2	Whether no rule was found for the authorization request. This should only be set to true in the default rule to e.g. forward policy decisions to parent components. If set to true the value of the "allowed" field will be ignored. Optional, defaults to false.
3	Whether the whole local cache in the OPA authorizer plugin in NiFi should be invalidated. Optional, defaults to false.
4	An optional error message that is shown to the user when access is denied.

Access Policies

NiFi uses access policies to manage access to system-wide resources like the user interface.

Component Level Access Policies and Access Policy Inheritance

Component Level Access Policies allow managing granular access to components like process-groups and processors. Components can inherite access policies defined for parent components, e.g. a process group is the parent component for a contained processor component.

The payload field requestedResource contains the id, name and description of the original resource that was requested. In cases with inherited policies, this will be an ancestor resource of the current resource. For the initial request, and cases without inheritance, the requested resource will be the same as the current resource.

When an authorizer returns "resourceNotFound" as result instead of an authorization decision, NiFi will send an authorization request for the parent component. Access policy inheritance can be recursive up to the root component. If "resourceNotFound" is returned for an authorization request and the component doesn’t have a parent component, NiFi will deny access to the component.

To manage access for all process groups in the NiFi instance a rule has to be defined for the root process group which is identified by the resource name "NiFi Flow" and a resource id generated at random ("/process-groups/<uuid>").

default allow := {
  "resourceNotFound": true
} (1)

allow := {
  "allowed": true
} if {
  input.resource.name == "NiFi Flow"
  startswith(input.resource.id, "/process-groups")
} (2)

allow := {
  "allowed": false
} if {
  input.resource.id == "/process-groups/a10c311e-0196-1000-2856-dc0606d3c5d7"
  input.identity.name == "alice"
} (3)

1	The default rule should return `"resourceNotFound": true`. If this is not set, NiFi’s access policy inheritance will not work. Any values for the `allowed` field in the response will be ignored.
2	A rule that grants all users access to the root process group and thus to all components in the NiFi instance.
3	A rule that denies access to a specific process group for the user "alice". For this process group the default rego rule will not be applied and NiFi’s component inhertiance will not be used. All child components of this process group will also be authorized based on this rule unless a more granular rule overrides it.

Communication between NiFi nodes

To allow communication between NiFi nodes an additional rego rule is required:

allow := {
  "allowed": true
} if {
  input.identity.name == "CN=generated certificate for pod" (1)
  input.resource.id == "/proxy" (2)
}

1	The identity of NiFi nodes authenticated with TLS certificates provided by the secrets operator.
2	Only access to the `/proxy` API is required.

Caching

The OPA authorizer has a mechanism to cache results from OPA which can be configured in the NifiCluster spec (see above). To delete the whole cache add "dumpCache": true to the result.

allow := {
  "allowed": false
  "dumpCache": true
} if {
  ...
}

Encrypting sensitive properties on disk

Some flows require storing a sensitive property like a password or access token, which is then stored on disk. NiFi encrypts these properties in flow files.

The sensitive property encryption is configured using the properties keySecret, autoGenerate and algorithm in the spec.clusterConfig.sensitiveProperties configuration section of the NifiCluster resource.

The keySecret configures the name of the Secret object that holds the encryption key; it is required to specify a Secret name. The Secret needs to contain the key as a value to the nifiSensitivePropsKey key. You can either specify a key yourself - in which case the key needs to be at least 12 characters long - or just specify the name of the Secret and set autoGenerate to true (the default is false). If autoGenerate is false and no Secret with the given name in keySecret is found, the operator raises an error.

The algorithm property configures the encryption algorithm used to encrypt the sensitive properties. Consult the reference documentation for a list of supported algorithms.

Autogenerated sensitive properties key

Let the operator generate a Secret with the name nifi-sensitive-property-key:

sensitiveProperties:
  keySecret: nifi-sensitive-property-key
  autoGenerate: true

Custom sensitive properties key and sensitive properties algorithm

Create the Secret yourself:

apiVersion: v1
kind: Secret
metadata:
  name: nifi-sensitive-properties-key
stringData:
  nifiSensitivePropsKey: my-encryption-key

Configure the Secret and a different sensitive properties algorithm:

sensitiveProperties:
  keySecret: nifi-sensitive-property-key
  algorithm: nifiArgon2AesGcm256

Upgrading sensitive properties algorithm

Please make sure to backup any flows before upgrading the sensitive properties algorithm!

The sensitive properties algorithm can be changed via the nifi.sh CLI tool as described in the Apache NiFi documentation.

Assuming that you deployed a cluster with this:

sensitiveProperties:
  keySecret: nifi-sensitive-property-key
  algorithm: nifiArgon2AesGcm256

If you want to change the algorithm to nifiPbkdf2AesGcm256, you have to run the following command on each NiFi Pod:

/stackable/nifi/bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256

Be careful with the notation used in the NiFiCluster nifiPbkdf2AesGcm256 versus the setting in the NiFi CLI NIFI_PBKDF2_AES_GCM_256!

Alternatively, you can use this shell script to automatically execute this in each pod via kubectl (make sure to edit the NAMESPACE and STATEFULSET_NAME accordingly):

NAMESPACE="default"
STATEFULSET_NAME="simple-nifi-node-default"
COMMAND="/stackable/nifi/bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256"

kubectl get pods -n "$NAMESPACE" --no-headers -o custom-columns=":metadata.name" | grep "^$STATEFULSET_NAME" | \
while read pod; do
  echo "Running on $pod"
  kubectl exec -n "$NAMESPACE" -c "nifi" "$pod" -- sh -c "$COMMAND"
done

Afterwards, update your NiFiCluster to the required algorithm nifiPbkdf2AesGcm256:

sensitiveProperties:
  keySecret: nifi-sensitive-property-key
  algorithm: nifiPbkdf2AesGcm256

Finally, apply the updated NiFiCluster and restart / delete the StatefulSet:

kubectl apply -n "$NAMESPACE" -f <nifi-yaml>
kubectl delete -n "$NAMESPACE" statefulsets ${STATEFULSET_NAME}

Host Header Check

NiFi checks the host header of incoming requests and rejects them if they are passing through a proxy that is not on an allow-list configured in the nifi.web.proxy.host property.

A patch applied during the build of the SDP container image for NiFi allows turning off this check by adding nifi.web.proxy.host=* to the properties. The Host header check for NiFi clusters created by the operator is disabled by default but can be enabled in the NiFi configuration. In this case the list of allowed hosts defaults to Kubernetes Services used by NiFi and can be extended with custom entries.

spec:
  clusterConfig:
    hostHeaderCheck:
      allowAll: false
      additionalAllowedHosts:
      - example.com:1234