kubernetes.operator.job.autoscaler.backlog-processing.lag-threshold |
5 min |
Duration |
Lag threshold which will prevent unnecessary scalings while removing the pending messages responsible for the lag. |
kubernetes.operator.job.autoscaler.catch-up.duration |
5 min |
Duration |
The target duration for fully processing any backlog after a scaling operation. Set to 0 to disable backlog based scaling. |
kubernetes.operator.job.autoscaler.enabled |
false |
Boolean |
Enable job autoscaler module. |
kubernetes.operator.job.autoscaler.history.max.age |
86400000 ms |
Duration |
Maximum age for past scaling decisions to retain. |
kubernetes.operator.job.autoscaler.history.max.count |
3 |
Integer |
Maximum number of past scaling decisions to retain per vertex. |
kubernetes.operator.job.autoscaler.metrics.busy-time.aggregator |
MAX |
Enum |
Metric aggregator to use for busyTime metrics. This affects how true processing/output rate will be computed. Using max allows us to handle jobs with data skew more robustly, while avg may provide better stability when we know that the load distribution is even.
Possible values: |
kubernetes.operator.job.autoscaler.metrics.window |
10 min |
Duration |
Scaling metrics aggregation window size. |
kubernetes.operator.job.autoscaler.restart.time |
3 min |
Duration |
Expected restart time to be used until the operator can determine it reliably from history. |
kubernetes.operator.job.autoscaler.scale-down.max-factor |
0.6 |
Double |
Max scale down factor. 1 means no limit on scale down, 0.6 means job can only be scaled down with 60% of the original parallelism. |
kubernetes.operator.job.autoscaler.scale-up.grace-period |
1 h |
Duration |
Duration in which no scale down of a vertex is allowed after it has been scaled up. |
kubernetes.operator.job.autoscaler.scale-up.max-factor |
100000.0 |
Double |
Max scale up factor. 2.0 means job can only be scaled up with 200% of the current parallelism. |
kubernetes.operator.job.autoscaler.scaling.effectiveness.detection.enabled |
false |
Boolean |
Whether to enable detection of ineffective scaling operations and allowing the autoscaler to block further scale ups. |
kubernetes.operator.job.autoscaler.scaling.effectiveness.threshold |
0.1 |
Double |
Processing rate increase threshold for detecting ineffective scaling threshold. 0.1 means if we do not accomplish at least 10% of the desired capacity increase with scaling, the action is marked ineffective. |
kubernetes.operator.job.autoscaler.scaling.enabled |
true |
Boolean |
Enable vertex scaling execution by the autoscaler. If disabled, the autoscaler will only collect metrics and evaluate the suggested parallelism for each vertex but will not upgrade the jobs. |
kubernetes.operator.job.autoscaler.stabilization.interval |
5 min |
Duration |
Stabilization period in which no new scaling will be executed |
kubernetes.operator.job.autoscaler.target.utilization |
0.7 |
Double |
Target vertex utilization |
kubernetes.operator.job.autoscaler.target.utilization.boundary |
0.4 |
Double |
Target vertex utilization boundary. Scaling won't be performed if the current processing rate is within [target_rate / (target_utilization - boundary), (target_rate / (target_utilization + boundary)] |
kubernetes.operator.job.autoscaler.vertex.exclude.ids |
|
List<String> |
A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented. |
kubernetes.operator.job.autoscaler.vertex.max-parallelism |
200 |
Integer |
The maximum parallelism the autoscaler can use. Note that this limit will be ignored if it is higher than the max parallelism configured in the Flink config or directly on each operator. |
kubernetes.operator.job.autoscaler.vertex.min-parallelism |
1 |
Integer |
The minimum parallelism the autoscaler can use. |