kafka_connect_watcher package

Subpackages

Submodules

kafka_connect_watcher.aws_emf module

AWS EMF Publishing management for cluster & connectors

kafka_connect_watcher.aws_emf. handle_watcher_emf ( config , watcher ) [source]
Return type :

None

kafka_connect_watcher.aws_emf. init_emf_config ( config ) [source]
Return type :

None

kafka_connect_watcher.aws_emf. publish_cluster_metrics ( cluster , metrics ) [source]
Return type :

None

kafka_connect_watcher.aws_emf. publish_clusters_emf ( cluster ) [source]
Return type :

None

kafka_connect_watcher.aws_emf. publish_connector_metrics ( cluster , connector_name , connector_metrics , metrics ) [source]
Return type :

None

kafka_connect_watcher.aws_emf. publish_watcher_emf_metrics ( config , watcher , metrics ) [source]

kafka_connect_watcher.cli module

kafka_connect_watcher.cli. start_watcher ( ) [source]

kafka_connect_watcher.cluster module

Models Connect cluster

class kafka_connect_watcher.cluster. ConnectCluster ( cluster_config , watcher_config ) [source]

Bases: object

The ConnectCluster manages connection and scan of the connectors status execution. It also collects metrics about itself.

property api : Api
property cluster : Cluster
emf_high_resolution ( ) [source]
Return type :

bool

property hostname : str
property name : str
property port : int

kafka_connect_watcher.config module

Configuration loader

class kafka_connect_watcher.config. Config ( config_file_path = None , configuration = None ) [source]

Bases: object

Represents the configuration & settings from the execution.

property config : dict
property original_config : dict
set_scan_intervals ( ) [source]
Return type :

int

class kafka_connect_watcher.config. EmfConfig ( config ) [source]

Bases: object

kafka_connect_watcher.connectors_eval module

kafka_connect_watcher.connectors_eval. evaluate_connector_status ( queue ) [source]
Return type :

None

kafka_connect_watcher.error_rules module

Error handling rules

class kafka_connect_watcher.error_rules. AutoCorrectRule ( config , watcher_config ) [source]

Bases: object

Actions to take when errors are detected.

map_notify_targets ( watcher_config ) [source]

Maps the notification_channels to the rule notify targets

property original_config : dict
process ( cluster , connector ) [source]
class kafka_connect_watcher.error_rules. EvaluationRule ( rule_definition , watcher_config ) [source]

Bases: object

Error handling rule

config_key : str = 'evaluation_rules'
execute ( connect ) [source]

Scans the connectors, matches the ones invalid and not healthy. When the connector status is RUNNING, we check all the tasks too to be sure. When paused, if we ignore paused connectors, skip

Return type :

None

filter_out_connector ( connector_name , cluster ) [source]
property original_config : dict

kafka_connect_watcher.logger module

Logging management.

kafka_connect_watcher.logger. setup_logging ( ) [source]

kafka_connect_watcher.notifications module

Manages different channels of communications

class kafka_connect_watcher.notifications. Notifications ( notifications_def ) [source]

Bases: object

property definition : dict

kafka_connect_watcher.threads_settings module

kafka_connect_watcher.tools module

kafka_connect_watcher.tools. import_regexes ( to_import ) [source]
Return type :

list [ re.Pattern ]

kafka_connect_watcher.watcher module

Main entrypoint of the python watcher

class kafka_connect_watcher.watcher. Watcher [source]

Bases: object

The Watcher class is the entry point to the program cycling over the different connect cluster, collecting metrics, handling exceptions and graceful shutdowns.

exit_gracefully ( pid , pelse ) [source]
run ( config ) [source]
kafka_connect_watcher.watcher. process_cluster ( queue ) [source]
kafka_connect_watcher.watcher. process_error_rules ( handling_rule , connect_cluster , watcher ) [source]

Module contents