patroni.postgresql.mpp package¶
Submodules¶
Module contents¶
Abstract classes for MPP handler.
MPP stands for Massively Parallel Processing, and Citus belongs to this architecture. Currently, Citus is the only supported MPP cluster. However, we may consider adapting other databases such as TimescaleDB, GPDB, etc. into Patroni.
-
class
patroni.postgresql.mpp.
AbstractMPP
(config: Dict[str, Union[str, int]])¶ Bases:
abc.ABC
An abstract class which should be passed to
AbstractDCS
.Note
We create
AbstractMPP
andAbstractMPPHandler
to solve the chicken-egg initialization problem. When initializing DCS, we dynamically create an object implementingAbstractMPP
, later this object is used to instantiate an object implementingAbstractMPPHandler
.-
__init__
(config: Dict[str, Union[str, int]]) → None¶ Init method for
AbstractMPP
.- Parameters
config – configuration of MPP section.
-
_abc_impl
= <_abc._abc_data object>¶
-
_get_handler_cls
() → Iterator[Type[patroni.postgresql.mpp.AbstractMPPHandler]]¶ Find Handler classes inherited from a class type of this object.
- Yields
handler classes for this object.
-
abstract property
coordinator_group_id
¶ The group id of the coordinator PostgreSQL cluster.
-
get_handler_impl
(postgresql: Postgresql) → AbstractMPPHandler¶ Find and instantiate Handler implementation of this object.
- Parameters
postgresql – a reference to
Postgresql
object.- Raises
PatroniException
: if the Handler class haven’t been found.- Returns
an instantiated class that implements Handler for this object.
-
abstract property
group
¶ The group for a given MPP implementation.
-
group_re
: Any¶
-
is_coordinator
() → bool¶ Check whether this node is running in the coordinator PostgreSQL cluster.
- Returns
True
if MPP is enabled and the group id of this node matches with thecoordinator_group_id
, otherwiseFalse
.
-
is_enabled
() → bool¶ Check if MPP is enabled for a given MPP.
Note
We just check that the
_config
object isn’t empty and expect it to be empty only in case ofNull
.- Returns
True
if MPP is enabled, otherwiseFalse
.
-
is_worker
() → bool¶ Check whether this node is running as a MPP worker PostgreSQL cluster.
- Returns
True
if MPP is enabled and this node is known to be not running as the coordinator PostgreSQL cluster, otherwiseFalse
.
-
property
k8s_group_label
¶ Group label used for kubernetes DCS of the MPP cluster.
- Returns
A string representation of the k8s group label of a given MPP implementation.
-
property
type
¶ The type of the MPP cluster.
- Returns
A string representation of the type of a given MPP implementation.
-
-
class
patroni.postgresql.mpp.
AbstractMPPHandler
(postgresql: Postgresql, config: Dict[str, Union[str, int]])¶ Bases:
patroni.postgresql.mpp.AbstractMPP
An abstract class which defines interfaces that should be implemented by real handlers.
-
__init__
(postgresql: Postgresql, config: Dict[str, Union[str, int]]) → None¶ Init method for
AbstractMPPHandler
.- Parameters
postgresql – a reference to
Postgresql
object.config – configuration of MPP section.
-
_abc_impl
= <_abc._abc_data object>¶
-
abstract
adjust_postgres_gucs
(parameters: Dict[str, Any]) → None¶ Adjust GUCs in the current PostgreSQL configuration.
- Parameters
parameters – dictionary of GUCs, with key as GUC name and the corresponding value as current GUC value.
-
abstract
bootstrap
() → None¶ Bootstrap handler.
Is called when the new cluster is initialized (through
initdb
or a custom bootstrap method).
-
group_re
: Any¶
-
abstract
handle_event
(cluster: patroni.dcs.Cluster, event: Dict[str, Any]) → None¶ Handle an event sent from a worker node.
- Parameters
cluster – the currently known cluster state from DCS.
event – the event to be handled.
-
abstract
ignore_replication_slot
(slot: Dict[str, str]) → bool¶ Check whether provided replication slot existing in the database should not be removed.
Note
MPP database may create replication slots for its own use, for example to migrate data between workers using logical replication, and we don’t want to suddenly drop them.
- Parameters
slot – dictionary containing the replication slot settings, like
name
,database
,type
, andplugin
.- Returns
True
if the replication slots should not be removed, otherwiseFalse
.
-
abstract
schedule_cache_rebuild
() → None¶ Cache rebuild handler.
Is called to notify handler that it has to refresh its metadata cache from the database.
-
abstract
sync_meta_data
(cluster: patroni.dcs.Cluster) → None¶ Sync meta data on the coordinator.
- Parameters
cluster – the currently known cluster state from DCS.
-
-
class
patroni.postgresql.mpp.
Null
¶ Bases:
patroni.postgresql.mpp.AbstractMPP
Dummy implementation of
AbstractMPP
.-
_abc_impl
= <_abc._abc_data object>¶
-
property
coordinator_group_id
¶ The group id of the coordinator PostgreSQL cluster.
- Returns
always
None
.
-
group_re
: Any¶
-
-
class
patroni.postgresql.mpp.
NullHandler
(postgresql: Postgresql, config: Dict[str, Union[str, int]])¶ Bases:
patroni.postgresql.mpp.Null
,patroni.postgresql.mpp.AbstractMPPHandler
Dummy implementation of
AbstractMPPHandler
.-
__init__
(postgresql: Postgresql, config: Dict[str, Union[str, int]]) → None¶ Init method for
NullHandler
.- Parameters
postgresql – a reference to
Postgresql
object.config – configuration of MPP section.
-
_abc_impl
= <_abc._abc_data object>¶
-
adjust_postgres_gucs
(parameters: Dict[str, Any]) → None¶ Adjust GUCs in the current PostgreSQL configuration.
- Parameters
parameters – dictionary of GUCs, with key as GUC name and corresponding value as current GUC value.
-
bootstrap
() → None¶ Bootstrap handler.
Is called when the new cluster is initialized (through
initdb
or a custom bootstrap method).
-
group_re
: Any¶
-
handle_event
(cluster: patroni.dcs.Cluster, event: Dict[str, Any]) → None¶ Handle an event sent from a worker node.
- Parameters
cluster – the currently known cluster state from DCS.
event – the event to be handled.
-
ignore_replication_slot
(slot: Dict[str, str]) → bool¶ Check whether provided replication slot existing in the database should not be removed.
Note
MPP database may create replication slots for its own use, for example to migrate data between workers using logical replication, and we don’t want to suddenly drop them.
- Parameters
slot – dictionary containing the replication slot settings, like
name
,database
,type
, andplugin
.- Returns
always
False
.
-
schedule_cache_rebuild
() → None¶ Cache rebuild handler.
Is called to notify handler that it has to refresh its metadata cache from the database.
-
sync_meta_data
(cluster: patroni.dcs.Cluster) → None¶ Sync meta data on the coordinator.
- Parameters
cluster – the currently known cluster state from DCS.
-
-
patroni.postgresql.mpp.
get_mpp
(config: Union[Config, Dict[str, Any]]) → patroni.postgresql.mpp.AbstractMPP¶ Attempt to load and instantiate a MPP module from known available implementations.
- Parameters
config – object or dictionary with Patroni configuration.
- Returns
The successfully loaded MPP or fallback to
Null
.
-
patroni.postgresql.mpp.
iter_mpp_classes
(config: Optional[Union[Config, Dict[str, Any]]] = None) → Iterator[Tuple[str, Type[patroni.postgresql.mpp.AbstractMPP]]]¶ Attempt to import MPP modules that are present in the given configuration.
- Parameters
config – configuration information with possible MPP names as keys. If given, only attempt to import MPP modules defined in the configuration. Else, if
None
, attempt to import any supported MPP module.- Yields
tuples, each containing the module
name
and the imported MPP class object.