openrvdas.logger.transforms.transform
The biggest thing that the abstract parent class Transform now does is help with (optional) type checking of the child class' inputs. In the past, an explicit, but very awkward, form of type checking was used, with child classes passing the Transform class a list of input_format and output_format specifications.
That is now deprecated in favor of using Python's type hints. Type hints should be specified for the child class' transform() method. Then the transform() method can call self.can_process_record(record) to see whether it's one of the input types it can handle. If not, it can "return self.digest_record(record) to have the parent class try to deal with it:
E.g.: def transform(self, record: Union[int, str, float]): if not self.can_process_record(record): # inherited from BaseModule() return self.digest_record(record) # inherited from BaseModule() return str(record) + '+'
If no type hints are specified, can_process_record() will return True for all records except those of type "None" or "list". The logic is that digest_record() will return a None when given a None, and when given a list, will iteratively apply the transform to every element of the list and return the resulting list.
Note that the child class can explicitly call super().__init__() or such to initialize the type checking and set its debugging level. If it is not explicitly initialized, it will be done implicitly the first time can_process_record() or digest_record() are called, but with the default of quiet=False.
1#!/usr/bin/env python3 2""" 3The biggest thing that the abstract parent class Transform now does is help 4with (optional) type checking of the child class' inputs. In the past, an 5explicit, but very awkward, form of type checking was used, with child classes 6passing the Transform class a list of input_format and output_format specifications. 7 8That is now deprecated in favor of using Python's type hints. Type hints should be 9specified for the child class' transform() method. Then the transform() method can 10call self.can_process_record(record) to see whether it's one of the input types it 11can handle. If not, it can "return self.digest_record(record) to have the parent 12class try to deal with it: 13 14E.g.: 15 def transform(self, record: Union[int, str, float]): 16 if not self.can_process_record(record): # inherited from BaseModule() 17 return self.digest_record(record) # inherited from BaseModule() 18 return str(record) + '+' 19 20If no type hints are specified, can_process_record() will return True for all 21records *except* those of type "None" or "list". The logic is that digest_record() 22will return a None when given a None, and when given a list, will iteratively 23apply the transform to every element of the list and return the resulting list. 24 25Note that the child class can explicitly call super().__init__() or such 26to initialize the type checking and set its debugging level. If it is not explicitly 27initialized, it will be done implicitly the first time can_process_record() or 28digest_record() are called, but with the default of quiet=False. 29""" 30 31from logger.utils.base_module import BaseModule # noqa: E402 32 33 34################################################################################ 35class Transform(BaseModule): 36 """ 37 Base class Transform about which we know nothing else. 38 39 Passes arguments quiet, encoding and encoding_errors up to BaseModule 40 """ 41 ############################ 42 def __init__(self, **kwargs): 43 super().__init__(**kwargs) 44 self._initialize_type_hints() 45 46 ############################ 47 def _initialize_type_hints(self): 48 """ Retrieve any type hints for child transform() method so we can 49 check whether the type of record we've received can be parsed 50 natively or not.""" 51 super()._initialize_type_hints(module_type='transform', 52 module_method=self.__class__.transform)
36class Transform(BaseModule): 37 """ 38 Base class Transform about which we know nothing else. 39 40 Passes arguments quiet, encoding and encoding_errors up to BaseModule 41 """ 42 ############################ 43 def __init__(self, **kwargs): 44 super().__init__(**kwargs) 45 self._initialize_type_hints() 46 47 ############################ 48 def _initialize_type_hints(self): 49 """ Retrieve any type hints for child transform() method so we can 50 check whether the type of record we've received can be parsed 51 natively or not.""" 52 super()._initialize_type_hints(module_type='transform', 53 module_method=self.__class__.transform)
Base class Transform about which we know nothing else.
Passes arguments quiet, encoding and encoding_errors up to BaseModule
quiet - if type checking should log type errors or operate silently.
Two additional arguments govern how records will be encoded/decoded
from bytes, if desired by the Writer subclass when it calls
_encode_str() or _decode_bytes:
encoding - 'utf-8' by default. If empty or None, do not attempt any
decoding and return raw bytes. Other possible encodings are
listed in online documentation here:
https://docs.python.org/3/library/codecs.html#standard-encodings
encoding_errors - 'ignore' by default. Other error strategies are
'strict', 'replace', and 'backslashreplace', described here:
https://docs.python.org/3/howto/unicode.html#encodings
mirror_to - Optional Writer to which all records read or transformed
by this module (if it is a Reader or Transform) will be
"mirrored" (copied). Mirroring happens asynchronously via
a queue and background thread to minimize impact on the
primary data flow. Writers cannot be mirrored.