hibou.preprocessing.sync

hibou.preprocessing.sync(*args, **kwargs)

Synchronizes the two pd.DataFrame by averaging the inputs data and the targets data to the given timestep. This function can also filter the data and extract the variables of interest.

Note

If the filter on inputs data failed, neither inputs data, nor targets_data will be stored.

Warning

By default the inputs_var and targets_var will be all the known variables. You should pass values to extract only the variables the artificial neural network needs.

Warning

During the synchronization, the timestamps stored is the end of the averaging period. And the fluxes should have the same convention.

Parameters

inputspd.DataFrame

The inputs data for the neural network.

targetspd.DataFrame

The targets data for the neural network..

filtersFilters, optionnal, keyword-only

By default: Filters(). The filters to apply on the data, by default, no filter is applied. Both inputs and targets data will be pass to theses filters. In case the filters failed, neither inputs nor targets data are stored.

filters_inputsFilters, optionnal, keyword-only

By default: Filters(). The filters to apply on the inputs data only, by default, no filter is applied. Be careful as no targets data will be given to these filters. Note that these filters are only used with the KEEP_ALL synchronization method.

inputs_varIterable[str], optionnal, keyword-only

By default, variables will contains all known variables for the inputs data. An iterable of strings that contains the names of the variables to extract for the inputs.

targets_varIterable[str], optionnal, keyword-only

By default, variables will contains all known variables for the targets data. An iterable of strings that contains the names of the variables to extract for the targets.

timesteptimedelta, optionnal, keyword-only

By default: timedelta(minutes=30).

methodMethod, optionnal, keyword-only

By default: Method.ON_BOTH. The method of synchronization can be:

• Method.ON_BOTH : keep inputs data even if there is no targets data

• Method.KEEP_ALL : keep all data whenever there is targets or not

Please note that targets alone are never kept.

missing_sample_allowanceint, optionnal, keyword-only

By default: 100. The maximum percentage of missing data allowed in the calculation of the mean. Must be between 0 and 100.

Returns

tuple

A tuple that contains the synchronized (and filtered) data: (inputs data, targets data, timestamps).

Raises

UnknownMethodError

If the given method is not recognized.