hydroserverpy

Getting Started

HydroServer Python Client

The hydroserverpy Python package provides an interface for managing HydroServer data and metadata, loading observations, and performing data quality control. This guide will go over how to install the package and connect to a HydroServer instance.

Installation

You can install the package via pip:

pip install hydroserverpy

Connecting to HydroServer

To connect to HydroServer, you need to initialize the client with the instance of HydroServer you’re using and your user credentials if you want to access and modify your own data. If you don’t provide authentication credentials you can read public data, but you will not be able to create or modify any data.

Example: Anonymous User

from hydroserverpy import HydroServer

# Initialize HydroServer connection.
hs_api = HydroServer(
    host='https://playground.hydroserver.org'
)

Example: Basic Authentication

from hydroserverpy import HydroServer

# Initialize HydroServer connection with credentials.
hs_api = HydroServer(
    host='https://playground.hydroserver.org',
    username='user@example.com',
    password='******'
)

Funding and Acknowledgements

Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003).

Subpackages

Module contents

class hydroserverpy.HydroServer(host, username=None, password=None, apikey=None, api_route='api')[source]

Bases: object

Connects to a HydroServer instance and used to interact with HydroServer’s Data Management API endpoints.

Parameters:

host (str) – The base URL or host of the HydroServer API.
username (Optional[str]) – The username for basic authentication, if required.
password (Optional[str]) – The password for basic authentication, if required.
apikey (Optional[str]) – The API key for authentication, if using API key authentication.
api_route (str) – The API route to use, default is ‘api’.

property dataloaders

Data Loader Endpoint.

Returns:: An instance of DataLoaderEndpoint.
Return type:: DataLoaderEndpoint

property datasources

Data Source Endpoint.

Returns:: An instance of DataSourceEndpoint.
Return type:: DataSourceEndpoint

property datastreams

Datastream Endpoint.

Returns:: An instance of DatastreamEndpoint.
Return type:: DatastreamEndpoint

property observedproperties

Observed Property Endpoint.

Returns:: An instance of ObservedPropertyEndpoint.
Return type:: ObservedPropertyEndpoint

property processinglevels

Processing Level Endpoint.

Returns:: An instance of ProcessingLevelEndpoint.
Return type:: ProcessingLevelEndpoint

property resultqualifiers

Result Qualifier Endpoint.

Returns:: An instance of ResultQualifierEndpoint.
Return type:: ResultQualifierEndpoint

property sensors

Sensor Endpoint.

Returns:: An instance of SensorEndpoint.
Return type:: SensorEndpoint

property things

Thing Endpoint.

Returns:: An instance of ThingEndpoint.
Return type:: ThingEndpoint

property units

Unit Endpoint.

Returns:: An instance of UnitEndpoint.
Return type:: UnitEndpoint

class hydroserverpy.HydroServerETL(service, data_file, data_source)[source]

Bases: object

run()[source]

The run function is the main function of this class. It reads in a data file and parses it into observations, which are then posted to HydroServer. The run function also updates the DataSource object with information about the sync process.

:param self :return: None

class hydroserverpy.HydroServerQualityControl(datastream_id, observations)[source]

Bases: object

Quality control operations for HydroServer observations.

Parameters:

datastream_id (Union[UUID, str]) – The ID of the datastream.
observations (pd.DataFrame) – DataFrame containing ‘timestamp’ and ‘value’ columns.

add_points(points, index=None)[source]

Adds new points to the observations, optionally at specified indices.

Parameters:

points (List[List[Union[str, float]]]) – List of points to be added.
index (Optional[List[int]]) – Optional list of indices at which to insert the points.

Return type:

None

change_values(index_list, operator, value)[source]

Changes the values of observations based on the specified operator and value.

Parameters:

index_list (List[int]) – List of indices for which values will be changed.
operator (str) – The operation to perform (‘MULT’, ‘DIV’, ‘ADD’, ‘SUB’, ‘ASSIGN’).
value (Union[int, float]) – The value to use in the operation.

Return type:

None

datastream_id: Union[UUID, str]

delete_points(index_list)[source]

Deletes points from the observations at the specified indices.

Parameters:: index_list (List[int]) – List of indices for which points will be deleted.
Return type:: None

drift_correction(start, end, gap_width)[source]

Applies drift correction to the values of observations within the specified index range.

Parameters:

start (int) – Start index of the range to apply drift correction.
end (int) – End index of the range to apply drift correction.
gap_width (float) – The width of the drift gap to correct.

Returns:

DataFrame after applying drift correction.

Return type:

pd.DataFrame

fill_gap(gap, fill, interpolate_values)[source]

Fills identified gaps in the observations with placeholder values and optionally interpolates the values.

Parameters:

gap (Tuple[int, str]) – Tuple containing the time value and unit for identifying gaps.
fill (Tuple[int, str]) – Tuple containing the time value and unit for filling gaps.
interpolate_values (bool) – Whether to interpolate values for the filled gaps.

Returns:

DataFrame of points that filled the gaps.

Return type:

pd.DataFrame

filter(data_filter)[source]

Executes the applied filters and returns the resulting DataFrame.

Parameters:: data_filter (Dict[str, Union[float, int]]) – Dictionary containing filter operations and their values.
Return type:: None

find_gaps(time_value, time_unit)[source]

Identifies gaps in the observations based on the specified time value and unit.

Parameters:

time_value (int) – The time value for detecting gaps.
time_unit (str) – The unit of time (e.g., ‘s’, ‘m’, ‘h’).

Returns:

DataFrame containing the observations with gaps.

Return type:

pd.DataFrame

interpolate(index_list)[source]

Interpolates the values of observations at the specified indices using linear interpolation.

Parameters:: index_list (list[int]) – List of indices where values will be interpolated.
Return type:: None

property observations: DataFrame

Returns the observations DataFrame, filtered if a filter has been applied.

Returns:: Observations DataFrame.
Return type:: pd.DataFrame

shift_points(index_list, time_value, time_unit)[source]

Shifts the timestamps of the observations at the specified indices by a given time value and unit.

Parameters:

index_list (List[int]) – List of indices where timestamps will be shifted.
time_value (int) – The amount of time to shift the timestamps.
time_unit (str) – The unit of time (e.g., ‘s’ for seconds, ‘m’ for minutes).

Return type:

None