ocean_data_gateway.readers.erddap.stations¶

class ocean_data_gateway.readers.erddap.stations(kwargs)[source]¶

Bases: ocean_data_gateway.readers.erddap.ErddapReader

Inherits from ErddapReader to search for 1+ stations or dataset_ids.

kw¶

Contains space and time search constraints: min_time, max_time.

Type: dict, optional

variables¶

variables is None for this class since we read search by dataset_id or station name.

Type: None

approach¶

approach is defined as ‘stations’ for this class.

Type: string

Attributes

dataset_ids: Find dataset_ids for server.
meta: Rearrange the individual metadata into a dataframe.

Methods

`all_variables`()	Return a DataFrame of allowed variable names.
`check_variables`(variables[, verbose])	Checks variables for presence in database list.
`count`(url)	Small helper function to count len(results) at url.
`data`()	Read in data for all dataset_ids.
`data_by_dataset`(dataset_id)	Return the data for a single dataset_id.
`find_dataset_id_from_station`(station)	Find dataset_id from station name.
`meta_by_dataset`(dataset_id)	Return the catalog metadata for a single dataset_id.
`search_variables`(variables)	Find valid variables names to use.

__init__(kwargs)[source]¶

Parameters

kwargs (dict) –

Can contain arguments to pass onto the base ErddapReader class (known_server, protocol, server, parallel). The dict entries to initialize this class are: * kw: dict, optional

Contains time search constraints: min_time, max_time. If not input, all time will be used.

dataset_ids: string, list, optional Use this option if you know the exact dataset_ids for the data you want. These need to be the dataset_ids corresponding to the databases that are being searched, so in this case they need to be

the ERDDAP server’s dataset_ids. If you know station names but

not the specific database uuids, input the names as “stations” instead.
stations: string, list, optional Input station names as they might be commonly known and therefore can be searched for as a query term. The station names can be input as something like “TABS B” or “8771972” and has pretty good success.

Notes

The known_server needs to match the station name or dataset_id you are searching for.

Methods

`__init__`(kwargs)	param kwargs Can contain arguments to pass onto the base ErddapReader class
`all_variables`()	Return a DataFrame of allowed variable names.
`check_variables`(variables[, verbose])	Checks variables for presence in database list.
`count`(url)	Small helper function to count len(results) at url.
`data`()	Read in data for all dataset_ids.
`data_by_dataset`(dataset_id)	Return the data for a single dataset_id.
`find_dataset_id_from_station`(station)	Find dataset_id from station name.
`meta_by_dataset`(dataset_id)	Return the catalog metadata for a single dataset_id.
`search_variables`(variables)	Find valid variables names to use.

Attributes

`dataset_ids`	Find dataset_ids for server.
`meta`	Rearrange the individual metadata into a dataframe.

all_variables()¶

Return a DataFrame of allowed variable names.

Returns
Return type: DataFrame of variable names and count of how many times they are present in the database.

Notes

This list is specific to the given ERDDAP server. If you are using an user-input server, it will have its own known_server name and upon running this function the first time, you should get a variable list for that server.

Examples

>>> import ocean_data_gateway as odg
>>> odg.erddap.ErddapReader(known_server='ioos').all_variables()
                                   count
variable
air_pressure                        4028
air_pressure_10011met_a                2
air_pressure_10311ahlm_a               2
air_pressure_10311ahlm_a_qc_agg        1
air_pressure_10311ahlm_a_qc_tests      1
...                                  ...
wind_speed_windbird_qc_agg             1
wind_speed_windbird_qc_tests           1
wind_to_direction                     55
wmo_id                               954
z                                  37377

Or for a different known_server:

>>> odg.erddap.ErddapReader(known_server='coastwatch').all_variables()
              count
variable
abund_m3          2
ac_line           1
ac_sta            1
adg_412           8
adg_412_bias      8
...             ...
yeardeployed      1
yield             1
z                 3
z_mean            2
zlev              6

check_variables(variables, verbose=False)¶

Checks variables for presence in database list.

Parameters

variables (string, list) – String or list of strings to compare against list of valid variable names.
verbose (boolean, optional) – Print message if variables are matches instead of passing silently.

Returns

Nothing is returned. However, there are two types of behavior
if variables is not a valid variable name(s), an AssertionError is – raised and search_variables(variables) is run on your behalf to suggest valid variable names to use.
if variables is a valid variable name(s), nothing happens.

Notes

This list is specific to the ERDDAP server being used.

Examples

Check if the variable name ‘sal’ is valid:

>>> odg.erddap.ErddapReader(known_server='ioos').check_variables('sal')
AssertionError                            Traceback (most recent call last)
<ipython-input-13-f8082c9bfafa> in <module>
----> 1 odg.erddap.ErddapReader(known_server='ioos').check_variables('sal')
~/projects/ocean_data_gateway/ocean_data_gateway/readers/erddap.py in check_variables(self, variables, verbose)
    572         salinity_qc                                       954
    573         sea_water_practical_salinity                      778
--> 574         soil_salinity_qc_agg                              622
    575         soil_salinity                                     622
    576         ...                                               ...
AssertionError: The input variables are not exact matches to ok variables for known_server ioos.
Check all parameter group values with `ErddapReader().all_variables()`
or search parameter group values with `ErddapReader().search_variables(['sal'])`.
 Try some of the following variables:
                                                count
variable
salinity                                          954
salinity_qc                                       954
sea_water_practical_salinity                      778
soil_salinity_qc_agg                              622
soil_salinity                                     622
...                                               ...
sea_water_practical_salinity_4161sc_a_qc_tests      1
sea_water_practical_salinity_6754mc_a_qc_tests      1
sea_water_practical_salinity_6754mc_a_qc_agg        1
sea_water_practical_salinity_4161sc_a_qc_agg        1
sea_water_practical_salinity_10091sc_a              1

Check if the variable name ‘salinity’ is valid:

>>> odg.erddap.ErddapReader(known_server='ioos').check_variables('salinity')

count(url)¶: Small helper function to count len(results) at url.

data()¶

Read in data for all dataset_ids.

Returns
Return type: A dictionary with keys of the dataset_ids and values the data of type pandas DataFrame.

Notes

This is either done in parallel with the multiprocessing library or in serial.

data_by_dataset(dataset_id)¶

Return the data for a single dataset_id.

Returns
Return type: A tuple of (dataset_id, data), where data type is a pandas DataFrame

Notes

Data is read into memory.

property dataset_ids¶

Find dataset_ids for server.

Notes

The dataset_ids are found by querying the metadata through the ERDDAP server. Or, if running with stations() and input dataset_ids, they are simply set initially with those values.

find_dataset_id_from_station(station)¶

Find dataset_id from station name.

Parameters: station (string) – Station name for which to search for dataset_id

property meta¶

Rearrange the individual metadata into a dataframe.

Notes

This should exclude duplicate entries.

meta_by_dataset(dataset_id)¶: Return the catalog metadata for a single dataset_id.

search_variables(variables)¶

Find valid variables names to use.

Parameters: variables (string, list) – String or list of strings to use in regex search to find valid variable names.
Returns
Return type: DataFrame of variable names and count of how many times they are present in the database, sorted by count.

Notes

This list is only specific to the ERDDAP server.

Examples

Search for variables that contain the substring ‘sal’:

>>> odg.erddap.ErddapReader(known_server='ioos').search_variables('sal')
                                                count
variable
salinity                                          954
salinity_qc                                       954
sea_water_practical_salinity                      778
soil_salinity_qc_agg                              622
soil_salinity                                     622
...                                               ...
sea_water_practical_salinity_4161sc_a_qc_tests      1
sea_water_practical_salinity_6754mc_a_qc_tests      1
sea_water_practical_salinity_6754mc_a_qc_agg        1
sea_water_practical_salinity_4161sc_a_qc_agg        1
sea_water_practical_salinity_10091sc_a              1

Find available variables sorted by count:

>>>  odg.erddap.ErddapReader(known_server='ioos').search_variables('')
                                                    count
variable
time                                                38331
longitude                                           38331
latitude                                            38331
z                                                   37377
station                                             37377
...                                                   ...
sea_surface_wave_from_direction_elw11a3t01wv_qc...      1
sea_surface_wave_from_direction_elw11b2t01wv            1
sea_surface_wave_from_direction_elw11b2t01wv_qc...      1
sea_surface_wave_from_direction_elw11b2t01wv_qc...      1
sea_water_pressure_7263arc_a                            1