phicore.io.PhiDataFile

class phicore.io.PhiDataFile(fullpath: str, mode: str = 'r', force: bool = False)

__init__(fullpath: str, mode: str = 'r', force: bool = False)

Defines the structure of some archived data and methods associated to Input and Output.

Parameters:

• fullpath (str) – path to the file. If the full path contains the token “{date}” it will be replaced by the current date
• mode (str) – the mode in which to open the file see documentation of open
• force (bool) – overwrite a file even if it exists

Methods

__init__(fullpath, mode, force)	Defines the structure of some archived data and methods associated to Input and Output.
create_dataset(name, data, fletcher32, …)	Create a new dataset see h5py.Group.create_dataset
create_group(name, location)	Create a new dataset see h5py.Group.create_group
get_attrs(location)	Returns all attrs in specified location, as a dict.
list_xarray(location)	List valid xarrays in designated folder.
open(mode, backend[, filters])	Open the hdf5 file
read_xarray(location, index, …] = (), …)	Read an xarray from hdf5
write_attrs(attrs, location)	Write attributes to a h5 node
write_xarray(data, location, chunks, …)	Write an xarray to hdf5

create_dataset(name: str, data, fletcher32: bool = True, complib: str = 'blosc:lz4', complevel: int = 0, chunks: bool = None, backend: str = 'pytables', **args)

Create a new dataset see h5py.Group.create_dataset

Parameters:

• args (kwargs) – see keyword arguments from h5py.Group.create_dataset
• fletcher32 (bool) – use fletcher32 checksums
• complib (str) – compression library to use (see pytables.Filters)
• complevel (str) – the compression level (see pytables.Filters)
• chunks (bool) – chunk shape, to enable auto-chunking set to True or None with h5py, or to None with Pytables
• backend (str) – the backend to use

create_group(name: str, location: Optional[str] = None)

Create a new dataset see h5py.Group.create_group

get_attrs(location: Optional[str] = None) → Dict[str, Any]

Returns all attrs in specified location, as a dict.

Parameters:location (str) – location path inside the hdf5 file. If None, get_attrs returns head attributes.

list_xarray(location: str = '/data/') → List[str]

List valid xarrays in designated folder. Returns full path.

Parameters:location (str) – Where to look for xarrays. Default if not specified is “/data/”

open(mode: Optional[str] = None, backend: str = 'h5py', filters=None)

Open the hdf5 file

Parameters:

• mode (str) – mode in which to open the file. By default, use the mode setting provided at initialization
• backend (str) – hdf5 backend to use {“h5py”, “pytables”}
• filter (obj) – An instance of the Filters class (for compression, etc)

read_xarray(location: str, index: Tuple[int, ...] = (), chunks: Tuple[int, ...] = (), backend: str = 'h5py', mmap: bool = False)

Read an xarray from hdf5

Only one of index, chunks can be provided at a time.

Parameters:

• location (str) – path in the hdf5 file
• index (tuple) – tuple of slices specifying the subset of the dataset to load
• chunks (tuple, optional) – If chunks is provided, it is used to load the new dataset into dask arrays. chunks=() loads the dataset with dask using a single chunk for all arrays.
• backend (str) – the backend to use, one of {‘hdf5’, ‘pytables’}
• mmap (bool, default=False) –

  if True return a memory map of the data. To obtain a numpy array it is sufficient to slice or perform calculations with the obtained object.

  Note

  this option is not compatible with index or chunks,

  and returns a namedtuple (with the idential fields) instead of a real DataArray.

Returns:

X – returns an xarray.DataArray if mmap=False and a namedtuple with the same fields otherwise

Return type:

{xarray.DataArray, namedtuple}

write_attrs(attrs: dict, location: Optional[str] = None) → None

Write attributes to a h5 node

Parameters:

• attrs (dict) – the attributes to set
• location (str) – location path inside the hdf5

write_xarray(data, location: str = '/data/', chunks: bool = None, backend: str = 'pytables', complib: str = 'blosc:lz4', complevel: int = 0, **args) → None

Write an xarray to hdf5

Parameters:

• data (xarray.DataArray) – the data to save
• location (str) – path in the hdf5 file in which to save
• args (kwargs) – other keyword arguments to pass to h5py.Group.create_array
• fletcher32 (bool) – use fletcher32 checksums
• complib (str) – compression library to use (see pytables.Filters)
• complevel (str) – the compression level (see pytables.Filters)
• chunks (bool) – chunk shape, to enable auto-chunking set to True or None with h5py, or to None with Pytables
• backend (str) – the backend to use